Located at Berkeley Lab, NERSC is the principal scientific computing facility for the US Department of Energy (DOE) Office of Science. NERSC's mission is to accelerate scientific discovery for the Office of Science through HPC and data. Any research project funded by the Office is eligible for access to NERSC systems. More than 7000 scientists, including many early-career scientists and students, use NERSC for basic, non-classified scientific research that involves supercomputing and massive data sets: Modeling the Earth's climate, studying the properties of novel materials, probing the origin and evolution of the Universe, analyzing data from particle accelerators, understanding protein structures, and more.
This diverse population of users brings with it the challenges of ever-evolving goals, expectations, requirements, ability, software practices and preferences. Efforts to support Jupyter at NERSC began in the context of a tectonic shift in user workloads driven by the explosion of data from experimental and observational science facilities, the accelerating application of artificial intelligence (AI) to complex science problems, and the fusion of experimental data and simulation.

How It Started

By 2015, we had observed with increasing regularity that users were trying to use SSH tunnels to launch and connect to their own Jupyter notebooks on Edison, a previous generation supercomputer. One user even published a blog post \citep{a} about how to do it. NERSC recognized that Jupyter notebooks and similar tools were a part of the emerging data science landscape we would need to engage, understand, and support. Faced with the challenge of how we would authenticate users and launch, manage, and proxy their notebooks, we turned to JupyterHub, which had been released just a few months prior to address just those issues. JupyterHub has a highly extensible, relatively deployment-agnostic design built on powerful high-level abstractions (spawners, authenticators, services). It is developed by a robust open-source community clearly invested in its development and propagation. From the perspective of an organization taking on the challenge of supporting a platform for a diverse and demanding user base, these characteristics represent potential strategic leverage.