Introduction
The National Energy Research Scientific Computing Center
(NERSC), located at
Berkeley Lab, is the principal scientific computing facility for the
US Department of Energy (DOE)
Office of Science. More than 7,000 scientists, including many early-career scientists and students, use NERSC for basic non-classified research that involves supercomputing and massive data: modeling Earth's climate, studying properties of materials, probing the evolution of the Universe, analyzing data from collisions of subatomic particles, understanding protein structures, and more.
Evolution in NERSC's workload over the past 5 years has been driven by the explosion of data in all of these science areas. Experimental and observational science facilities (microscopes, telescopes, genome sequencers, particle accelerators) increasingly need leadership computing to make sense of all the data they generate. Artificial intelligence (AI) models designed to solve complex problems are trained using both massive data sets and supercomputer simulations. Jupyter has become the de facto platform for data science and AI, so it seems natural for data-intensive science users to expect Jupyter to work on a supercomputer.
In 2016, NERSC began exploring whether we could rely on Jupyter to provide a rich user interface to the new Cori supercomputer alongside traditional command-line interface (CLI) and remote desktop (i.e., NX) access modes. Today around 700 unique users per month are using Jupyter on Cori, a figure that has tripled over the last three years. In a typical month about 3,000 unique users login into Cori via CLI or NX, so in some sense, 20-25% of user interaction with Cori now goes through Jupyter. Jupyter has become indispensable, a primary point of entry to Cori and other systems for a substantial fraction of all NERSC users.
How It Started and How It's Going
In 2015, we observed with increasing regularity that users were trying to use SSH tunnels to launch and connect to their own Jupyter notebooks on
Edison, a previous generation supercomputer. One user even published a
blog post about how to do it. NERSC recognized that Jupyter notebooks and similar tools were a part of the emerging data science ecosystem we would need to engage, understand, and support. Faced with the challenge of how we would authenticate users and launch, manage, and proxy their notebooks, we began discussions with our colleagues at U.C. Berkeley (Fernando Pérez, Min Ragan-Kelly, and others) who were growing and expanding the Jupyter ecosystem to include institutional deployments. This led us to JupyterHub, which had been released a few months earlier, to address just those issues. JupyterHub provides a managed multi-user Jupyter service to enable access to computational environments and resources. It has a highly extensible, deployment-agnostic design built on powerful high-level abstractions (spawners, authenticators, services), and is developed by a robust, broad open-source community. From the perspective of an organization taking on the challenge of supporting any platform for a diverse and demanding user base, these characteristics represent potential strategic leverage.