Julia, Python and R - These are not only programming languages but also name givers to a computational tool that has become a rising star in the world of science: Ju-pyt-er notebooks. Working in the style of a narrative, the web application helps data scientists to combine code, explanatory text and multimedia resources in one document. Jupyter notebooks make science more visual and easily understandable. They allow researchers to access ICOS data directly and work with it in a well-documented and transparent way.
Karolina Pantazatou is a junior scientific programmer at the ICOS Carbon Portal. She has been working with the web-application for several years and concludes: “The notebooks are a great way of exploring ICOS data and sharing computational ideas.” She explains that the software is not only a great aid for scientists but also for students and people working in education. Moreover, Jupyter notebooks can be an effective gate-opener to the general public and bear great potential for knowledge exchange outside and within the ICOS community.
Simone Pieber, postdoctoral researcher at Empa, a research institution in Switzerland, benefits from the interactive features of Jupyter notebooks. Her colleagues operate the high-alpine ICOS station at Jungfraujoch, 3850 meters above sea level. “It is extremely valuable to obtain measurements at this peculiar location because it allows insights on the atmosphere’s composition. The measurements are also representative of background conditions in Central Europe,” Simone explains. “We operate a laser-based instrument at Jungfraujoch which was developed at Empa roughly 12 years ago. It provides real time data on carbon dioxide and stable isotope ratios. These data allow me to characterize regional air masses which are transported to Jungfraujoch,” she adds.
For her data analysis, Simone Pieber uses a range of tools and software, including Jupyter. “The Jupyter notebooks allow accessing simulations from the ICOS Carbon Portal. I can combine these data with the simulation we generate at Empa and further data for my analysis.” Currently, Simone uses Jupyter to access the simulation output from the ICOS Carbon Portal. “I really appreciate the Jupyter notebooks as they enable me to work interactively with my collaborator Ute Karstens who is a scientist at the ICOS Carbon Portal. For example, if she has new updates on the simulations, or when she thinks there is something interesting to look into, she can make modifications in the corresponding Jupyter notebook which I can easily implement afterwards. That’s very, very useful!”, Simone explains.
Jupyter notebooks make scientific collaboration easier
Julia Kelly, PhD student of Lund University, attended a course on how to use Jupyter notebooks as a teaching tool last year and is very much convinced of them. “I honestly think this tool is amazing, I see a lot of potential in it,” says Julia. “The problem with programming often is that, if you are not experienced, you look at the script and find yourself confronted with a wall of text and numbers. Students get frustrated with programming assignments because they do not know what is going on. With Jupyter notebooks, you can break the script down into different sections and, suddenly, what you have to do becomes much clearer,” she explains. The notebooks can work as an exercise platform where students can work individually or in groups. Another advantage offered by Jupyter is that students can submit reports and assignments. This makes it easy for teachers to grade reports and keep up to date with the learning process of the class.
Julia Kelly’s research involves upscaling the estimates of greenhouse gas fluxes. “A lot of my science work is about analysing data and that is a challenging process because there are many different factors you have to take into account. Any software that makes this process easier, is already a big help. With Jupyter notebooks, you can mix different components together – text, graphs, code – and present them in a nice and easily understandable way. This is very convenient for group work. You can run pieces of the code and, at the same time, discuss with your partners what is going on. Another plus: You can send Jupyter notebooks to someone else and they can run and read through them in their own time. They can ask you questions and add comments and follow along with your strain of thought. All of these things are very useful and bring us one step closer to FAIR data,” Julia states.
ICOS constantly tries to improve the FAIRness of data and Jupyter notebooks
By aiming to tick all the boxes of the FAIR concept, the ICOS Carbon Portal strives to make ICOS data more equitable. The acronym reflects what scientific data should be: findable, accessible, interoperable and reusable. Karolina Pantazatou emphasises on the fact that ICOS Carbon Portal constantly tries to improve the FAIRness of data. In the same spirit, ICOS also strives to improve the FAIRness of Jupyter notebooks. She explains: “ICOS open access Jupyter notebooks are stored and made available for download on GitHub. GitHub handles the version control of notebooks, keeping track of changes and providing the means to access earlier and current versions. Each one of these notebooks contains a detailed description of the implemented workflow. Additionally, all notebooks are described with rich metadata in external, markdown documents on GitHub. These documents also include information regarding the license. Notebooks usually need to be run in other Jupyter instances than the ones they were developed in. To ensure interoperability and reusability, we have stored a docker image file with the names of all Python modules used in the notebooks. This way, users may create a new Jupyter instance based on the content of the aforementioned docker file and then run the notebooks. Caution is needed when selecting which version of a Python module to install, as the notebooks might not work if the wrong version is installed.” All the above simplify the process of sharing computational narratives and make Jupyter a powerful tool for young as well as established scientists.
Jupyter notebooks can work as gate-opener for general public
In a simplified version, Jupyter notebooks can also be used as an interactive tool to inform the general public on climate research. By hiding the underlying code, they can turn into fascinating documents packed with interactive maps, graphs videos and quizzes. Karolina Pantazatou has developed educational notebooks using ICOS data for the Swedish Science Centres, an organisation dedicated on stimulating children's, student's and the general public's interest in science and scientific innovations. “We created Jupyter notebooks for high school and university students to promote how programming can be used to monitor the effects of climate change. However, Jupyter notebooks can also be a powerful tool to communicate recent developments in science and research results in an engaging and playful way,” she says.
Jupyter as web-based interactive development environment for notebooks, code, and data have had a surprising rise in popularity over the last couple of years. This is because of their user-friendliness and universal usability. For ICOS, they are a great tool to make climate change research more effective and accessible within and outside of the science community. They can be a key to open science!
GENERAL SUMMARY
Jupyter Notebooks are a computational tool that help researchers to create and share documents containing live code, equations, visualizations and narrative text.
Main advantages for scientists:
• Possibility to share work within a project group
• Makes the process of documenting workflows easy
• Introduces a completely new way of presenting scientific output/research results in e.g. conferences or lectures
• Enables other researchers (from the same or different discipline) to make use of past work and findings
Main advantages for the educational sector:
• Jupyter notebooks can serve as an exercise platform for students to work individually or in a group
• Students can write and submit reports in the form of a notebook
• Introduces users/students to how to work with data in a way that supports the FAIR principles from an early stage
Main advantages for the public:
• Helps to inform the general public about developments in science using a combination of intuitive, interactive visualisations and explanatory text
• Supports the connection between science and a broad audience by explaining scientific phenomena in a playful way, e.g. quizzes