ICOS evaluated by experts: “A global power in the greenhouse gas and climate change arena”

25 May 2021
Evaluation Report cover

ICOS has been successfully set up and has established itself as a global power in the greenhouse gas and climate change arena. A Five-Year Evaluation Report of ICOS states that this has been possible through the strong commitment of the scientific community, and ICOS’s engagement with all key global organisations that influence international policymaking. By producing standardised, high-precision and long-term data on greenhouse gases in Europe, ICOS has become the base of a scientific value-chain supporting global policy- and decision-making to combat climate change and its impact.

ICOS has been evaluated by an external Evaluation Committee at the end of its five-year implementation period. The evaluation was organised around five areas: management, financial management, internal engagement and integration, data and user expectations, and international cooperation. A set of 36 key performance indicators (KPIs) were also established and evaluated during the process. 

In general, the Evaluation Committee found that ICOS completed its implementation phase very successfully, with a well-established governance as well as operational and financial management processes. It has made a significant progress in providing temporal and spatial data on greenhouse gases in Europe, all channeled robustly and efficiently through the ICOS Carbon Portal. Also, a high degree of integration was found across the different elements of ICOS. The outputs of ICOS in the form of data, publications, as well a scientific and outreach events have been rising strongly.

The Chair of the Committee, Andrew Harrison from Diamond Light Source, UK, says he was surprised to find how many different parts ICOS integrates: “You go across countries, across domains, funneling all the data through one Carbon Portal. It is a huge challenge to get this complex system to work together, and we at the Evaluation Committee were impressed by how much has been achieved by ICOS in this relatively short time.” 

The work continues – next steps on providing top-quality greenhouse gas data

Next, ICOS will build on the achievements and improve the areas identified by the evaluation, utilising the recommendations, Key Performance Indicators and criteria developed in the process. 

Andrew Harrison names a few important factors he finds crucial for continued success of ICOS: “I believe that extending the spatial coverage beyond the current member countries is going to be critical for the future. Extension of the station network in all three domains is needed to answer the key questions on global GHG behaviour, and will provide a basis for national and European-scale annual GHG budgets and accountings.”

The second thing Andrew Harrison finds equally crucial is to have a long-term plan on how ICOS will be maintained and developed technically and scientifically. “How are you going to maintain and upgrade the existing technology and equipment, and how you can identify and develop new technology and make sure that it is funded. A state-of-the-art infrastructure is necessary to provide reliable results and the keenest insights into the greenhouse gas balances.”

The report and the respective Evidence Report providing further insight into the evaluation findings will next be the source-to-go for all the decision makers of ICOS. The General Assembly has already discussed the findings; it prioritised for instance the development of a technological step-up-improvement process of ICOS, continuing the facilitation of cross-domain research initiatives such as the Drought studies, and further working with the KPI’s established. The General Assembly will also transmit the evaluation results to the European Strategic Forum on Research Infrastructures, ESFRI, as a role model for the permanent monitoring of Landmark RIs.

ICOS Director General Werner Kutsch is very pleased with the assessment: “This external view on our organisation shows the strength of the ICOS community which has proved that setting up such a complex distributed infrastructure is possible. Some 15 years ago this was a dream, now we have evidence. Additionally, I am grateful that the Evaluation Committee provided us pointers to improve ICOS further. That allows us to develop even better services for scientists as well as for policy makers, to fight the climate crisis.” 

A review process in the midst of the pandemic 

The review was based on documentation and data provided by ICOS and on surveys of a wide range of ICOS staff and stakeholders.  A two-day online meeting between the Evaluation Committee and many of those surveyed provided even deeper insights.

In addition to Andrew Harrison as a Chair, the committee included an expert for each ICOS domain and for finance. The members were Leif Anderson, Department of Marine Sciences, University of Gothenburg, Sweden, Ulla Wandinger, Leibniz Institute for Tropospheric Research, Germany, Dan Yakir, Weizmann Institute, Israel, and Maki Yamada, Swedish Research Council, Sweden. 

This being the first time a distributed infrastructure was evaluated, the committee and the ICOS Head Office were figuring out much of the process along the way, as well as the KPI’s to measure. Further, the pandemic forced the team to work fully virtually: “It would have been so much easier to see the people, visit stations, feel the atmosphere and see daily interactions.” All these experiences gave good ground to provide recommendations for the actual review process as well.

Read the Evaluation Report.

The Evaluation report has also a complementary ‘Evidence Report’ compiled by ICOS ERIC, providing further insight and details in the areas evaluated. Read the Evidence Report.