ICOS data flow

ICOS produces quality-controlled observational data, results from modelling related to greenhouse gases, as well as synthesis reports. These data come from national measurement networks, from ICOS Thematic Centers and from modelers in the greenhouse gas research community.

We produce about 25-30 TB of sensor data per year, along with a bit under 1 GB of processed data products and 5-20 TB of elaborated data products. The ICOS processed data products are mostly time-series of observed variables, for example atmospheric carbon dioxide concentrations, given for every 30-minute interval. Every observation station delivers such time series for 50-100 variables, representing the measurements made every day of every year.

On this page:

Data flow at ICOS

The following diagram shows the standardised ways of handling data within ICOS.

 

data flow

The diagram shows the flow of data within ICOS. The explanations below relate to the numbers in the diagram.


Image explained:


1. Data are collected at ICOS measurement stations. There are more than 170 sites in 16 countries, making up three networks of Atmosphere, Ecosystem and Ocean stations. Some sites or stations are equipped with up to 100 data-collecting instruments and sensors.

2. The ‘raw’ data are stored in a safe repository as soon as possible. Environmental observation data are very precious, because, if the data are lost, it is not possible to go back and re-measure. Therefore, copies of all ‘raw’ data are transferred to safe, long-term storage at a trustworthy data centre. Typically, this is done within 24 hours of collecting the data.

3. The observation data are then passed on to the Thematic Centres for expert processing. Each station sends its sensor data to the relevant Thematic Centre – Atmosphere, Ecosystem or Ocean – for processing and quality control.

4. In addition to the three Thematic Centres, the Central Analytical Laboratories (CAL) is a part of the ICOS Central Facilities. CAL consists of Flask and Calibration Laboratory and the Central Radiocarbon Laboratory. They provide gas analyses and calibration gases.

5. The ICOS Thematic Centres take care of and process the observations following standardised procedures. At the Thematic Centres, the data are checked, quality controlled and, if needed, gap-filled. Some data types, for example, fluxes, require quite extensive processing. Finally, the data are ag-gregated into half-hourly or hourly averages.

6. Quality-controlled data are delivered by the Thematic Centres. When ready to be released, the finalized observation data are transferred to the Carbon Portal. How often new data become available varies, but the aim is to reduce as much as possible the time between collection and release.

7. The Carbon Portal is responsible for organising and taking care of all ICOS datasets. The Carbon Portal is the ‘one-stop shop’ for ICOS data prod-ucts and offers advanced search, visualization and downloading services. The portal is also responsible for all ICOS data management (such as assigning identifiers, DOIs, to make them easily cited), for tracking how ICOS data are used and for long-term archiving. Finally, it will also provide user-community support.

8. Users can freely and openly access the ICOS data and may also contribute. Everyone who wants to access, view or download ICOS data products is welcome to do so. Any use of data should be properly referenced and acknowledged. Some ICOS data users, including atmospheric and ecosystem modellers, also use the Carbon Portal to distribute their research results. If you have a data product that is based on ICOS data, you are invited to make your data product available through the Carbon Portal.

9. All ICOS data products are safely stored in the ICOS repository. Copies of all data products that are handled by the Carbon Portal are stored in a safe, long-term manner in the ICOS repository. This also includes all metadata that are necessary to make sense of the data themselves. The repository is based on the B2SAFE service provided by the Euro-pean Data Infrastructure (EUDAT).

10. Descriptions of the ICOS data products and their contents must be easily found. ICOS is operating its own data catalogue, but we also share in-formation about all the ICOS data products and how to access them with other data providers, including environmental and climate-data portals.

11. ICOS data can be effectively and quickly sent to other computing centres. Our collaboration with European e-infrastructure providers, such as EUDAT and EGI, makes it easy to transfer ICOS data to and from high-performance computing centres.