Guidelines: Numerical Data CAMS - Copernicus · the data license: CAMS data license (document...

Guidelines: Numerical Data CAMS

Issued by: METEO FRANCE

Date: 05/06/2019

Copernicus Atmosphere Monitoring Service

Summary :

The aim of the user guide of Regional Air Quality data production is to give a general description of CAMS production and Data Service and to explain the users : what are the new file naming conventions implemented and what kind of data are available with the Data Service.

The different types of products are described and elements about availability of the products are given.

Table of contents1. Site of access...............................................................................................22. Production CAMS (portfolio)...........................................................................23. Description of the files...................................................................................3

3.1. NEW File Naming Convention used according to the WMO standard...............34. More about files :..........................................................................................9

4.1. Online data............................................................................................94.2. Archived data.........................................................................................94.3. Note: File format and concentration unit..................................................104.4. About domain extent.............................................................................10

5. Access through data download service...........................................................115.1. Registration and key generation..............................................................115.2. Key validity duration.............................................................................125.3. Requesting the data download service.....................................................12

Guidelines: Numerical Data CAMS | 1

This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centrefor Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.


1. Site of access

Direct data access througth the CAMS website https://www.regional.atmosphere.copernicus.eu .

Access through data download service https://download.regional.atmosphere.copernicus.eu. Requires a registration and a key generation (see 5.1 in this document). This registration implies the acceptance ofthe data license: CAMS data license (document available on the website).

2. Production CAMS (portfolio)

NRT DATA AVAILABILITY: Grib2 or Netcdf files of the ENSEMBLE and of all the 7 models CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM

The files are available for 10 pollutants + 4 varieties for pollens (Birch Pollen, Olive Pollen, Grass Pollen and Ragweed Pollen). Pollen data are available during a specific period de-pending on the type of pollen.

The Grib2/Netcdf files are available for 4 days of Forecast and one day of Analysis.

7 Models : CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM

The ENSEMBLE is the median of the 7 models.

FILES Availability of data at HH:MM UTC

Forecast (0 – 24h) for 7 Models 05:15




Forecast (0 – 24h) for ENSEMBLE 06:30




Analysis (-24H-1H) for 7 Models 11:45

Analysis (-24H-1H) for ENSEMBLE 12:00


http://www.regional.atmosphere.copernicus.eu/doc/CAMS_data_license_final.pdf

http://download.regional.atmosphere.copernicus.eu/

http://www.regional.atmosphere.copernicus.eu/


3. Description of the filesFiles are created as follows

1 file per Day (24H) per Pollutant per Model at Surface for Forecast and Analysis. 1 file per Day (24H) per Pollutant per Model at All Levels for Forecast.

Days: Forecast: (0H-24H) ,(25H-48H),(49H-72H),(73H-96H) Analysis: (-24H-1H) Yesterday

Pollutants: O3, CO, NO2, SO2, PM25, PM10, PANS, NMVOC, NO, NH3, BIRCH POLLEN,OLIVE POLLEN, GRASS POLLEN, RAGWEED POLLEN. Pollen data are available during a specific period depending on the type of pollen.Models : ENSEMBLE, CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM

Levels: Surface (0m), 50, 250, 500, 1000, 2000, 3000, 5000 m

Files are available in Grib2 format and Netcdf format for model ENSEMBLE for both Forecast (surface as well as other levels ) & Analysis (surface as well as other levels).Birch pollen forecasts, olive pollen, grass pollen ang ragweed pollen are available at surface level only.

Files are available only in Netcdf format for models :CHIMERE, EMEP, EURAD, LOTOS-EUROS, MATCH, MOCAGE, SILAM for both Forecast (surface as well as other levels) & Analysis (surface as well as other levels).Birch pollen forecasts, olive pollen, grass pollen and ragweed pollen are available at surface level only.

3.1. NEW File Naming Convention used according to the WMO standardReference document: WMO Manual on the Global Telecommunication System, Vol. 1 (See in document: paragraph procedures and file naming convention)(http://www.wmo.int/pages/prog/www/WIS/Publications/WMO_386/WMO_386_Vol_I_en.pdf)

Example : For the pollutant NMVOC forecast issued by model SILAM for the reference date 2016-06-28T00:00:00Z for the data of first 24 Hours [0H to 24H] for all the levels Surface to 5000m the new filename is :

W_fr-meteofrance,MODEL,SILAM+FORECAST+ALLLEVELS+NMVOC+0H24H_C_LFPW_20160628000000.nc


http://www.wmo.int/pages/prog/www/WIS/Publications/WMO_386/WMO_386_Vol_I_en.pdf

callto:2000,%203000,%205000

callto:50,%20250,%20500,%201000


3.1.1 NOMENCLATURE of Filename :

Symbol Meaning

W WMO product identifier pflag

_ delimiter of fields

fr location indicator (fr= FRANCE)

- delimiter within the field

meteofrance production center

, delimiter used in the product identifier

MODEL data designator

, delimiter used in the product identifier

Name of model Possible values: ENSEMBLE, CHIMERE, EMEP, EURAD, LOTOSEUROS, MATCH, MOCAGE, SILAM

+ Type of data composite

Type of data Possible Values: FORECAST/ANALYSIS/FORECASTREPORT/ANALYSISREPORT

+ Type of level composite

Type of level Possible values:ALLLEVELS (0, 50, 250, 500, 1000, 2000, 3000, 5000 m)SURFACE (0m)

+ Type of Pollutant composite

Type of Pollutant Possible values: O3, CO, NO2, SO2, PM25, PM10, PANS, NMVOC, NO, NH3, BIRCHPOLLEN, OLIVEPOLLEN, GRASSPOLLEN, RAGWEEDPOLLEN

+ Type of Timeline composite

Type of Timeline Timeline of Forecast data of 24H:0H24H ,25H48H ,49H72H, 73H96H Timeline of Analysis data of 24H : -24H-1H)


C Pflag C


LFPW Country code


Reference Date & Time Fixed length Date & Time in the format YYYYMMDDHHMMSS

. delimiter of format

Type of File Format File extension of format: nc (for Netcdf format) / grib2 (for GRIB2 format) / txt (for reports)


callto:2000,%203000,%205000

callto:50,%20250,%20500,%201000


3.1.2 NETCDF+ GRIB2 File Names examples :

3.1.2.1 ENSEMBLE : NETCDF+ GRIB2

FORECAST

1 file per DAY [24H] per pollutant , ALLLEVELS or SURFACE

ALLLEVELS : 0, 50, 250, 500, 1000, 2000, 3000, 5000 meters. POLLUTANTS provided :O3, CO, NO2, SO2 ,PM25 , PM10, PANS, NMVOC,

NO,NH3 TIMELINE of Forecast data:0H24H ,25H48H ,49H72H, 73H96H

Examples:

W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+ALLLEVELS+O3+0H24H_C_LFPW_20160629000000.ncW_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+ALLLEVELS+O3+0H24H_C_LFPW_20160629000000.grib2

SURFACE: 0 m

POLLUTANTS provided : O3, CO, NO2, SO2 ,PM25 , PM10, BIRCHPOLLEN, OLIVEPOL-LEN, GRASSPOLLEN, RAGWEEDPOLLEN.Pollen data are available during a specific period depending on the type of pollen.

TIMELINE of Forecast data:0H24H ,25H48H ,49H72H, 73H96H

Examples:

W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+SURFACE+O3+0H24H_C_LFPW_20160629000000.ncW_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+SURFACE+O3+0H24H_C_LFPW_20160629000000.grib2

Report Of Ensemble Forecast Production:

Reports in the form of text files gives the information of number of models involved in the production of Forecast of Model Ensemble

Example of Filename:

W_fr-meteofrance,MODEL,ENSEMBLE+FORECASTREPORT+0H24H_C_LFPW_20160615000000.txtW_fr-meteofrance,MODEL,ENSEMBLE+FORECASTREPORT+49H72H_C_LFPW_20160615000000.txt



ANALYSIS :

1 file per DAY [24 H] per pollutant, ALLLEVELS or SURFACE for D-1


NO,NH3. TIMELINE of Analysis data: -24H-1H


W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+ALLLEVELS+O3+-24H-1H_C_LFPW_20160628000000.ncW_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+ALLLEVELS+O3+-24H-1H_C_LFPW_20160628000000.grib2

SURFACE : 0m POLLUTANTS provided :O3, CO, NO2, SO2 ,PM25 , PM10, PANS, NMVOC,

NO,NH3. TIMELINE of Analysis data: -24H-1H


W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+SURFACE+O3+-24H-1H_C_LFPW_20160628000000.ncW_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+SURFACE+O3+-24H-1H_C_LFPW_20160628000000.grib2

Report Of Ensemble Analysis Production:

Reports in the form of text files gives the information of number of models involved in the production of Analysis of Model Ensemble


W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSISREPORT+-24H-1H_C_LFPW_20160618000000.txt



3.1.2.2 MODELS ( CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM) : only NETCDF

FORECAST :

1 file per DAY [24 H] per pollutant per model for ALLEVELS / SURFACE


NO,NH3 TIMELINE of Forecast data:0H24H ,25H48H ,49H72H, 73H96H

Examples:

W_fr-meteofrance,MODEL,EMEP+FORECAST+SURFACE+SO2+0H24H_C_LFPW_20160620000000.nc

SURFACE : 0mPOLLUTANTS provided : O3, CO, NO2, SO2 , PM25, PM10, BIRCHPOLLEN, OLIVEPOLLEN, GRASSPOLLEN, RAGWEEDPOLLEN.Pollen date are available during a specific period depending on the type of pollen.

TIMELINE of Forecast data:0H24H ,25H48H ,49H72H, 73H96H

Examples:

W_fr-meteofrance,MODEL,SILAM+FORECAST+SURFACE+O3+73H96H_C_LFPW_20160629000000.nc

ANALYSIS :

1 file per DAY [24 H] per pollutant per model, ALLLEVELS or SURFACE for DAY -1


NO,NH3 TIMELINE of Analysis data: -24H-1H

Examples:

W_fr-meteofrance,MODEL,EURAD+ANALYSIS+ALLLEVELS+CO+-24H-1H_C_LFPW_20160615000000.nc

W_fr-meteofrance,MODEL,MATCH+ANALYSIS+ALLLEVELS+PM10+-24H-1H_C_LFPW_20160612000000.nc



SURFACE : 0meters POLLUTANTS provided :O3, CO, NO2, SO2 ,PM25 , PM10, PANS, NMVOC,

NO,NH3 TIMELINE of Analysis data: -24H-1H

Examples:

W_fr-meteofrance,MODEL,EURAD+ANALYSIS+SURFACE+CO+-24H-1H_C_LFPW_20160615000000.nc

W_fr-meteofrance,MODEL,MATCH+ANALYSIS+SURFACE+PM10+-24H-1H_C_LFPW_20160612000000.nc

# NOTE : Netcdf files for models are not available if the model is absent that particular day.



4. More about files :

4.1. Online data

Online data relate to data less than 30 days old.

Forecasts and analyses are available in Grib Edition 2 or/and Netcdf Format depending on the model :

Available file format for Ensemble data are Grib Edition 2 and Netcdf . Available file format for partners data are Netcdf only.

4.2. Archived data

Archived data means data older than 30 days old. This Regional Air Quality archive is available since October 1 st , 2015 .

Below are a few more particularities about Regional Air Quality Archived production :

• Forecasts and analyses are currently available in Grib Edition2 format for archived products.

• Analyses issued from Ensemble model are available FOR ALL SPECIES since March 2016. Before this date, only O3 and NO2 species were produced by a sufficient number of models to generate reliable Ensemble products. So please, don'tuse analysis data from other pollutants than O3 and NO2 before March 2016.

Note that archived analyses from partner models could be missing, occasionnaly, in case of production problems. In such a case, please use Ensemble analyses instead.In any case, in order to get long and complete series of data, please select Ensemble products.

• Analyses are available for levels 50, 250, 500, 1000, 2000, 3000, 5000 meters since June 22, 2017.

WARNING : Should you request an all levels analysis at an earlier date, then you will get a file named :W_fr-meteofrance,MODEL,[MODEL_NAME]+ANALYSIS+ALLLEVELS+[POLLUTANT]+-24H-1H_C_LFPW_[REQUESTED_DATE].[file_format]with the surface only.



4.3. Note: File format and concentration unit

Unit used to define pollutant concentration is closely linked with the file format.Below are the list of available format and the concentration unit used for each species.

Data format Unit SpeciesGRIB Edition 2 kg/m3 CO|NH3|NMVOC|NO|NO2|O3|PANS|PM10|PM25|SO2GRIB Edition 2 m-3 BIRCHPOLLEN | OLIVEPOLLEN | GRASSPOLLEN |

RAGWEEDPOLLENNetCDF micro g/

m3CO|NH3|NMVOC|NO|NO2|O3|PANS|PM10|PM25|SO2

NetCDF grains/m3 BIRCHPOLLEN | OLIVEPOLLEN | GRASSPOLLEN |RAGWEEDPOLLEN

4.4. About domain extent

The domain extent is :

• Since October 1 st , 2015 until June 12 th , 2019 :

in longitude: from-24.95W to 44.95E.→ in longitude: from-24.95W to 44.95E. in latitude: from 30.05N to 69.95N→ in longitude: from-24.95W to 44.95E.

• Since June 12 th , 2019 :

in longitude: from-24.95W to 44.95E.→ in longitude: from-24.95W to 44.95E. in latitude: from 30.05N to 71.95N→ in longitude: from-24.95W to 44.95E.



5. Access through data download service

5.1. Registration and key generation

To interact with our data download service you must provide a valid key/token.This can be done in two steps :

Step1 – Registration/getting an account

1) First you have to register on the Regional data server registration page.Fill in the form and accept the term of the CAMS Data License.2) A mail will be sent to you recapping your user profile and explaining you how to generateyour token.3) To use CAMS regional APIs, you have now to generate this valid token. See below forthe procedure.

Step2 – Key generation

The user gets a key by making a request to an authentication web service, accessible via HTTP, by giving his account ID (UID) and the associated password (pwd):

https://geoservices.regional.atmosphere.copernicus.eu/services/GetAPIKey?user-name=YOUR_UID&password=YOUR_PASSWORD

The user then receives a reply with an XML document containing the key:

<Token>__Tiyodhhsd4G4q8856sqY884g59W6T54gSg__</Token>

Note: double underscores are part of the token.

NB: 1- Please note that the GetAPIKey web service does not verify the validity of the UID and associated password.

In practice, the service provides a key (without sending an error message) even if the user has provided a wrong information, but in this case, the key will be invalid. The real access control is performed during the service invocation.

2- if you decide to change the password sent to you during the registration step, you will have to generatea new token.

3- This unique key/token allows the use of all CAMS Regional APIs.


https://www.regional.atmosphere.copernicus.eu/doc/CAMS_data_license_final.pdf

https://data.regional.atmosphere.copernicus.eu/openwis-user-portal/srv/en/user.register.get


5.2. Key validity duration

A key has an unlimited validity duration, until the revocation of access rights to the re -source.

5.3. Requesting the data download service

The data download service is a REST API where users can request the desired products.

Example of a request:https://download.regional.atmosphere.copernicus.eu/services/CAMS50?&token=...&grid=0.1&model=ENSEMBLE&package=FORECAST_O3_SURFACE&time=0H24H&referencetime=2016-07-04T00:00:00Z&format=GRIB2

Below are the full-wording and the possible parameter values used for the API request (the “|” character means one choice between many values).

Element of the request

Authorized values

Base URL https://download.regional.atmosphere.copernicus.eu/services/CAMS50?

Token (unique value)

&token=YOUR_KEY

Grid size (in degree)

&grid=0.1

Model name &model=CHIMERE|EMEP|ENSEMBLE|EURAD|LOTOSEUROS|MATCH|MOCAGE|SILAM

Package (composite parameter)

&package=type_species_level

package type FORECAST|ANALYSIS|ANALYSISREPORT|FORECASTREPORT

package species CO|NH3|NMVOC|NO|NO2|O3|PANS|PM10|PM25|SO2|BIRCHPOLLEN|OLIVEPOLLEN|GRASSPOLLEN|RAGWEEDPOLLEN

ALLSPECIES (for analysis and forecast reports only)

package level SURFACE|ALLLEVELS

Time For analysis products:-24H-1HFor forecast products:0H24H25H48H49H72H73H96H



Reference time (ISO 8601 format)

&referencetime=YYYY-MM-DDT00:00:00Z Note: this is the requested date. Archive (in GRIB2 only) is available from Oct1st, 2015.

Format of the file

&format=GRIB2|NETCDF|TXT

Remark 1: Not all combinations of the above parameters are meaningful. E.g, you can’trequest an “ANALYSIS” product with a “FORECAST” time value.

The API manages some error cases by responding with an HTTP error code. Developpers using this API must know them and must catch them in their code:

Error case Error code (HTTP) and optional message (XML)

Incomplete request or missing parameter 400 : Parameter missing, parameter model is mandatory

Invalid token 401 : Parameter token is invalid. Please specify &token=YOUR_KEY

Erroneous parameter value 403 : Values of parameters are wrong.

Data not available 404 : Not Found

Connection refused (may be due to number of requests limitation)

503 : Please try again later

Remark 2: (Reminder) The valid HTTP response code, when the request succeeds, is 200.

Remark 3: When automating the download requests, it is important to consider the daily data availability hour as described in chapter 2. It means that it is of no interest to request the data sever too early, leading to large number of erroneous access requests...



Some examples of valid requests are therefore (ignore line returns):

https://download.regional.atmosphere.copernicus.eu/services/CAMS50?&token=__M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg__&grid=0.1&model=ENSEMBLE&package=FORECAST_O3_SURFACE&time=0H24H&referencetime=2016-07-04T00:00:00Z&format=GRIB2

https://download.regional.atmosphere.copernicus.eu/services/CAMS50?&token=__M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg__&grid=0.1&model=ENSEMBLE&package=ANALYSISREPORT_ALLSPECIES_SURFACE&time=-24H-1H&referencetime=2016-06-25T00:00:00Z&format=TXT

https://download.regional.atmosphere.copernicus.eu/services/CAMS50?&token=__M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg__&grid=0.1&model=ENSEMBLE&package=FORECASTREPORT_ALLSPECIES_ALLLEVELS&time=0H24H&referencetime=2016-07-02T00:00:00Z&format=TXT

https://download.regional.atmosphere.copernicus.eu/services/CAMS50?&token=__M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg__&grid=0.1&model=ENSEMBLE&package=ANALYSIS_PM10_ALLLEVELS&time=-24H-1H&referencetime=2017-06-20T00:00:00Z&format=GRIB2



CONTACT: For any questions please contact:

Copernicus Support mail contact: [email protected]

Copernicus CAMS and C3S Support (Public Service Desk web site):https://copernicus-support.ecmwf.int/


https://copernicus-support.ecmwf.int/

mailto:[email protected]

Guidelines: Numerical Data CAMS - Copernicus · the data license: CAMS data license (document...

Documents

Transcript of Guidelines: Numerical Data CAMS - Copernicus · the data license: CAMS data license (document...