TIGGE LAM PLAN Eighth meeting of the GIFS-TIGGE Working Group Geneva, 22 to 24 February 2010
Slide 1 GO-ESSP Paris. June 2007 Slide 1 (TIGGE and) the EU Funded BRIDGE project Baudouin Raoult...
-
Upload
nickolas-dawson -
Category
Documents
-
view
217 -
download
0
Transcript of Slide 1 GO-ESSP Paris. June 2007 Slide 1 (TIGGE and) the EU Funded BRIDGE project Baudouin Raoult...
GO-ESSP Paris. June 2007
Slide 1
Slide 1
(TIGGE and) the EU Funded BRIDGE project
Baudouin Raoult
Head of Data and Services Section
ECMWF
GO-ESSP Paris. June 2007
Slide 2
Slide 2
THORPEX was established in May 2003 by the 14th WMO Congress as a ten-year international global atmospheric research and development programme
THORPEX is under the auspices of the WMO Commission for Atmospheric Sciences (CAS)
THORPEX Goals:
- To reduce and mitigate natural disasters
- To fully realise the societal and economic benefits of improved weather forecasts
GO-ESSP Paris. June 2007
Slide 3
Slide 3
TIGGE
THORPEX Interactive Grand Global Ensemble
Key objectives of TIGGE:
- Enhanced collaboration on development of ensemble prediction, internationally and between operational centres and universities
- New methods of combining ensembles from different sources and of correcting for systematic errors
- A deeper understanding of the feasibility of interactive ensemble system responding dynamically to changing uncertainty
- The development of a prototype future Global Interactive Forecasting System (GIFS)
GO-ESSP Paris. June 2007
Slide 4
Slide 4
The TIGGE core dataset
Global ensemble forecasts to around 14 days generated routinely at different centres around the world
Outputs collected in near real time and stored in a common format for access by the research community
Easy access to long series of data is necessary for applications such as bias correction and the optimal combination of ensembles from different sources
GO-ESSP Paris. June 2007
Slide 5
Slide 5
Homogeneity of the TIGGE database
Homogeneity is paramount for TIGGE to succeed-The more consistent the archive the easier it will be to develop applications
There are three aspects to homogeneity:-Common terminology (parameters names, file names,…)
-Common data format
-Definition of an agreed list of products (Parameters, Steps, levels, …)
NCEP ECMWF
precipitation precip TP
Mean sea level pressure prmsl MSL
Temperature at 2 meters t2m 2T
GO-ESSP Paris. June 2007
Slide 6
Slide 6
Completeness
The objective is to have 100% complete datasets at the Archive Centres
Completeness may not be achieved for two reasons:
- The transfer of the data to the Archive Centre fails
- Operational activities at a data provider are interrupted and back filling past runs is impractical
Incomplete datasets are often very difficult to use
Most of the current tools used for ensemble forecasts assume a fixed number of members from day to day
- These tools will have to be rewritten
GO-ESSP Paris. June 2007
Slide 7
Slide 7
Strong governance
Precise definition of:- Which products: list of parameters, levels, steps, units,…
- Which format: GRIB2
- Which transport protocol: UNIDATA’s LDM
- Which naming convention: WMO file name convention
Only exception: the grid and resolution- Choice of the data provider
- Best possible model output
One must do what one requires from others- Sample dataset available
- Various GRIB2 tools, “tigge_check” validator, …
- Scripts that implement exchange protocol
GO-ESSP Paris. June 2007
Slide 8
Slide 8
Phased implementation of the archive
Phase 1: multiple instances, low development effort
ECMWF NCAR CMA
Phase 1ArchiveCentres
GO-ESSP Paris. June 2007
Slide 9
Slide 9
Building the TIGGE database
Three archive centres: CMA, NCAR and ECMWF
Ten data providers:
- Already sending data routinely: ECMWF, JMA (Japan), UK Met Office (UK), CMA (China), NCEP (USA)
- Coming soon: BOM (Australia), CPTEC (Brazil), KMA (Korea), MSC (Canada), Météo-France (France)
Exchanges using UNIDATA LDM, HTTP and FTP
- Firewall issues with China
Operational since 1st of October 2006
43 TB, growing by ~ 1 TB/week
GO-ESSP Paris. June 2007
Slide 10
Slide 10
Quality Assurance
GO-ESSP Paris. June 2007
Slide 11
Slide 11
Accessing TIGGE
Portals at NCAR and ECMWF. CMA portal to be developed
ECMWF portal offers:
- Access to offline data
- Aggregation along any axis (date, level, parameter, origin, ensemble, …)
- Provision of multi-model data on a single grid (regridding to any lat/lon grid)
- Sub-area selection
- Reduces volumes to be downloaded by many order of magnitude
GO-ESSP Paris. June 2007
Slide 12
Slide 12
Phased implementation of the archive
Phase 2: distributed approach, higher development effort
GO-ESSP Paris. June 2007
Slide 13
Slide 13
BRIDGE
Bridge is a 2 years project funded by the EC under the FP6-IST programme.
It will demonstrate the benefits of GRID technology for international cooperation, in particular between Europe and China
This work focuses on the development of interoperable Grid infrastructures (CNGrid, GRIA)
Three applications: Aircraft Design, Meteorology, Drug Design
GO-ESSP Paris. June 2007
Slide 14
Slide 14
Meteo scenario: exploring TIGGE phase 2
Distributed processing on distributed data, across the two GRID middleware
Each site hosts only part of the data
Each site offers basic operations on the data (e.g. computing an average)
Strategy: minimize data transfers
- Run operations at data location
- Decompose operations in simpler ones
- Most of the time intermediate results are much smaller
GO-ESSP Paris. June 2007
Slide 15
Slide 15
EPS Products
Examples
- Ensemble mean
- Standard deviation
- Clustering
- Probability of weather events
- Extreme Forecast Index
- EPSgram
Some products can be decomposed in simpler operations on a subset of members
- Ensemble means
Some products need all the members
- Clustering
GO-ESSP Paris. June 2007
Slide 16
Slide 16
Example 1: ensemble mean
Data requests describes 10 fields
6 are available from ECMWF
4 are available from CMA
S1 = sum(6 ECMWF fields) performed at ECMWF
S2 = sum(4 CMA fields) performed at CMA
Intermediate results and associated metadata moved to site were user invoked request: (S1,6) and (S2,4)
Final result A=(S1 + S2)/(6+4) computed locally and returned to user
GO-ESSP Paris. June 2007
Slide 17
Slide 17
Example 2: EPSGram
Send lat/lon location at each sites
Receive list of values back
Compute distributions
Generate plot
GO-ESSP Paris. June 2007
Slide 18
Slide 18
Non-decomposable operation
Data service
Data
Data repository
ECMWF/GRIA
Data service
Data1
Data repository
CMA/GOS
Operation service
Result
Data 2
GO-ESSP Paris. June 2007
Slide 19
Slide 19
Decomposable operation
Data service
Data
Data repository
ECMWF/GRIA
Operation Service
Result 1 Result 2
Result
DWD/GRIA
Operation service
Result
Data service
Data
Data repository
CMA/GOS
Operation service
Result
GO-ESSP Paris. June 2007
Slide 20
Slide 20
Operations as Workflows
Study a selection of EPS products
Classify them into decomposable and non-decomposable
For decomposable operations- Implement sub-operations as web services
- Deploy several instances of these web services
- Describe full operation as a workflow
- User will not see the workflow, but the high level operation
There may be several ways to execute a workflow- Because the same sub-operations are offered by several sites
- Choose the “cheapest” workflow
- Cost of a workflow based on amount of data moved
GO-ESSP Paris. June 2007
Slide 21
Slide 21
Conclusion
TIGGE Phase 1 is progressing well
- Strong governance
- Very good working relationship between CMA, NCAR and ECMWF
BRIDGE project gives us the funding to explore how to implement TIGGE Phase 2