A Proposed National ESMF-based Modeling System (NEMS)
description
Transcript of A Proposed National ESMF-based Modeling System (NEMS)
1
A Proposed National ESMF-basedModeling System (NEMS)
Louis Uccellini31 July 2007
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle
Per
cent
Use
d
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
2
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
GD
AS
GF
S an
al
NA
M an
al
CFS
RTOFS
SR
EF NAM
AQ
GFSHUR
RD
AS
Current (2007)
GENS/NAEFS
Current - 2007
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
~ Executive Summary ~
3
Forces for Change• Increasing emphasis on ensemble approaches
– Multi-model ensembles• SREF• NAEFS• Climate Forecast System
• Entering the NPOESS era– More rapid access to
hyperspectral data– GPS soundings– Higher resolution surface
radiance data• All models run within ESMF
– Models run concurrently – Hybrid vertical coordinate– Coupled– Spanning all scales
• Operational Earth System model – more explicit hydro, climate and ecosystems applications
Model Region 1
Model Region 2
Global/Regional Model Domain
ESMF-based System
~ Executive Summary ~
4
CFSMFS
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
WAV
CFS & MFS
GENS/NAEFSGFS
Next Generation PrototypePhase 4 - 2015
Regional
Rap Refresh
Global
HURSREF
Reforecast
Hydro / NIDIS/FF
Hydro
NAM
GDAS
RDAS
RTOFS RTOFSAQ
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
AQ
Computing factor: 81
~ Executive Summary ~
5
Overview
• A conceptual prototype for a National next-generation production suite
• ESMF-based Modeling System– Planned capabilities– Planned components– Criteria for inclusion
• Summary
“Model of the day” is not a scientifically supportable solution for the future
6
Concurrent execution of global and regional forecast models (Phase 1)
Model Region 1
Model Region 2
Global/Regional Model Domain
• Common Modeling Infrastructure is ESMF-based• In-core Solution
– Single executable for analysis, all model domains– No file writes to (or reads from) disk except for saving output– Provides
• Concurrent execution of ensembles– Single executable, multiple members– “On-the-fly” breeding throughout the forecast from all members
• More efficient execution of rapid updating– In-core updating for analysis increments – Regional (CONUS, Alaska, Hawaii, Caribbean, Guam & Puerto Rico) – Global (if requirements and resources)
Analysis
7
Concurrent execution of global and regional forecast models (Phase 2)
Model Region 1
Model Region 2
Global/Regional Model DomainAnalysis
Local Solution
• Real time boundary and initial conditions available hourly
– “On-demand” downscaling to local applications• Similar to current hurricane runs but run either
– Centrally at OR– Locally (B.C, I. C. retrieved from on-line data)
• No boundary or initial conditions older than 1 hour – Flexibility for “over capacity” runs (e.g. Fire Wx, Hurricane)
• Using climate fraction must be planned• No impact on remainder of services
• For NEXTGEN: A consistent solution from global to local with a single forecast system and ensembles providing estimate of uncertainty
8
Analysis--------------
OtherForecastSystems
Physics(1,2,3)
ESMF Utilities(clock, error handling, etc)
Post processor & Product GeneratorVerification
Resolution change
1-11-21-32-12-22-3
ESMF Superstructure(component definitions, “mpi” communications, etc)
Multi-component ensemble+
Stochastic forcing
Coupler
Dynamics(1,2)
Application Driver
National ESMF*-based Modeling System (NEMS)(uses standard ESMF compliant software)
* Earth System Modeling Framework (NCAR/CISL, NASA/GMAO, Navy (NRL), NCEP/EMC), NOAA/GFDL
2, 3 etc: NCEP supported thru NUOPC, NASA, NCAR or NOAA institutional commitmentsComponents are: Dynamics (spectral, FV, NMM, FIM, ARW, FISL, COAMPS…)/Physics (GFS, NRL, NCAR, GMAO, ESRL…)
9
Planned NEMS Capabilities• Components and capabilities of the NEMS infrastructure
– Configuration control• Domain• Resolution (horizontal, vertical)• Standardized fixed field generation (WPS - topography, land use, etc)• Tracer definition
– Observations ingest, formatting, QC, etc libraries– Nesting (static and moving, telescoping, 1-way, 2-way)– Concurrent ensemble execution (single executable, multiple members)– Data assimilation (3D-var and advanced techniques)– In-core updating for analysis increments and boundary conditions– Model dynamics and physics including
• Atmosphere• Ocean• Land surface and hydrology• Air Quality and trace gases
– Post-processor and product generator– Standard operational verification– Documentation for operational and research users
10
Planned NEMS Capabilities (cont)• Modeling Research
– Global and regional– Institutionally supported components
• Atmosphere – GFS (NCEP)– NOGAPS (Navy)– FV (NASA, GFDL)– NMM (NCEP)– ARW (ESRL, NCAR, AFWA)– COAMPS (Navy)– FIM (ESRL)– FISL (NCEP)
• Ocean– MOM4 (GFDL)– HYCOM (NCEP, Navy)
• Land surface and hydrology– Noah (NCEP)– VIC (Princeton, U. Wash)– MOSAIC (NASA)– Sacramento (OHD)– Smirnova LSM (ESRL)
• Air Quality and trace gases– CMAQ (EPA, ARL)– GOCART (NASA)– NAAPS (Navy)
Under constructionWill include in future
11
Planned NEMS Capabilities (cont)• Operational Models (NCEP only)
– Global Forecast System• GFS
– Global Ensemble (GENS, NAEFS)• GFS
– North American (NAM)• NMM
– Short-range Ensemble (SREF)• NMM• ARW• Physics diversity
– High Resolution Window (HRW)• NMM• ARW
– Air Quality (AQ)• CMAQ
– Land Surface & Hydrology (LIS)• Noah
– Rapid Refresh (RR)• Dynamics TBD• GSD physics
– Hurricane (HUR)• NMM for hurricanes• HYCOM
– Seasonal Climate Forecast (CFS)• GFS for climate• MOM4
Will include in Operational NEMS
12
Criteria for Inclusion in NEMS• Research
– Adherence to ESMF standards– Institutional support for code– Participation in system evolution
• Operations– Research criteria plus:– Standard “Transition to Operations” criteria
• NCEP: http://www.emc.ncep.noaa.gov/– Forecast performance benefits– IT compatibility– Efficiency– Sustainability
• AMOP (Navy)• AFWA IPT process (USAF)
13
Supplementaries
14
Enhance NWS forecast services
• Provide greater focus on high-impact events• Increase information content in NWS guidance products
– Probabilistic and ensemble methods
• Accelerate accuracy of numerical guidance• Take on additional environmental information service
responsibilities• Increase analysis and verification services for NWS field
offices– Efficient Grid Initialization– Analysis of Record (and RTMA)– Gridded Verification
15
Additional forcing and enabling factors
• Computing– ~1,000 processors 10,000 processors– Emphasis on highly scalable processes– Ensembles scale linearly and “perfectly”
• Common Modeling Infrastructure– ESMF-based system
• Provides maximum flexibility for dynamics and physics components• Common post-processor and product generator, configured as an ESMF component
– Concurrent coupling– Concurrent ensemble runs
• Observations (number and availability)– Advanced Polar and Geostationary sounders (~100 X greater)
• NPOESS (<60 minutes globally) – 2015 (or later)• METOP (1-4) – 2007 • NPP (90-120 minutes globally) – 2009• GOES-R – 2013 (or later)
– Next-generation Doppler radar• Next-Generation Air Traffic-control System (NEXTGEN)
– Geographically consistent solutions– Global to terminal scales– At least hourly updating globally
• “Completing the Forecast,” “Fair Weather” reports
16
Additional forcing and enabling factors (cont)
• Advanced probabilistic (post-processing) techniques becoming available through NAEFS project– Bias correction– 2nd moment correction– Classical (frequentist) and Bayesian techniques
• Advanced dissemination strategies– E.g. NOMADS (“Fat server/Thin Client” technology)
• Maturing, ensemble-based, probabilistic systems offer the most potential benefit– Hypothetical ideal state may be
• Combination of dynamics (N~2) and physics ( M~3) components and/or
• Stochastic physics within the best model– Managed (N, M) component diversity based on controlled
experiments and assessment of value-added• CFS & DEMETER work (EMC & CPC)• Components institutionally supported (operational or major
research institution)
17
Conceptual Prototype:Numerical Forecast Guidance (1)
• Information should be optimally combined from all available sources– Domestic and international
models (global, e.g. NAEFS and regional)
– Same product format for all time scales (unified post-processing)
• Progress in numerical forecast system development should not be constrained by post-processing– Improved products come from
development of improved systems
Impact of Models on Day 1 Precipitation Scores
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Th
reat
Sco
re
Human(HPC)
ETA
Linear(Human(HPC))Linear (ETA)
• Robust training and outreach program must– Accompany new probabilistic-
based system– Support NWS Field Operations,
commercial sector and international users
– Support advanced dissemination of forecast information on all time and space scales
18
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
GD
AS
GF
S an
al
NA
M an
al
CFS
RTOFS
SR
EF NAM
AQ
GFSHUR
RD
AS
Current (2007)
GENS/NAEFS
Current - 2007
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
19
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
Refo
recast
NA
M an
al
CFS
SR
EF NAM
GFS
WAV
HUR
Next Generation PrototypePhase 1 - 2009
3-hourly GDAS (2)
1-hourly RDAS (6)
GENS/NAEFS
RTOFSAQ
GF
S A
nal
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
Added• 1-Hourly RDAS• 3-Hourly GDAS• Reanalysis/ Reforecast
Computing factor: 3
20
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
Refo
recast
GF
S A
nal
NA
M A
na
l
CFS & MFS
GFS
WAV
HUR
GENS/NAEFS
Next Generation PrototypePhase 2 - 2011
GDAS
SR
EF
RDAS
RTOFSHydro / NIDIS AQ
NAM
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
AQ
Added• Hydro/NIDIS products Moved• GFS ½ h earlier Expanded• Hurricane & wave products Incorporated• Multi-domain rapid updating + Fire Weather
Computing factor: 9
21
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
Refo
recast
GF
S A
nal
NA
M A
na
l
CFS & MFS
GFS
WAV
HUR
GENS/NAEFS
Next Generation PrototypePhase 3 - 2013
GDAS
SREF
RDAS
RTOFS
NAM
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
AQHydro / NIDIS/FFAQ
Computing factor: 27
Added• Flash flood products Moved• SREF concurrent to NAM Expanded• Reforecast capability
22
CFSMFS
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Pe
rce
nt
Us
ed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
WAV
CFS & MFS
GENS/NAEFSGFS
Next Generation PrototypePhase 4 - 2015
Regional
Rap Refresh
Global
HURSREF
Reforecast
Hydro / NIDIS/FF
Hydro
NAM
GDAS
RDAS
RTOFS RTOFSAQ
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
AQ
Computing factor: 81
Added• Hourly GDAS Moved• GFS concurrent to NAM & SREF Expanded• Hurricane capability (hires)
23
CFS & MFS
NCEP Production SuiteWeather, Ocean & Climate Forecast Systems
Version 3.0 April 9, 2004
0
20
40
60
80
100
0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00
6 Hour Cycle: Four Times/Day
Per
cen
t U
sed
RUCFIREWXWAVESHUR/HRWGFSfcstGFSanalGFSensETAfcstETAanalSREFAir QualityOCEANMonthlySeasonal
CFSMFS
WAVGFSRegional
Rap Refresh
GlobalSREFReforecast
Hydro
NAM
GDAS
RDAS
RTOFS
RT
OF
S
CFS & MFSAQ Hydro / NIDIS/FF AQ
GENS/NAEFS
>100% of 2015 computing
Next Generation PrototypeFinal – 2017+
NCEP Production SuiteWeather, Ocean, Land & Climate Forecast Systems
GLOBAL NEXTGEN
HU
R
Computing factor: > 240
ECOSYSTEMS
SPACE WEATHER
HENS
24
EVOLUTION of the Next-Generation NCEP Production Suite (1)
Phase Date Computer
Power*Human
ResourcesImplementation
of new services
Date**
1 2009 3 +8(2 - 2008
3- 2007
2 – 2008
1 - 2009)
Increased forecast accuracy with hourly RDAS/LDAS, 3 hourly GDAS/GLDAS (2), and
Advanced Data Assimilation (2010)
Reanalysis-Reforecast (5)
HABs (1)
2010
2 2011 9 +6(2 – 2007
1 – 2008
3 – 2009)
ESMF-based in-core system (5)
Downscaled 4 Domain RR with Fire Wx
Global hourly Aviation products
Land-HYDRO-NIDIS seasonal products (1)
2013
3 2013 27 +3(1- 2010
2 – 2011)
Concurrent NAM, SREF
Coupled Land-Hydro & Flash Flood (FF) guidance (1)
Biogeochemical tracers (2)
2016
* Relative to NCEP’s 2007 computer
** If no new resources or new unfunded items
25
EVOLUTION of the Next-Generation NCEP Production Suite (2)
Phase Date Computer
Power*Human
ResourcesImplementation
of new services
Date**
4 2015 81@ +5(2 – 2011)
(3-2010)
Hourly GDAS+ Concurrent GFS
+
Fully coupled global atmosphere-ocean
Advanced global ensemble system
Hurricane ensemble (1)@
Dynamic storm surge ensemble (1)@
NEXTGEN support@
2018
Final 2017
(+)
240 +2(2 – 2015)
Concurrent GEFS/NAEFS
Hurricane ensemble
Dynamic storm surge ensemble
Full ecosystem support (2)
NEXTGEN support
2021
* Relative to NCEP’s 2007 computer
@ additional 3x computing upgrade in 2009required
+ If positive upgrade to services
** If no new resources or new unfunded items