1

NCEP Ensemble Prediction System

Yuejian ZhuEnsemble team leader

EMC/NCEP/NWS/NOAA

Acknowledgements: EMC ensemble team members

2

Responsibilities of Ensemble Development (NCEP)- Assess, model, communicate uncertainty in numerical forecasts

• Present uncertainty in numerical forecasting– Tasks

• Design, implement, maintain, and continuously improve ensemble systems– Sciences

• Initial value related uncertainty• Model related forecast uncertainty

– Ensemble systems• Global – GEFS / NAEFS / NUOPC• Regional – SREF / HREF / NARRE-TL / HWAF ensemble• Climate – Contributions to future coupling CFS configuration• NAEFS/GEFS downscaled• Ocean wave ensemble (MMA/EMC)

• Statistical correction of ensemble forecasts– Current tasks

• Correct for systematic errors on model grid, correct ensemble spread.• Downscale information to fine resolution grid (NDFD)• Combine all forecast info into single ensemble/probabilistic guidance

• Probabilistic product generation / user applications– Contribute to design of probabilistic products– Support use of ensembles by

• Internal users (NCEP Service Center, WFOs, OHD/RFC forecasters and et al.)• External users (research, development, and applications)

Uncertainties &disagreements

Ensemble forecast is widely used in daily weather forecast

3

December 2012 was 20 anniversary of both NCEP and ECMWF global ensemble

operational implementation

4

Each ensemble member evolution is given by integrating the following equation

where ej(0) is the initial condition, Pj(ej,t) represents the model tendency component due to parameterized physical processes (model uncertainty), dPj(ej,t) represents random model errors (e.g. due to parameterized physical processes or sub-grid scale processes – stochastic perturbation) and Aj(ej,t) is the remaining tendency component (different physical parameterization or multi-model).

Reference: Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, Y. Zhu, 2005:"A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems“ Monthly Weather Review, Vol. 133, 1076-1097

T

tjjjjjjjj dtteAtedPtePdeeTe

00 )],(),(),([)0()0()(

Description of the main ensemble systems

Operation: NCEP-1992; ECMWF-1992; MSC-1998

Initial problem Model problem

5

Initial uncertainty

TSrelocation

Modeluncertainty

Resolution Forecast length

Ensemble members

Daily frequency

1992.12 BV None None T62L18 12 2 00UTC

1994.3 T62L18 16 10(00UTC)4(12UTC)

00,12UTC

2000.6 T126L28(0-2.5)T62L28(2.5-16)

10

2001.1 T126(0-3.5)T62L28(3.5-16)

2004.3 T126L28(0-7.5)T62L28(7.5-16)

00,06,12,18UTC

2005.8 TSR T126L28

2006.5 ETR 14

2007.3 20

2010.2 STTP T190L28

2012.2 T254L42 (0-8)T190L42 (8-16)

Evolution of NCEP GEFS configuration

6

Estimating and Sampling Initial Errors:The Breeding Method - 1992

• DATA ASSIM: Growing errors due to cycling through NWP forecasts• BREEDING: - Simulate effect of obs by rescaling nonlinear perturbations

– Sample subspace of most rapidly growing analysis errors• Extension of linear concept of Lyapunov Vectors into nonlinear environment• Fastest growing nonlinear perturbations• Not optimized for future growth –

– Norm independent– Is non-modal behavior important?

Courtesy of Zoltan Toth

References

1. Toth and Kalnay: 1993 BAMS2. Tracton and Kalnay: 1993 WAF3. Toth and Kalnay: 1997 MWR

7

P1

N1

P2

N2

P#, N# are the pairs of positive and negative

P1 and P2 are independent vectors

Simple scaling down (no direction change)

Ensemble Transform with Rescaling(2006 )

P1, P2, P3, P4 are orthogonal vectors

No pairs any more

To centralize all perturbed vectors (sum of all vectors are equal to zero)

Scaling down by applying mask,

The direction of vectors will be tuned by ET.

Rescaling

ANL

ANLANL

Bred Vector(2006)

P1 forecast

P4 forecastP3 forecast

P2 forecast

t=t1t=t0 t=t0t=t2 t=t2t=t1

Rescaling

2006

References:

1. Wei and et al: 2006 Tellus2. Wei and et al: 2008 Tellus

2Dmask

8

Fcst/guess 3hrs 9hrs6hrs

Relocate TS to

Observed position

GDAS(SANL)

FCST

GFS TS relocation

PC

N

6hrs fcst

Use ens. track information

Use GFS track information

Use GFS track information

Use ens. track

information

To separate into env. flow (EF)Strom, and storm perturbation (SP)Relocate TS to observed position

FCST

Ensemble TS relocation

Ens. rescalingFor SP (p+n)

Ens. rescalingFor EF (p+n)

Combined

2005

Reference:

Liu and et al: 2006 AMS conference extended paper 9

Tropical Storm Relocation

Example of Hurricane Track Improvement

Ivan (09/14)

Without relocation

With relocation

10

Stochastic Total Tendency Perturbation (STTP)(Hou, Toth and Zhu, 2006)

NCEP operation – Feb. 2010

htthtjtj

N

jjiii XXXXtwXX 6006

1,

'

Simplification: Use finite difference form for the stochastic term

Modify the model state every 6 hours:

Where w is an evolving combination matrix, and is a rescaling factor.

);();(,...,1

, tXTwtXTt

Xjj

Njjiii

i

Formulation:

Reference:

1. Hou and et al: 2008 AMS conference extended paper2. Hou and et al: 2010 in review of Tellus

11

STTP (continuous)Generation of Stochastic Combination Coefficients:• Matrix Notation (N forecasts at M points) S (t) = P(t) W(t) MxN MxN NxN• As P is quasi orthogonal, an orthonormal matrix W ensures orthogonality for S.• Generation of W matrix: (Methodology and software provided by James Purser).

– a) Start with a random but orthonormalized matrix W(t=0);– b) W(t)=W(t-1) R0 R1(t)

• R0, R(t) represent random but slight rotation in N-Dimensional space

Random walk (R1) superimposed on a periodic Function (R0)

wij(t) for i=14, and j=1,14

12

13

What do we expect from ensemble forecast?

What is relatively good ensemble forecast?

The Relative Measure of Predictability (RMOP)

The shading indicates the RMOP of the ensemble mean 500-hPa height at each grid point, compared to ensemble forecasts of 500-hPa height over the previous 30 days. These are in 10% increments as indicated by the color bar at the bottom of the graphic. Shading at 90% indicates that at least 9 of 10 ensemble forecasts in the past 30 days had fewer ensemble members in the same "bin" as the ensemble mean than the present forecast. In this case, the trough in the eastern US is 90% predictable relative to ensemble forecasts in the past 30 days.

The blue numbers over each box represents the percentage of time that a forecast with the given degree of predictability has verified over the past 30 days. Here, over the 90% predictability box we see that only 72% of the forecasts with 90% relative predictability at 120 hours have verified in the same climatological bin as the observed 500-hPa height at 120 hours over the past 30 days. Note that in general, the values are generally lower than the RMOP numbers below the bar. This is because:

The underlying forecast model is imperfect, The initial conditions are imprecise, and The atmosphere behaves chaotically.

We can expect verification percentages to decrease with

Increasing forecast lead time, During the warm season, and During relatively unpredictable regimes in all seasons.

In this example, we see that only 72% of the forecasts with 90% relative predictability at 120 hours have verified in the same climatological bin as the observed 500-hPa height, over the past 30 days of ensemble forecasts.

Application of theory

To illustrate the use of RMOP, consider the graphic of RMOP from the ensemble run of 00 UTC 17 December 2003 valid 00 UTC 22 December 2003 (a 120-hr forecast), found right:

Reference:Toth, Zhu and Marchok, 2001: WAF

15

NAEFS products – Metagram (examples)

Dot area is proportional to the weighting applied to that member

•= ens. mean position* = observed position

An experimental multi-model

product

Courtesy of Tom Hamill

Day at which forecast loses useful skill (AC=0.6) N. Hemisphere 500hPa height calendar year means

2007 2008 2009 2010 2011 20123

4

5

6

7

8

9

10

11

GFS GEFS NAEFS

17

NH Anomaly Correlation for 500hPa HeightPeriod: January 1st – December 31st 2012

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

GFS GEFS NAEFS

Forecast (days)

Anom

aly

Corr

elati

on

GFS – 8.0d

GEFS – 9.5d

NAEFS – 9.85d

18

Which ensemble is better?

NCEP/GEFS CMC/GEFS

NCEP - single model CMC - multi-physics (parameterizations)

19

NH 500hPa height

NH 1000hPa height

Spring 2011

Spring 2011

Spring 2012

Spring 2012

10-day forecast:

5750

5350

8470 78

72

=1.14 =1.06

=1.2 =1.08

Last implementation:Reduced RMS errors

Increasing spread

Improvement of ensemble

spread

20

Ensemble SpreadThen Now

Courtesy of Dr. Trevor Alcott 21

Ensemble RangeThen Now

Courtesy of Dr. Trevor Alcott22

0

50

100

150

200

250

300

350

400

450

0 12 24 36 48 72 96 120 144 168

GEFS2011 GEFS2012Tr

ack

erro

r(N

M)

Forecast hoursCASES-2011: 393 356 314 274 238 182 141 98 71 49CASES-2012: 439 399 357 315 277 219 178 143 119 94

Atlantic, AL01~19 (06/01~12/31/2011)Atlantic, AL01~19 (05/01~11/27/2012)

GEFS-11 GEFS T190L28 (operational run) GEFS-12 GEFS T254L42 (operational run)

23

1 2 3 4 5 6 7 80

50

100

150

200

250

GFS GEFS EC_det EC_ens

ECMWF made better forecast

Track Forecast Error for Atlantic 2012 Season

FCST HOURS 0 12 24 36 48 72 96 120 Cases 170 151 137 123 110 90 73 57

GFS – NCEP deterministic forecastGEFS – NCEP ensemble forecastEC_det – ECWMF deterministic forecastEC_ens – ECMWF ensemble forecast

NCEP made better forecast

24

Interactions between DA and EPS

Ideally, EPS and DA systems should be consistent for best performance of both.

DA provides best estimates of initial uncertainties, i.e. analysis error covariance, for EPS.

EPS produces accurate flow dependent forecast (background) covariance for DA.

)( aP

DA EPSfP

Best analysis error variances

Accurate forecast error covariance

25

Next GEFS Configurations• Time for implementation – Q2FY14• Model

– NCEP Global Forecast System (GFS) version 10 – will be implemented in the same time with GEFS– Semi-lagrangian scheme for dynamics– Check the track page (milestone has all listed items)

• Horizontal resolution– Current T254 (~50km) for 0-192 hours, T190 (~70km) for 192-384 hours– Plan for T574 SL (dynamic) and T382 (physics) (~34km) for all 16 days forecast

• Vertical resolution– Current 42 hybrid levels– Plan for 64 hybrid levels to match with hybrid DA and GFS deterministic forecast

• Memberships and runs– 21 members per run– 4 runs per day at 00, 06, 12, 18UTC

• Time step for integration– 900s for dynamic and 450s for physics

• Resource required for this configuration– Current operation on WCOSS – 84 nodes for integration and 12 nodes for post process (about 46 minutes) – This configuration will use 252 nodes for integrations and 30 nodes for post process (about 65 minutes)– Note: WCOSS has totally 640 nodes for maximum usage.

• Output– Additional: output every 3hr and 0.5 degree pgrb files for first 8-day for NAEFS data exchanges and downstream

application, such as SREF 26

Operation~50km

High resolutionSL testing

~33km

20121023 20121024

06UTC

00UTC

18UTC

12UTC

06UTC

18UTC

12UTC

00UTC

New high resolution ensemble has about

12 hours in advance to

predict Sandy

turned to North-west

27

Background !!!

28

Random Model Error – Stochastic Schemes• Stochastically-perturbed physics tendencies (SPPT) – operational

ECMWF scheme.• Stochastic total tendency perturbation (STTP) – operational NCEP

scheme• Vorticity confinement (VC) – under development at UKMet and

ECMWF• Stochastically-perturbed boundary-layer humidity (SHUM)• Perturbed convective trigger on SAS

– Testing on HWRF ensemble and global system

• Questions and issues– Will these schemes help to increase ensemble spread?– Will these schemes help to reduce missing forecast for extreme weather

events?– What is the future of physical parameterizations?

• Deterministic or probabilistic?29

Early study from Zoltan Toth: BAMS 1992

In general, breeding method is more conception, and SV is more practical.

Since we don’t know the size of initial uncertainties, we believe that smaller initial perturbations will be better (if it grows faster and catch up forecast errors)

30

Two-scale Lorenz ‘96 model

slow large-scale variables xi (i=1,2,…I)

fast small-scale variables yi,j (i=1,2,…I; j=1,2,…J)

Let I=36 and J=10 in this study. Thus, the slow large-scale variables xi could be thought of some atmospheric quantity in 36 sectors of a latitude circle, so that each large sector covers 10 degrees of longitude, while the fast small-scale variables yi,j can represent the values of some other quantity in 36*10 sectors, so that each small sector covers 1 degree of longitude in one large sector.

yi,j

31

,)(

,)(

,2,1,1,,

1,211

iyjijijijiji

J

jjiiiii

i

xbhcF

bccyyycby

dtdy

ybhcFxxxx

dtdx

Courtesy of Jessie Ma

Results from Lorenz experiments

32

0 24 48 72 96 120 144 168 192 216 240 264 288 312 336 3600

1

2

3

4

5

RM

SE

slow mode fast mode

h=0

0 24 48 72 96 120 144 168 192 216 240 264 288 312 336 3600

1

2

3

4

5

RM

SE

h=0.1

0 24 48 72 96 120 144 168 192 216 240 264 288 312 336 3600

1

2

3

4

5

RM

SE

lead time (hr)

h=0.5

0 24 48 72 96 120 144 168 192 216 240 264 288 312 336 3600

1

2

3

4

5

RM

SE

lead time (hr)

h=0.8

No interaction

High interaction

Courtesy of Dr. Jessie Ma

Experiments for 2009 Operational ImplementationT126L28 vs. T190L28 resolution, Nov. 2007 Cases

SPS works with both resolutions

--- T126L28--- T126L28 + SP--- T190L28--- T190L28 + SP

CRPSS ROC

33

Ensemble size test from 5 to 80 members

Comparing T126L28 with 80 members to T190L28 with 20 members

An Effect Configurations of Ensemble Size and Horizontal Resolution for NCEP GEFS

Juhui Ma, Yuejian Zhu, Panxin Wang and Richard Wobus

AAS 2012

34

Northern Hemisphere 500hPa Geopotential Height

CRPSS PAC

RMSE/SPREAD

Significant test for RMSE

35

Northern Hemisphere 500hPa Geopotential Height

RMSE/SPREADAC

CRPSS

RMSE AC CRPSS

20T190 1-5d 3-5d

80T126 12-16d 13-16d 11-16d

36

Unperturbed Analysis - 4D-VAR (TL1279L91)EDA-based perturbation - the difference between the perturbed (perturb all obs and sea-surface T and use SPPT to simulate random model error) and unperturbed first-guesses (TL399L91) SV-based perturbation - initial singular vectors (T42L62)

Latest ECMWF ensemble forecast system

EDA1

EDA2

EDA10…

…

SV1SV2SV3SV4SV5

mem1

mem3mem2

+

……

……

mem50

Initial Condition = Unperturbed Analysis + EDA-based perturbation + SV-based perturbation

SV6SV7SV8SV9

SV10

+

37

Buizza R, Leutbecher M, Isaksen L, et al. 2010. Combined use of EDA- and SV-based perturbations in the EPS.Newsletter n. 123, ECMWF, Shinfield Park, Reading RG2-9AX, UK, pg 22–28.

Similar to EnKF, growing slowly,

good CRPS scores

38

P. Houtekamer, ARMA

CMC’s Multi-model EPS for the assimilation (current)# Deep convection Surface scheme Mixing length Vertical mixing parameter

12345

Kain & Fritsch OldkuoRelaxed Arakawa Schubert Kuo Symétrique Oldkuo

ISBA ISBA

force-restoreforce-restoreforce-restore

BougeaultBlackadar Bougeault Blackadar Bougeault

1.00.85 0.851.0 1.0

678910

Kain & Fritsch Kuo SymétriqueRelaxed Arakawa SchubertKain & FritschOldkuo

force-restore ISBAISBA ISBAISBA

Blackadar Bougeault Blackadar Blackadar Bougeault

0.85 0.851.0 0.851.0

1112131415

Relaxed Arakawa SchubertKuo SymétriqueOldkuoKain & FritschKuo Symétrique

force-restoreforce-restore force-restore force-restore

ISBA

Blackadar Bougeault Blackadar Bougeault Blackadar

1.0 0.85 0.85 1.0 1.0

1617181920

Relaxed Arakawa SchubertKuo Symmetric Kain & FritschOldkuoRelaxed Arakawa Schubert

ISBA force-restore

ISBAISBA

force-restore

Bougeault Bougeault Blackadar BougeaultBlackadar

0.85 1.0

0.85 0.851.0

21222324

Relaxed Arakawa SchubertOldkuoKain & FritschKuo Symétrique

ISBA force-restoreforce-restore

ISBA

Blackadar BougeaultBlackadarBougeault

0.85 1.0 1.0 0.85

39

Changes to the 16 day forecast systemP. Houtekamer, ARMA/MSC, Canada

• Like in the EnKF :– Use of a more recent version of the model and the model physics,– Removing an old surface scheme,– 20 minute time step,– Use of a topography filter,

• No perturbation of model physics when convection is active,• No longer ramping down the stochastic physics in the

tropics.• With these changes the system is a lot more robust (on

occasion with the currently operational system we have to rerun an integration).

• Implementation: Jan/Feb 2013

40

IC uncertainty

Physics uncertainty

Model resolution

Forecast length

Ensemble members

Daily frequency

2001 BV Multi-model (MM, Eta and RSM)

48km 63hr 10 09, 21z

2003 Multi-model + Multi-physics (MMMP, add ETA_kf)

15

2004 MMMP (add more diverse physics schemes to Eta)

40km 87hr

2005 MMMP (add NMM and ARW)

21

2006 03,09,15,21z

2009 BV + downscaled ETR

MMMP (less Eta and more WRF, add more physics scheme to RSM)

32km 87hr 21 03,09,15,21Z

2012 BV +DownscaledETR

NEMS-NMMBWRF-NMMWRF-ARW

22km 87hrs 21 03,09,16,21Z

Evolution of NCEP SREF configuration

41

SREF system upgrade (Aug. 21, 2012) • Model Change 1. Model adjustment (eliminate Eta and RSM legacy models and add new

NEMS-based NMMB model) 2. Model upgrade (two existing WRF cores from v2.2 to version 3.3) 3. Resolution increase (from 32km/35km to 16km) 4. All models run with 35 levels in the vertical and 50 mb model top.

• IC diversity improvement 1. More control ICs (NDAS -> NMMB, GDAS -> NMM, RAP blended @ edges

w/GFS -> ARW) 2. More IC perturbation diversity (a mix of Breeding, ETR as well as a

Blending of the two) 3. Diversity in land surface initial states (NDAS, GFS, and RAP).

• Physics diversity improvement 1. More diversity of physics schemes (flavors from NAM, GFS, NCAR and

RAP)

42

List of the physics schemes Member(Model)

IC

IC perturb.

LBC physics Land surface

conv

mp

lw

sw

pbl

Sfc layer

stochastic

model

initial perturb.

nmmb_ctl NDAS BV gfs BMJ FER GFDL GFDL MYJ MYJ no NOAH NAM nonmmb_n1 gefs1

nmmb_p1 gefs2 nmmb_n2 gefs3 SAS GFS GFDL GFDL GFS MYJ no NOAH nmmb_p2 gefs4 nmmb_n3 gefs5 BMJ WSM6 GFDL GFDL MYJ MYJ NOAH nmmb_p3 gefs6 nmm_ctl GFS Blend gfs BMJ FER

(newEta)

GFDL GFDL MYJ M_Obuhov (Janjic Eta)

no NOAH GFS no

nmm_n1 gefs1 nmm_p1 gefs2 nmm_n2 gefs3 SAS FER

(new Eta)GFDL GFDL MYJ M_Ouhov

(janjic Eta)no NOAH

nmm_p2 gefs4 nmm_n3 gefs5 KF

(new Eta)

FER(new Eta)

GFDL GFDL MYJ M_obuhov(janjic Eta)

no NOAH

nmm_p3 gefs6 arw_ctl RAP ETR gfs KF

(newEta)

FER(newEta)

GFDL GFDL MYJ M_obuhov(Janjic Eta)

no NOAH RAP no

arw_n1 gefs1 arw_p1 gefs2 arw_n2 gefs3 BMJ FER

(new Eta)

GFDL GFDL MYJ M_obuhov(Janjic Eta)

no NOAH

arw_p2 gefs4 arw_n3 gefs5 BMJ FER

(new eta)GFDL GFDL MYJ M_Obuhov

(Janjic Eta)no NOAH

arw_p3 gefs6 43