Overview of Ensemble Forecasting

Steven L. Mullen

Univ. of Arizona

COMET Faculty 99 CoursePresented by Steve MullenWednesday, 9 June 1999

Benefactors• Dave Baumhefner, NCAR

• Joe Tribbia, NCAR

• Ron Errico, NCAR

• Tom Hamill, NCAR

• Harold Brooks, NSSL

• Chuck Doswell, NSSL

• Dave Stensrud, NSSL

• Eugenia Kalnay, NCEP-UO-UM-?

• Steve Tracton, NCEP

• Zoltan Toth, NCEP

• Ron Gelaro, NRL

• Rolf Langland, NRL

• Jeff Anderson, GFDL

• Mike Harrison, UKMO

• Tim Palmer, ECMWF

• Roberto Buizza, ECMWF

• Peter Houtekamer, AES

Presentation Overview

• Philosophy and Benefits of Ensembles

• Estimate of Initial Uncertainty

• Design of Initial Perturbations for EPS

• Inclusion of Model Uncertainty in EPS

• Ensemble Size

• Integration of EPS and Data Assim System

• Model Validation

• Evaluation and Utility of EPS

• Classroom Activities

Philosophy and Benefitsof Ensemble Forecasting

• Initial Condition Uncertainty (ICU)

• Probability Density Function (PDF) of initial conditions about “Truth”

• GOAL: predict evolution of PDF

• Gives information on 1st & 2nd moments Forecast uncertainty from dispersion

• Thought to be most applicable to MRF (6-10 day) and seasonal (30-90 day) forecasts

• Beneficial to SRF (06 h-2 day) for QPF

• KEY: IC error versus model error More skillful model, more beneficial PIC

• Now includes dispersion from uncertainty in initial state and model formulations

Univ Utah Ensemble12 km inner grid

Univ Utah Ensemble12 km inner grid

Precipitation Dispersion32 km NSSL Mixed Ensemble

Oct 97-Dec 97

1

2

3

4

5

6

7

8

0 3 6 9 12 15 18 21 24 27 30 33 36

forecast time (h)

rms

(mm

)

12 h

6 h

3 h

1 h

Perturbation Design

• What is the goal?

1) Robust estimate of PDF? 2) Sample extremes of PDF?3) Make up for deficiency in EPS?

• Requirements1) Properly constrained by estimates

of analysis error2) Equally-likely probability

for each perturbation field• What are some of the attributions of

current perturbation schemes for global ensemble models?

Dave Baumhefner, in progress

Ranked Probability Scoreby Model and Perturbation

0.2

0.4

0.6

0.8

24h 48hFcst Time

Grand EnsETA DiffETA BredRSM Bred

Ranked ProbabilitySkill Score

Relative to Climatology

0.0

0.1

0.2

0.3

0.4

0.5

24h 48hFcst Time

RP

SS

Grand EnsETA DiffETA BredRSM BredETA OpnlMeso ETA

Perturbation DesignConclusions

• Perturbation methods control dispersion characteristics out to 5-7 days

• SV: linear growth 1-3 days

• Random: classic error growth curve

• Random: project onto SVs 1-5 days

• BV: unique, different than analysis error, but has improved with recent changes

• Perturb strategy is unimportant after 5-7 days, once growth is strongly nonlinear

Model Uncertainties

• Specification of Subgrid Scale Processes

• GOAL: improve transient variability and increase ensemble

dispersion

• Methodologies / Philosophies1) Fixed during model integration:

different parameterization schemeschange tunable parameters 2) Stochastic element during integration:

to a scheme’s tunable parameters to model tendencies directly

• What are some of the attributes?

Rank Histogram24 h Rain Totals

24h Rank ECMWF

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Fixed

Stoch

Stochastic Cb Parameterization

Model Uncertainties Conclusions

• Increases dispersion

• Changes predictability estimates

• Model validation issues?

Model Validation

• Major Challenge for Mesoscale LAMs• Inclusion of stochastic dynamics/physics into

model requires consideration ofamplitude spatial scaletemporal scale

• Statistics for model and observations are currently lacking, so need for

long-term model integrationsbetter utilization of obs networkin absence of obs statistics, validate by comparison with explicit models

• GOAL: model PDFs match obs PDFs

Ensemble Size (N)

• Increased N or finer model resolution

• Partitioning N among perturbed IC’s and different physics parameterizations

• Depend on model, forecast objective etc.

• Choice is not always clearResolution of complex terrain

• Larger N always decreases sampling uncertaintyDiminishing returns N exceeds 10-20

• N sets limits on resolution of PDF1% event requires N of 200 or larger

• Large N warranted for accurate EPSModel with good climateAbility to simulate phenomenonSound perturbation strategy

EPS and Data Assimilation System

• Current status of Data Assimilation 3DVAR and OI techniques

homogeneousisotropic

flow independent• Kalman filter and 4DVAR can account

for these shortcomingsKalman filter expensive

4DVAR lacks cycling

• Ensemble of perturbed 6h SRFs may provide an alternative to 4DVAR

inexpensivecontains cycling

• Houtekamer and Mitchell (1998) study

Utility of EPS

• Challenge: convey info in ensemblesReduce flow dimensionality

clusters, EOFs, indices, envelopes User friendly and flexible

wide spectrum of needs and abilities

“problem of day” changes

• Enhance utility by stat. post-processingMLR MOS-techniques

Kalman filteringAI-neural

networks

• Rigorous assessment of stat. significance

• Cost-benefit analysis

Neural Net Post-ProcessingReliability Diagram 0.25”

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Forecast Probability

Obs

erve

d Fr

eque

ncy NET

RAW

MOS

NET(MOS)

Cost-Benefit AnalysisPrecipitation

Fav SitesReal-Time Ensemble Products

• NCEP MRF Ensembles

CDC Boulderwww.cdc.noaa.gov/~map/maproom/ENS/ens.html

NCEP Ensemble Homepagesgi62.wwb.noaa.gov:8080/ens/enshome.html

Univ of Utahwww.met.utah.edu/jhorel/html/models/model_ens.html

• MOS for MRF Ensembles

Penn Statewww.essc.psu.edu/~rhart/ensemble/ensmos.html

• Short-Range Mixed Ensembles

NSSL/NOAAvicksburg.nssl.noaa.gov/mm5/ensemble/index_all.html

• SAMEX? NCEP ETA/RSM?

Ask Kelvin D. and Steve T., respectively!

Univ. Utah

Univ. Utah

MRF Ensemble MOSfrom Penn State

NSSL Experiment Ensemble Model Physics/Uncertainty

FNMOC/UA Products

Classroom ActivitiesAppropriate for Undergrads

• Probabilistic ForecastingQPF

Use MOS thresholds

MAX-MIN

Credible Interval Forecasts

(e.g. Prob. within 2oF)

Be willing to stumble and be humbled!

• Hands-On NWPBarotropic Model Experiments