BRINGINGITALLTOGETHER:APROTOTYPEFORAPROBABILSITICNATIONALBLEND

Acknowledgements:KevinKelleher,MattPeroutka,Yuejian Zhu,JohnWagner,GearyLayne,JeffreyCraven

1

ZoltanTothOAR/ESRL/GSD

RomanKrzysztofowiczUniversityofVirginia

MarkAntolikNWS/MDL

Malaquias PenaNWS/NCEP/EMC&IMSG

MelissaPetty,GearyLayneOAR/ESRL/GSD

AMS2017AnnualMeeting

OUTLINE / SUMMARY• Role & relevance of NWP & SPP in forecasting

– NWP provides predictive information– SPP fixes NWP miscalibration - Potentially major user impact

• Proposed framework for National Blend of Models (NBM)– Combine & calibrate guidance for NWP prognostic variables– Derive user variables from calibrated prognostic variables

• A prototype for probabilistic NBM– Bayesian Processor of Ensemble (BPE)

• Combines multiple ensemble / high res forecasts & obs/analyzed climate• Unified processing of all continuous variables• Comprehensive set of output variables

• BPE status– Algorithm, software developed– Ongoing transfer to MDL– Comparative test against EKDMOS next

2

STATISTICAL POST-PROCESSING (SPP) IN NWP• NWP is basis of weather forecasting

– Provides all predictive information beyond 6 hrs• Data assimilation, numerical model, ensemble approach

• Systematic errors limit utility of NWP– Forecasts are miscalibrated (mislabeled)

• Statistical Post-processing– Re-labels forecast information

• Traditional approach to SPP– Process each NWP guidance product individually

• NAM-MOS, GFS-MOS, NAEFS-EKDMOS, etc• Multitude of products overwhelm forecasters

• Birth of NWS National Blend of Models (NBM)– Merge & calibrate forecasts

3

• Drift of model solutions from real to model world– 1st moment calibration

• Uncertainty not captured well by single/ens fcsts– 2nd moment calibration

• Abundance of guidance products– Fusing all predictive information

Lead-time dependent behavior ofNWP Model Prog Variables (MPV) on model grid

• Missing variables– Limited model resolution in space, processes, variables

Simultaneous relationships btwUser Specific Variables on fine & MPVs on model grid

PROBLEMS WITH NWP FORECASTS

4

5

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

USERSPECIFICVARIABLES

OBJECTIVEOFSTATISTICALPOST-PROCESSINGMultiplelead

tim

es

Uncalibratedmodelvariables Calibrateduservariables

Coarsermodelgrid

Finescalegrid

6

USERSPECIFICVARIABLES

Modelgridpoints (M) <<(N)Finescalegridpoints

COMMONAPPROACHTOPOST-PROCESSING1-Step,directapproach

Multiplelead

tim

es(L)

Uncalibratedmodelvariables Calibrateduservariables

Multitudesoflead-timeandfine-scalegriddependentrelationships

O(10-100)

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

Cost:LxMx(N/M);Retrainforeachmodelupdate

7

NWPANALYSIS

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

CALIBRATEDMODELPRO

GVARS

Mo d e l g r i d ( M )

CalibrationToolbox

CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead

tim

es

(Re)forecastsample NWP(re)analysis

Cost:LxM

8

NWPANALYSIS

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

CALIBRATEDMODELPRO

GVARS

Mo d e l g r i d

CalibrationToolbox

CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead

tim

es

(Re)forecastsample NWP(re)analysis

9

NWPANALYSIS

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

CALIBRATEDMODELPRO

GVARS

Mo d e l g r i d ( M )

CalibrationToolbox

CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead

tim

es

(Re)forecastsample NWP(re)analysis

N/Msavingsfromfixing

modelproblemsonmodelgrid

10

NWPAN

ALYSES

USERSPECIFICVARIABLES

Mo d e l g r i d Finescalegrid

DerivationToolbox

DERIVATIONOFUSERVARIABLES

Noneedforhindcasts

NWPreanalysis Finescale(re)analysis

Single, instantaneousrelationship

11

NWPAN

ALYSES

USERSPECIFICVARIABLES

Modelgrid Finescalegrid

DerivationToolbox

DERIVATIONOFUSERVARIABLES

Noneedforhindcasts

NWPreanalysis Finescale(re)analysis

Single, instantaneousrelationship

Cost:Mx(N/M)Retrainonlyforreanalysisupdate

12

NWPANALYSIS

FORECASTDAY-1

FORECASTDAY-2

FORECASTDAY-N

CALIBRATEDMODELPRO

GVARS

USERSPECIFICVARIABLES

Mo d e l g r i d Finescalegrid

CalibrationToolbox

DerivationToolbox

PUTTINGITTOGETHER:2-STAGEPOST-PROCESSING

Noneedforhindcasts

Multiplelead

tim

es

(Re)forecastsample NWPreanalysis Finescale(re)analysis

Single,simultaneousrelationship

1-STEPvs2-STAGEAPPROACHTOPOST-PROCESSING• Disadvantagesof1-stepapproach

– Addressescoarsescalemodelproblemonfinescalegrid– Redundancy– Separatederivationofuservariablesforeachleadtime- Redundancy– Requireshindcast sample

• Benefitsof2-stageapproach– Addressesindependentproblemsseparately

• Betterunderstanding/separatemetricstofocusonissues– Modulardesigneasescommunityinvolvement,supportscollaboration– Shareddevelopmentofderivationtoolbox

• Vastarrayof“perfectprog”toolsavailable– UPP,DAforwardmodels,satelliteproductgeneration,etc

– Calibratedmodelvariablescanbeusedindownstreamcoupledappls• Liberatesdownstreamusersfromeverchangingmodelversionspecificbiases

– Developandusereanalysis– basedrelationships• One-waycoupledusermodels(eg,hydro)canbecalibratedwithreanalysis

– Simplifiesforecastediting– Modifycalibratedmodelprog variables• Multitudeofconsistentuservariablesautomaticallyderivedw/oaddition.edits

– Thoroughanalysisofsystematicmodelerrorsformodeldevelopers 13

RECOMMENDED APPLICATION• Calibrate all model prognostic variables

– Lead-time dependent behavior eliminated• NWP (re)analysis as proxy for truth• Calibrated forecasts statistically behave like analyses• Systematic error estimates useful for model developers

– NWP-based predictive info fused on model grid– Scales with number of model gridpoints x leadtimes

• Derive additional user variables– Fine scale analysis as proxy for truth (RUA)– Instantaneous relationships between analyses of

• Model Prog Vars (MPV) & User Specific Variables (USV)– No lead-time dependency - No need for hindcasts

– Toolbox with various types of routines• UPP, DA forward models and other physical & statistical methods

– Scales with number of fine scale gridpoints14

Mode l P rognost i c Va r i ab le sREALTIMENWPFORECASTEnsemble,HighRes.ControlHindcast,(Re)analysis

FineScaleUserVar.

Reanalysis

15

Calibrate&ConsolidateNWPOutput

CalibratedModelProgVariables

CalibratedUser

VariablesForecastEditor

NWSFORE

CASTER

S

DeriveFineScaleUserVars.

6D-DATACUBE

Product Generator

Pregenerated Products,InteractiveServices

EXTERN

ALUSERS

3rd PartyAppls.

CoupledModels

Climate&ForecastDistributionsEnsembleScenarios

H ISTOR ICAL DATA

16

BayesianProcessorofEnsemble

CalibratedModelProgVariables

CalibratedUser

VariablesProposediEdit

NWSFORE

CASTER

S

PlannedBays.Deriv.,UPP,etc

6D-DATACUBE

BPEProduct Generator,othertools

Pregenerated Products,InteractiveServices

EXTERN

ALUSERS

3rd PartyAppls.

CoupledModels

Climate&ForecastDistributionsEnsembleScenarios

Mode l P rognost i c Va r i ab le sREALTIMENWPFORECASTEnsemble,HighRes.ControlHindcast,(Re)analysis

FineScaleUserVar.

ReanalysisHISTOR ICAL DATA

BAYESIANPROCESSOROFENSEMBLE- BPE

1) Estimateclimatedistribution

2) Assessskillin&combineallpredictors

3) Castforecastinclimatecontext

4) Offermultipleoutputformats17

!

• Quantifies&reducesuncertainty• Calibrates(de-biases)ensemble

Krzysztofowicz &Toth 2008,Krzysztofowicz 2010

BAYESIANPROCESSOROFENSEMBLE(BPE)BASICS• Useclimatologyasbasis– linkwclimateprediction

– Fitparametricdistributiontolargeclimatesample• Calibrationensured,extremeshandledgracefullyw/olargehindcast

– Expressforecastsasanomaliesfromclimate• Noneedforlargehindcast sampletodefineobs.Climatology

• Transformeachvariableintonormalspace– UnifiedapproachforALLcontinuousvariables

• Assesspredictiveinformationinguidanceproducts– Multilinearregression,1st &2nd momentsaspredictors

• Wellestablishedtechnology(MOS);Forecastinformationmaximized

• Processcurrentforecast– Calibratedforecastasmodifiedclimatedistribution(percentiles)

• Mosteconomicstorage– 2modifiedparsofclimatedistribution– Prob.dist.,quantiles,posteriorensemble(scenarios/jointprob)

18

ConnectsNWPoutputw.usersviavalueaddedsteps• Role

– Repositoryofauthoritativeprobabilisticguidance• Forecastermodificationviaeditortool

• Function– Holdsanswertoallquestionsaboutfutureenviron.

• Accessinfoviainterrogatortool• Content

– Calibratedprog.vars.frompost-processingtool– Calibrateduservariables– incl.climatepercentiles

• Format– 6D - Space(3),time,variables,uncertainty

• Implementation– Phase1 – Only 2-foldexpans.ofcurrentNDFD

• Climatologicaldistributions– negligiblecost• 2BPEposteriorparameterstodescribefcst distribs

– Phase2• Ensemblemembersforjointprobabilities&scenarios

6DNDFD– CENTRALTENETOFFORECASTPROCESS

19

NATIONAL BLEND OF MODELS –BRINGING ALL THE PIECES TOGETHER

• Modular/comprehensive/expandable framework• Academic research & NOAA operations• Array of theory based techniques• End-to-end workflow w up-to-date software• Variety of forecast products & climate info• All model prog vars processed in unified way• Calibrated and informative guidance• Comprehensive array of output formats• Serving diverse applications

20

BACKGROUND

2

OUTLINE / SUMMARY• Role & relevance of NWP & SPP in forecasting

– NWP provides predictive information– SPP fixes NWP miscalibration - Potentially major user impact

• Proposed framework for National Blend of Models (NBM)– Combine & calibrate guidance for NWP prognostic variables– Derive user variables from calibrated prognostic variables

• A prototype for probabilistic NBM– Bayesian Processor of Ensemble (BPE)

• Combines multiple ensemble / high res forecasts & obs/analyzed climate• Unified processing of all continuous variables• Comprehensive set of output variables

• BPE status– Algorithm, software developed– Ongoing transfer to MDL– Comparative test against EKDMOS next

22

CURRENT GFS CONFIGURATION

23

GFSModel UPPGFS

COMPONENTS

OUTPUTRawModelPrognosticVariables

BiasedUser

Variables

APPLICATIONS Coupled Models & Other Downstream User Applications

NGGPS SYSTEM DESIGN CONSIDERATIONSComponent functionalities, links, software infrastructure

24

CalibrationStat.Post-Pr.

NWPModel

DerivationUPP,SPP,etc

NGGPSCOMPONENTS

OUTPUTRawModelPrognosticVariables

CalibratedModelProgVariables

CalibratedFineScaleUserVars

APPLICATIONS Coupled Models & Other Downstream User Applications

Two-

way

Cou

plin

g

One

-way

Cou

plin

g

BPEPROJECTSTATUS• OMMversion- Oneensemble,Multiplecontrol&Multipleauxiliary

predictors– Algorithm&documentationcomplete(U.Va)– Codescomplete(GSD)– Off-linetesting(GSD,ongoing)– TransitiontoMDL(Oct16)– TestingatMDL(Nov16)

25

• MMMversion– Multipleensemblesadded– Algorithm&documentation(U.Va,Sep16)– Codescomplete(GSD,Nov16)– Off-linetesting(GSD,Jan17)– ComparisonwEKDMOSinoperationalenvironment(MDL,Mar17)

• Assessquality&computationalspeedatobservationsites

• Finalreport(All,May2017)

LikelihoodFunctionEstimator