GIVE ME ODDSGIVE ME ODDS

FORECASTING

ENSEMBLE

COMAP Symposium 00-1Presented by Steve TractonWednesday, 15 December

1999

KEY POINTSKEY POINTS

• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL FORMULATION

• WEATHER FORECASTING, THEREFORE, IS INHERENTLY STOCHASTIC, NOT DETERMINISTIC IN NATURE

• ENSEMBLE PREDICTION - REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL NWP (“WAVE OF THE FUTURE”) - CONSISTS OF MULTIPLE PREDICTIONS FROM SLIGHTLY DIFFERENT INITIAL CONDITIONS AND/OR WITH VARIOUS VERSIONS OF MODELS, THE OBJECTIVES BEING TO:

– IMPROVE SKILL THROUGH ENSEMBLE AVERAGING, WHICH ELIMINATES NON-PREDICTABLE COMPONENTS

– PROVIDE RELIABLE INFORMATION ON FORECAST UNCERTAINTIES (E.G., PROBABILITIES) FROM THE SPREAD (DIVERSITY) AMONGST ENSEMBLE MEMBERS

• REALITY - POSITIVE RESULTS ON BOTH COUNTS WITH OPERATIONAL GLOBAL MODEL ENSEMBLE SYSTEM; EXPERIMENTAL REGIONAL MODEL ENSEMBLES ENCOURAGING (OPERATIONAL EARLY 2000?)

– NET RESULT - ENHANCE UTILITY OF NWP FOR VIRTUALLY ALL APPLICATIONS

• REALIZING THE PRACTICAL UTILITY OF ENSEMBLES ACCOMPLISHED VIA A VARIETY OF NEW PRODUCTS DESIGNED TO CONDENSE AND MAXIMIZE INFORMATION CONTENT FOR USERS; USER FEEDBACK ESSENTIAL AND ENCOURAGED!!!

KEY CONSIDERATIONSKEY CONSIDERATIONS

• STRATEGIES FOR CREATING ENSEMBLES

– PROCDEDURES FOR GENERATING INITIAL STATE PERTURBATIONS

• RANDOM

• TIME LAGGING

• ANALYSES FROM OTHER CENTERS

• “BREEDING”

• SINGULAR VECTORS

– PERTURBING MODEL (E.G., CONVECTIVE PARAMETERIZATION) AND/OR MULTI-MODEL ENSEMBLES

– MODEL CONFIGURATION?

• RESOLUTION

• PHYSICAL SOPHISTICATION

• DOMAIN

– ENSEMBLE SIZE

NOTE: OPTIMUM STRATEGY UNKNOWN (NO CONCENSUS)!!

IDEAL: EFFECTIVE/EFFICIENT SAMPLING OF ALTERNATIVE SCENARIOS, I.E., PROBABILITY DISTRIBUTIONS. LIMITED COMPUTER RESOURCES GENERALLY REQUIRE COMPROMISES RELATIVE TO PERCEIVED OPTIMUM, E.G., MODEL RESOLUTION VERSUS ENSEMBLE SIZE)

KEY CONSIDERATIONS(CONT.)KEY CONSIDERATIONS(CONT.)

• PRODUCT DEVELOPMENT

OBJECTIVE:CONDENSE LARGE AMOUNTS OF OUTPUT INTO A “USER FRIENDLY” FORM THAT PROVIDES RELIABLE ESTIMATES OF THE RANGE AND LIKLIHOOD OF ALTERNATIVE SCENARIOS

– PRODUCTS CAN RANGE FROM DISPLAY OF ALL FORECASTS THROUGH MEANS/SPREAD AND CLUSTERS TO FULL PROBABILITIY DISTRIBUTIONS DISPLAYED IN VARIOUS FORMATS

• STATISTICAL POSTPROCESSING (E.G., BIAS CORRECTIONS, CALIBRATION OF PROBABILITIES

• ENSEMBLE OUTPUT STATISTICS

• ADDITIONAL/ALTERNATIVE PRODUCTS CONTINUAL INTERACTION AMONGST CONTINUAL INTERACTION AMONGST DEVELOPERS AND USERSDEVELOPERS AND USERS

• VALIDATION

– STANDARD SKILL SCORES

– MEASURES OF SPREAD

– MEASURES OF RELIABILITY

• EDUCATION AND TRAINING

– COMET SYMPOSIUM

– TRAINING MODULES

– ON SITE VISITS

– WEB BASED

– ??

N-AWIPS GRAPHICAL PRODUCTSN-AWIPS GRAPHICAL PRODUCTS(GEMPAK META FILES)(GEMPAK META FILES)

• SPAGHETTI CHARTS– 500 Z– 1000Z– 1000/500 TCK– MSLP– 850 T– 700 RH

• SPREAD– 1000 Z– 500 Z

• CLUSTERS– 1000 Z– 500 Z

• PROBABILITIES– 500 Z > THRESHOLDS– 700 RH > 70%– TCK <540– 250 V > THRESHOLDS– 850 T > 0C

• MSLP CENTERS

PRODUCT DEVELOPMENT INCLUDESPRODUCT DEVELOPMENT INCLUDES

• PROBABILITIES

– VIRTUALLY ALL RELEVANT AND MODEL DERIVED PARAMETERS, E.G.,

• SEVERE WEATHER INDICES• AVIATION WINDS > THRESHOLD• SENSIBLE WEATHER ELEMENTS (MODEL

DERIVED/INFERRED• CIRCULATION INDICES (E.G., BLOCKING)

• EXPANDED CLUSTERED PARAMETERS AND FOR SPECIALIZED REGIONS

• VERTICAL PROFILES

• METEOGRAMS

• ENSEMBLE DERIVED MOS

• TROPICAL STORM TRACKS

– DIRECT FROM ENSEMBLES– BACKGROUND FOR GFDL MODEL ENSEMBLES

SOME APPLICATIONSSOME APPLICATIONS

• FORECASTS OF ENSEMBLE MEAN, SPREAD, PROBABILITY DISTRIBUTIONS, ETC. OF ANY MODEL FIELD/PARAMETER OR QUANTITIES DERIVED THEREFROM ENHANCE THE ENHANCE THE UTILITY OF FORECASTSUTILITY OF FORECASTS

• APPLICABLE TO MODELS FROM VERY SHORT RANGE CLOUD SCALE THROUGH REGIONAL MESOSCALE SHORT RANGE AND GLOBAL MEDIUM RANGE TO COUPLED OCEAN/ATMOSPHERE CLIMATE PREDICTION SYSTEMS

• IMPROVE DATA ASSIMILATION SYSTEMS

• ADAPTIVE/TARGETED OBSERVATIONS

• DATA SETS FOR FUNDAMENTAL RESEARCH ON PREDICTABILITY ISSUES

NOTE: LARGE CURRENT USER COMMUNITY (OPERATIONAL GLOBAL SYSTEM) INCLUDES NCEP SERVICE CENTERS, WFO’S, USAF, OH, PRIVATES/BROADCASTERS

CLUSTER ANALYSISCLUSTER ANALYSIS

OBJECTIVELY GROUP TOGETHER FORECASTS WHICH ARE SIMILAR ACCORDING TO SOME CRITERIA

GOAL: IDENTIFY EXTREMES, GROUPINGS (CLUSTERS) WITHIN ENVELOPE OF POSSIBILITIES (“ATTRACTORS”)

• ISSUES:

– QUANTITY

• MSLP• 500 Z• ETC.

– MEASURE

• ANOMALY CORRELATION• CIRCULATION PARAMETERS• PATTERN RECOGNITION• PHASE-SPACE MEASUREMENTS

– REGION

EVALUATION/VERIFICATIONEVALUATION/VERIFICATION

• SITUATIONAL AND PHENOMENOLOGICAL CASE STUDIES (E.G., CYCLOGENESIS, FLOOD POTENTIAL)

• STATISTICAL

– STANDARD AC, RMS, SCORES (E.G., APPLIED TO ENSEMBLE MEAN VS. CONTROL, RELATIVE CLOSENESS OF MEMBERS TO ANALYSIS)

– “TALAGRAND” (VERIFICATION RANK) DIAGRAMS - MEASURES OF BIASES IN DISTRIBUTION OF ENSEMBLE MEMBERS INCLUDING FREQUENCY OF OUTLIERS)

– BRIER, RANKED PROBABILITY SCORES (PROBABILITY SKILL SCORES)

– RELIABILITY DIAGRAMS (OBSERVED VERSUS FORECAST FREQUENCIES; ENABLES CALIBRATION OF PROBABILITIES)

– MOS VERSUS ENSEMBLE POPS

– RELATIVE OPERATING CHARACTERISTICS (ROC); (EXPLICIT COMPARISON OF THE RELATIVE UTILITY OF DETERMINISTIC AND ENSEMBLE PREDICTIONS)

SHORT RANGE ENSEMBLE FORECASTING SHORT RANGE ENSEMBLE FORECASTING (SREF)(SREF)

• OBJECTIVE: DEVELOP A REGIONAL MODEL, SHORT-RANGE (0-3 DAYS) ENSEMBLE PREDICTION SYSTEM TO PROVIDE OPERATIONALLY RELEVANT AND USEFUL GUIDANCE ON THE PROBABILITY DISTRIBUTION OF WEATHER ELEMENTS OR EVENTS, ESPECIALLY FOR QPF

• GOAL: IMPLEMENT INITIAL OPERATIONAL PRODUCTION OF A REGIONAL MODEL BASED ENSEMBLE SYSTEM AND PRODUCT SUITE (SREF-I) BY ~ JANUARY, 2000

– TARGET SYSTEM:• ETA PLUS RSM MULTI-MODEL• 10 MEMBER• 40 KM RESOLUTION • ~ETA DOMAIN• RUN TWICE PER DAY• PERTURBATIONS; REGIONAL “BREEDING”

– PRODUCT SUITE:• ENSEMBLE MEAN/SPREAD CHARTS• SPAGHETTI CHARTS• PROBABILITY CHARTS• METEOGRAMS

• STATUS MILESTONES• CONDUCT PILOT STUDIES (10/96-3/98)• PARTICIPATE IN STORM AND MESOSCALE ENSEMBLE

EXPERIMENT (SAMEX) (3/98-11/98)

• SOME ISSUES– ALTERNATIVE PERTURBATION STRATEGIES– TRADEOFFS; RESOLUTION, ENS SIZE/DOMAIN– PRODUCT DEVELOPMENT– VALIDATION PROCEDURES– DATA/PRODUCT DISSEMINATION– EDUCATION AND TRAINING

STATUS/MILESTONES

COMPLETED (PILOT STUDIES)

• MAJOR TASKS/ACCOMP MAJOR TASKS/ACCOMP (CONT.)(CONT.)

– ILLUSTRATE SIGNIFICANCE ILLUSTRATE SIGNIFICANCE

OF UNCERTAINTIES IN SREFOF UNCERTAINTIES IN SREF

– DEMONSTRATE THE DEMONSTRATE THE

POTENTIAL POTENTIAL OF SREF TO OF SREF TO

PROVIDE OPERATIONALLY PROVIDE OPERATIONALLY

USEFUL INFORMATIONUSEFUL INFORMATION

– PROVIDE BASIS FOR A PROVIDE BASIS FOR A

PROTOTYPE OPERATIONAL PROTOTYPE OPERATIONAL

SYSTEMSYSTEM

STATUS/STATUS/MILESTONESMILESTONES

COMPLETED (PILOT STUDIES)COMPLETED (PILOT STUDIES)

• SOME KEY FINDINGSSOME KEY FINDINGS

– ENHANCED DIVERSITY OF ENHANCED DIVERSITY OF

SOLUTIONS (SPREAD) SOLUTIONS (SPREAD)

WITH:WITH:

• MULTI-MODEL ENSEMBLEMULTI-MODEL ENSEMBLE

• HIGHER RESOLUTION HIGHER RESOLUTION

• GLOBAL BRED (VS GLOBAL BRED (VS

“RANDOM”)“RANDOM”)

• REGIONAL ENHANCEMENTREGIONAL ENHANCEMENT

STATUS/MILESTONESSTATUS/MILESTONESCOMPLETED (PARTICIPATE IN SAMEX)COMPLETED (PARTICIPATE IN SAMEX)

• BOTTOM LINE:BOTTOM LINE:

– MUCH GAINED, MUCH GAINED, ACCOMPLISHED, LEARNEDACCOMPLISHED, LEARNED

– RESULTS GENERALLY RESULTS GENERALLY FAVORABLEFAVORABLE

– SOME DISSAPPOINMENTS SOME DISSAPPOINMENTS RELATIVE TO RELATIVE TO EXPECTATIONS, EXPECTATIONS, BUTBUT WE WE UNDERSTAND WHYUNDERSTAND WHY

– REMAIN COMMITED TO REMAIN COMMITED TO BASIC STRATEGYBASIC STRATEGY

STATUS/STATUS/MILESTONESMILESTONES

COMPLETED (PARTICIPATE IN SAMEX)COMPLETED (PARTICIPATE IN SAMEX)

• SOME KEY FINDINGSSOME KEY FINDINGS

– IMPROVED ENS MEAN IMPROVED ENS MEAN

SKILL, RELIABILITY, RPSS SKILL, RELIABILITY, RPSS

WITH MULTI-MODEL WITH MULTI-MODEL

APPROACH APPROACH

– INSUFFICENT SPREAD INSUFFICENT SPREAD

• DOMAIN TOO SMALL - DOMAIN TOO SMALL -

NEGATIVE IMPACT OF NEGATIVE IMPACT OF

BC’SBC’S

– PRECIPITATION FORECASTS PRECIPITATION FORECASTS

“WOEFUL”“WOEFUL”

• WEAK FORCING, WEAK FORCING,

SUMMER LIKE PATTERN SUMMER LIKE PATTERN

SAMEX DOMAIN

LARGE

SMALL

SAMEX SYSTEM:

MULTI-MODEL (ETA/RSM)10 MEMBERS32 KM RESOLUTIONSAMEX DOMAINREGIONAL ENHANCEMENT