Verification of the Cooperative Institute for Precipitation Systems‘

Analog Guidance Probabilistic Products

Chad M. Gravelle and Dr. Charles E. GravesSaint Louis University - Department of Earth and Atmospheric Sciences

John P. GaganNOAA/NWS Springfield, MO

Fred H. GlassNOAA/NWS St. Louis, MO

Michael S. EvansNOAA/NWS Binghamton, NY

2009 National Weather Association Annual Meeting - Norfolk, Virginia20 October 2009

What:• If the current state of the atmosphere resembles a previous

state then the two are termed analogs, and for a period of time, the current state may evolve in a similar fashion as the past state (Lorenz 1969).

• Recent research has had success using the “perfect prog” approach to find analogs.– Hanson (2007)– Root et al. (2007)– Diomede et al. (2008)– Evans and Murphy (2008)

Why:• Provides a conditional climatology of the forecast• Confidence in NWP model output• Historical Impacts - NCDC Storm Data, COOP snow event maps,

snowfall potential, etc.• Historical framework

The What and Why of Analogs

CIPS Analog Guidance - The Big Picture

• Search the 29-yr North American Regional Reanalysis (NARR) dataset against the model forecast (GFS212-40km) for potential analogs.– 6 months over the winter season (OCT - MAR)– 6 h temporal resolution– ~21,112 potential analogs (29 winters, 6 months, 4 per day)

• Remove “duplicate” times by choosing the “best” analog over a 24-h period. 1984011512, 1984011518, 1984011600, 1984011606

• Refine and rank the resulting analogs.

• Create products that are useful for winter weather guidance.

300 HGHT500 HGHT700 FRNT850 HGHT850 TMPC850 TMPC

850 FRNT850

THTEADV

2m TMPCPMSLPWTR

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

• Establish a basic set of metrics to assess future improvements in the analog system.

• Do the top analogs consistently identify snowfall potential (e.g., does the >6” 50% probability guidance capture the area of >6” of snowfall?)?

• Assess the significance of the probabilistic snowfall guidance (e.g., what does a >4” 30% probability of snow indicate?).

• What thresholds and probabilities are the most reliable in providing snowfall guidance?

Goals of the Verification Study

Verification statistics were determined using the following:

• 35 organized (>2”) snow events occurred during the winter of 2008-2009 east of the Rocky Mountains.– 26 events occurred in one of the Midwest domains.– 14 events occurred in one of the East Coast domains.– 5 events occurred in both Midwest and East Coast domains.

• Four GFS forecasts (F036, F048, F060, F072) and the run’s associated CIPS Analog Guidance Probabilistic Products were used for each event and domain where applicable; 180 total forecast runs.

• Seasonal verification statistics represent the average over the 180 forecast runs.

Winter 08-09 Organized Snow Event Verification Data

• Winter storm affected the CONUS from the mid-Mississippi River Valley through New England between 27 and 29 January 2009.

• Continental United States mid-latitude cyclone track.

East Coast Domain 1 Case - 27-29 January 2009

• Area of >8” snow fell from western New York into interior New England with amounts >12” in higher elevations.

• A mix of snow, sleet, and freezing rain fell to the south of the heavy snow.

• Analogous to the winter storm that affected the Northeast from 15-17 December 2007 (inset). Similar track, snowfall amounts, and precipitation type. COOP Snow Event for the 72-h period ending 1200 UTC

20090129

3rd Best Analog

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

4360 1319

66714844

POD: 0.867 (1.0)FAR: 0.232 (0.0)THREAT SCORE: 0.687

(1.0)BIAS: 1.13 (1.0)

ObservedYes No

Fore

cast Yes

No

>40% probability:

Winter 08-09 Probabilistic Snow Guidance Results

• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).

• How well do the predicted probabilities of an event correspond to their observed frequencies?

• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?

• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

Attribute Diagrams

WWRP/WGNE Forecast Verification Research

• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).

• How well do the predicted probabilities of an event correspond to their observed frequencies?

• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?

• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

Attribute Diagrams

WWRP/WGNE Forecast Verification Research

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 40%.

• Observed Relative Frequency: 0.395

• Example of nearly perfect reliability.

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 50%.

• Observed Relative Frequency: 0.678

• Example of under forecasting.

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 30%.

• Observed Relative Frequency: 0.214

• Example of over forecasting.

Winter 08-09 Probabilistic Snow Guidance Results

• POD results indicate at a given snowfall threshold that the area enclosed by the lower percentiles (10-40%) contain the majority of the threshold snowfall.

• The threat scores indicate that for all thresholds there is a maximum in the 30-50% percentile range.

• The attribute diagrams reveal an expected “overforecasting” of guidance probabilities at all thresholds. However, results between 30-70% have the most reliability.

• In all thresholds, higher guidance probabilities indicate higher relative observed frequency of snowfall (i.e., good resolution).

• Though not examined, we speculate that higher guidance probabilities at low thresholds (i.e., 80% at 4”) may be a better indicator of higher end snowfall potential than low probabilities at high thresholds (i.e., 30% at 8”).

Conclusions

NWA 2009 Analog Posters:

P3.19 Using Regional Historical Analogs as Guidance for a Midwestern Winter Weather Event

P3.22 Use of a Historical Analog-Based Winter Storm Guidance Package for Forecasting a Central New York Snow Event

Questions or comments? gravelle@eas.slu.edu or gravesce@slu.edu

The analog guidance can be found at:http://www.eas.slu.edu/CIPS/ANALOG/analog.php

Questions