Edward Tollerud1, Tara Jensen2, John Halley Gotway2, Huiling Yuan1,3, Wally Clark4, Ellen Sukovich4, Paul

Oldenburg2, Randy Bullock2, Gary Wick4 (also Tressa Fowler, Barb Brown, Matt Pocernich, Paul Schultz, Chris

Harrop, Isidora Jankov)

1 ESRL/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO

3 CIRES, Boulder, CO 4 ESRL/PSD, Boulder, CO

Evaluation of QPF during the HMT-West Winter Exercise: A DTC/HMT Collaboration

with USWRP

Wally Clark

http://esrl.noaa.gov/gsd/fab

USWRP/Test-bed Workshop 2010

HMT-DTC/USWRP Collaboration

Goals and Objectives

Implementation and demonstration of new verification capabilities for high-resolution NWP

Develop DTC capabilities for ensemble model probabilistic prediction and verification

Data Impact Studies Impact studies of model physics and

parameterizations relevant to HMT research

Collaborating Agencies and Motivations

GSD: High-resolution ensemble NWP and verificationPSD, USWRP: Atmospheric Rivers and QPF; Severe eventsNCAR/RAL: MET-based real-time and object–based verification; HWT interactionsRFCs, WFOs, CA DWR: Operational reservoir management

Duality of Purpose: A Tale of Two Communities

1)Research for the USWRP, HMT, and etc.

Very strong initial focus on QPF and extreme precipitationImportant observational componentWhat physical mechanisms lead to damaging rainfall?What verification tools can provide valuable diagnostic

information?(eg. MODE vis a vis atmospheric rivers)

2) Operational Forecast Guidance for RFCs, CDWR, NWS, and etc.

Baseline comparison with previous products (i.e, GFS forecasts)Establish usefulness of high-resolution WRF ensemble forecastsWhat is value of ensemble mean?What are limits to forecast lead times for QPF?

HMT Winter Exercise – Online Real-time Verification

1)MODE and standard scores2)Selection of Vx data: gages, Stage IV, daily, 6h3)Choice of format: valid time, initialization time, monthly summary, time series, bar plots, boxplots, variable thresholds, regional selections

Real-time QPF verification for HMT-West

USWRP/Test-bed Workshop 2010

• Initialized 12 UTC 1/17/2010, Verif. Stage IV, 0.1 in. threshold• Good performance for ensemble mean (black ) and GFS (brown) • Better performance overall for heavy rainfall periods• http://verif.rap.ucar.edu/eval/hmt/2010/graphics/

18 January 2010

19 January 2010

Wind turning: diurnal forcing or pure happenstance??

• Constant Valid Time: 1200 UTC 01/20/2010

• Spatial Frequency Bias, .01 inch threshold

• Full domain, .01 threshold

• Verified with 6h Stage IV

•Not large change with lead time except longest (4-5 day)

•Anomalously Large bias for ensemble mean

Real-time QPF verification for HMT-West

USWRP/Test-bed Workshop 2010

USWRP/Test-bed Workshop 2010

• January Summary, ETS

•24h forecast, full domain

•Verified at 24h gage sites

• Segregated by threshold

• Dramatic performance hit at higher thresholds

•No real ensemble member advantages in statistics but large intger-quartile range

Real-time QPF verification for HMT-West

USWRP/Test-bed Workshop 2010

Real-time QPF verification for HMT-West

• January Summary, FAR

•24h forecast, full domain

•Verified at gage sites

• Segregated by threshold

•Good Performance by Ensemble Mean, poor for GFS

PODY verified at points with 24h gage observations

PODY verified at Stage IV grid points (6h)

Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Fields – 6h forecasts valid at 1200 UTC 20 January 2010

Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Objects – 6h forecasts valid at 1200 UTC 20 January 2010

72h 48h 24h SSM/IMODE Object Comparison of

GFS Forecasts with SSM/I Observation for 25 February, 2004 (Clear Cut Case)

Area=312Area=369 Area=306 Area=127

Wally Clark

Object-based Atmospheric River Verification of Integrated Water Vapor

http://esrl.noaa.gov/gsd/fab

Proposed Future Activities

USWRP/Test-bed Workshop 2010

Expand and maintain real-time QPF verification demonstration system and analyze results (HMT-West and eventually HMT-East)

Develop and assess prototype MODE-based spatial verification techniques to verify Eastern Pacific Atmospheric River events in near-real-time

Addition of new ensemble-based probabilistic techniques to the MET, including spatial ensemble verification prototypes

Investigate data impacts WRT verification data sets and initialization data

Prioritized Tasks: Details

Adapt HWT-based probability scoring for HMTIncorporate GFS into skill scores for base-liningAdd SREF to ensemble setDisplay Quantitative attributes from MODEAdd wind verification for HMT-WestBuild 6h gage verification optionDevelop MODE-based ensemble verification

utilitiesInclude Variable duration option for verification

periods

Prioritized Tasks: Details (Continued)

Develop Credible Downscaling Applications for model Comparisons

Extend Metviewer to Other Menu-Driven Procedures

HMT-East Prototype ProductsIntroduce Time-series Verification OptionsPublication of Winter Exercise Studies

Collaboration/Leverage with other DTC Tasks and Testbeds

Probability and the HWTVerification of winds: QNSE Metviewer and Ensemble Processing for METEnsemble Product Development, post-processing

with DETHMT, ARs, and MET/MODE

Presentations and Publications Meeting/Workshop Presentations:

BACIMO Workshop (Halley Gottway et al.)WRF User’s Conf. (Tollerud et al.; Halley Gottway

et al.)Testbed Workshop (Tollerud et al.; Clark et al.)HMT West Tutorial (Jensen and Tollerud)Verification Colloquium (Tollerud et al.)

Presentations planned for Future Weather Analysis and Forecasting; HydrologyQPF3 and GRAPES

Formal Publication Plans

• Diagnostic Results from a Real-time QPF Verification System for the HMT Winter Exercise

• Assessing the Performance of a High-resolution WRF ensemble Forecast System Relative to GFS Forecasts

• Object-based Verification for Historical Pacific Atmospheric Rivers

• Techniques for Spatial Verification using IWV and moisture Flux Objects

• QPF Verification Uncertainty and Other Impacts of Precipitation Dataset Choice