Edward Tollerud1, Tara Jensen2, John Halley Gotway2, Huiling Yuan1,3, Wally Clark4, Ellen Sukovich4, Paul
Oldenburg2, Randy Bullock2, Gary Wick4 (also Tressa Fowler, Barb Brown, Matt Pocernich, Paul Schultz, Chris
Harrop, Isidora Jankov)
1 ESRL/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO
3 CIRES, Boulder, CO 4 ESRL/PSD, Boulder, CO
Evaluation of QPF during the HMT-West Winter Exercise: A DTC/HMT Collaboration
with USWRP
Wally Clark
http://esrl.noaa.gov/gsd/fab
USWRP/Test-bed Workshop 2010
HMT-DTC/USWRP Collaboration
Goals and Objectives
Implementation and demonstration of new verification capabilities for high-resolution NWP
Develop DTC capabilities for ensemble model probabilistic prediction and verification
Data Impact Studies Impact studies of model physics and
parameterizations relevant to HMT research
Collaborating Agencies and Motivations
GSD: High-resolution ensemble NWP and verificationPSD, USWRP: Atmospheric Rivers and QPF; Severe eventsNCAR/RAL: MET-based real-time and object–based verification; HWT interactionsRFCs, WFOs, CA DWR: Operational reservoir management
Duality of Purpose: A Tale of Two Communities
1)Research for the USWRP, HMT, and etc.
Very strong initial focus on QPF and extreme precipitationImportant observational componentWhat physical mechanisms lead to damaging rainfall?What verification tools can provide valuable diagnostic
information?(eg. MODE vis a vis atmospheric rivers)
2) Operational Forecast Guidance for RFCs, CDWR, NWS, and etc.
Baseline comparison with previous products (i.e, GFS forecasts)Establish usefulness of high-resolution WRF ensemble forecastsWhat is value of ensemble mean?What are limits to forecast lead times for QPF?
HMT Winter Exercise – Online Real-time Verification
1)MODE and standard scores2)Selection of Vx data: gages, Stage IV, daily, 6h3)Choice of format: valid time, initialization time, monthly summary, time series, bar plots, boxplots, variable thresholds, regional selections
Real-time QPF verification for HMT-West
USWRP/Test-bed Workshop 2010
• Initialized 12 UTC 1/17/2010, Verif. Stage IV, 0.1 in. threshold• Good performance for ensemble mean (black ) and GFS (brown) • Better performance overall for heavy rainfall periods• http://verif.rap.ucar.edu/eval/hmt/2010/graphics/
18 January 2010
19 January 2010
Wind turning: diurnal forcing or pure happenstance??
• Constant Valid Time: 1200 UTC 01/20/2010
• Spatial Frequency Bias, .01 inch threshold
• Full domain, .01 threshold
• Verified with 6h Stage IV
•Not large change with lead time except longest (4-5 day)
•Anomalously Large bias for ensemble mean
Real-time QPF verification for HMT-West
USWRP/Test-bed Workshop 2010
USWRP/Test-bed Workshop 2010
• January Summary, ETS
•24h forecast, full domain
•Verified at 24h gage sites
• Segregated by threshold
• Dramatic performance hit at higher thresholds
•No real ensemble member advantages in statistics but large intger-quartile range
Real-time QPF verification for HMT-West
USWRP/Test-bed Workshop 2010
Real-time QPF verification for HMT-West
• January Summary, FAR
•24h forecast, full domain
•Verified at gage sites
• Segregated by threshold
•Good Performance by Ensemble Mean, poor for GFS
PODY verified at points with 24h gage observations
PODY verified at Stage IV grid points (6h)
Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Fields – 6h forecasts valid at 1200 UTC 20 January 2010
Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Objects – 6h forecasts valid at 1200 UTC 20 January 2010
72h 48h 24h SSM/IMODE Object Comparison of
GFS Forecasts with SSM/I Observation for 25 February, 2004 (Clear Cut Case)
Area=312Area=369 Area=306 Area=127
Wally Clark
Object-based Atmospheric River Verification of Integrated Water Vapor
http://esrl.noaa.gov/gsd/fab
Proposed Future Activities
USWRP/Test-bed Workshop 2010
Expand and maintain real-time QPF verification demonstration system and analyze results (HMT-West and eventually HMT-East)
Develop and assess prototype MODE-based spatial verification techniques to verify Eastern Pacific Atmospheric River events in near-real-time
Addition of new ensemble-based probabilistic techniques to the MET, including spatial ensemble verification prototypes
Investigate data impacts WRT verification data sets and initialization data
Prioritized Tasks: Details
Adapt HWT-based probability scoring for HMTIncorporate GFS into skill scores for base-liningAdd SREF to ensemble setDisplay Quantitative attributes from MODEAdd wind verification for HMT-WestBuild 6h gage verification optionDevelop MODE-based ensemble verification
utilitiesInclude Variable duration option for verification
periods
Prioritized Tasks: Details (Continued)
Develop Credible Downscaling Applications for model Comparisons
Extend Metviewer to Other Menu-Driven Procedures
HMT-East Prototype ProductsIntroduce Time-series Verification OptionsPublication of Winter Exercise Studies
Collaboration/Leverage with other DTC Tasks and Testbeds
Probability and the HWTVerification of winds: QNSE Metviewer and Ensemble Processing for METEnsemble Product Development, post-processing
with DETHMT, ARs, and MET/MODE
Presentations and Publications Meeting/Workshop Presentations:
BACIMO Workshop (Halley Gottway et al.)WRF User’s Conf. (Tollerud et al.; Halley Gottway
et al.)Testbed Workshop (Tollerud et al.; Clark et al.)HMT West Tutorial (Jensen and Tollerud)Verification Colloquium (Tollerud et al.)
Presentations planned for Future Weather Analysis and Forecasting; HydrologyQPF3 and GRAPES
Formal Publication Plans
• Diagnostic Results from a Real-time QPF Verification System for the HMT Winter Exercise
• Assessing the Performance of a High-resolution WRF ensemble Forecast System Relative to GFS Forecasts
• Object-based Verification for Historical Pacific Atmospheric Rivers
• Techniques for Spatial Verification using IWV and moisture Flux Objects
• QPF Verification Uncertainty and Other Impacts of Precipitation Dataset Choice
Top Related