Transcript of Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring...
- Slide 1
- Evaluation and Comparison of Multiple Convection-Allowing
Ensembles Examined in Recent HWT Spring Forecasting Experiments
Israel Jirak, Steve Weiss, and Chris Melick Storm Prediction Center
WoF Workshop, April 3, 2014
- Slide 2
- Convection-allowing ensembles ( ~ 4-km grid spacing) can
provide important information to forecasters regarding the
uncertainty of storm intensity, mode, location, timing, etc. on the
outlook to watch scale These ensembles will play an important role
in the ability of SPC to provide a more continuous flow of
probabilistic hazard information in support of WoF 2 March 201229
June 2012 24-h neighborhood prob UH 25 m 2 /s 2 24-h ensemble max
10-m Wind Speed Convection-Allowing Ensembles Overview
- Slide 3
- An experimental real-time Storm-Scale Ensemble Forecast (SSEF)
system has been produced for the NOAA Hazardous Weather Testbed
(HWT) Spring Forecasting Experiment (SFE) by OU CAPS since 2007
through CSTAR funding Comprised of 4-km convection-allowing WRF and
ARPS members: 2007: 10 members; 2008: 10 members; 2009: 20 members;
2010: 26 members (full CONUS to 30 h); 2011: 50 members (full CONUS
to 36 h); 2012: 28 members; 2013: 27 members at 00Z (to 48 hours)
and 8 members at 12Z (to 18 hours) Primarily examine explicit storm
attributes, especially hourly maximum fields (HMFs):Updraft
Helicity, Updraft Speed, 10-m AGL Wind Speed, 1-km AGL Reflectivity
Ensemble display approaches include spaghetti plots, ensemble max,
and neighborhood probabilities (more on next slide) SPC began
processing deterministic high-resolution runs from EMC and NSSL as
the Storm-Scale Ensemble of Opportunity (SSEO) in 2011 The 4-km
AFWA ensemble was made available to SPC in 2012 Convection-Allowing
Ensembles History
- Slide 4
- Traditional ensemble probabilities of HMFs from high-resolution
models are not especially useful, owing to poor agreement among
members at the grid point of these fields. Applying a binary
neighborhood approach to a storm-scale ensemble improves the
statistical results of HMFs in forecasting severe weather ROI=20-40
km Sigma=30 grid points Same approach can also be applied to
observations (e.g., radar reflectivity) for verification purposes
03 May 2008 (Harless 2010) Convection-Allowing Ensembles
Neighborhood Probabilities
- Slide 5
- HM Updraft Helicity > 25 m 2 s -2 SSEO 24-hr fcst valid 00Z
28 April
- Slide 6
- Convection-Allowing Ensembles Neighborhood Probabilities
Grid-Point Probability HM Updraft Helicity >25 m 2 s -2 SSEO
24-hr fcst valid 00Z 28 April
- Slide 7
- Convection-Allowing Ensembles Neighborhood Probabilities 40-km
Neighborhood Probability HM Updraft Helicity >25 m 2 s -2 SSEO
24-hr fcst valid 00Z 28 April
- Slide 8
- Convection-Allowing Ensembles Neighborhood Probabilities 40-km
Neighborhood Smoothed Prob HM Updraft Helicity >25 m 2 s -2 SSEO
24-hr fcst valid 00Z 28 April
- Slide 9
- OU/CAPS Storm-Scale Ensemble Forecast (SSEF) System Since 2007;
36-hr forecasts from 00z; 12z runs began in 2013 Primarily WRF-ARW;
4-km grid spacing; forecasts to 60hrs in 2014 Multi-physics,
multi-initial conditions: applies SREF perturbations to NAM ICs
Advanced physics, 3DVAR & radar data assimilation; available
for HWT/SFE SPC Storm-Scale Ensemble of Opportunity (SSEO) Since
2011; 36-hr forecasts at 00z & 12z; 7 members (2 time-lagged)
Multi-model (ARW, WRF-NMM & NMM-B), multi-physics; ~4-km grid
spacing Uses available deterministic models at SPC to process as an
ensemble Basic data assimilation through NDAS; available year-round
in SPC Air Force Weather Agency (AFWA) Ensemble Since 2012; 60-hr
forecasts at 00z & 12z; 10 members; 4-km grid spacing Single
model (WRF-ARW), multi-physics, multi-initial conditions Cold start
from downscaled global model forecasts (GFS, UM, CMC) No data
assimilation; available year-round in SPC Convection-Allowing
Ensembles System Comparison
- Slide 10
- The Fractions Skill Score (FSS) was calculated for neighborhood
probability (ROI=40 km; =40 km) of updraft helicity 25 m 2 s -2 for
the SSEO/SSEF versus practically perfect hindcasts of preliminary
severe weather reports (ROI=40 km; =120 km) during SE2011 The SSEO
had higher fractions skill score (FSS) for neighborhood
probabilities of UH 25 m 2 /s 2 during SFE2011 than the SSEF The
number of members included in the SSEF did not seem to have a
strong impact on the statistical results for neighborhood
probabilities of UH during SE2011 when verified against severe
weather reports Convection-Allowing Ensembles SFE2011 Results Nprob
UH 25 m 2 /s 2 SSEO SSEF 24 memFSS = 0.84FSS = 0.68 3-hr [NPRS]:UH
25 m 2 s -2 valid 06Z on 02 June 2011 w/ verifying reports and
practically perfect hindcast FSS 3-h periods SFE2011
- Slide 11
- Even for 6-h QPF, the SSEO received the highest subjective
ratings relative to other operational and experimental models and
ensembles during SFE2011 Statistically, the probabilistic QPF
forecasts (>0.5) from the SSEO were typically as good as (if not
better than) the SSEF during SFE2011 at various lead times
Convection-Allowing Ensembles SFE2011 Results from Tara Jensen, DTC
from Dave Novak, WPC SSEO favored over CAPS ensemble
- Slide 12
- During SFE2012, the SSEO outperformed the SSEF and AFWA in
terms of FSS for neighborhood probabilities of reflectivity 40 dBZ
(bug later found in SSEF) Subjective ratings by the SFE2012
participants of HMF ensemble forecasts tended to favor the SSEO
forecasts of UH over the SSEF and AFWA forecasts
Convection-Allowing Ensembles SFE2012 Results Hourly FSS Nprob Refl
40 dBZ 3-hr ensemble forecast ratings (max, nprob) of UH
- Slide 13
- The quality of the AFWA forecasts was less consistent than the
SSEO forecasts Some UH forecasts from the AFWA ensemble were very
good (bottom left) while others were poor (bottom right)
Convection-Allowing Ensembles SFE2012 Results
- Slide 14
- The impact of radar data assimilation in the CAPS SSEF was
evident in the first 4 hours of the 12 UTC-initialized forecast.
Otherwise, there was little statistical difference in the FSS among
the 00 and 12 UTC SSEO and SSEF. Subjective ratings of 00Z ensemble
HMFs were again favorable for the SSEO during SFE2013
Convection-Allowing Ensembles SFE2013 Results Hourly FSS Nprob Refl
40 dBZ 3-hr ensemble HMF forecast ratings (max, nprob)
- Slide 15
- Why is a poor mans ensemble (i.e., SSEO) performing as well as
formally designed ensembles? Lets consider some aspects of
configuration for convection-allowing ensembles Single model vs.
multi-model Number of members Initial conditions and IC/LBC
perturbations Physics Convection-Allowing Ensembles
Configuration
- Slide 16
- Even with the same initial conditions, clustering often occurs
by model core Generally more confident in a solution if different
model cores are in agreement Is a multi-model approach a good way
to address uncertainty in a convection-allowing ensemble?
Convection-Allowing Ensembles Configuration: Single model vs.
multi-model WRF-NMM WRF-ARW SSEO 21Z on 16 April 2011 3-hr
spaghetti plot of UH 25 m 2 s 2
- Slide 17
- Is the success of the SSEO a fortuitous balance of members with
an underforecast bias and those with an overforecast bias (Row and
Correia, 2014 AMS); not necessarily a result of using multiple
model cores? Neighborhood verification of radar reflectivity
reveals members with lower biases (e.g., NAM Nest) versus those
with higher biases (e.g., NSSL-WRF) Biases will change with
upcoming HiResW upgrade, so we may learn more this spring
Convection-Allowing Ensembles Configuration: Single model vs.
multi-model
- Slide 18
- For the way convection-allowing ensembles are currently
configured, there does not appear to be a huge benefit to running
more than ~10 members Clark et al. (2011) objectively identified
the point of diminishing returns at 9 members for 6-hr QPF at f30
and 2-km scale Convection-Allowing Ensembles Configuration: Number
of members Could run additional members to more effectively sample
the forecast PDF, but is it worth the additional computational
cost? Use a larger neighborhood? from Clark et al. (2011)
- Slide 19
- Currently, all members of the SSEO are initialized from the NAM
(including two time-lagged members), so diversity primarily arises
from multi-model/physics AFWA approach (single model, 3 different
IC/LBCs) often leads to higher, overconfident probabilities SSEF
approach not an obvious improvement over single, unperturbed IC
(i.e. SSEO), suggesting ICs not properly perturbed at this scale
Convection-Allowing Ensembles Configuration: Initial conditions and
IC/LBC perturbations AFWAOBS high probs nothing observed
- Slide 20
- In four test runs for May 2013, Kong et al. (2014) found larger
domain-average ensemble spread for multiple fields by directly
using LBCs from SREF members rather than extracting perturbations
and applying to the NAM (current strategy) NSSL-WRF ensemble this
spring will directly utilize IC/LBCs from selected SREF members
Convection-Allowing Ensembles Configuration: Initial conditions and
IC/LBC perturbations Ensemble Spread from Kong et al. (2014)
- Slide 21
- Though spread in an ensemble with physics-only diversity is
less than that from an ensemble that also includes IC/LBC
perturbations, the contribution to spread from physics diversity
can be large, including for instability fields (Clark et al. 2010)
Convection-Allowing Ensembles Configuration: Physics diversity from
Clark et al. (2010)
- Slide 22
- Convection-allowing ensembles (~4-km grid spacing) can provide
useful information to forecasters regarding the uncertainty of
storm intensity, mode, location, timing, etc. in generating
outlooks on Day 1; setting the stage for the continuous flow of
probabilistic hazard information down toward the warning scale The
SPC SSEO has proven to be as useful/skillful as formally designed
convection-allowing ensembles, which raises questions about
effective/proper configuration of these types of systems NSSL is
running eight 00Z members this spring with only IC/LBC diversity
directly from 21Z SREF members CAPS is planning to run an
experimental 4-km EnKF SSEF system this year in near real-time for
comparison with traditional SSEF forecasts Convection-Allowing
Ensembles Summary