Ensemble-4DWX update: focus on calibration and verification
-
Upload
aretha-anthony -
Category
Documents
-
view
41 -
download
0
description
Transcript of Ensemble-4DWX update: focus on calibration and verification
© 2009 UCAR. All rights reserved.ATEC-4DWX IPR, Aug 11-12 2009
National Security Applications Program Research Applications Laboratory
Ensemble-4DWX update: focus on calibration and verification
Tom Hopson, Josh Hacker, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Reminder of status in April 2009
• Running WRF V3.0.1.1
• Had just finished running E-4DWX for ATC during Dec 2008 and Jan 2009, then returned configuration to DPG
• Configured and ran E-4DWX for supporting UT Division of Air Quality
• Improved MYJ and YSU PBL height diagnosis and PBL mixing
• Added new graphics and improve post-processing flexibility (installation for GMOD and plotting historical case archive) and computing parallelisms
• Fully automated calibration just being tested
• Calibrated variables: 2-m T
ATEC-4DWX IPR, Aug 11-12 2009
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Progress since April 2009 (1 of 2)
• Improved scripting to manage the computational resources
• Increased stations from 26 to 34 (regional)
• Calibration hindcast increased from roughly 600 to over 1000 points– Increased hindcast size allows calibration of 2 additional lead-times (36hr, 42hr)
• Included new skill measure for ensemble skill-spread relationship
• Corrected over-dispersion of calibration process
• Calibrated variables: 2-m T, 2-m Td (new)
ATEC-4DWX IPR, Aug 11-12 2009
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Progress since April 2009 (2 of 2)
• Examined results from CAD study at ATC (earlier talk)
• Using performance of ensemble members to optimize DPG’s single-model configuration
• Submitted proposal for brief paper in BAMS but was rejected. Have not decide whether to revise and resubmit
• Continuing to work on scientific paper for MWR
• Downtime of HPC (recent):May 13-20, Agami4 failureJune 16-18, SGI nodeJuly 11-19, power outage caused Agami4 failureJuly 29 - August 4, Agami4 failure
ATEC-4DWX IPR, Aug 11-12 2009
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Calibration and verification
• Ensemble calibration to correct predicted distribution
• Calibration is needed for users capable of decision making with probabilistic guidance. Will be needed for foreseeable future
• Verification of different ensemble characteristics is easily completed when performing calibration
ATEC-4DWX IPR, Aug 11-12 2009
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Calibration: review and recap
• Calibration (and verification) is now fully-automated
• Utilizes “persistence” if available
• 34 sites in and near DPG
• Full calibration for all sites ~ 1X per week for each weather variable
• Using lookup tables ~ every hour (8 hrs, was 1hr)
ATEC-4DWX IPR, Aug 11-12 2009
Pro
babi
lity
calibration
Temperature [K]
Pro
babi
lity
Temperature [K]
“spread” or “dispersion”
“bias”obs
obs
ForecastPDF
ForecastPDF
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
3-hr dewpoint time series
Before Calibration After Calibration
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
42-hr dewpoint time series
Before Calibration After Calibration
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
NCAR/RAL - National Security Applications Program10
obs
Blue is “raw” ensembleBlack is calibrated ensembleRed is the observed value
Notice: significant change in both “bias” and dispersion of final PDF
(also notice PDF asymmetries)
ATEC-4DWX IPR, Aug 11-12 2009
PDFs: raw vs. calibrated
NCAR/RAL - National Security Applications Program11ATEC-4DWX IPR, Aug 11-12 2009
Troubled rank histograms
1 2 3 4 5 6 7 8 9 10Ensemble #
1 2 3 4 5 6 7 8 9 10Ensemble #
Co
un
ts0
1020
30
Co
un
ts0
1020
30
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
3-hr dewpoint rank histograms
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
42-hr dewpoint rank histograms
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
RMSE of ensemble members
3hr Lead-time 42hr Lead-time
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Significant calibration regressors
3hr Lead-time 42hr Lead-time
ATEC-4DWX IPR, Aug 11-12 2009
Station DPG S01
© 2009 UCAR. All rights reserved.
National Security Applications Program Research Applications Laboratory
Future plans
Refine lookup table process to decrease computation time (required as calibration data size continues to increase and new weather variables are added)
Implement for a) wind speed, b) wind direction, c) precipitation,d) surface pressure
Diagnose most informative model set to use operationally
Develop scheme for model points without surface observations over whole model domain
ATEC-4DWX IPR, Aug 11-12 2009