PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff...
-
date post
20-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff...
![Page 1: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/1.jpg)
PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS
Jeff Baars
Cliff Mass
Mark Albright
University of Washington, Seattle, Washington
This work was supported in part by the DoD Multidisciplinary University Research Initiative (MURI) program administered by the Office of Naval Research Under Grant
N00014-01-10745.
![Page 2: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/2.jpg)
The point of it all…
Vislocky and Fritsch (1997), using 1990-1992 data, saw that an average of 2 or more MOS’s (CMOS) outperformed individual MOS’s and many human forecasters in a forecasting competition.How has the story changed since then?And how well do CMOS, MOS & the NWS perform during extreme conditions? In different seasons? In different regions?
![Page 3: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/3.jpg)
Data
July 1 2003 – Jan 1 2004 (6 months).30 stations, all at major WFO sites.Maximum and minimum temperature, and POP.
![Page 4: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/4.jpg)
Data (con’t)Consensus MOS (CMOS)– simply an average of 4 MOS’s: AMOS, EMOS, MMOS, NMOS.12Z-issued forecast from NWS matched against previous 00Z forecast from models.
NWS has 00Z model data available, and has added advantage of watching conditions develop since 00Z.Models of course can’t look at NWS, but NWS looks at models.
Forecasts going out 48 (60) hours, so in the analysis there are:
Two maximum temperatures (MAX-T),Two minimum temperatures (MIN-T), and Four 12-hr POP forecasts.
![Page 5: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/5.jpg)
Data (con’t)NWS MOS definition for MAX-T and MIN-T and for POP.Observed precipitation data converted to binary rain/no-rain data for Brier Score calculations.
![Page 6: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/6.jpg)
Maximum and Minimum Temperature
![Page 7: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/7.jpg)
Total MAE for the 6 forecasts
Maximum temperature
Minimum temperature
![Page 8: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/8.jpg)
MAE, temperature, by forecast period
![Page 9: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/9.jpg)
Bias Time Series, all stations
![Page 10: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/10.jpg)
MAE, Maximum temperature period 1, by station
WestMtnWest
South-west
EastSouth Mid-
west
![Page 11: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/11.jpg)
MAE by forecast period, large departure from climatology
![Page 12: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/12.jpg)
Probability of Precipitation
![Page 13: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/13.jpg)
Total Brier Score for all 6 forecasts
![Page 14: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/14.jpg)
Brier Scores by forecast period
![Page 15: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/15.jpg)
Brier Scores by forecast period, large departure from climatology
![Page 16: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/16.jpg)
ConclusionCMOS shows equal or superior forecast skill compared to NWS and individual MOS’s when all time periods are considered.
True for max and min temperatures and POPs.
The NWS forecasts show superior forecast skill for max. temperatures during large departures from climatology.http://www.atmos.washington.edu/~jbaars/mos_vs_nws.html
![Page 17: PERFORMANCE OF NATIONAL WEATHER SERVICE FORECASTS VERSUS MODEL OUTPUT STATISTICS Jeff Baars Cliff Mass Mark Albright University of Washington, Seattle,](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d475503460f94a228d3/html5/thumbnails/17.jpg)
Future WorkSimple weighting correction to CMOS.Remove worst model from CMOS.