Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research...
-
Upload
hilary-brooks -
Category
Documents
-
view
224 -
download
0
Transcript of Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research...
![Page 1: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/1.jpg)
Weather Forecast
Verification
Training- Recent Advancements -
Pertti NurmiMeteorological Research & Development
Finnish Meteorological Institute
Laurie WilsonMeteorological Service of Canada
1
trainingmodules
2
0. Introduction
![Page 2: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/2.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
2680 + 302800
= 96,8 %2680 + 302800
= 96,8 %
TornadoTornado observed
forecastYes No fc S
Yes 30 70 100
No 20 2680 2700
obs S 50 2750 2800
Verification(training)History, 1884
Tornados in USA”The Finlay case”
0
2750 + 02800
= 98,2 %
Never forecasta Tornado
2750 + 02800
= 98,2 %
Never forecasta Tornado !
TornadoTornado observed
forecastYes No fc S
Yes 0 0 0
No 50 2750 2800
obs S 50 2750 2800
Hypothesis
![Page 3: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/3.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Goals of Verification Training: Understand the basic properties and relationships of
(common) verification measures
Learn to extract useful information from (graphical) verification results
Increase interest in forecast verification and the methods
Apply verification in everyday fc practice at every fc office
0
Generally HEAVILY neglected topic in training activities !
![Page 4: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/4.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
”WMO WWRP/WGNE JWGV” (*
Harold Brooks(NSSL, USA; not shown)
Barbara Brown(NCAR, USA; Chair)
Barbara Casati(MSC, Canada)
Ulrich Damrath(DWD, Germany)
Beth Ebert(BMRC, Australia)
Anna Ghelli (ECMWF)
Pertti Nurmi (FMI, Finland)
David Stephenson(U. Reading, UK; not shown)
Clive Wilson (UKMO, UK)
Laurie Wilson(MSC, Canada)
0
*) WMO ~ World Meteorological Organization WWRP ~ World Weather Research Program WGNE ~ Working Group on Numerical Experimentation JWGV ~ Joint Working Group on Verification
InternationalActivities
![Page 5: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/5.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Goals of JWGV: • Cooperation between forecasters, users and verifiers
• Development and application of verification methodologies
• Promote awareness of the importance of verification via training
Recent activities of JWGV:• Research and training workshops on verification methods • Recommendations for verification methodologies
• Test-bed activities
• Beijing 2008 Olympics Real-Time Forecast Verification (RTFV)
• THORPEX and TIGGE
• EUMETCAL verification training modules
”WMO WWRP/WGNE JWGV”
2
0
1
InternationalActivities
![Page 6: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/6.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html
JWGV website Google search:
“Forecast > verification”
gives this as the first prompt !
0
![Page 8: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/8.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
www.ecmwf.int/newsevents/meetings/workshops/2007/jwgv/lecture_notes/index.html
1
![Page 9: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/9.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
1
Argentina, Australia , Brasil, Canada, Checz Republic, China, Croatia, Denmark, Ethiopia, Finland, France, Germany, Hong Kong, India, Italy, Kenya, Lithuania, Nigeria, Norway, Romania, Senegal, Slovakia, South Africa, South Korea, Sweden, Switzerland, Ukraine, USA
31 students,One / country
![Page 10: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/10.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Components:
i. Tutorial ~ 2 days
ii. Scientific workshop and presentations ~ 3 days
i. Tutorial contents:
• Use/interpretation of “standard” verification methods• Application of new methodologies (e.g. spatial forecast verification)
• Hands-on laboratory exercises
• ”R” software (verification components) training
ii. Workshop contents:
• Invited keynotes & contributed presentations
• User-oriented forecast verification
• Verification of extreme/ high impact weather, EPS, uncertainty …
1
![Page 11: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/11.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Tutorial contents:
i. Introduction to R (Matt Pocernich)
ii. Basic Verification Concepts (Barb Brown)
iii. Verification of Continuous Predictands (Barbara Casati)
iv. Verification of Categorical Predictands (Pertti Nurmi)
v. Verification of Probabilistic Predictands and Ensemble Forecasts (Beth Ebert)
vi. Confidence Intervals and Hypothesis Tests (Simon Mason)
vii. Projects (Laurie Wilson)
1
![Page 12: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/12.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
1
www.r-project.org
(i)
![Page 25: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/25.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Project No. 1: Verification of ensemble-based PoP forecasts (Dataset: Tong/ Hong Kong)
Calculate the predictand… Verify the POP forecasts... Prepare and plot a reliability table... Compute the BS…
Project No. 2: Temperature verification (Dataset: Diaconu/ Romania)
Choose a verification problem… Look at the data… Produce and examine scatter plots…Choose verification scores…
Project 3: Cloudiness forecasts in 3 categories (Dataset: Diaconu/ Romania)
Choose a verification issue... Look at the data… Produce contingency tables... Choose verification measures…
Project No. 4: Wind speed forecasts(Dataset: Moreau/ Canada)Choose the verification problem… Look at the data… Plot
forecast-observation scatterplots... Choose verification measures... Calculate the scatter index...
Project No. 5 Climatological analysis(Dataset: Korecha/ Ethiopia)Produce box and whisker plots… Calculate the annual
statistics… Do the same for each month separately...
Project No. 6: Precipitation forecasts(Dataset: Suaya/ Argentina)Look at the data… Prepare histogram plots… Pose the
relevant verification questions… Carry out the verification and produce graphs to display your results…
1(vii) Projects
Laurie WilsonEnvironment Canada
deterministic
![Page 26: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/26.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
1 “Probably the best trainingcourse I ever had.
Thank you to all of you! “
Overall evaluationby students:
> 2.5 / 3
![Page 27: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/27.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
2What’s included in the modules:
1. Introduction Goals and general guidelines
2. Continuous variables
3. Categorical events
4. Probability forecasts
Glossary + References
+ Practical exercises ~ Quiz !
finished spring 2006
finished winter 2007
finished summer 2007
CAL trainingmodules
![Page 29: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/29.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
2 www.eumetcal.org
Demonstration on Sat 10:10-11:00 WELCOME TO TEST
![Page 30: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/30.jpg)
www.eumetcal.org/resources/ukmeteocal/verification/www/english/courses/msgcrs/index.htm
… produced by Pertti Nurmi, Laurie Wilson & Sigbritt Näsman, 2006-07
CAL modules !
2
Demonstration on Sat 10:10-11:00 WELCOME TO TEST
![Page 31: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/31.jpg)
www.eumetcal.org/resources/ukmeteocal/verification/www/english/courses/msgcrs/index.htm
2
”Help button”
”Glossary button”
”Communication tool”
”Return to Index”… open browser !
![Page 32: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/32.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
Jolliffe and Stephenson, Eds. (2003): Forecast Verification:A Practitioner’s Guide, Wiley & Sons.
Nurmi (2003): Recommendations on the verification of local weather forecasts. ECMWF Tech. Mem. 430.
Wilks (2006): Statistical Methods in the Atmospheric Sciences, Chapter 7 (Forecast Verification). Academic Press.
WMO Joint Working Group on Verification Website: www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html
ECMWF Verification Methods Training Workshop Lectures: www.ecmwf.int/newsevents/meetings/workshops/2007/jwgv/
lecture_notes/index.html
EUMETCAL Forecast Verification Learning Modules: www.eumetcal.org/resources/ukmeteocal/verification/www/
english/courses/msgcrs/index.htm
2
0
1
References:
![Page 33: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/33.jpg)
5 July 2007CALMet - Beijing
Forecast Verification Training
There is nothing as difficult as
forecasting the weather (correctly) ...
… … exceptexcept (proper)(proper) forecast forecast
verification !verification !- Allan Murphy, RIP- Allan Murphy, RIP
Training is one key issue !
Finishing motto
![Page 34: Weather Forecast Verification Training - Recent Advancements - Pertti Nurmi Meteorological Research & Development Finnish Meteorological Institute Laurie.](https://reader035.fdocuments.us/reader035/viewer/2022062423/5697bfbf1a28abf838ca32a7/html5/thumbnails/34.jpg)
Pertti Nurmi
Meteorological Research & Development
Finnish Meteorological Institute
Thank You