National Precipitation Verification Unit
description
Transcript of National Precipitation Verification Unit
![Page 1: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/1.jpg)
National Precipitation Verification Unit
Letitia L. Soulliard
RSIS & NOAA/NWS/OCWWS
RFC Verification Workshop
August 15, 2007
Salt Lake City, UT
![Page 2: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/2.jpg)
2
What is the NPVU?
• National• Precipitation• Verification• Unit
• Verifies the QPF from the NGM, NAM, GFS, HPC, Conus RFCs, Conus WFO’s (NDFD), and Puerto Rico (NDFD)
![Page 3: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/3.jpg)
3
Who oversees the NPVU?
• Office of Climate, Water, and Weather ServicesPerformance Branch (Douglas Young)
• Located at :Hydrometeorological Prediction Center/NCEP (Ed
Danaher)
![Page 4: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/4.jpg)
4
What is the History of the NPVU?
• 32km Verification – Started October 2000• Eastern RFCs
Had day 1 totals starting in August 2000 Had 6hr amounts starting October 2000
• Western RFCs CN and CB started in November 2000 NW started in January 2001
• Mosaic of all RFCs Started January 2001
• NDFD Conus WFO’s - Started October 2004 Puerto Rico – Started June 2005
• 4km Verification – Started October 2004
![Page 5: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/5.jpg)
5
How does the NDFD relate to the RFC data?
• It doesn’t
• Different domain areas make it impossible to directly compare
• Both are individually compared to HPC and the models but not to each other
![Page 6: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/6.jpg)
6
Why should you use the NPVU’s products?
• To see if there is an improvement over HPC’s forecast
• To see if HPC is improving upon the models• To see which model might be better for
each season or for each RFC area• To view QPF products for other RFCs and
view continuity issues
![Page 7: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/7.jpg)
7
What are the different NPVU Products found on the web?
• QPF Verification Statistics• Comparison Plots• RFC “Live” Data• Archive Data
RFC QPEHPC QPFOffline request form
![Page 8: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/8.jpg)
8
What is verified by the NPVU and when is it updated?
• 12Z QPF for RFCs, WFO’s and HPC• 00Z QPF for Models
• Updated Once a Month approximately around the 10th for the previous month
• E-mail notification
![Page 9: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/9.jpg)
9
How does the data get from the RFCs to the NPVU?
RFC’s QPF & QPE
AWIPS WANNetworkControlFacility
NPVUFTPIBMSP
Offices ofSystems
Operations SBN
dbnet
![Page 10: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/10.jpg)
10
How are the forecast and observation grids modified in order to be compared?NPVU Receives: To 32km grid by: To 4km grid by:
4km QPE from RFC Spatial Average No Conversion Needed
10km QPF from RFC Area Preservation Technique
Area Preservation Technique
1x1 Degree GFS Area Preservation Technique
Linear Interpolation
32km NAM Change coverage area Linear Interpolation
90km NGM Area Preservation Technique
Linear Interpolation
32km HPC No Conversion Needed Linear Interpolation
5km NDFD Spatial Average Area Preservation Technique
![Page 11: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/11.jpg)
11
What statistical scores are calculated by the NPVU?
NPVUStatistics
32km / 4km
NPVUStatistics
32km / 4km
CategoricalContingency
Table
CategoricalContingency
Table
ContinuosBoth, Obs, ForContinuos
Both, Obs, For
Graphs
BiasPODFARTS
ETSThreshold Distribution
Graphs
BiasPODFARTS
ETSThreshold Distribution
Text
HSS
Text
HSS
Threshold Values (in.)
X >= 0.01X >= 0.10X >= 0.25X >= 0.50X >= 1.00
Threshold Values (in.)
X >= 0.01X >= 0.10X >= 0.25X >= 0.50X >= 1.00
Interval Values (in.)
0.00 <= X < 0.010.01 <= X < 0.100.10 <= X < 0.250.25 <= X < 0.500.50 <= X < 1.00
1.00 <= X
Interval Values (in.)
0.00 <= X < 0.010.01 <= X < 0.100.10 <= X < 0.250.25 <= X < 0.500.50 <= X < 1.00
1.00 <= X
Graphs
MEMAE
RMSEVolume Bias
Interval Distribution
Graphs
MEMAE
RMSEVolume Bias
Interval Distribution
Text
x and of For and Obs Bias of the
Envelope for CategoriesReduction of Variance
Linear CorrelationCoefficient
Text
x and of For and Obs Bias of the
Envelope for CategoriesReduction of Variance
Linear CorrelationCoefficient
![Page 12: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/12.jpg)
12
What do all these numbers really mean?
• Online Help Guide• http://www.hpc.ncep.noaa.gov/npvu/help/
![Page 13: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/13.jpg)
13
What is the difference between the Continuous Interval Types?• Observed – Forecast Pair
Obs: 0.2”For: 0.3”
• Observed OnlyPair is put in A
• Forecast OnlyPair is put in B
• Observed and Forecast (Both)Pair is put in A & B
B0.25 – 0.499
A0.10 – 0.2499
![Page 14: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/14.jpg)
14
How are Categorical Scores Calculated?
• Contingency Table
• High-Low Misses
![Page 15: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/15.jpg)
15
What are some common confusions about the NPVU statistics?
• What verification periods are covered?All verification is done at 6hr intervalsDay 1 is four 6hr periods
• Why create an interval distribution graphic?Important for identifying how significant
the statistics really are
![Page 16: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/16.jpg)
16
How is the Interval Distribution Useful?
![Page 17: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/17.jpg)
17
What constraints are there in the statistics?
• Small sample sizes can result in anomalous scores
• The eastern RFCs differ from the western RFCs due to Mountain Mapper
• Re-mapping can introduce small discrepancies
![Page 18: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/18.jpg)
18
Which RFCs are considered eastern and which are western?
Eastern RFCs
ABRFCLMRFCMARFCMBRFCNCRFCNERFCOHRFCSERFCWGRFC
Western RFCs
CBRFC
CNRFC
NWRFC
APRFC
![Page 19: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/19.jpg)
19
What are the differences between the Eastern RFCs and the Western RFCs?
Eastern RFCs
• 00Z and 12Z Forecast• 18Z forecast
AB, MA, MB and NC RFC
• Forecast out to 24hr Some forecast more (but
not verified)
• QPE are stage III multi-sensor data
Western RFCs
• 12Z forecast ONLY (due to MM restrictions)
• Forecast out to 72hr (and verified)
• QPF is made for points• QPE is gauge data• Both are gridded using
Mountain Mapper
![Page 20: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/20.jpg)
20
What is Mountain Mapper and how does it affect the NPVU's product/process?
• Developed at CBRFC to convert point data into gridded data
• Uses PRISM precipitation Climatology• Since both QPE and QPF are run through this
they are both “biased” toward the climatology where as the models and HPC are not
![Page 21: National Precipitation Verification Unit](https://reader036.fdocuments.us/reader036/viewer/2022062518/5681451a550346895db1dae3/html5/thumbnails/21.jpg)
21
What is the NPVU doing about this Mountain Mapper Issue?
• HPC provided forecast run through Mountain MapperFor the CNRFC, CBRFC, and NWRFC areasSince January 2003
• Statistics have been run for each month, warm and cool season, and annual
• Mountain Mapper has not improved HPC’s forecast as much as expected