Calculation of Instantaneous Frequency and Instantaneous Bandwidth (1)
The IPWG Inter-comparison Effortipwg/meetings/bologna-2016/Bologna2016_Orals/5 … · a national...
Transcript of The IPWG Inter-comparison Effortipwg/meetings/bologna-2016/Bologna2016_Orals/5 … · a national...
The IPWGInter-comparison EffortChris Kidd, Beth Ebert, John Janowiak,Joe Turk, Daniel Vila, Matt Sapiano, Shoichi Shige, Estelle de Coning, Ralph Ferraro, Vincenzo Levizzani, Phil Arkin
The Whys and WhereforesIPWG objectives:
#2) Establish standards for validation and independent verification of precipitation measurements
#3) Foster the exchange of data on inter-comparisons of operational precipitation measurements from satellites
• Need to provide regional-scale validation of precipitation products.
• Building upon work of the GPCP Algorithm Inter-comparison Programme (AIP1-3) and the WetNet Precipitation Inter-comparison Projects (PIP1-3).
• On-going inter-comparisons using common graphical, descriptive and statistical analysis of daily, 0.25° resolution products.
• Algorithm developers make products available for comparisons done in local ‘validation’ centres.
BESTEFFORTBASIS8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
GV-IPWG synergies
Criteria GV program IPWG
Type of validation Priority on physical, also statistical
Has focused on descriptive and statistical
Source of validation data Arranged for and collected by principle investigators
Doesn't request. IPWG participants free to contribute
Source of observational data Specific satellite-based products
participants provide products directly to validation groups
Types of Validation data Gauge, radars and specialist instrumentation, diverse in specific locations
Conventional gauge and/or radar networks, usually part of a national network
Types of observational data single-sensor, instantaneous, full-resolution datasets
Blended satellite sensor products, time/area averaged.
AfterTurk&Arkin,BAMS2008
Bothapproachesarecomplementary
GV=GroundValidation
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
IPWG Inter-comparison regionsNearreal-timeinter-comparisonofmodel&satelliteestimatesvsradar/gauge
IPWG- http://www.isac.cnr.it/~ipwg/8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Accesing IPWG intercomparisons
http://www.isac.cnr.it/~ipwg/
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
MainIPWGInter-comparison
Websites
Regional Web pages - Japan
Dailyimagesandstatisticsandsummaryplots!Probablythebestrunwebpageatpresent!
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Regional Web pages - US
Dailyimagesandstatistics.(night-timecolour scheme?!)
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Regional Web pages - Europe
Limiteddailyimagesandstatistics.
BUT– NEW!– instantaneouscomparisons
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Processing systemInitialsetup:SettingofdatesCleaningoutold/decayeddata
Acquiringdata:SearchingexistingdataListingmissingdataCreationof.netrcfileftpdatasources
Remappingofdata:…toregionalgridor5kmPSGprojection…
Resultsgeneration:StatisticalanalysisGraphicaloutput
Webpages:GenerateHTMLfilesCopyingtoserver
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Processing checksforeachday(d0-d0-31)
dn=dn+1
setd0=today
foreachday(d0-d0-31)
foreachproduct(p1-pn)
if(productforday)!exist
addto.netrcfile
foreachdatasource(s0-sn)
ftpdatasource(4k)
N
Y
foreachproduct&day
remaptoPSGusingLUTs&standardiseformat
standardisefilename
foreachproduct&day
Generatestatistics
Generateplots
foreachproduct&day
generateHTMLfiles
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Processing checksforeachday(d0-d0-31)
dn=dn+1
setd0=today
foreachday(d0-d0-31)
foreachproduct(p1-pn)
if(productforday)!exist
addto.netrcfile
foreachdatasource(s0-sn)
ftpdatasource(4k)
YN
foreachproduct&day
remaptoPSGusingLUTs&standardiseformat
standardisefilename
foreachproduct&day
Generatestatistics
Generateplots
foreachproduct&day
generateHTMLfiles
Setuplistofpastdates/daysUsuallyokay:
sometimesneedstweaking
Checksforaproductsresults:Okayifnoresults,butnotifbaddata
Preparesproductsintocommon
formatUsuallyokay…
Generatesoutputs:
Okayifthereisrain…
GeneratesrawHTML:Occasionalissues
withserver
FTPrunsseveraltimes:
4Kbufferlimitonmacros
AutomatedsystemstheyareNOT!8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
What’s new?Growing interest in the inter-comparison of precipitation retrievals at the full-spatial/instantaneous resolution, but:• Requires much more processing;• Data often in different formats, resolutions, sampling;• More comparisons – orbital (~2-3/sat) vs daily (1)Nevertheless• Europe – same region as daily estimates and US –
mapping done for eastern US (better quality radar)• About 32,000 comparisons/year/region for GPM alone• Assumes a ‘uniform’ 15 km footprint• EU radar (15 min at 52°N), US radar (2 min at 37.5°N).
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
MainIPWGInter-comparison
Websites
European Instantaneous Inter-comparisons
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Regional analysis: Western EuropeAMSR2 GMI SSMIS
ATMSDPR-NS MHS-V04MHS-V03
0.00.10.20.30.40.50.60.70.80.9Correlation
EUMETSAT Conference, 26-30 September 2016Goddard Space
Flight Center
Regional analysis: United States
AMSR2 GMI SSMIS
ATMSDPR-NS MHS-V04MHS-V03
0.00.10.20.30.40.50.60.70.80.9Correlation
EUMETSAT Conference, 26-30 September 2016Goddard Space
Flight Center
Inter-satellite retrieval comparisons
GMI-AMSR2 338cases 6,919,090fovsGMI-SSMIS 1673cases 11,718,314fovsGMI-MHS 1732cases 4,466,330fovsGMI-ATMS 444cases 1,156,309fovs
(basedupon<=5kmfov matchups)
GPM orbit crosses all
constellation satellite orbits thus allowing inter-satellite comparisons to be made. 60s,nadircrossingmatchups(2015)
EUMETSAT Conference, 26-30 September 2016Goddard Space
Flight Center
Instantaneous matchupsGMI AMSR2
2015
-04-17
050
3UTC
NAtla
ntic
GMI ATMS
2015
-06-07
160
9UTC
Papu
aNew
Guine
a
GMI MHS
2015
-05-12
230
8UTC
PacificOcean
GMI SSMIS
2015
-01-23
103
4UTC
Indian
Ocean
GMI SSMIS20
15-03-13
071
4UTC
NorthAtla
ntic
GMI AMSR2
2015
-02-09
235
3UTC
NPacific(Ha
waii)
EUMETSAT Conference, 26-30 September 2016Goddard Space
Flight Center
00.11.010100GMIrainrate(mmh-1)
GMIrainrate(mmh-1)00.11.010100
GMIrainrate(mmh-1)00.11.010100
GMIrainrate(mmh-1)00.11.010100
100
10
1.0
0.1
0
AMSR
2
106
105
104
103
102
101
#ob
servations
Instantaneous retrievals GMI vs sensor100
10
1.0
0.1
0
SSMIS
100
10
1.0
0.1
0
ATMS
100
10
1.0
0.1
0
MHS
-V03
GMIrainrate(mmh-1)00.11.010100
100
10
1.0
0.1
0
MHS
-V04
<=60smatch<=5kmco-location
Intercomparison findingsStatistical metrics:• are at best imprecise measures of performance, and are
very different between daily/instantaneous – which are the best ones to use?
• rain/no-rain boundary is visually important, but has little effect on some statistics (e.g. correlation)
Resolution: • previously 0.25/daily, but each instantaneous product has
different resolutionsNumber of products: • there are many more instantaneous products than daily
products (per overpass vv per day).
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Conclusions• First and foremost, the IPWG inter-comparisons are carried
out on a ‘best effort’ basis;
• Many challenges face ‘inter-comparisons’ – particularly from ‘not-just’ rain products (i.e. POPs, error bars, high-resolution/instantaneous);
• Great care is needed in the interpretation of inter-comparisons, yet alone the statistics;
• Observational capability is not the same as retrieval ability (e.g. SSMIS ≠ GPROF);
• The inter-comparisons are very reliant upon the contributors (or lack of – modellers take note!).
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
HELP!Need for guidelines/best practices on:
• Resolutions:- Are the retrievals Gaussian weighted or ’flat’?- What is the ‘true’ retrieval resolution (incl. channel weights)?- Do we need a ‘standard’ retrieval resolution (yes!) ?
• Over-sampling at edge-of-scan (conicals/imagers)
• Co-located channel fovs (e.g. SSMI vs SSMIS/etc)
• Deterioration of GV data with resampling (e.g. MRMS EqArea-LatLon-EqArea)
• True impact of parallax?
8th IPWGand5th IWSSMworkshop,3-7October2016Bologna,Italy
Advice….• Developers: • always look at your results – instantaneous
retrievals, not just averages!• For instantaneous estimates, map the log10 of
the rainrate – it can be very revealing!• Users:• Statistics are useful, but rarely convey the true
usefuless of the product
3-hour
day
5-day
Month
VAR vs. HQ (mm/hr) Feb. 2002 30°N-S
Huffman 2/10
Fine-scale data allows users to decide the averaging strategy
Space-timedependency
At full resolution the ‘accuracy’ of estimated rain is low; averaging over time and space improves the picture
IPWG#5,Hamburg,11-15October2010
Statistical chasm….
Perfo
rmance
Temporal/spatialscaleGoodphysicalcoincidence Goodsamplingpoorsampling
The future of IPWG validation regions
Suggested analysis levels:• real time analysis: 'correctness' of near real-time
products: should these be easily available (cics?)• 'historical' analysis: 'authorised' release of quality-
controlled products • high-resolution analysis: similar set-up to daily/0.25
degree products (including instantaneous)• 'training' data sets: example data sets from a
comprehensive range of precipitation products –common naming convention and format
4th InternationalPrecipitationWorkingGroup(IPWG)Workshop13-17October2008Beijing,China
Personal recommendations
Evaluation of combined techniques:• individual component products for combined algorithm• actual combination techniques
Standardisation of precipitation product format• simple 2D arrays with accompanying text files• common format –variable size• quantitative resolution – 0.1 mm/day (WMO standard)• error fields can be accompanying files• Naming conventions standardised
4th InternationalPrecipitationWorkingGroup(IPWG)Workshop13-17October2008Beijing,China
TheIPWGgauntlet:….togenerateglobal1km,1minuterainfallestimates….
Multi-product comparisons
Satellite(andmodel)precipitationestimatesaregenerallysimilar,but
differindetail
EarthObservationfortheWaterCycle,ESRIN,20-23October2015GoddardSpaceFlightCenter