Object-based Spatial Verification for Multiple Purposes
-
Upload
kenyon-hendrix -
Category
Documents
-
view
19 -
download
0
description
Transcript of Object-based Spatial Verification for Multiple Purposes
The Centre for Australian Weather and Climate ResearchA partnership between CSIRO and the Bureau of Meteorology
Object-based Spatial Verification for Multiple Purposes
Beth Ebert1, Lawrie Rikus1, Aurel Moise1, Jun Chen1,2, and Raghavendra Ashrit3
1 CAWCR, Melbourne, Australia2 University of Melbourne, Australia3 NCMRWF, India
www.cawcr.gov.au
Object-based spatial verification
2
FORECAST OBSERVATIONS
Verifying attributes of objects
3
Other examples
4
HIRLAM cloud
AVHRR satellite
Climate features (SPCZ)Jets in vertical plane
Convective initiation
Vertical cloud comparison
What does an object approach tell us?
• Errors in• Location
• Size
• Intensity
• Orientation
• Results can• Characterize errors for individual forecasts
• Show systematic errors
• Give hints as to source(s) of errors
• I will discuss CRA, MODE, "Blob"• Not SAL, Procrustes, Composite (Nachamkin), others
5
OBS
FCST
6
Contiguous Rain Area (CRA) verification
• Find Contiguous Rain Areas (CRA) in the fields to be verified– Choose threshold– Take union of forecast and
observations– Use minimum number of points
and/or total volume of parameter to filter out insignificant CRAs
Observed Forecast
• Define a rectangular search box around CRA to look for best match between forecast and observations
• Displacement determined by shifting forecast within the box until MSE is minimized or correlation coefficient is maximized
• Error decomposition MSEtotal = MSEdisplacement + MSEintensity + MSEpattern
Ebert & McBride, J. Hydrol., 2000
Heavy rain over India
Met Office global NWP model forecasts for monsoon rainfall, 2007-2012
7
Ashrit et al., WAF, in revision
Heavy rain over India
8
CRA threshold: 10 mm/d 20 mm/d 40 mm/d 10 mm/d 20 mm/d 40 mm/d
Errors in Day 1 rainfall forecasts
Heavy rain over India
9
Error decomposition (%) of Day 1 rainfall forecasts
Climate model evaluation
10
Delage and Moise, JGR, 2011 added a rotation component
Can global climate models reproduce features such as the South Pacific Convergence Zone?
Climate model evaluation
"Location error" = MSEdisplacement + MSErotation
"Shape error" = MSEvolume + MSEpattern
Applied to 26 CMIP3 models11
etc.
Climate model evaluation
Correcting the position of ENSO EOF1 strengthens model agreement on projected changes in spatial patterns of ENSO driven variability in temperature and precipitation
12 Power et al., Nature, 2013
13
Method for Object-based DiagnosticEvaluation (MODE) (Davis et al. MWR 2006)
Identification
Merging
Matching
Comparison
Measure attributes
Convolution – threshold process
Summarize
Fuzzy Logic Approach
Compare forecast and observed attributes
Merge single objects into clusters
Compute interest values*
Identify matched pairs
Accumulate and examine comparisons across
many cases
*interest value = weighted combination of attribute matching
CRA & MODE – what's the difference?
14
CRA MODE
Convolution filter N Y
Object definition Rain threshold Rain threshold
Object merging N Y
Matching criterionMSE or correlation coefficient
Total interest of weighted attributes
Location error X- and Y- error Centroid distance
Orientation error Y Y
Rain area YY, incl. intersection, union, symmetric area
Rain volume Y Y
Error decomposition Y N
Comparison for tropical cyclone rainfall
15
CRA MODE
Chen, Ebert, Brown (2014) – work in progress
Westerly jets
"Blob" defined by percentile of local maximum of zonal mean U in reanalysis Y-Z plane
16
5th percentile 10th percentile 15th percentile
Rikus, Clim. Dyn., submitted
Westerly jets
17
Westerly jets
18
Global reanalyses show consistent behaviour except 20CR.
Can be used to evaluate global climate models.
Future of object-based verification
• Routinely applied in operational verification suite• Other variables• Climate applications
19
Future of object-based verification
Ensemble prediction – match individual ensemble members
20
8 ensemble members
Johnson & Wang, MWR, 2012, 2013
Prob(object)=7/8
Brie
r sk
ill s
core
Ensemble calibration approaches
Future of object-based verification
Weather hazards
21
Tropical cyclone structure
Pollution cloud, heat anomaly
Blizzard extent and intensity
Flood inundation
Fire spread
WWRP High Impact Weather Project
Thank you
The Centre for Australian Weather and Climate ResearchA partnership between CSIRO and the Bureau of Meteorology
Thank youwww.cawcr.gov.au
Extra slides
23
25
Spatial Verification Intercomparison Project
• Phase 1 – understanding the methods
• Phase 2 – testing the methods
• "MesoVICT" – precipitation and rain in complex terrain
• Deterministic & ensemble forecasts
• Point and gridded observations including ensemble observations
• MAP D-PHASE / COPS dataset
Core
Determ. precip+ VERA anal
+ JDC obs
Tier 1
Det
erm
. win
d
+ V
ERA a
nal
+ JD
C o
bs
Ensemble precip
+ V
ERA anal
+ JD
C obs
Ensemble wind+ VERA anal
+ JDC obs
Tier 2a
Tier 2b
Deter
m. w
ind
+ V
ERA e
nsem
ble
+ JD
C obs
Determ. precip
+ VERA ensem
ble
+ JDC obs
Ensemble wind
+ VERA ensem
ble
+ JDC obs En
sem
ble
prec
ip
+ V
ERA e
nsem
ble
+ JD
C obs
Tier 3
Oth
er v
aria
ble
s ense
mble
+ V
ER
A e
nse
mble
+ JD
C o
bs
Sensi
tivit
y t
est
sto
meth
od p
ara
mete
rs
MODE – total interest
26
M
i jiji
M
i jijijij
wc
FwcI
1 ,,
1 ,,,
M = number of attributes
Fi,j = value of object match (0-1)
ci,j = confidence, how well a given attribute describes the forecast error
wi,j = weight given to an attribute
Attributes:•centroid distance separation
•minimum separation distance of object boundaries
•orientation angle difference
•area ratio
•intersection area
Tropical cyclone rainfall
27
CRA:•Displacement & rotation error•Correlation coefficient•Volume•Median, extreme rain•Rain area•Error decomposition
MODE:•Centroid distance & angle difference•Total interest •Volume•Median, extreme rain•Intersection / union / symmetric area