Atmospheric Science Department University of Missouri - Columbia A Procrustes Shape Analysis...

Atmospheric Science DepartmentUniversity of Missouri - Columbia

A Procrustes Shape Analysis Verification Tool

Steve Lack Sakis Micheas Neil Fox Chris Wikle


Methodology

• Procrustes Scheme– In Greek mythology, Damastes, was a thief

who often went by the name Procrustes and offered travelers a bed to rest

– The twist was he would “fit” the travelers to the bed…stretching them if they were too short and cutting their limbs off if they were too tall

Journal of Hydrology article forthcoming Micheas et al.


Methodology

• So in this context we will fit forecast cells to truth cells by shifting location, resizing the object, and rotating them if necessary

• Origin use is in biology (anatomy) for lining up jaw bones (landmarks) or other structure for classification

• Works with gridded data of any dimensions for spatial objects, originally designed for ensembles


Methodology

• Define a penalty function– Takes into account

max,mean,min intensity, location fitting

– May be user defined (apply weights)

– The lower the penalty the better the forecast

kjijkjk

j zercz jk̂ˆˆˆ

kkkk SSSSSSRSSD maxminavg


Advantages

• Reduction of dimensionality for speed (less than 1 minute to run on a fair sized domain)

• Takes into account shape information with intensity data

• Can alter the penalty function to get more meaningful data

• Error decomposition• User defined thresholds


Shortcomings

• Memory issue when running a large domain with a lot of identifiable cells– Can break down domain however

• The total penalty function adds error components that may not of the same magnitude– Can refine these in the future

• Relative comparison (interpretation)


Results

• Entire domain penalty was calculated by using 0.10” (memory issue)

• Minimum size of cell set to 10 pixels

• Divided domain into NE,NW,SE,SW quadrants

• Recalculated penalties based on 0.01” again minimum size set to 10 pixels


13 May 2005 – WRF4NCAR


13 May 2005 – Stage 2


Fits – WRF4NCAR


Total Penalties – Entire Domain

WRF4NCAR WRF4NCEP WRF2CAPS

RSS (1*10^3) 5.9151 5.4487 4.8603

SStot (1*10^3) 1.7531 2.3083 1.9005

# Cells

(Truth = 17)

26 61 39

Tot Error (1*10^5)

2.6607 5.7858 2.8416


Error Decomposition – Entire Domain


Min Intensity 2 4 4

Mean Intensity 8926 26825 9079.9

Max Intensity 2521220 5463000 2702100

Translation 4.6711 22.669 11.596

Rotation 0 0 0

Dilation 428290 7736700 1176000


NW Domain- 13 May 2005(clockwise NCAR, NCEP, CAPS, Stage 2)


Total Penalties – NW Domain


RSS (1*10^3) 5.8255 5.7583 5.34

SStot (1*10^3) 3.4659 3.8712 3.4335

# Cells

(Truth = 39)

50 33 37

Tot Error (1*10^4)

0.96153 1.8417 1.1106


Error Decomposition – NW Domain


Min Intensity 0 0 0

Mean Intensity 104.78 297.67 148.31

Max Intensity 3685 12361 5618

Translation 32.488 13.969 15.15

Rotation 0 0 0

Dilation 797540 691740 758880


1 June 2005 – WRF4NCAR


Stage 2


Total Penalties


RSS (1*10^3) 4.3458 3.8631 4.7510

SStot (1*10^3) 1.7761 2.3113 1.8553

# Cells

(Truth = 48)

59 109 49

Tot Error (1*10^5)

3.8423 3.4230 3.4501


Error Decomposition


Min Intensity 18 13 11

Mean Intensity 16151 12394 14009

Max Intensity 363710 326030 326240

Translation 5.5781 8.139 6.3785

Rotation 0.0001 0 0

Dilation 1085000 1945500 1316600


NW Domain- 01 June 2005(clockwise NCAR, NCEP, CAPS, Stage 2)


Total Penalties – NW Domain


RSS (1*10^4) 1.1838 1.6253 1.4684

SStot (1*10^3) 2.9221 3.9807 2.9628

# Cells

(Truth =30)

53 50 58

Tot Error (1*10^4)

1.7542 3.8873 2.3150


Error Decomposition – NW Domain


Min Intensity 0 0 0

Mean Intensity 184.91 425.37 60.646

Max Intensity 5519 22194 8405

Translation 59.107 12.883 30.551

Rotation 0 0 0

Dilation 3217400 357250 827400


Conclusions

• A lot of weight is given to intensity forecasts as well as position/shape

• June 1 forecast are all handled equally well by the models

• WRF4NCAR handles intensity the best for May 13 although WRF2CAPS is close on position

• WRF4NCEP underestimates intensity in most cases


Acknowledgements

• Stat Group: Sakis Micheas, Chris Wikle, Yong Song, Kent Peng

• Rains Group: George Limpert and Neil Fox

Award # ATM-0434213

Atmospheric Science Department University of Missouri - Columbia A Procrustes Shape Analysis...

Documents

Transcript of Atmospheric Science Department University of Missouri - Columbia A Procrustes Shape Analysis...