Verification of Numerical Weather Prediction systems employed by the Australian Bureau

of Meteorology over East Antarctica duringthe 2009-10 summer season.

Neil AdamsCAWCR,Tasmania-Antarctica RegionBureau of Meteorology,Hobart, Australia.

1. Model performance over the 2009/10 summer.

2. Precipitation forecasting during an extreme event at Casey.

3. The future of polar prediction within Australia.

S1=100∗∑ eg

∑GL

eg=∣ X f − X o x ∣∣ X f − X o

y ∣

G L=max ∣ X f

x ∣ ,∣ X o

x ∣max ∣ X f

y ∣ ,∣ X o

y ∣

Where:

and:

x

y

Verification - The S1 skill Score.

Xf : Forecast field,

Xo: Observed field,

: zonal resolution,: meridional resolution.

The S1 score measures the agreement between both the forecast and observed data and the gradient of the forecast and observed data.

A score of 0 implies a perfect forecast and a score of 100 implies no skill in the forecast.

S1 skill scores from PolarLAPS were compared with those from the available ECMWF forecasts. Both ECMWF and PolarLAPS were verified against ERA-Interim analyses at 1.50 resolution.

ECMWF domain – data at 1.50 resolution.

PolarLAPS domain – data at 0.250 resolution.

The comparison was done on reduced domains to maximise the coincidence of the two forecast systems.

[ERA analyses interpolated to the PolarLAPS domain using a lagrangian bicubic formulation].

Reduced PolarLAPS domain.

Reduced ECMWF domain.

Note the strong seasonalvariation in the S1 score – most

pronounced with polarLAPS.

Time-Step ECMWF PolarLAPS

+024HR 18.0 26.1

+048HR 27.5 40.6

+072HR 37.8 53.0

+096HR 47.9 62.6

+120HR 57.1 69.8

Time-Step ECMWF PolarLAPS

+024HR 16.0 23.6

+048HR 24.6 37.2

+072HR 34.1 49.4

+096HR 43.7 59.5

+120HR 53.0 66.9

ECMWF and PolarLAPS average S1 skill scores.

(I January 2006 - 30 April 2010).

ECMWF and PolarLAPS average summer season skill scores.

2006/07,2007/08,2008/09,2009/10(1 November – 31 March)

2006/07 2007/08 2008/09 2009/10

30

35

40

45

50

55

+072HR-ECMWF+072HR-polarLAPS

2006/07 2007/08 2008/09 2009/10

50

55

60

65

70

+120HR-ECMWF+120HR-polarLAPS

2006/07 2007/08 2008/09 2009/10

40

45

50

55

60

65

+096HR-ECMWF+096HR-polarLAPS

Mean summer season S1 scores – ECMWF and PolarLAPS (verified against ERA-Interim).

2006/07 2007/08 2008/09 2009/10

15

20

25

30

+024HR-ECMWF+024HR-polarLAPS

2006/07 2007/08 2008/09 2009/10

20

25

30

35

40

45

+048HR-ECMWF+048HR-polarLAPS

Points to note:

S1 scores calculated at the analysis resolution of 1.5o,

PolarLAPS initialised off NCEP-GFS at 1.0o (last few months at 0.5o),

Only capturing ability of ECMWF and PolarLAPS to model the synoptic scale features – not meso-scale. (Australia only has access to 1.5o ECMWF),

PolarLAPS has no data assimilation so no high resolution information in the initial state or lateral boundary conditions,

Artefact of the cold-start initialisation are spin-up errors causing phase shifts and timing errors in system movements.

So – S1 score doesn't validate the high resolution information contained within the PolarLAPS forecasts and penalises the timing/phase errors.

Phase errors degrade S1Skill score, however long lead time on significant events still highly useful guidance.

Bias CorrectedRMS Error:

Forecast Period:25 January to25 April 2010.

Temperature

Pressure

As with the S1 skill score, the single station statistics such as RMSE or bias corrected RMSE don't necessarily convey how useful an NWP system is in supporting weather forecasting.

Another verification method is basedaround model climatology.

Long term data set 1988 - present.

January 25 to April 25 2010.

January 25 to April 25 2010.

January 25 to April 25 2010.

January 25 to April 25 2010.

January 25 to April 25 2010.

January 25 to April 25 2010.

120 km

Precipitation forecasting.In the 24 hours to 11 am (0300 UTC) on 20 April 2010 Casey measured 34.2 mm of water equivalent snow – a record for Casey Station !!!.

Model accumulated precipitation in the 24 hours to 0300 UTC:ACCESS-G : 2.9 mm,NCEP-GFS : 5.0 mm,PolarLAPS : 12.8 mm,ECMWF : 16.0 mm.AMPS : 5.3 mm (8.0 mm from previous run).(data from runs made 1200 UTC 18 April 2010).

PolarLAPS

+

Model verification still unsatisfactory but does guide future development:

Antarctic forecasting is dominated by:the wind field,snowfall,cloud cover, base and thickness,

Modelling requires adequate resolution,appropriate physical schemes,and the best possible initial conditions.

The Australian Polar Prediction System (APPS).

A business case was submitted to the Bureau Executive in June 2010 for the APPS to include:

A polar NWP system built around the Australian Community Climate Earth System Simulator (ACCESS) atmospheric model (UKMO-UM) – ACCESS-P,A sea-ice forecasting system based on CICEv4 using ACCESS-P output as forcing,A 30 year re-analysis using ACCESS-P.

The Australian Polar Prediction System (APPS) - cont.

The executive meeting held in late June met to discuss the business case and has agreed to support the NWP component.

A polar NWP system built around the Australian Community Climate Earth System Simulator (ACCESS) atmospheric model (UKMO-UM) – ACCESS-P,A sea-ice forecasting system based on CICEv4 using ACCESS-P output as forcing,A 30 year re-analysis using ACCESS-P.

The Executive are interested in theidea of a sea-ice analysis and

forecasting system and will revisitfunding of this initiative in the

2011/12 financial year.No mention at all wasmade of the bid for are-analysis project !

The Australian Polar Prediction System(APPS) – approximate time-line:

2010: ACCESS-P running over the same domain as PolarLAPS – initialised from ACCESS-G output,

2011: Optimising paramaterisation schemes for polar prediction,

2011: Porting of the ACCESS 4-Dimensional Variational Assimilation (4DVar) to the rotated ACCESS-P grid,

2012: Resolution increase, 2013: partial coupling of sea-ice forecast

system.

Thank you.