Caio A. S. Coelho, D. B. Stephenson, F. J. Doblas-Reyes (*) and M.

Balmaseda (*) Department of Meteorology, University of Reading and ECMWF (*)

E-mail: c.a.d.s.coelho@reading.ac.uk

The skill of empirical and combined/calibrated coupled multi-model South American seasonal

predictions during ENSO

Aim:

•

To produce improved probability rainfall forecasts for S. America

Strategy:• Stage 1: Nino-3.4 index, 1 model (Coelho et al. 2003,2004)• Stage 2: Equatorial Pac. SST, 7 models (Stephenson et al. 2005)• Stage 3: S. American rainfall, 3 models (Coelho et al. 2005a,b)

Plan of talk

1. Issues2. Conceptual framework (“Forecast Assimilation”)3. Examples of application: 0-d (Nino-3.4)

1-d (Eq. Pac. SST) 2-d (S. Amer.

rainfall) Downscaling

4. Conclusions

1. Issues

• Why do forecasts need it?• How to do it?• How to get good probability

estimates?

Calibration

Combination • Why to combine?• How to combine?

2. Conceptual framework

)y(p

)x(p)x|y(p)y|x(p

i

iiiii

Data Assimilation “Forecast Assimilation”

)x(p

)y(p)y|x(p)x|y(p

f

fffff

y

Modelling the likelihood p(x|y)

Forecast MAE

(C)

MAESS (%)

BS BSS

(%)

Uncert

(C)

Climatol. 1.16 0 0.25 0 1.19

Empirical 0.53 55 0.05 79 0.61

ECMWF 0.57 51 0.18 29 0.33

Integrated 0.31 74 0.04 81 0.32

MAESS = [1- MAE/MAE(clim.)]*100%

Empirical ECMWF

Integrated

BSS = [1- BS/BS(clim.)]*100%

Example 1: Dec Niño3.4 forecasts (5-month lead)

Example 2: Equatorial Pacific SST

Forecast Brier Score (BS)

BSS

(%)

Climatol p=0.5 0.25 0

Ensemble (ENS) 0.19 24

Integrated (INT) 0.17 31

)0YPr(p tt

SST anomalies: Y (°C)Forecast probabilities: p

DEMETER: 7 coupled models; 6-month lead

BSS = [1- BS/BS(clim.)]*100%

Y 0YOBS OBS INT ENS

1BS0)op(n

1BS

n

1k

2kk

Brier Score as a function of longitude

Forecast assimilation reduces (i.e. improves) the Brier score in the eastern and western equatorial Pacific

ENS - - - INT

Example 3: South American rainfall anomalies

(mm/day)

ENSO composites: 1959-200116 El Nino years13 La Nina years

• Empirical model (EMP):

ASO SST DJF

• Multi-model ensemble (ENS):

3 DEMETER coupled models

ECMWF, Meteo-France, Met Office

1-month lead

Start: Nov DJF

• Integrated (INT) forecast

Combines EMP and ENS

OBS(El Nino)

EMP(El Nino)

ENS(El Nino)

INT(El Nino)

OBS(La Nina)

EMP(La Nina)

ENS(La Nina)

INT(La Nina)

Mean Anomaly Correlation Coefficient (ACC)

Generally low skill (c.f. ACC<0.31)Larger skill in ENSO years than in neutral yearsCalibration and combination improves skill

EMP ENS INT

Correlation score for S.American rainfall

Comparable level of deterministic skillHigher skill in the tropics and southeastern S. America

Brier Skill Score for S. American rainfall

Forecast assimilation improves the Brier Skill Score (BSS) in the tropics

limcBS

BS1BSS )0YPr(p tt

EMP ENS INTENS

Why has the skill been improved?

• How well calibrated the forecasts are (reliability)

• Ability to discriminate between different observed situations (resolution)

Forecast skill depends on:

Brier Score decomposition

1BS0)op(n

1BS

n

1k

2kk

)o1(o)oo(Nn

1)op(N

n

1BS

l

1i

2ii

l

1i

2iii

iNkk

ii1i o

N

1)p|o(po

n

1kkon

1o

reliability resolution uncertainty

Reliability component of the BSS

Forecast assimilation improves reliability over many regions

limc

reliabreliab BS

BSBSS

EMP ENS INT

Resolution component of the BSS

Forecast assimilation improves resolution in the tropics

limc

resolresol BS

BSBSS

INTENSEMP

Example 4: Downscaling of rainfall anomalies

• Multi-model ensemble (ENS):

3 DEMETER coupled models

ECMWF, Meteo-France, Met Office

1-month lead

Start: Nov DJF

Forecast Correlation Brier Score

ENS 0.57 0.22

INT 0.74 0.17

South Box: DJF rainfall anomalies (1-month lead)ENS

INT

Forecast assimilation substantially improves forecast skill

- - - Observation Forecast

Forecast Correlation Brier Score

ENS 0.62 0.21

INT 0.63 0.18

North Box : DJF rainfall anomalies (1-month lead)ENS

INT

Forecast assimilation slightly improves forecast skill

- - - Observation Forecast

• Forecast assimilation improves the skill of probability forecasts

• South America rainfall example: - empirical and integrated predictions have

comparable level of deterministic skill - improved reliability and resolution in the tropics; - improved reliability in subtropical and central

regions - higher skill in ENSO years than neutral years - tropical and southeastern South America are the

two most predictable regions- first step towards an integrated system for South

America

4. Conclusions:

•Coelho C.A.S., 2005: “Forecast Calibration and Combination: Bayesian Assimilation of Seasonal Climate Predictions”. PhD Thesis. University of Reading, 178 pp. • Coelho C.A.S., D. B. Stephenson, F. J. Doblas-Reyes and M. Balmaseda, 2005a: “From Multi-model Ensemble Predictions to Well-calibrated Probability Forecasts: Seasonal Rainfall Forecasts over South America 1959-2001”. CLIVAR Exchanges No 32, Vol. 10, No 1, 14-20.• Coelho C.A.S., D. B. Stephenson, M. Balmaseda, F. J. Doblas-Reyes and G. J. van Oldenborgh, 2005b: “Towards an integrated seasonal forecasting system for South America”. Submitted to J. Climate.

•Stephenson, D. B., C.A.S. Coelho, F. J. Doblas-Reyes, and M. Balmaseda, 2005:“Forecast Assimilation: A Unified Framework for the Combination of Multi-Model Weather and Climate Predictions.” Tellus A, Vol. 57, 253-264.• Coelho C.A.S., S. Pezzulli, M. Balmaseda, F. J. Doblas-Reyes and D. B. Stephenson, 2004: “Forecast Calibration and Combination: A Simple Bayesian Approach for ENSO”. Journal of Climate. Vol. 17, No. 7, 1504-1516.

• Coelho C.A.S., S. Pezzulli, M. Balmaseda, F. J. Doblas-Reyes and D. B. Stephenson, 2003: “Skill of Coupled Model Seasonal Forecasts: A Bayesian Assessment of ECMWF ENSO Forecasts”. ECMWF Technical Memorandum No. 426, 16pp. Available at http://www.met.rdg.ac.uk/~swr01cac

More information …

Forecast assimilation improves reliability in the western Pacific

Reliability as a function of longitudeReliability as a function of longitude

ENS

- - - INT

Resolution as a function of longitude

Forecast assimilation improves resolution in the eastern Pacific

ENS - - - INT