Caio A. S. Coelho
description
Transcript of Caio A. S. Coelho
Caio A. S. Coelho
Is a group of forecastsbetter than one?
Department of Meteorology, University of Reading
• Introduction• Combination in meteorology• Bayesian combination• Conclusion and future
directions
Plan of talk
Pierre-Simon Laplace(1749-1827)
C)yy(yy MSLSLS
The Pioneer of combination
xpya a
p
•
••
•
•
•
•
•
•
•
•• •
•• •
• xy
•
•
)1757,ichcovBos(SituationofMethod:y
squareLeast:y
MS
LS
))x(,x(CCwhere
Laplace (1818): Combination of two estimators
)xmin( 2
|)xmin(|
slope residual
+ numerical = methods
Have combined forecasts better skill thanindividual forecasts?
In modern times…
Forecasts from
•••
Forecast combination literature
• Trenkler and Gotu (2000): ~600 publications
(1970-2000)• Widely applied in Economics and
Meteorology
• Overlap of methods in these areas
• Combined forecasts are better than individual forecasts
Some issues
• What is the best method for combining?
• Is it worth combining unbiased forecasts with biased forecasts?
DEMETER coupled model forecasts
Nino-3.4 index (Y)
Period: 1987-99
9 members
Jul -> Dec
5 months lead
DEMETER web page: http://www.ecmwf.int/research/demeter
ECMWFMeteo-France (MF)Max Planck Institut (MPI)),(N~Y 2
tt
December Nino-3.4 raw forecasts
]ˆ96.1ˆ,ˆ96.1ˆ[:.I.P%95 tttt
Xt
tt
sˆ
Xˆ
),(N~Y 2tt
Forecast combination in meteorology
:wandw
:F
:M
:X
)t(Xww)t(F
io
M
1iiio
Linear combination of M ensemble mean forecasts X
constants
models
ensemble mean
combined forecast
• Bias-removed multimodel ensemble mean forecast (Uem)
• Regression-improved multimodel ensemble mean forecast(Rem)
• Regression-improved multimodel forecast (Rall)
Combination methods used in meteorology
Kharin and Zwiers (2002)
M
1iiio )t(Xww)t(F
How to estimate wo and wi ?
Kharin and Zwiers combined forecasts
2iii
n
1i
2i
2mRe
2mRe
)mReY(
n
1s
)s,m(ReN~Y
2iii
n
1i
2i
2Rall
2Rall
)RallY(
n
1s
)s,Rall(N~Y
]XXX[X
)s,Uem(N~Y
321
2X
Y: Observed December Nino-3.4 index
X: Ensemble mean forecast of Y for December
Thomas Bayes (1701-1761)
The process of belief revision on any event Y consists in updating the probability of Y
when new information X becomes available
The Bayesian approach
)xX(p
)Y(p)Y|xX(p)xX|Y(p
Example: Ensemble mean (X=x=27C)
Likelihood:p(X=x|Y)
Prior:p(Y)
Posterior:p(Y|X=x)
)C,Y(N~Y b
1TT
111T
obba
)SGCG(CGL
C)LGI()CGSG(D
)]YY(GX[LYY
)S],YY[G(N~Y|X o
Prior:
Likelihood:
Posterior:
1YYXYSSG
YGXGYo T
YYXX GGSSS
)D,Y(N~X|Y a
calibration
Bayesian multimodel forecast (B)
bias
July and December Reynolds OI V2 SST (1950-2001)
Nino-3.4 index observational data
Nino-3.4 indexmean values:Jul: 27.1CDec: 26.5Cr: 0.87
R2 =0.76
tyeartheforindex4.3NinoJuly
tyeartheforindex4.3NinoDecemberY
50.1C14.14),(N~|Y
t
t
1o
o2
t0t1ott
),(N~Y:iorPr 2otot t1oot
All combined forecasts
Uem
Rall
Rem
B
Why are Rall and B so similar?
Combined forecasts in Bayesian notationForecast
Likelihood Prior
Uem
Rem
Rall
B
)S,YY(N~Y|X o
)S],YY[G(N~Y|X o
)S],YY[G(N~Y|X o
)S],YY[G(N~Y|X o
)S,Y(N~Y YY
)S,Y(N~Y YY
),(N~Y 2otot
)C(Uniform
B priorRem and
Rall prior
)C,Y(N~Y b
)S],YY[G(N~Y|X oBayesian
combinationPrior:Likelihoo
d:
Skill and uncertainty measuresForecast MSE
(C)2
Skill Score(%)
Uncert.(C)2
Climatology 1.98 0 1.20
ECMWF (raw) 1.64 18 0.49
ECMWF (bias-removed) 0.18 91 0.49
ECMWF (regression-improved)
0.22 89 0.47
MF (raw) 0.27 86 0.39
MF (bias-removed) 0.22 89 0.39
MF (regression-improved) 0.20 90 0.46
MPI (raw) 15.24 -669 0.49
MPI (bias-removed) 1.52 23 0.49
MPI (regression-improved) 1.06 46 0.86
Uem 0.25 88 1.79
Rem 0.46 77 0.56
Rall 0.12 94 0.38
B 0.11 94 0.23Skill Score = [1- MSE/MSE(climatology)]*100%
Conclusions and future directions
• Forecast can be combined in several different ways
• Combined forecasts have better skill than individual forecasts
• Rall and B had best skill for Nino-3.4 example
• Inclusion of very biased forecast has not deteriorated combination
• Extend method for South America rainfall forecasts