Characterization of Forecast Error using Singular Value Decomposition
description
Transcript of Characterization of Forecast Error using Singular Value Decomposition
Characterization of Forecast Error using Singular Value Decomposition
Andy Moore and Kevin SmithUniversity of California Santa Cruz
Hernan ArangoRutgers University
Outline
• An overview of singular value decomposition (SVD)• Flavors of SVD• Duality of SVD• Norms• Unstable jet• California Current
Singular Value Decomposition (SVD)
Ae eSquare Matrix:
Right singular vectors: Au vLeft singular vectors: T A v u
T 2A Au u T 2AA v vTA UΣV
A generalization of eigenvectors for rectangularmatrices.
-1A EΛERectangular Matrix:
Σ Important rank/dimension info
Think Covariance!
u1
u2 v1
v2
A
p pd dt Mx x
SVD and “Model” Errors
Perfect model:
( )d dt M t x x εImperfect model:
( ) ( ) ( )pt t t x x x
( )d dt t x M x ε
Errors:
TLM:
Tangent linearmodel
Error
State vector: T, , , ,T S u v x
0
( ) (0, ) (0) ( , ) ( )t
t t t d x M x M ε
Singular Value Decomposition (SVD)
ComplimentaryFunction (CF)
Particular integral(PI)
SVD of CF – Singular vectors of initial conditions (i.e. ECMWF EFS)
SV 1
SV 2
Initial ConditionCovariance
at t=0
SV 1
SV 2
Final TimeCovariance
at t=t
Singular Value Decomposition (SVD)
ComplimentaryFunction (CF)
Particular integral(PI)
SVD of PI – Stochastic optimals (SO)
SO 1 (q1)
SO 2(q2)
Model ErrorCovariance
at t~0
Model ErrorCovariance
at t=t
SO 1 (q1)
SO 2(q2)
0
( ) (0, ) (0) ( , ) ( )t
t t t d x M x M ε
Duality of SVD
Fastest growingperturbations
Dynamics ofmeander and
eddy formation
Fastest growingerrors
Most predictablepatterns
Fastest lossof predictability
Flavors of SVD
( )d dt t x M x ε
?
Flavors of SVD
0 (0)d dt x M x εInitial condition error:
Find the 0 that maximizes:T ( ) ( )t t x P x
Subject to the constraint:
T T0 0 (0) (0) 1 ε Cε x C x
Equivalent to the generalized eigenvalue problem:
T (0)= (0) M PM x C xand SVD of: 1 2 1 2P MC
FInal time norm
Initial time norm
Illustrative Example – A Zonal Jet600km
360k
m500m deep, f=10-4, =0, x-15km, z=100mEastward Gaussian jet, 40km width, 1.6ms-1
SV time interval = 2 days. Energy norm, P=C.
TM PMSVD:
T
dt dt M P MSVD:
T dtM PMSVD:
SVD:| '| T 'ct t t
t te dtdt M PM
Initial SV
Forcing SV
Stoch Opt (white)
Stoch Opt (red)
tc= 2 days
Periodic Channel & Zonal Jet
x
y
xz
x
y
xz
Initial Final
Conservation of wave action(or pseudomomentum):
Doppler shifting of (ku) isaccompanied by increase in E(Buizza and Palmer, 1995).
E
ku
Baroclinically Unstable Jet
1000km
2000
km
x=10km, f=-10-4, =1.6×10-11
t=0 t=50 days
SST SST
SH
Initial Condition Singular Vectors
Singular Vector #12 Singular Vector #12
Singular Vector #11Singular Vector #11
SSH SSH
SSH SSH
t=0 t=2 days
t=0 t=2 days
Energy norm at initial and final time
The Forecast Problem
SV 1
SV 2
Analysis ErrorCovariance
at t=0
SV 1
SV 2
Forecast ErrorCovariance
at t=t
t=0 t=T
forecast
Forecast initialcondition error=
analysis error
1T 1
ax E x
Tf fM FM
Ea F
Perform SVD on:
subject to:
(Ehrendorfer & Tribbia, 1998)
?
fM
The Inverse Analysis Error Covariance, (Ea)-1
1 1 T 1 a GE RD G
1 T 1 T D G R G VTV
InverseAnalysis
errorcovariance Hessian matrix
Hessian matrix Primal spaceLanczos vectorexpansion from
4D-Var
The number Lanczos vectors= number of 4D-Var inner-loops
PriorErrorCov.
Adjointof
ROMS
Tangentof
ROMS
ObsErrorCov.
The Forecast Error Covariance, F
Experience in numerical weather prediction atECMWF suggests that F=E is a good choice(Buizza and Palmer, 1995).
We will assume the same here…
… more on this later however…
Evolved Analysis Error Covariance
t0 ta tf
(Ea)-1 (Ea)-1
Ma Mf
Analysiscycle
(4D-Var)
Forecastcycle
Evolved Analysis Error Covariance
1 T T T Ta
a a a a e eM E M M VTV M V TV
We actually need the analysis error at the end of theanalysis cycle:
so we need the time evolved Lanczos vectors, Ve.
T 1 V D V I T 1 e eV D V Ibut
Reorthonormalize using Gramm-Schmidt:T T Te e g gV TV V PTP V
where: g eV V P T g gV V Iand
Hessian Singular VectorsT T f fx M EM xFind the x that maximizes forecast error
Subject to the constraint that 1T 1
ax E x(Barkmeijer et al, 1998)
Solve the equivalent eigenvalue problem:
1 2 -1 -T T T -1 T 1 2 e f f eS L P V M EM V P L S w w
where TT LSL (Cholesky factorization of T)
and1 2 T T ew S L PV x A x and x A w
where A+ is the right generalized inverse, and AA I
The dimension ofthe problem is reduced to the #of 4D-Var inner-loops wholespectrum.
1 1 T T a e eE V P T P VBut
Baroclinically Unstable Jet:Identical Twin 4D-Var
Strong constraint primal 4D-Var1 outer-loop, 15 inner-loops2 day assimilation windowPerfect T obs everywhere onday 0, day 1, day 2Initial conditions only adjustedBalance operator applied
rms error in T
rms error in u rms error in v
rms error in SSH
Cycle #
Cycle #
Cycle #
Cycle #
4D-Var
No assim
Forecast4D-Var
No assim
Forecast
4D-Var
No assim
Forecast4D-Var
No assim
Forecast
Singular Values of 2 Day Jet Forecasts
log10
Cycle #
SV # SV1 SV1
SVnSVn
SV1 SV1
SVnSVn
Rugby Ball
SV1 SV1
SVnSVn
Cigar
SV1 SV1
SVnSVn
Hessian Singular Vectors
SV #1 SV #1
SV #1
Initial SSH Final SSH Initial SSH Final SSH
Initial SSH Final SSH
CYCLE #1 CYCLE #20
CYCLE #40
Hessian Singular Vectors
SV #1
SV #1
CYCLE #1
CYCLE #20
Initial SSH Final SSHt=2 daysForecast SSH
The California Current
30km, 10 km, 3 km & 1km grids, 30- 42 levels
Veneziani et al (2009)Broquet et al (2009)
ERA40 and CCMP forcing
SODA openboundaryconditions
fb(t), Bf
bb(t), Bb
xb(0), Bx
Previous assimilationcycle
Observations (y)
CalCOFI &GLOBEC
SST &SSH
Argo
TOPP Elephant Seals
Ingleby andHuddleston (2007)
Data from Dan Costa
Observations
4D-VarAnalysis
Posterior
Observations
4D-VarAnalysis
Posterior
Observations
4D-VarAnalysis
Posterior
prior prior prior
Sequential 4D-Var
8 day 4D-Var cyclesoverlapping every 4 days
30 Year Reanalysis of California Current1980-2010
Obs:Pathfinder, AMSR-E, MODIS, EN3, AvisoForcing: ERA40, ERA-Interim, CCMP (25 km)Analysis every 4 days, 8 day overlapping assim cycleshttp://www.oceanmodeling.ucsc.edu
Initial cost J
Final cost J
+ Final NL J
Moore et al. (2012)
CCS: Hessian SVs
Jan1999
June1999
Cycle #
Dec1999
10 kmCCS ROMS
log10 SV #
Spring
SV1 SV1
SVnSVn
Autumn
SV1 SV1
SVnSVn
Spring
SV1 SV1
SVnSVn
Autumn
SV1 SV1
SVnSVn
CYCLE #1
SV SSH initial
SV SSH final
Forecast SSH
10 kmCCS ROMS
CYCLE #23
SV SSH initial
SV SSH final
Forecast SSH
10 kmCCS ROMS
The Forecast Error Covariance
TT
T
aa aE I G B I G
d dR
d d
KKM M
KK
Recall that we can express the forecast error cov. as:
Posteriorerror
covarianceTangentLinear4D-Var
AdjointLinear4D-Var
Tf fffE DMM
Forecasterror
covariance
diag , , ,f b f aD E B B B
Controlpriors
Tangentlinearmodel
where:
1 2 1 T T T 1 2 ff fS L V D VL S w wM M
So the control SVD problem becomes:
(computational cost equals (# inner-loops)2)
Summary
• SVD provides information about forecast error growth.• Growing directions of the forecast error covariance error ellipsoid vary with time• SV structures become smaller scale• Flow and/or error dependent regimes• Future work: - explicit forecast error covariance - model error and weak constraint - control singular vectors