Chapter 5: Exploratory Methods for

Spatio-Temporal Data

Jim Faulkner

Space-Time Reading GroupUniversity of Washington

Feb 5, 2018

Outline

1. Spectral Analysis

2. Empirical Covariance Functions

3. Empirical Orthogonal Function Analysis

4. Principal Oscillation Patterns

5. Canonical Correlation Analysis

1 / 31

Spectral Analysis

Why use spectral analysis?

I Partition overall variation into different scales, e.g., seasonal,annual, decadal

I Determine which components account for the most variation

I Simplifies correlation structure – spectral transformationdecorrelates

I Can allow for reduced-rank approximations

2 / 31

3.5.1 Spectral Representations via Orthogonal

Series

Consider a process that can be written as a function of time, f (t),over the interval (a, b). Define a sequence of spectral basis functionsφk(t) for k = 0, 1, . . . , to be orthogonal over the interval (a, b) if∫ b

a

φk(t)φl(t)dt =

{0, k 6= l ,

δ, k = l ,(3.118)

where δ > 0. These functions are orthonormal if δ = 1.

3 / 31

Now we expand f (t) in terms of these basis functions:

f (t) =∞∑k=0

αkφk(t), (3.119)

where αk are the weights or spectral coefficients of the expansion.

αk =

∫ b

a

f (t)φk(t)dt k = 1, 2, . . . , (3.120)

The spectral coefficients are the projection of f (t) onto theorthogonal basis functions.

4 / 31

Trigonometric Series ExpansionConsider f (t) defined on interval (−1/2, 1/2). Let φk(t) correspondto the orthonormal trigonometric basis functions sin(2πkt) andcos(2πkt). We can write

f (t) =a02

+∞∑k=0

{ak cos(2πkt) + bk sin(2πkt)} (3.122)

The Fourier coefficients are:

ak = 2

∫ 12

− 12

f (t) cos(2πkt)dt, k = 0, 1, . . . (3.123)

bk = 2

∫ 12

− 12

f (t) sin(2πkt)dt, k = 0, 1, . . . (3.124)

5 / 31

Using Euler’s relationship, e±i2πkt ≡ cos(2πkt)± i sin(2πkt), we cansimplify :

f (t) =∞∑k=0

αke±i2πkt − 1/2 ≤ t ≤ 1/2, (3.125)

where φk(t) ≡ e±i2πkt and αk ≡ ak + ibk are both complex functions.This result is the Fourier Transform or the spectral representationtheorem.

6 / 31

3.5.2 Discrete-Time Spectral Expansion

Let {Yt : t = 1, . . . ,T} be a times series and define{φk(t) : t = 1, . . . ,T ; k = 1, . . . , pα} to be a complete set of basisfunctions. Define Y ≡ (Y1, . . . ,YT )′, α ≡ (α1, . . . , αpα), andΦ ≡ (φ1, . . . ,φpα), where φk ≡ (φk(1), . . . , φk(T ). Then thespectral expansion of Y

Yt =

pα∑k=1

αkφk(t) (3.126)

can be written in matrix notation as

Y = Φα (3.127)

7 / 31

Multiply both sides of (3.127) by Φ to obtain the spectral coefficientvector,

α = (Φ′Φ)−1

Φ′Y (3.128)

which is in the form of a least-squares estimator. When the basisfunctions are orthonormal, Φ′Φ = I , then (3.128) becomes

α = Φ′Y (3.129)

Note that the operation Φ′Y can be carried out using the FastFourier Transform (FFT), and the operation Φα can be carried outusing the inverse FFT.

8 / 31

Figure 5.1. SST anomalies one location through time.

9 / 31

Figure 5.4. SST anomalies in tropical Pacific through time.

10 / 31

Empirical Covariance Functions

Assume we have observations Zt ≡ (Z (s1; t), . . . ,Z (sm; t)′) fort = 1, . . . ,T . An m ×m empirical (averaged over time) lag-τ spatialcovariance matrix is given by

C(τ)Z ≡

1

t − τ

T∑t=τ+1

(Zt − µZ )(Zt−τ − µZ )′, τ = 0, 1, . . . ,T (5.1)

where the empirical spatial mean is given by

µZ ≡1

T

T∑t=1

Zt

11 / 31

Figure 5.6. Lag-0 covariance and correlation along equator for SST.

12 / 31

Figure 5.7. Lag-6 covariance and correlation along equator for SST.

13 / 31

The estimated spatio-temporal covariance at spatial lag h and timelag τ is given by

CZ (h; τ) ≡ 1

|Ns(h)|1

|Nt(τ)|×∑

si ,sj∈Ns(h)

∑t,r∈Nt(τ)

(Z (si ; t)− µ(si))(Z (sj ; r)− µ(sj))

where Ns(h) refers to pairs of spatial locations with spatial lag withinsome tolerance of h, Nt(τ) refers to pairs of time points with timelag within some tolerance of τ , and |N(·)| refers to the cardinality(number of elements) in the set N(·). Also,

µZ (si) ≡1

T

T∑t=1

Z (si ; t)

14 / 31

Figure 5.9. SST spatio-temporal covariance.

15 / 31

5.3 Empirical Orthogonal Function (EOF) Analysis

I EOFs are eigenvectors from eigen (spectral) decomposition ofcovariance matrix

I In discrete formulation is PCA, in continuous is Karhunen-Loeveexpansion.

I Typically used

1. Diagnostically to find principal spatial structures and how thosevary

2. To reduce dimensionality in spatio-temporal data sets and toreduce noise

16 / 31

5.3.1 Spatially Continuous Formulation

Consider data {Zt(s) : s ∈ Dt , t = 1, 2, . . . } from a continuousspatial process measured at discrete time intervals. We want to findan optimal separable orthogonal decomposition:

Zt(s) =∞∑k=1

αt(k)φk(s), (5.17)

such that var(αt(1)) > var(αt(1)) > . . . , and cov(αt(i), αt(k)) = 0for all i 6= k .

17 / 31

The solution is the Karhunen-Loeve expansion, which allows thedecomposition:

C(0)Z (s, r) =

∞∑k=1

λkφk(s)φk(r), (5.18)

where {φk(·)} are the eigenfunctions and {λk} are the eigenvalues ofthe Fredholm integral equation:∫

Ds

C(0)Z (s, r)φk(s)ds = λkφk(r) (5.19)

This can be solved numerically, but is difficult and usually not done inpractice.The kth ”amplitude” times series {αt(k) : t = 1, 2, . . . } is

αt(k) =

∫Ds

Zt(s)φk(s)ds, t = 1, 2, . . . (5.22)

18 / 31

5.3.2 Spatially Discrete Formulation

I Let Zt ≡ (Z1(s1), . . . ,Zt(sm))′ and define the kth discrete EOFto be ψk ≡ (ψk(s1), . . . , ψk(sm)), where ψk is the vector in thelinear combination at(k) = ψ′kZt , for k = 1, . . . ,m.

I In general, ψk is the vector that maximizes var(at(k)) subject tothe constraints ψ′kψk = 1 and cov(at(k), at(j)) = 0 for allj 6= k . This is equivalent to solving the eigen decomposition ofC

(0)Z :

C(0)Z = ΨΛΨ′

where Ψ ≡ (ψ1, . . . ,ψm) is the m ×m orthonormal matrix ofeigenvectors (Ψ′Ψ = I ), Λ = diag(λ1, . . . , λm) is the m ×mdiagonal matrix of eigenvalues decreasing down the diagonal,var(at(k) = λk), and Zt is centered with mean 0.

19 / 31

Figure 5.17. SST first and second EOFs.

20 / 31

Figure 5.18. SST third and fourth EOFs.

21 / 31

5.5 Principal Oscillation Patterns (POPs)

I Spectral decomposition of the propogator matrix for thefirst-order dynamical system provides information aboutunderlying dynamics

I POP analysis based on spectral decomposition of propogatormatrix of a spatio-temporal process expressed as a first-orderdynamical system

I Consider the first-order discrete linear system

Zt = MZt−1 (5.26)

where Zt ≡ (Zt(s1), . . . ,Zt(s1))′, and M is the m ×m, full-rankbut non-symmetric propogator matrix.

22 / 31

I In short, if we calculate the normalized SVD M = WΛV′ then

Zt = WV′Zt = Wat (5.28)

where at ≡ V′Zt are the POP coefficients.

I The columns of W, {wk} are the principal oscillation patternsand the columns of V, {vk} are called the adjoint bases.

I We can form the recursion at = Λat−1 or at(k) = λkat−1, whichresults in solutions at(k) = (λk)t .

I These coefficients evolve according to at(k) = λtkeiφk t , where

γk = |λk |.I Components of interest are at(k), γk , φk , a0(k)/e andτk = −1/ ln(γk).

23 / 31

5.5.1 Calculation of POPs

I To account for uncertainty in the data, we write the process as afirst-order vector-autoregressive process:

Zt = MZt−1 + ηt (5.33)

where ηt has mean 0 with uncorrelated elements.

I Under second-order stationarity, M = C(1)Z (C(0)

Z )−1

I A method-of-moments estimator is M = C(1)Z (C(0)

Z )−1

I Calculate eigenvalue decomposition of M, which results inM = WΛW−1

I Set V′ = W−1, then the POP-coefficient estimates can beobtained from at = V′Zt

24 / 31

Figure 5.23. SST POP for 10th eigenvalue. a) real part, b) negativeimaginary part, c) negative real part, c) imaginary part.

25 / 31

5.6 Spatio-Temporal Canonical Correlation

Analysis (CCA)

I CCA obtains linear combinations of two sets of random variableswhose correlations are maximal

I Can apply to two random variables indexed by space and time

I Suppose we have two data sets {Zt ≡ (Zt(s1), . . . ,Zt(sm))′}and {Xt ≡ (Xt(x1), . . . ,Xt(x`))′} with a possibly differentspatial domain but the same temporal domain (t = 1, . . . ,T )

I The kth canonical correlation is defined as

rk ≡ corr(ξ′kZt ,ψ′kXt) =

ξ′kC(0)Z ,Xψ

′k

(ξ′kC(0)Z ξk)1/2(ψ′kC(0)

X ψk)1/2(5.36)

26 / 31

I For k = 1 we can write:

r 21 =[ξ′1(C(0)

Z )−1/2C(0)Z ,X (C(0)

X )−1/2ψ1]2

(ξ′1ξ1)(ψ

′1ψ1)

where ξ1 ≡ (C(0)Z )1/2ξ1 and ψ1 = (C(0)

X )1/2ψ1.

I Turns out that r 21 is the largest singular value in the singularvalue decomposition of

(C(0)Z )−1/2C(0)

Z ,X (C(0)X )−1/2

where ξ1 and ψ1 are the left and right singular vectors.

I The vectors ψ1 and ξ1 can then be calculated and the timeseries of canonical variables at(1) ≡ ξ′1Zt and bt(1) ≡ ψ′1Xt canbe obtained.

27 / 31

Figure 5.26. First CCA patterns for SST and Mallard counts

28 / 31

Figure 5.27. First canonical variables for SST and Mallard counts

29 / 31

References

I Cressie, N., and C. K. Wikle. 2011. Statistics forspatio-temporal data. John Wiley & Sons, New Jersey.

30 / 31