Parameter Estimation in the Spatio-Temporal Mixed Effects ...€¦ · Parameter Estimation in the...

Parameter Estimation in the Spatio-TemporalMixed Effects Model –

Analysis of Massive Spatio-Temporal Data Sets

Matthias KatzfußAdvisor: Dr. Noel Cressie

Department of StatisticsThe Ohio State University

September 17, 2010

Matthias Katzfuß (OSU Statistics) STME Parameter Estimation September 17, 2010 1 / 23

Outline

1 Introduction: The STME Model

2 Parameter EstimationEM EstimationBayesian Estimation

3 Application: Analysis of CO2 Data

4 Conclusions

Introduction: The STME Model

Outline

4 Conclusions

Notation

• Hidden spatio-temporal process yt(s) at time t and location s

• Measurementszt(si ,t) = yt(si ,t) + εt(si ,t)

i = 1, . . . , nt

t = 1, . . . ,T

• In vector notation: z1:T := [z′1, . . . , z′T ]′, where

zt := [z(s1,t), . . . , z(snt ,t)]′

• Goal: Predictyt(s0); t ∈ {1, . . . ,T}

Motivating Example: Remote-Sensing Data

Example: Global satellite measurementsof CO2

Challenges of global remote-sensing data:

• Massiveness• Need dimension reduction

• Sparseness• Need to take advantage of spatial

and temporal correlations

• Nonstationarity• Need a flexible model

Spatio-Temporal Mixed Effects Model (Cressie et al., 2010)

Process Model:yt(s) = x(s)′βt + b(s)′ηt + γt(s)

• x(s)′βt : large-scale trend

• b(s) := [b1(s), . . . , br (s)]′: vector of known spatial basis functions

• ηt = Hηt−1 + δt ; t = 1, 2, . . .• η0 ∼ Nr (0,K0)

• δt ∼ Nr (0,U)

• γt(s) ∼ N(0, σ2γvγ(s)): fine-scale variation

Unknown parameters: θ :={{βt}, σ2

γ ,K0,H,U}

Previous Approaches to Massive S-T Data Sets

• Many ad-hoc methods used outside the statistics literature(non-optimal, no measures of uncertainty)

• Other statistical spatio-temporal dimension-reduction models are lessgeneral (e.g., Nychka et al., 2002)

• STME model: Parameter estimation via binned-method-of-moments(Kang et al., 2010):• Many arbitrary choices have to be made• Estimates have to be modified to be valid• Does not fully exploit temporal dependence in the data

Parameter Estimation

Outline

4 Conclusions

Parameter Estimation EM Estimation

Outline

4 Conclusions

Maximum-Likelihood Estimation

• Goal: Findθ̂ML = arg max

θf (z1:T |θ)

where recall zt = Xtβt + Btηt + γt + εt

• Problem: Likelihood f (z1:T |θ) is quite complicated

• Solution: Expectation-maximization algorithm (Dempster et al.,1977)• Maximization: “Complete-data likelihood” f (η1:T ,γ1:T |θ) is easy to

maximize

• Expectation: Eθ( f (η1:T ,γ1:T |θ) | z1:T ) is obtained via FRS, a rapidsequential updating technique based on the Kalman filter (Kalman,1960)

EM Estimation (Katzfuss & Cressie, 2010)

The EM algorithm:

• Choose initial value θ[0]

• For l = 0, 1, 2, . . . (until convergence):

1. E-Step: Run FRS with θ[l ] to obtain Eθ[l ]( f (η1:T ,γ1:T |θ) | z1:T )

2. M-Step: θ[l+1] = arg maxθ

Eθ[l ]( f (η1:T ,γ1:T |θ) | z1:T )

3. Go back to 1.

Properties of the resulting estimates:

• Parameter estimates guaranteed to be valid

• Here, convergence to a (possibly local) maximum of the likelihoodfunction

Parameter Estimation Bayesian Estimation

Outline

4 Conclusions

Bayesian Inference

• Parameters θ have a prior distribution

• Obtain posterior distribution of unknowns yt(s0) and θ given the dataz1:T using Bayes’ Theorem

• In almost all cases, have to approximate posterior by sampling from it

• “Shrinkage”: Biased, but more efficient estimators

Priors and Posteriors

Prior distributions:

• “Standard” priors on {βt} and σ2γ

• Covariance matrices K0 and U: Multiresolutional Givens-angle prior(Kang & Cressie, 2009)• Control extreme eigenvalues• Shrink off-diagonal elements toward zero

• Propagator matrix H: Shrink off-diagonal elements depending on howfar corresponding basis functions are apart

Posterior distribution:

• Samples of posterior distribution obtained using MCMC

Application: Analysis of CO2 Data

Outline

4 Conclusions

The Data

Mid-tropospheric CO2 on May 1-4, 2003, as measured by AIRS (nt ≈ 14K )

400Day 1 Day 2

Day 3 Day 4

Statistical Analysis

• Trend: x(s) = [1 lat(s)]′

• Make predictions on a hexagonal grid of size 57, 065 for each day

• Basis functions: r = 380 bisquare functions at 3 spatial resolutions

−1.0 −0.5 0.0 0.5 1.0

Bisquare function in one dimension

EM Results

Predictions using EM

Standard errors using EM

EM computation time: 16 iterations × one minute each = 16 min total

Bayesian Results

Posterior means

Posterior standard deviations

1,500 MCMC iterations × 15 seconds each = 6.25 hours total

Estimates of the Propagator Matrix

50 100 150 200 250 300 350

350−1

−0.5

Conclusions

Outline

4 Conclusions

Conclusions

• STME Model• Scalable and flexible technique for analysis of massive, nonstationary

spatio-temporal data sets• Provides uncertainty quantification• Here, successful use on CO2 satellite data

• Parameter estimation:• EM Estimation: Fast, easy• Bayesian estimation: Better prediction (≈ 10% for AIRS data), more

accurate uncertainty assessment

Conclusions

References

• Cressie, N., Shi, T., & Kang, E. L. (2010). Fixed rank filtering for spatio-temporaldata. Journal of Computational and Graphical Statistics. Forthcoming.

• Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum Likelihood fromIncomplete Data via the EM Algorithm. Journal of the Royal Statistical Society,Series B, 39(1), 1–38.

• Kalman, R. (1960). A new approach to linear filtering and prediction problems.Journal of Basic Engineering, 82(1), 35–45.

• Kang, E. L., & Cressie, N. (2009). Bayesian inference for the spatial randomeffects model. Department of Statistics Technical Report No. 830. The OhioState University.

• Kang, E. L., Cressie, N., & Shi, T. (2010). Using temporal variability to improvespatial mapping with application to satellite data. Canadian Journal of Statistics.Forthcoming.

• Katzfuss, M., & Cressie, N. (2010). Spatio-Temporal Smoothing and EMEstimation for Massive Remote-Sensing Data Sets. Department of StatisticsTechnical Report No. 840. The Ohio State University.

• Nychka, D. W., Wikle, C., & Royle, J. (2002). Multiresolution models fornonstationary spatial covariance functions. Statistical Modelling, 2, 315-331.

Parameter Estimation in the Spatio-Temporal Mixed Effects ...€¦ · Parameter Estimation in the...

Documents

Transcript of Parameter Estimation in the Spatio-Temporal Mixed Effects ...€¦ · Parameter Estimation in the...

Spatio-temporal variability of global soil moisture productsearth.esa.int/dragon-2015-programme/montzka-spatio-temporal... · • Soil moisture data ... China . Spatio-temporal ...

Logical Process Modeling of Spatio-Temporal Application · PDF fileLogical Process Modeling of Spatio – Temporal Application ... Logical Process Modeling of Spatio-Temporal ... the

Spatio temporal 2

A51 Spatio Temporal Features1

Spatio Temporal Model

SPATIO-TEMPORAL LAND USE/LAND COVER CHANGES … · SPATIO-TEMPORAL LAND USE/LAND COVER CHANGES ANALYSIS AND MONITORING IN THE VALENCIA MUNICIPALITY, SPAIN . SPATIO-TEMPORAL LAND USE/LAND

Aalborg Universitet Aspects of temporal and spatio-temporal ...

Spatio-temporal differences in model outputs and parameter space as determined by calibration extent

Spatio-Temporal Building Population Estimation for Highly ... - Spatio-Temporal Building... · Spatio-Temporal Building Population Estimation for Highly Urbanized Areas Using GIS

Dynamic spatio-temporal landscape models spatio-temporal landscape models ... account for transient adaptation phases.A static model ... the spatio-temporal patterns are not explicitly

Analysing Spatio-temporal Regression Data: A Case Study in ... fileThomas Kneib Spatio-Temporal Regression Data † Spatio-temporal regression models should allow { to account for

A Parameter-Free Spatio-Temporal Pattern Mining …climatechange.cs.umn.edu/eddies/media/papers/...A parameter-free spatio-temporal pattern mining model to catalog global ocean dynamics

Spatio-temporal resolution of primary processes of ... · Spatio-temporal resolution of primary ... starting at 3.8˚A13 and now at a 1.9˚Aresolution. 14 ... Spatio-temporal resolution

Spatio Temporal Economics

Subjective Preference of Spatio-Temporal Rate in …ywang/Research/Yong_ICME04_2.5.pdfSubjective Preference of Spatio-Temporal Rate in Video Adaptation ... spatio-temporal rates used

Motion-based Adaptive Streaming in WebRTC using Spatio ...home.ku.edu.tr/~mtekalp/globecom2017.pdf · parameter value (quality level adaptation). C. Mixed Spatio-Temporal Resolution

Subjective Preference of Spatio-Temporal Rate in Video ... · spatio-temporal rates used in adaptation, knowledge about subjective preference of spatio-temporal rates is needed. In

Spatio-temporal Negotiation Protocols

Moving object detection using spatio-temporal …ash/spatio-temporal multilayer compound...Multimed Tools Appl DOI 10.1007/s11042-016-3698-2 Moving object detection using spatio-temporal

Spatio-temporal compression of trajectories in road … - Spatio-temporal compression...Spatio-temporal compression of trajectories in road ... Therefore, our focus is on lossy spatio-temporal