Sequential Monte Carlo for static Bayesian models with ... · Background Independent SMC...

Background Independent SMC Examples Summary

Sequential Monte Carlo for static Bayesianmodels with independent MCMC proposals

Dr Christopher DrovandiQueensland University of Technology and

Australian Centre of Excellence for Mathematical andStatistical Frontiers

Collaborators: Leah South, Tony Pettitt, Adam ClementsAcknowledgments: ARC funding, EcoSta organisers

17th June, 2017

Chris Drovandi

EcoSta 2017 1 / 29


Outline

1 BackgroundBayesian Posterior SamplingMarkov Chain Monte CarloSequential Monte Carlo

2 Independent SMCIndependent MCMC Proposals within SMCRecycling

3 ExamplesBEGE ModelChallenging ODE Example

4 Summary

Chris Drovandi

EcoSta 2017 2 / 29


Bayesian Posterior Sampling

Interest is in sampling from the posterior π(θ|y), where

π(θ|y) =f (y |θ)π(θ)

Z

and Z =∫

Θ f (y |θ)π(θ)dθ.

Z is referred to as the evidence and is useful for model choice.

Chris Drovandi

EcoSta 2017 3 / 29


Markov Chain Monte Carlo

MCMC methods construct en ergodic Markov chain with theposterior as its limiting distrubution.

A common MCMC algorithm is Metropolis Hastings (MH-)MCMC, where proposals θ∗ are accepted with probability

min

(1,

f (y|θ∗)π(θ∗)q(θ|θ∗)

f (y|θ)π(θ)q(θ∗|θ)

),

where q(·) is the proposal density.

Chris Drovandi

EcoSta 2017 4 / 29


Markov Chain Monte Carlo

Some Limitations:

Difficult to automate and adapt the method

Need to tune the proposal distribution for good performance.

Convergence can be difficult to assess

Can have difficulty exploring irregular posteriors (egmulti-modality)

Standard MCMC is a serial algorithm

Chris Drovandi

EcoSta 2017 5 / 29


Sequential Monte Carlo (Chopin (2002) and Del Moralet al. (2006))

SMC methods can be a useful alternative to MCMC in someapplications as they are...

Naturally adaptive

Easily parallelisable

More capable of dealing with multimodal or complexdistributions

Able to produce an estimate of the unknown normalisingconstant

Chris Drovandi

EcoSta 2017 6 / 29


Sequential Monte Carlo

Basic idea:

Moving a population of N particles through a sequence ofdistributions (starting with one easy to sample from andfinishing at the target posterior)

Can introduce the effect of either the data (data annealing) orthe likelihood (likelihood annealing) sequentially

Chris Drovandi

EcoSta 2017 7 / 29



In likelihood annealing, the power posteriors are defined by

πt(θ|y) ∝ f (y |θ)γtπ(θ),

where 0 = γ0 < γt < γT = 1 and 0 < t < 1.

At each iteration, the following steps are applied

reweighting

resampling

moving to avoid particle degeneration, for example by severalruns of an MCMC kernel (largest impact on estimates, highestcost). We can make use of population of particles. ‘Standard’approach is multivariate normal random walk.

Chris Drovandi

EcoSta 2017 8 / 29



Denote the collection of particles representing target t asθi

t ,Wit Ni=1. Re-weight step:

w it+1 = W i

t f (y |θit)γt+1−γt

Effective sample size (ESS) can be estimated asESS = 1/

∑i (W

it+1)2. The sequence of γt can be selected

adaptively to maintain a particular ESS, e.g. N/2.

Boost ESS back up to N via resampling. Multinomial Re-sampling.

(Diversify Particles) Apply MCMC kernel with πt+1-invariantdistribution Rt+1 times. Determine Rt+1 adaptively by performing1 iteration on each particle and inspecting acceptance rate.

Chris Drovandi

EcoSta 2017 9 / 29


Sequential Monte Carlo - Estimating the Evidence

SMC provides convenient estimate of evidence, Z

We can write Z as:

Z =ZT

Z0=

T−1∏t=0

Zt+1

Zt,

with Z0 = 1. It is easy to show that

Zt+1/Zt ≈N∑i=1

w it+1.

Call this the “standard” SMC estimator.

Chris Drovandi

EcoSta 2017 10 / 29


Independent Proposals

We form efficient independent proposals in SMC

Independent proposals achieve uniform ergodicity rather thanstandard geometric ergodicity

Generally difficult to construct efficient independent proposalsin MCMC

In SMC, we can use population of particles

Obtain estimates of evidence with less bias and lower variancecompared with standard SMC estimator

Chris Drovandi

EcoSta 2017 11 / 29


Independent Proposals

Benefits of independent proposals in SMC

Takes advantage of the population of particles to formefficient independent proposals

Take advantage of the parallelisable nature of SMC

Re-use all information generated in the SMC process (betterestimates of posterior quantities and evidence compared withstandard SMC)

Chris Drovandi

EcoSta 2017 12 / 29


Our Independent Proposal

Our efficient independent proposals are based on a mixture(Pearson, 1894; Dempster et al., 1977) of Gaussian copulas

Copulas (Sklar, 1959) model dependence between parameterswhile also modelling their marginals separately

Requires choice of marginal distributions

Basic concept

Marginals are transformed to U [0, 1] using the marginal CDFsand then to N (0, 1).

Dependence is modelled with a multivariate Gaussian mixturemodel.

We can simulate from the resulting density and also evaluate it.

Chris Drovandi

EcoSta 2017 13 / 29


Recycling all ProposalsIn importance sampling (IS, Neal (2001)), the followingunnormalised weights are used for samples θiNi=1 drawn fromimportance distribution qφ(·)

ωi =f (y|θi )π(θi )

qφ(θi )for i = 1, . . . ,N.

Normalised weights are denoted ΩiNi=1.

ESS of the weighted set is ESS = 1/∑N

i=1(Ωi )2

Better importance distribution → higher ESS.The evidence can be obtained by averaging the unnormalisedweights

Z =1

N

N∑i=1

ωi .

Chris Drovandi

EcoSta 2017 14 / 29


Recycling all Proposals

We can use independent proposals qφt (·) for t = 0, . . . ,T asimportance distributions.

T+1 separate estimators of the evidence

Z =1

Nt

Nt∑i=1

ωit , where ωi

t =f (y|θi

t)π(θit)

qφt (θit)

A single evidence estimator

Z =1∑T

k=0 Nk

T∑t=0

Nt∑i=1

ωit

This estimator doesn’t take into account the different ESS’s fromdifferent temperatures.

Chris Drovandi

EcoSta 2017 15 / 29


Recycling all Proposals

A novel importance sampling method (similar to Nguyen et al.(2014))

Define λt = ESSt∑Tt=0 ESSt

Weights are scaled by λt

The evidence estimator is

Z =T∑t=0

λtRtN

RtN∑m=1

ωmt .

We refer to this method as CIS.

Chris Drovandi

EcoSta 2017 16 / 29


“bad environment-good environment” (BEGE) ExampleBEGE model (Bekaert et al 2015):

rt+1 = µt + ut+1,

ut+1 = σpωp,t+1 − σnωn,t+1,

ωp,t+1 ∼ Γ(pt , 1),

ωn,t+1 ∼ Γ(nt , 1),

pt = p0 + ρppt−1 +φ+p

2σ2p

u2t Iut≥0 +

φ−p2σ2

p

u2t (1− Iut≥0),

nt = n0 + ρnnt−1 +φ+n

2σ2n

u2t Iut≥0 +

φ−n2σ2

n

u2t (1− Iut≥0),

θ = (p0, n0, ρp, ρn, φ+p , φ

+n , φ

−p , φ

−n , σp, σn) with µt = 0.

Likelihood is computable but involves lots of numerical integrations(cdf), and finite differencing (pdf), quite slow for large datasets.

Chris Drovandi

EcoSta 2017 17 / 29


BEGE Example - Data

0 1000 2000 3000 4000 5000 6000 7000

Re

turn

s

-0.1

-0.05

0

0.05

0.1

0.15

S&P 500 daily returns from January 1990 to July 2016.

Chris Drovandi

EcoSta 2017 18 / 29


BEGE Results

50 independent runs with N = 1000 particles for both randomwalk (RW) and independent (IND) proposals.

RW requires 3.7 times more log-likelihood calculations thanIND - IND much more effective at diversifying particles.

ESS for usual SMC is N = 1000.

After ‘power posterior’ (Gramacy et al 2010, Nguyen et al2014) recycling ESS is 2670 on average.

After recycling all independent proposals ESS is 3450 onaverage.

Chris Drovandi

EcoSta 2017 19 / 29


BEGE Example - Evidence Results

Method

RW IND IND CISIP

Log E

vid

ence E

stim

ate

×104

2.2097

2.2098

2.2098

2.2099

2.2099

2.2100

2.21

Chris Drovandi

EcoSta 2017 20 / 29


BEGE Example - Evidence Results

Method

RW IND IND CISIP

log(VAR·TLL)

5

6

7

8

9

10

Chris Drovandi

EcoSta 2017 21 / 29


Challenging ODE Example

This challenging ODE example pushes the limits of what cancurrently be achieved with our independent proposal.

Nonlinear ODE system for modelling biochemical pathways

Used for investigating the enzymatic activation of protein Rinto its active form Rpp by enzyme S .

Chris Drovandi

EcoSta 2017 22 / 29


Challenging ODE ExampleThis four-dimensional system of coupled ODEs has been describedin Geyer (1991) and investigated from a Bayesian context inGirolami (2008):

dSdt = −k1S

dDdt = k1S

dRdt = − V1RS

Km1+R + V2RppKm2+Rpp

dRppdt = V1RS

Km1+R −V2Rpp

Km2+Rpp .

Rpp is observed for 20 time points based on parameterconfiguration by Girolami (2008)Denote observations by y and assume y(t) ∼ N (Rpp(t), σ2).We set σ = 0.02.Nine parameters (very complex posterior)

Chris Drovandi

EcoSta 2017 23 / 29


Failure of Independent Proposal - Challenging ODEExample

In this example we found that the independent proposal failed toprovide proper tail coverage.Posterior estimates:

-6 -4 -20

0.5

1

1.5

(a) log k1

-4 -2 0 20

0.5

1

(b) logV1

-10 -5 0 50

0.2

0.4

(c) logKm1

-6 -4 -2 00

0.5

1

(d) logKm2

-4 -2 00

0.5

1

1.5

(e) logV2

MCMCRWIND

Chris Drovandi

EcoSta 2017 24 / 29


ODE Example - Evidence Results

Method

RW IND IND CISIP

Lo

g E

vid

en

ce

Estim

ate

21

21.5

22

Chris Drovandi

EcoSta 2017 25 / 29


ODE Example - Evidence Results

Method

RW IND IND CISIP

log(VAR·TLL)

8

9

10

11

12

Chris Drovandi

EcoSta 2017 26 / 29


Independent SMC Summary

Benefits:

Efficient MCMC proposal within SMC

Reuse all proposals in estimating the posterior and evidence

Significant improvement in posterior inference compared to norecycling or accepted particle recycling, if independentproposals cover the tails of the targetPrecise estimates of the evidence via IS identities

Challenges:

Difficulty achieving tail coverage with current independentproposals

Chris Drovandi

EcoSta 2017 27 / 29


Future Work

Need for efficient independent proposals with improved tailcoverage

Defensive mixture distributionsMore flexible copula models

Further variance reduction through randomised quasi-MonteCarlo sample from independent proposal.

Chris Drovandi

EcoSta 2017 28 / 29


Key References

Chopin, N. (2002). A sequential particle filter method for static models.Biometrika, 89(3):539-552.

Nguyen, T. L. T., Septier, F., Peters, G. W., and Delignon, Y. (2014).Improving SMC sampler estimate by recycling all past simulated particles. In2014 IEEE Workshop. In Statistical Signal Processing (SSP), pages p117-120.

Del Moral, P., Doucet, A., and Jasra, A. (2006). Sequential Monte Carlosamplers. Journal of the Royal Statistical Society: Series B (StatisticalMethodology), 68:411-436.

Bekaert, G., Engstrom, E., and Ermolov, A. (2015). Bad environments, goodenvironments: A non-Gaussian asymmetric volatility model. Journal ofEconometrics, 186:258-275.

South, L. F., Drovandi, C. C., and Pettitt, A. N. (2017). SequentialMonte Carlo for static models with independent MCMC proposals.Revision Submitted to Statistics and Computing.https://eprints.qut.edu.au/view/person/Drovandi,_Christopher.htm

Chris Drovandi

EcoSta 2017 29 / 29

https://eprints.qut.edu.au/view/person/Drovandi,_Christopher.htm

Sequential Monte Carlo for static Bayesian models with ... · Background Independent SMC...

Documents

Transcript of Sequential Monte Carlo for static Bayesian models with ... · Background Independent SMC...