Post on 01-Feb-2016
description
NCAF January Meeting, Aston University, Birmingham.
Tutorial on Particle filters
Keith Copsey
Pattern and Information Processing Group
DERA Malvern
K.Copsey@signal.dera.gov.uk
NCAF January Meeting, Aston University, Birmingham.
Outline
Introduction to particle filters
– Recursive Bayesian estimation Bayesian Importance sampling
– Sequential Importance sampling (SIS)
– Sampling Importance resampling (SIR) Improvements to SIR
– On-line Markov chain Monte Carlo Basic Particle Filter algorithm Examples Conclusions Demonstration
NCAF January Meeting, Aston University, Birmingham.
Particle Filters
Sequential Monte Carlo methods for on-line learning within a Bayesian framework.
Known as
– Particle filters
– Sequential sampling-importance resampling (SIR)
– Bootstrap filters
– Condensation trackers
– Interacting particle approximations
– Survival of the fittest
NCAF January Meeting, Aston University, Birmingham.
Recursive Bayesian estimation (I)
Recursive filter:
– System model:
– Measurement model:
– Information available:
)|( ),( 11 kkkkkk xxpxfx
)|( ),( kkkkkk xypxhy
),,( 1 kk yyD
)( 0xp
NCAF January Meeting, Aston University, Birmingham.
Seek:
– i = 0: filtering.
– i > 0: prediction.
– i<0: smoothing.
Prediction:
– since:
)|( kik Dxp
1111 )|,()|( kkkkkk dxDxxpDxp
11111 )|()|()|( kkkkkkk dxDxpxxpDxp
Recursive Bayesian estimation (II)
NCAF January Meeting, Aston University, Birmingham.
Update:
where:
– since:
kkkkkk dxDxypDyp )|,()|( 11
kkkkkkk dxDxpxypDyp )|()|()|( 11
)|(
)|()|()|(
1
1
kk
kkkkkk Dyp
DxpxypDxp
Recursive Bayesian estimation (III)
NCAF January Meeting, Aston University, Birmingham.
Classical approximations
Analytical methods:
– Extended Kalman filter,
– Gaussian sums… (Alspach et al. 1971)
• Perform poorly in numerous cases of interest
Numerical methods:
– point masses approximations,
– splines. (Bucy 1971, de Figueiro 1974…)
• Very complex to implement, not flexible.
NCAF January Meeting, Aston University, Birmingham.
Perfect Monte Carlo simulation (I)
Introduce the notation
Represent posterior distribution using a set of samples or particles.
Random samples are drawn from the posterior distribution.
),,( 0:0 kk xxx
N
ikxkk dx
NDxp i
k1:0:0 )(
1)|(
:0
ikx :0
NCAF January Meeting, Aston University, Birmingham.
Easy to approximate expectations of the form:
– by:
kkkkk dxDxpxgxgE :0:0:0:0 )|()())((
N
i
ikk xg
NxgE
1:0:0 )(
1))((
Perfect Monte Carlo simulation (II)
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (I)
Take p(x)=Gamma(4,1) Generate some random samples Plot histogram and basic approximation to pdf
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0 20 40 60 80 100 120 140 160 180 2000
2
4
6
8
10
12
200 samples
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (II)
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
0.3
0.35
500 samples 1000 samples
NCAF January Meeting, Aston University, Birmingham.
Random samples and the pdf (III)
0 5 10 15 20 250
0.05
0.1
0.15
0.2
0.25
0 5 10 15 20 250
0.05
0.1
0.15
0.2
0.25
200000 samples5000 samples
NCAF January Meeting, Aston University, Birmingham.
Unfortunately it is often not possible to sample directly from the posterior distribution.
Circumvent by drawing from a known easy to sample proposal distribution giving:
Bayesian Importance Sampling (I)
)|( :0 kk Dxq
kkkk
kkk
kkkkkk
kkkk
kkkkk
kkkk
dxDxqDp
xwxg
dxDxqDxqDp
xpxDpxg
dxDxqDxq
DxpxgxgE
:0:0:0
:0
:0:0:0
:0:0:0
:0:0:0
:0:0:0
)|()(
)()(
)|()|()(
)()|()(
)|()|(
)|()())((
NCAF January Meeting, Aston University, Birmingham.
Bayesian Importance Sampling (II)
where are unnormalised importance weights:
Now:
)( :0 kk xw
)|(
)()|()(
:0
:0:0:0
kk
kkkkk Dxq
xpxDpxw
kkkkk
kkk
kkkkk
kkkk
dxDxqxw
dxDxq
DxqxpxDpdxxDpDp
:0:0:0
:0:0
:0:0:0
:0:0
)|()(
)|(
)|()()|( ),()(
NCAF January Meeting, Aston University, Birmingham.
Bayesian Importance Sampling (III)
Giving:
so that:
where are normalised importance weights
– and are independent random samples from
N
i
ikk
ikN
i
ikk
N
i
ikk
ik
k xwxg
xwN
xwxgN
xgE1
:0:0
1:0
1:0:0
:0 )(~)(
)(1
)()(1
))((
)(~~:0ikk
ik xww
kkkkk
kkkkkkk dxDxqxw
dxDxqxwxgxgE
:0:0:0
:0:0:0:0:0 )|()(
)|())()(())((
ikx :0 )|( :0 kk Dxq
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (I)
Factorising the proposal distribution:
and remembering that the state evolution is modelled as a Markov process
obtain a recursive estimate of the importance weights:
k
jjjjkk DxxqxqDxq
11:00:0 ),|()()|(
),|(
)|()|(
:0
11
kkk
kkkkkk Dxxq
xxpxypww
NCAF January Meeting, Aston University, Birmingham.
Derivation of SIR weights
Since:
We have:
k
jjjk xxpxpxp
110:0 )|()()( and
k
jjjkk xypxDp
1:0 )|()|(
),|(
)|()|(),|(
1
)(
)(
)|(
)|()|(),|(
)()|(
1:0
11
1:01:0
:0
1:01
:01
11:01:0
:0:0
kkk
kkkkk
kkkk
k
kk
kkk
kkkkk
kkkk
Dxxq
xxpxypw
Dxxqxp
xp
xDp
xDpw
DxqDxxq
xpxDpw
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (II)
Choice of the proposal distribution:
Choose proposal function to minimise variance of (Doucet et al. 1999):
Although Common choice is the prior distribution:
),|( 1:0 kkk Dxxq
kw
),|(),|( 1:01:0 kkkkkk DxxpDxxq
)|(),|( 11:0 kkkkk xxpDxxq
NCAF January Meeting, Aston University, Birmingham.
Illustration of SIS:
Degeneracy problems:
– variance of importance ratios
increases stochastically over time (Kong et al. 1994; Doucet
et al. 1999).
Sequential Importance Sampling (III)
w
Time 19
w
Time 10
w
Time 1
)|(/)|( :0:0 kkkk DxqDxp
NCAF January Meeting, Aston University, Birmingham.
SIS - why variance increase is bad
Suppose we want to sample from the posterior
– choose a proposal density to be very close to the posterior
density
• Then
• and
So we expect the variance to be close to 0 to obtain reasonable estimates
– thus a variance increase has a harmful effect on accuracy
1)|(
)|(
:0
:0
kk
kkq Dxq
DxpE
01)|(
)|(
)|(
)|(var
2
:0
:0
:0
:0
kk
kkq
kk
kkq Dxq
DxpE
Dxq
Dxp
NCAF January Meeting, Aston University, Birmingham.
Sequential Importance Sampling (IV)
Illustration of degeneracy:
w
Time 19
w
Time 10
w
Time 1
NCAF January Meeting, Aston University, Birmingham.
Sampling-Importance Resampling
SIS suffers from degeneracy problems so we don’t want to do that!
Introduce a selection (resampling) step to eliminate samples with low importance ratios and multiply samples with high importance ratios.
Resampling maps the weighted random measure on to the equally weighted random measure
– by sampling uniformly with replacement from
with probabilities
Scheme generates children such that and satisfies:
)}(~,{ :0:0ikk
ik xwx} { 1-
:0 Nx j k},,1;{ :0 Nixi k
},,1;~{ Niwik
NNN
ii
1iN
iki wNNE ~)(
)~1(~)var( ik
iki wwNN
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (I)
Variety of resampling schemes with varying performance in terms of the variance of the particles :
– Residual sampling (Liu & Chen, 1998).
– Systematic sampling (Carpenter et al., 1999).
– Mixture of SIS and SIR, only resample when necessary (Liu &
Chen, 1995; Doucet et al., 1999).
Degeneracy may still be a problem:
– During resampling a sample with high importance weight may
be duplicated many times.
– Samples may eventually collapse to a single point.
)var( iN
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (II)
To alleviate numerical degeneracy problems, sample smoothing methods may be adopted.
– Roughening (Gordon et al., 1993).
• Adds an independent jitter to the resampled particles
– Prior boosting (Gordon et al., 1993).
• Increase the number of samples from the proposal distribution to M>N,
• but in the resampling stage only draw N particles.
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (III)
Local Monte Carlo methods for alleviating degeneracy:
– Local linearisation - using an EKF (Doucet, 1999; Pitt &
Shephard, 1999) or UKF (Doucet et al, 2000) to estimate the
importance distribution.
– Rejection methods (Müller, 1991; Doucet, 1999; Pitt & Shephard,
1999).
– Auxiliary particle filters (Pitt & Shephard, 1999)
– Kernel smoothing (Gordon, 1994; Hürzeler & Künsch, 1998; Liu &
West, 2000; Musso et al., 2000).
– MCMC methods (Müller, 1992; Gordon & Whitby, 1995; Berzuini et
al., 1997; Gilks & Berzuini, 1998; Andrieu et al., 1999).
NCAF January Meeting, Aston University, Birmingham.
Improvements to SIR (IV)
Illustration of SIR with sample smoothing:
w
Time 19
w
Time 10
w
Time 1
NCAF January Meeting, Aston University, Birmingham.
MCMC move step Improve results by introducing MCMC steps with invariant
distribution .
– By applying a Markov transition kernel, the total variation of
the current distribution w.r.t. the invariant distribution can only
decrease.
Introduces possibility of variable dimension state space through the use of reversible jump MCMC (de Freitas et al., 1999; Gilks & Berzuini, 2001)
)|( :0 kk Dxp
NCAF January Meeting, Aston University, Birmingham.
Ingredients for SMC
Importance sampling function
– Gordon et al
– Optimal
– UKF pdf from UKF at Redistribution scheme
– Gordon et al SIR
– Liu & Chen Residual
– Carpenter et al Systematic
– Liu & Chen, Doucet et al Resample when necessary
Careful initialisation procedure (for efficiency)
)|( 1ikk xxp
),|( 1:0 kikk Dxxp
ikx 1
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter - Schematic
Initialisation
Importancesampling step
Resamplingstep
0k
1 kk
)}(~,{ :0:0ikk
ik xwx
},{ 1:0
Nxi k
measurement
ky
Extract estimate, kx :0ˆ
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (I)
Initialisation
–
– For sample
– and set
In practice, to avoid having to take too many samples, for the first step we may want to ensure that we have a reasonable number of particles in the region of high likelihood
– perhaps use MCMC techniques
0k
Ni ,,1 )(~ 00 xpxi
1k
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (II)
Importance Sampling step
– For sample
– For evaluate the importance weights
– Normalise the importance weights,
N
j
jk
ik
ik www
1
/~
Ni ,,1 )|(~~1
ikk
ik xxpx
),(~1:0:0
ik
ik
ik xxx and set
Ni ,,1
)~|( ikk
ik xypw
NCAF January Meeting, Aston University, Birmingham.
Basic Particle Filter algorithm (III)
Resampling step
– Resample with replacement particles:
– from the set:
– according to the normalised importance weights,
Set
– proceed to the Importance Sampling step, as the next
measurement arrives.
N
),,1;( :0 Nixi k
),,1;~( :0 Nix i k ikw
~
1 kk
NCAF January Meeting, Aston University, Birmingham.
Example
On-line Data Fusion (Marrs, 2000).
NCAF January Meeting, Aston University, Birmingham.
Example - Sensor Deployment
Aim to reduce target sd below some threshold...
… and keep it there
… by placing the minimum number of sensors possible
Sensor positions chosen according to particle distribution.
NCAF January Meeting, Aston University, Birmingham.
Example - In-situ monitoring of growing semiconductor crystal composition
Si1-xGex
substrate
NCAF January Meeting, Aston University, Birmingham.
Book Advert (or put this in or your fired)
Sequential Monte Carlo methods in practice, Editors: Doucet, de Freitas, Gordon, Springer-Verlag (2001).
– Theorectical foundations - plus convergence proofs
– Efficiency measures
– Applications:
• Target tracking; missile guidance; image tracking; terrain referenced navigation; exchange rate prediction; portfolio allocation; ellipsometry; electricity load forecasting; pollution monitoring; population biology; communications and audio engineering.
ISBN=0-387-95146-6, Price=$79.95.
NCAF January Meeting, Aston University, Birmingham.
Conclusions
On-line Bayesian learning a realistic proposition for many applications.
Appropriate for complex non-linear/non-Gaussian models
– don’t bother if KF based solution adequate. Representation of full posterior pdf leading to
– estimation of moments.
– estimation of HPD regions.
– multi-modality easy to deal with. Model order can be included in unknowns. Can mix SMC and KF based solutions
NCAF January Meeting, Aston University, Birmingham.
Tracking Demo
Illustrate a running particle filter
– compare with Kalman Filter
Running as we watch - not pre-recorded
Pre-defined scenarios, or design your own
– available to play with at coffee and lunch breaks.
Tracking Demo
NCAF January Meeting, Aston University, Birmingham.
2nd Book Advert
Statistical Pattern Recognition Andrew Webb, DERA ISBN 0340741643, Paperback: 1999: £29.99 Butterworth Heinemann
Contents:
– Introduction to SPR, Estimation, Density estimation, Linear
discriminant analysis, Nonlinear discriminant analysis - neural
networks, Nonlinear discriminant analysis - statistical methods,
Classification trees, Feature selction and extraction, Clustering,
Additional topics, Measures of dissimilarity, Parameter estimation,
Linear algebra, Data, Probability theory.