Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May...
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May...
![Page 1: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/1.jpg)
Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters
Prasanth JeevanMary Knox
May 12, 2006
![Page 2: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/2.jpg)
Background
• Optimal linear filters Wiener Stationary Kalman Gaussian Posterior, p(x|y)
• Filters for nonlinear systems Extended Kalman Particle
![Page 3: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/3.jpg)
Extended Kalman Filter (EKF)
• Locally linearize the non-linear functions
• Assume p(xk|y1,…,k) is Gaussian
![Page 4: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/4.jpg)
Particle Filter (PF)
• Weighted point mass or “particle” representation of possibly intractable posterior probability density functions, p(x|y)
• Estimates recursively in time allowing for online calculations
• Attempts to place particles in important regions of the posterior pdf
• O(N) complexity on number of particles
![Page 5: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/5.jpg)
Particle Filter Background [Ristic et. al. 2004]
• Monte Carlo Estimation
• Pick N>>1 “particles” with distribution p(x)Assumption: xi is independent
![Page 6: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/6.jpg)
Importance Sampling
• Cannot sample directly from p(x)• Instead sample from known importance
density, q(x), where:
• Estimate I from samples and importance weights
where
![Page 7: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/7.jpg)
Sequential Importance Sampling (SIS)
• Iteratively represent posterior density function by random samples with associated weightsAssumptions: xk Hidden Markov process, yk conditionally independent given xk
![Page 8: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/8.jpg)
Degeneracy
• Variance of sample weights increases with time if importance density not optimal [Doucet 2000]
• In a few cycles all but one particle will have negligible weights PF will updating particles that contribute little in
approximating the posterior
• Neff, estimate of effective sample size [Kong et. al. 1994]:
![Page 9: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/9.jpg)
Optimal Importance Density [Doucet et. al. 2000]
• Minimizes variance of importance weights to prevent degeneracy
• Rarely possible to obtain, instead often use
![Page 10: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/10.jpg)
Resampling
• Generate new set of samples from:
• Weights are equal after i.i.d. sampling
• O(N) complexity• Coupled with SIS,
these are the two key components of a PF
![Page 11: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/11.jpg)
Sample Impoverishment
• Set of particles with low diversity Particles with high
weights are selected more often
![Page 12: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/12.jpg)
Sampling Importance Resampling (SIR)
[Gordon et. al. 1993]
• Importance density is the transitional prior
• Resampling at every time step
![Page 13: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/13.jpg)
SIR Pros and Cons
• Pro: importance density and weight updates are easy to evaluate
• Con: Observations not used when transitioning state to next time step
![Page 14: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/14.jpg)
A Cycle of SIR
![Page 15: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/15.jpg)
Auxiliary SIR - Motivation[Pitt and Shephard 1999]
• Want to use observation when exploring the state space ( ’s) To have particles in regions
of high likelihood
• Incorporate into resampling at time k-1 Looking one step ahead to
choose particles
![Page 16: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/16.jpg)
ASIR - from SIR
• From SIR we had
• If we move the likelihood inside we get:
• We don’t have though
• Use , a characterization of given such as
![Page 17: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/17.jpg)
ASIR continued
• So then we get:
• And the new importance weight becomes:
![Page 18: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/18.jpg)
ASIR Pros & Cons
• Pro Can be less sensitive to peaked likelihoods and outliers by using observation Outliers - Model-improbable states that can result in a
dramatic loss of high-weight particles
• Cons Added computation per cycle If is a bad characterization of (ie.
large process noise), then resampling suffers, and performance can degrade
![Page 19: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/19.jpg)
Simulation Linear
• System Equations:
where v ~ N(0,6) and w ~ N(0,5)
![Page 20: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/20.jpg)
Simulation Linear10 Samples
![Page 21: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/21.jpg)
Simulation Linear50 Samples
![Page 22: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/22.jpg)
Simulation Linear
Table 1: Mean Squared Error Per Time Step
Number of Particles
Filter 10 50 100 1000
KF 0.0349 0.0351 0.0350 0.0352
ASIR 0.7792 0.0886 0.0417 0.0350
SIR 0.9053 0.0977 0.0496 0.0354
![Page 23: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/23.jpg)
Simulation Nonlinear
• System Equations:
where v ~ N(0,6) and w ~ N(0,5)
![Page 24: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/24.jpg)
Simulation Nonlinear10 Samples
![Page 25: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/25.jpg)
Simulation Nonlinear50 Samples
![Page 26: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/26.jpg)
Simulation Nonlinear100 Samples
![Page 27: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/27.jpg)
Simulation Nonlinear1000 Samples
![Page 28: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/28.jpg)
Simulation Nonlinear
Table 2: Mean Squared Error Per Time Step
Number of Particles
Filter 10 50 100 1000
EKF 812.08 826.20 827.94 838.75
ASIR 30.14 20.15 18.81 17.86
SIR 37.97 22.62 21.49 19.78
![Page 29: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/29.jpg)
Conclusion
• PF approaches KF optimal estimates as N
• PF better than EKF for nonlinear systems• ASIR generates ‘better particles’ in certain
conditions by incorporating the observation
• PF is applicable to a broad class of system dynamics Simulation approaches have their own
limitations Degeneracy and sample impoverishment
![Page 30: Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d4d5503460f94a2b589/html5/thumbnails/30.jpg)
Conclusion (2)
• Particle filters composed of SIS and resampling Many variations to improve efficiency
(both computationally and for getting ‘better’ particles)
• Other PFs: Regularized PF, (EKF/UKF)+PF, etc.