2012 mdsp pr03 kalman filter
-
Upload
nozomuhamada -
Category
Technology
-
view
138 -
download
1
Transcript of 2012 mdsp pr03 kalman filter
![Page 1: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/1.jpg)
Course Calendar Class DATE Contents
1 Sep. 26 Course information & Course overview
2 Oct. 4 Bayes Estimation
3 〃 11 Classical Bayes Estimation - Kalman Filter -
4 〃 18 Simulation-based Bayesian Methods
5 〃 25 Modern Bayesian Estimation Particle Filter
6 Nov. 1 HMM(Hidden Markov Model)
Nov. 8 No Class
7 〃 15 Supervised Learning
8 〃 29 Bayesian Decision
9 Dec. 6 PCA(Principal Component Analysis)
10 〃 13 ICA(Independent Component Analysis)
11 〃 20 Applications of PCA and ICA
12 〃 27 Clustering, k-means et al.
13 Jan. 17 Other Topics 1 Kernel machine.
14 〃 22(Tue) Other Topics 2
![Page 2: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/2.jpg)
Lecture Plan
Classical Bayesian Estimation Kalman Filter
1. Introduction
What is Kalman filter
Dynamical system representation
2. Discrete-time Gauss-Markov Model
Models, State and covariance transition
3. Bayesian Approach to Kalman Filter
State estimation problem, Deviation
![Page 3: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/3.jpg)
1. Introduction
Kalman filter is an optimal recursive data processing algorithm
It processes available measurement data to estimate the current value
of the variables of interest, by using
1) dynamics of system and measurement models,
2) statistical knowledge of system and measurement noises,
3) available information about initial conditions of state variables
Recursive: Not require all previous data by keeping in storage,
repeatedly process every time a new measurement is acquired
Real-time processing
3
![Page 4: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/4.jpg)
system
measurement device
System error noise : w(t)
Measurements
y(t) Optimal estimate
of state variables
Measurement
error noise : v(t)
unknown state
variables : x(t)
Deterministic
Input : u(t)
(Control)
Figure 1 Role of Kalman filter
Kalman filter
4
ˆ tx
![Page 5: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/5.jpg)
5
1 2 3 4
T
x t x t x t x t x t
Linearized continuous-time state-space model for motion in the z-direction
dispacement angle
( ) z t z t t t
M : Mass of carriage, m : mass of pendulum l : length of pendulum
g : gravitational constant
1 1
23 43232 2 2
3 3
2 4
434 4 4
1
2
3
4
0 1 0 0 0,
0 0 0
0 0 0 1 0 1 1,
0 0 0
1 0 0 0 ( )
x x M m gmga a
ax x bd M Mlu tx xdt
b bax x b M Ml
x
xy t z t
x
x
2)Dynamical state-space representation
3)Measurement equation
1)State space variables (Vector)
![Page 6: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/6.jpg)
Continuous-time state space model Deterministic linear time-invariant state space model is given by
In time-varing systems, all elements of matrices , , are function
(State Dynamic Model)
(Measurement Model)
where : n-vector, : m-vector, : -vector,
A B C
x tAx t Bu t
dt
y t Cx t
x u y l
of .t
6
Discrete-time state space model
1 1 (State Dynamic Model)
(Measurement Model)
where is integers (discrete-time index)
x t Ax t Bu t
y t Cx t
t
![Page 7: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/7.jpg)
2.1 Random signals(noises) are applied to a discrete-time state-space
system with random initial conditions. The input u(t) is deterministic
and the system noise w(t) to be zero-mean, white*1), random Gaussian
First-order Markov Process: State at t depends only on the previous
state at t-1. (State-space representation)
:mean value of 0
1 1 1
where 1 0, 1 and
0 0 0 , 0 0
,
where . :
ww
T
xx
x x
x t Ax t Bu t Ww t
w t R t
x Pr x x P
x t Pr x t x t P t
P t R t Cov x t E x t x t
2. Discrete-time Gauss-Markov Models
*1 white signal :=signal value is not correlated in time, thus this implies signal has equal
power at all frequencies. 7
![Page 8: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/8.jpg)
:
where, 0, vv
y t Cx t v t
v N R t
Measurement Model
The model is shown as the following block diagram.
u t
w t
1x t x t
v t
y tB
A
C
W
1z I
1 : delay operatorz
Figure 3 System diagram 8
![Page 9: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/9.jpg)
9
1
/2 1/2
1
1 1 1, exp
22
is D-dimensional random vector
:
: :
: Determinant of
Linear Transformation
, ,
T
xx xxD
xx
T
D
xx
xx xx
T
yy xx
Pr x x R x x R x xR
x x x
x E x
R Cov x E x x x x
R R
y Ax
Pr y y R A x AR A
Appendix: Multivariable Gaussian Density Distribution
![Page 10: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/10.jpg)
Since the Gauss-Markov model is characterized by a Gaussian
distribution, it is completely specified statistically by its mean and
variance.
1 1x t Ax t Bu t
y t Cx t
Mean
(State Variance)
1 1
(Measurement)
T T
ww
T
yy vv
P t AP t A WR t W
R t CP t C R t
Variance
2.2 Mean and Variance Transitions
10
![Page 11: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/11.jpg)
3. Bayesian Approach to Kalman filter [Problem]
/State-space system model
/Gauss-Markov model of the state transition mechanism.
/Bayesian optimal estimation (MAP solution)
Extracting the unobserved or hidden dynamic state variables x(t)
from noisy measurement data {y(τ): τ=0,1, …, t}.
We see that the dynamic state variable x(t) at time t is obtained
through the transition probability based on the previous state
(Markovian property), and the knowledge of the underlying
conditional probability.
11
In a random process the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property. A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a Markov process. Quote from Wikipedia
![Page 12: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/12.jpg)
Bayesian Approach – Prediction + Correction Scheme-
( )
1
1
: : 0 , 1 , , 1 ,
Bayesian Estimate:
ˆ arg
+ approach for computing
0 , 1 , , 1 ,
t
tx t
t
t
t
Data Y y y y t y t
x t Max Pr x t Y
Pr x t Y
y y y t y t
Y
Pr y t x tPr x t Y
Pr y t
Prediction Correction
1
t
t
Pr x t YY
12
Prediction Phase
Correction Phase
![Page 13: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/13.jpg)
13
1
1
(1)
, ( ) ( 1), 1 ? , ?
ˆ 1 ,
t
t
t
vv
ee
Pr y t x t Pr x t YPr x t Y
Pr y t Y
N Cx t R t N x t t P t t
N y t t R t
1
1
From the Gauss-Markov model, we have
,
ˆ 1 , 1
ˆ 1 ,
vv
t
t ee
Pr y t x t C t x t R t
Pr x t Y x t t P t t
Pr y t Y y t t R t
![Page 14: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/14.jpg)
1ˆ ˆPrediction: 1 1 1
ˆPrediction error: 1 1
1 1 1
tx t t E x t Y Ax t t
x t t x t x t t
Ax t t Ww t
1 1 1
1 1 1
T
T
ww
P t t E x t t x t t
AP t t A R t
Using the process model, we have (ignore the input term Bu(t-1))
State prediction error covariance
14
![Page 15: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/15.jpg)
1
1
1
Substituting these into Eq (1)gives
1exp
2
1ˆ ˆexp 1 1 1
2
1ˆ ˆexp 1 1
2
T
vv
T
T
ee
p x t Y t y t Cx t R y t Cx t
x t x t t P t t x t x t t
y t y t t R y t y t t
Rewrite the terms above as follows:
:
ˆ 1 1
ˆ : 1
ˆ 1
measurement noise v t y t Cx t
state estimation error x t t x t x t t
innovation e t y t y t t
y t Cx t t
15
![Page 16: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/16.jpg)
Optimal Bayesian estimator is the processor that maximizes the
posterior above. Taking natural logarithms of both sides of Eq.(2),
1
1 1
1
ln Pr ( ) ln Pr ln Pr
1 11 1 1
2 2
1R
2
t t t
T T
vv
T
ee
ln x t Y y t x t x t Y x t Y
LnK v t R t v t x t t p t t x t t
e t e t
The MAP estimate is obtained by differentiating eq.(3), setting it to
zero, and solving the equation. That is,
1
1
1
1exp
2
1exp 1 1 1
2
1exp (2)
2
T
vv
T
T
ee
p x t Y t v t R v t
x t t P t t x t t
e t R e t
16
(3)
![Page 17: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/17.jpg)
ˆ
1 1
ln Pr 0
ln Pr
1 1 0
MAP
x tx X
x t
T
vv
x t Y
x t Y
C R t y t Cx t P t t x t t
11 1
1 1
11 1
1 1
1
ˆ1 1
then,
ˆˆ 1
ˆ 1 1 (4)
T
vv
T
vv
T
MAP vv
T
vv
C R t C P t t x t
P t t x t t C R t y t
x t t X t C R t C P t t
P t t x t t C R t y t
This representation can be simplified by using a matrix inversion
lemma
1 1
1 1 1 1T T T
Matrix Inversion Lemma
A BD A A B I D A B D A
17
![Page 18: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/18.jpg)
11 1
1 1
1 1
1 1
It is possible to show 1
thus, 1
1
Substituting above into eq. (4),
ˆˆ ˆ1 1
ˆ =
T
vv
T
vv
T
vv
T
map vv
P t t C R t C P t t
P t t P t t C R t C I
P t t P t t I P t t C R t C
x t t X t P t t P t t x t t C R t y t
x t
1 1
1
ˆ1 1
ˆ ˆ = 1 1
ˆ = 1
where
is called .
T T
vv vv
T
vv
t P t t C R t Cx t t P t t C R t y t
x t t K t y t Cx t t
x t t K t e t
K t P t t C R t
Kalman gain
18
We can derive the following covariance-update relation.
1P t t I K t C P t t
![Page 19: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/19.jpg)
19
ˆ 0 0 , 0 0
ˆ ˆ1 1 1 1
1 1 1 1
ˆ ˆ1 1
1
ww
T
ee vv
x P
x t t Ax t t Bu t
P t t AP t t A R t
e t y t y t t y t Cx t t
R t CP t t C R
Initial Conditions :
Prediction :
Innovation :
Kalman Filter Algorithm
11
ˆ ˆ 1
1
T
ee
t
K t P t t C R t
x t t x t t K t e t
P t t I K t C P t t
Gain :
Update :
![Page 20: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/20.jpg)
1t
t y t
1y t
ˆ 1 1 , 1 1x t t p t t
ˆ 1 ,
1
x t t
p t t
ˆ ,x t t p t t
Measurement
Measurement
Correction process
Correction process
Prediction by Model
Prediction by Model
Figure 4
![Page 21: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/21.jpg)
Understanding Kalman filter operation and its Gain
( :Kalman gain)new old newˆ ˆX X KE K
Prediction: Use state space model
Correction: Use measurement
is small
is large
smallReliable model is small
large
largeReliable measurement is large
small
T
vv
new old
new new
vv
vv
K t P t t C R t
ˆ ˆK X X
ˆK X KE
P t tK
R t
P t tK
R t
1
![Page 22: 2012 mdsp pr03 kalman filter](https://reader034.fdocuments.us/reader034/viewer/2022052211/559752231a28abf65b8b45d8/html5/thumbnails/22.jpg)
22
References:
[1] J. Candy, “Model-based Signal Processing”, John Wiley/IEEE Press, 2006
[2] J. Candy, “ Bayesian Signal Processing Classical, Modern, and Particle Filtering Methods”,
John Wiley/IEEE Press, 2009
[3] S. Maybeck, “Stochastic Models, Estimation, and Control”, Vol. 1, Academic Press, 1979,
1999
[4] R. Brown, “Introduction to random signal analysis and Kalman filtering”, John Weily, 1983