Adaptive Rao-Blackwellized Adaptive Rao-Blackwellized Particle Filter and It’s EvaluParticle Filter and It’s Evaluation for Tracking in Surveillation for Tracking in Surveill

anceance

Xinyu Xu and Baoxin Li, Senior Member, IEEEXinyu Xu and Baoxin Li, Senior Member, IEEE

AbstractAbstract

In this paper, by proposing an adaptive In this paper, by proposing an adaptive RRao-ao-BBlacklackwellized wellized PParticle article FFilter (RBPF) for tracking in surveilter (RBPF) for tracking in surveillance, we show how to exploit the analytical relaillance, we show how to exploit the analytical relationship among state variables to improve the effitionship among state variables to improve the efficiency and accuracy of a regular particle filter (PF).ciency and accuracy of a regular particle filter (PF).

Introduction Introduction

Visual tracking is an important step in Visual tracking is an important step in many practical applications.many practical applications.

Generally, suppose we have an estimatorGenerally, suppose we have an estimator depending upon 2 variables R and depending upon 2 variables R and

L, the RB theorem reveals its variance L, the RB theorem reveals its variance satisfies:satisfies:

Non-negative

For the visual tracking problem, let For the visual tracking problem, let denote the state to be estimated and denote the state to be estimated and the observation, with subscript the observation, with subscript tt the time the time index.index.

The key idea of RBPF is to partition the The key idea of RBPF is to partition the original state-space into two parts and original state-space into two parts and . .

The justification for this decomposition The justification for this decomposition follows from the factorization of the follows from the factorization of the posterior probabilityposterior probability

RBPF for tracking in surveillanceRBPF for tracking in surveillance

a) Partition the state spacea) Partition the state space

In this paper ,using 8-D ellipse model to In this paper ,using 8-D ellipse model to describe the targetdescribe the target

The scale change is related to its position The scale change is related to its position alone alone yy axis, so the scale change can be es axis, so the scale change can be estimated conditional on the location comptimated conditional on the location components. The 8-D state space can separate ionents. The 8-D state space can separate into 2 groupsnto 2 groups

Root variables containing the motion information.

Leaf variables containing the scale parameters.

b) Overview of the methodb) Overview of the method In this work, root variables are propagated In this work, root variables are propagated

by a first order system motion model by a first order system motion model defined bydefined by

Conditional on the root variables, the leaf Conditional on the root variables, the leaf variables forms a linear-Gaussian variables forms a linear-Gaussian substructure specified bysubstructure specified by

transition matrixtransition matrix random noiserandom noise

A function encoding the conditional relation of L

Gaussian random noise

Since both color histogram and gradient cues Since both color histogram and gradient cues do not follow a linear-Gaussian relationship do not follow a linear-Gaussian relationship with state variable, the observation model is with state variable, the observation model is given in a general form:given in a general form:

The observationsThe observations form a linear form a linear relationship with state Lrelationship with state L

Image observation

Nonlinear function

Random noise

Gaussian random noise

Relationship between variablesRelationship between variables

The RBPF algorithmThe RBPF algorithm

Just like regular PF, RBPF represents the posJust like regular PF, RBPF represents the posterior density by a set of weighted particles:terior density by a set of weighted particles:

Each particle is represented by a tripletEach particle is represented by a triplet . .

The proposed RBPF algorithm will sample thThe proposed RBPF algorithm will sample the motion using PF, while apply Kalman filter e motion using PF, while apply Kalman filter to estimate the scale parameters andto estimate the scale parameters and conconditional on the motion state.ditional on the motion state.

(1)Propagate samples(1)Propagate samples

a) Sample the object motiona) Sample the object motion according to according to

After this step, we haveAfter this step, we haveminus sign is denotes the corresponding variable iminus sign is denotes the corresponding variable is a priori estimates a priori estimate

b) Kalman prediction for leaf states b) Kalman prediction for leaf states according toaccording to

According to the Kalman filter modelAccording to the Kalman filter model(4)and(6), we project forward the state a(4)and(6), we project forward the state and error covariance using:nd error covariance using:

After this step, we haveAfter this step, we have

Prediction for the mean of the leaf variables

Covariance for leaves

Observation prediction

a) Compute the color histogram a) Compute the color histogram for each for each sample ellipsesample ellipse Γ characterized by ellipse Γ characterized by ellipse centercenter and scale and scale

Pixels that are closer to the region center Pixels that are closer to the region center are given higher weights specified byare given higher weights specified by

(2)Evaluate weight for each particle(2)Evaluate weight for each particle

Kronecker delta function

b) Compute the gradient b) Compute the gradient for each sample for each sample ellipseellipse Γ characterized by ellipse centerΓ characterized by ellipse center

and scale and scalethe gradient of a sample ellipse is the gradient of a sample ellipse is computed as an average over gradients of computed as an average over gradients of all the pixels on the boundaryall the pixels on the boundary

where the gradient at pixelwhere the gradient at pixel is set to is set to the maximum gradient by a local search the maximum gradient by a local search along the normal linealong the normal line of the ellipse at of the ellipse at locationlocation

A simple operator is used to compute A simple operator is used to compute the gradient in the gradient in xx and and yy axis for pixel axis for pixel

finally, the gradient at point finally, the gradient at point is is computed ascomputed as

c) Compute the weightc) Compute the weight one isone is based on color histogram similarity based on color histogram similarity

between the hypothetical region and the between the hypothetical region and the target modeltarget model

pp stands for the color histogram of a stands for the color histogram of a sample hypothesis in the newly observed sample hypothesis in the newly observed image, and image, and qq represents the color represents the color histogram of target model.histogram of target model.

Another is based on gradientAnother is based on gradient

Notice that all the sample is divided by the Notice that all the sample is divided by the maximum gradient to normalize into maximum gradient to normalize into range[0,1], the final weight for each range[0,1], the final weight for each sample is given bysample is given by

(3)Select samples(3)Select samples

Resampling with replacementResampling with replacement

the latest measurements will be used to modifthe latest measurements will be used to modify the prediction PDF of not only the root variaby the prediction PDF of not only the root variables but also the leaf variables.les but also the leaf variables.

After this step,After this step,

(4)Kalman update (4)Kalman update for leaf variablesfor leaf variables

Kalman update is accomplished byKalman update is accomplished by

After this step, we haveAfter this step, we have

(5)Compute the mean state (5)Compute the mean state at time at time tt

Since resampling has been done, the mean stSince resampling has been done, the mean state can be simply computed as the average of ate can be simply computed as the average of the state particlesthe state particles

(6)Compute the new noise variance(6)Compute the new noise variance We found that when velocity is small and We found that when velocity is small and

constant, we only need a small noise variance to constant, we only need a small noise variance to reach the smallest MSE, if velocity changes reach the smallest MSE, if velocity changes dramatically, we need a much larger noise dramatically, we need a much larger noise variance to reach the lowest MSE.variance to reach the lowest MSE.

The noise varianceThe noise variance is computed by is computed by

Evaluation of the RBPF algorithmEvaluation of the RBPF algorithm

Evaluate the performance between RBPF Evaluate the performance between RBPF and PF.and PF.

Real data experimentReal data experiment

DiscussionDiscussion

Failure cases:Failure cases:when camera is not mounted higher when camera is not mounted higher than the target object…than the target object…

Computation cost:Computation cost:the same level of estimation the same level of estimation accuracy, RBPF needs far fewer accuracy, RBPF needs far fewer particles than PF dose; hence, it is particles than PF dose; hence, it is more efficient than PF.more efficient than PF.

ConclusionConclusion

Comparative studies using both Comparative studies using both simulated and real data have simulated and real data have demonstrated the improved demonstrated the improved performance of the proposed RBPF performance of the proposed RBPF over regular PF.over regular PF.

Future working: to find a proper Future working: to find a proper dependency model from a large dependency model from a large number of state variables.number of state variables.