Boosted Particle Filter: Multitarget Detection and Tracking

Fayin Li

Motivation and Outline

• For a varying number of non-rigid objects, the observation models and target distribution be highly non-linear and non-Gaussian.

• The presence of a large, varying number of objects creates complex interactions with overlap and ambiguities.

• How object detection can guide the evolution of particle filters?

• Mixture particle filter• Boosted objection detection• Boosted particle filter• Observation model in this paper

Multitarget Tracking Using Mixture Approach

• Given observation and transition models, tracking can be considered as the following Bayesian recursion:

• To deal with multiple targets, the posterior is modeled as M-component non-parametric mixture approach

• Denote

ttttt

ttttttt

tt

tttttt

dxyxpxyp

dxyxpxxpxyp

yyp

yxpxypyxp

)|()|(

)|()|()|(

)|(

)|()|()|(

1:0

11:011

1:0

1:0:0

M

jttjtjtt yxpyxp

1:0,:0 )|()|(

11:0111:0 )|()|()|( tttjttttj dxyxpxxpyxp

Mixture Approach and Particle Approximation• Then the prediction step

• And the updated mixture

• where

• and

• The new filtering is again a mixture of individual component filtering. And the filtering recursion can be performed for each component individually. The normalized weights is only the part of the procedure where the components interact.

M

jttjtjtt yxpyxp

11:01,1:0 )|()|(

M

jttjtjtt yxpyxp

1:0,:0 )|()|(

tttjttj

ttjttjttj

dxyxpxyp

yxpxypyxp

)|()|(

)|()|()|(

1:0

1:0:0

M

k ttktk

ttjtj

M

k tttkttktk

tttjttjtj

tjyyp

yyp

dxyxpxyp

dxyxpxyp

1 1:01,

1:01,

1 1:01,

1:01,

,)|(

)|(

)|()|(

)|()|(

Particle Approximation

• Particles filters are popular at tracking for non-linear and/or non-Gaussian Models.

• However they are poor at consistently maintaining the multi-modality of the target distributions that may arise due to ambiguity or the presence of multiple objects.

• In standard particle filter, the distribution can be represented by N particles . During recursion, first sample particles from an proposal distribution

with weight • Resample the particles based the weights to approximate

the posterior

Ni

it

it wx 1},{

),|(~ :01:0 tttit yxxqx

),|(

)|()|(

:01:0

11

ti

tit

it

it

itti

tit yxxq

xxpxypww

)|( :0 tt yxp

Particle Approximation

• Because each component can be considered individually in mixture approach, the particles and weights can be updated for each component individually.

• The posterior distribution is approximated by

• And the particle weight updated rule is

• And the mixture weights can be updated using particle weights

M

j Iitx

ittjtt

j

it

xwyxp1

,:1 )()|(

),|(

)|()|(~,~

~

1

11

tittj

it

itj

ittj

iti

t

Ij

jt

iti

t yxxq

xxpxypww

w

ww

m

jj Ii

ittjtjM

n tntn

tjtjtj

Ii

itttj ww

w

wwyyp ~,,~)|( 1,,

1 ,1,

,1,,1

Example

• A simple example governed by the equations

),|()|( 211 xtttt xxNxxp

),|()|( 22ytttt xyNxyp

Mixture Computation and Variation

• The number of modes is rarely known ahead and is unlikely to remain fixed.

• It may fluctuate as ambiguities arise and are resolved, or objects appear and disappear.

• It is necessary to recompute the mixture representation• Based on the particles and weights, we can use k-means

to cluster the sample set and update the number of modes, particles weights, and mixture weights.

• In stead of M modes, we can use M different likelihood distributions. When one or more new objects appear, they are detected and initialized with an observation model. Different observation model (data association) allow us track objects.

AdaBoost

• Given a set of weak classifiers

– None much better than random• Iteratively combine classifiers

– Form a linear combination

– Training error converges to 0 quickly– Test error is related to training margin

}1,1{)( :originally xjh

rated" confidence" },{)( also xjh

t

t bxhxC )()(

Adaboost Algorithm(Freund & Shapire) Weak

Classifier 1

WeightsIncreased

Weak classifier 3

Final classifier is linear combination of weak classifiers

W

Wtt log

2

1,

t

xhyt

t Z

eiDiD

iti )(

1

)()(

t

i

xhyt

ht

Z

eiDh ii )()(min

Weak Classifier 2

A variant of AdaBoost for aggressive feature selection G iven exam ple im ages (x1 ,y1) , … , (xn ,yn) w here y i = 0, 1 for negative and positive

exam ples respectively. In itialize w eights w 1 ,i = 1 /(2m ), 1 /(2 l) for train ing exam ple i, w here m and l are the

num ber of negatives and positives respectively. For t = 1 … T

1) N orm alize w eights so that w t is a d istribution 2) For each feature j train a classifier h j and evaluate its error j w ith respect to w t. 3) C hose the classifier h j w ith low est error. 4) U pdate w eights according to :

1,,1

i

titit ww

w here e i = 0 is x i is classified correctly, 1 o therw ise, and

1 t

t

t

T he final strong classifier is:

otherwise

xxhT

t

T

t ttt h0

2

1)(1)( 1 1 , w here )

1log(

t

t

Cascading Classifiers for Object Detection

• Given a nested set of classifier hypothesis classes

• Computational Risk Minimization. Each classifier has 100% detection rate and the cascading reduces the false positive rate

vs false neg determined by

% False Pos

% D

etec

tion

0 50

50

100

ObjectIMAGESUB-WINDOW

Classifier 1

F

T

NON-Object

Classifier 3T

F

NON-Object

F

T

NON-

Classifier 2T

F

NON-Object

Boosted Particle Filter

• Cascading Adaboost algorithm gets high detection rate but large number of false positives, which could be reduced by considering the motions of the objects (players).

• As with many particle filters, the algorithm simply proceeds by sampling from the transition prior without using the data information.

• Boosted Particle Filter uses the following mixture distribution as the proposal distribution for sampling

• Here qada is a Gaussian distribution and can be set dynamically with affecting the convergence of the particle filter. If there is overlap between a component of mixture particle filters and the nearest cluster detected by Adaboost, use the mixture proposal distribution, otherwise set = 0

)|( 1tt xxp

)|()1(),|(),|( 11:11*

tttttadatttB xxpyxxqyxxq

Observation Model • Hue-Saturation-Value (HSV) histogram is used to

represent the region containing the object. It has N = NhNs + Nv bins.

• Then a kernel density estimation of the color distribution at time t is given:

• Bhattacharyya coefficient is applied to measure the distance between two color histograms

• And the likelihood function is• If the object is represented by multiple regions, the

likelihood function will be

])([);( ndbxnk tt

2

1

10

** );();(1)](,[

N

ntt xnkxnkxKK

)](,[ *2

)|( txKKtt exyp

j tjj xKK

tt exyp)](,[ *2

)|(

Experiments and Conclusion

• Boosted particle filter works well no matter how many objects and adapts successfully to the changes (players come in and out).

• Adaboost detects the new players and BPF assigns the particles to them.

• Mixture components are well maintained even Adaboost fails.

• Object detection and dynamics are combined by forming the proposal distribution for the particle filter: the detections in current frame and the dynamic prediction from the previous time step.

• It incorporates the recent observations, which improves the robustness of the dynamics

• The detection algorithm gives a powerful tool to obtain and maintain the mixture representation.

Tracking Results

• Video 1 and Video 2