Oliver Williams
University of Cambridge
Andrew Blake
Microsoft Research,
Cambridge
Roberto Cipolla
University of Cambridge
A Sparse Probabilistic Learning Algorithm
for Real-Time Tracking
exploit temporal
coherence
object detector Self starting
Self recovering
Efficient
Robust Tracking
Background: Support Vector Tracking
Avidan [CVPR 2001]
SVM on object class
Treat score as function of
position
-100 -80 -60 -40 -20 0 20 40 60 80 100-3
-2
-1
0
1
2
3
Translation (pixels)S
VM
Sco
re
subimage
image gradient
Learn right versus left
Training for displacement
positive example
negative example
SVM
Translation (pixels)
SVM
Sco
re
-u
+u
Calibrate state update from SVM score
Relevance Vector
Machine
Probabilistic
Statistical filter
Want sparsity
Computational efficiency
Temporal Fusion
Tipping [NIPS 2000]
SVM in Bayesian setting
Learns continuous function from training set {zi,ti}
RVM Evaluation Equation
weights
kernel function
test subimagetraining subimage
Setting some wi to zero sparse solution
Bayesian Training
Sparsity encouraged by zero-mean prior :
Training data modelled by Gaussian likelihood function
hyperparameter
provided target
“true” underlying value noise parameter
-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5
Occam’s Razor
Sometimes, lower Occam
penalty paid by removing an
example…
…and explaining data with
more noise
-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5
Relevance Vector Machine
Posterior
Train RVM by finding hyperparameters
Maximise marginal likelihood
“Prune” vector i when i ! 0
Creating a Training Set
δX1
Select a few “seed” stills
Simulate translation, scaling and rotation
labelled training set
δX3
X2
RVM Tracking
X1 X2 X3 XN
4D RVM learns mapping
{zi}
{ti}
Input Space (images)
State space (u,v,s,)
R400
R4
RVM
RVM Tracking
Frame t Frame t+1
XtXt+1
xt x*t+1 xt+1
RVM Tracking
At each frame…
RVM
X
Statistical filter
Xt+1
Extract subimage
Random variable
New state estimate
Motion Model
x*t+1
xt+1
Initialization & Recovery
Algorithm trains from “seed”
subimages
Provided by localisation algorithm
Localisation algorithm also used for
Initialization (frame 0)
Validation
Re-initialization
Results: Computational Efficiency
Why track?
0 10 20 30 40 50 60 70 80 9010
1
102
103
CP
U t
ime
/ fra
me
(m
s)
Frame Number
Restarting
Results: Computational Efficiency
Localize with boosting [Viola, Jones 2001, Li et al. 2002]
0 10 20 30 40 50 60 70 80
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time (s)
CP
U U
tiliz
ation
Boosting
RVM
Automatic Camera Management
Use position/scale information to control digital pan and
zoom
Conclusions
Future Work
RVM sparsity-smoothness tradeoff
Robustness to illumination and occlusion
Tracking 3D motion
Hybrid RVM tracking is…
Self starting SVM/Boosting detector
Self recovering SVM/Boosting validation
Efficient Temporal fusion +
sparsity
Questions?
Results: Cars
Algorithm is not specific to any class of objects
cinematographer
START
Establishment
Shot
Medium View
Zoom Withdraw
Pan &
Withdraw
Pan
Detect
New Pos
Big Change
Lost
Lost
1D example
Tutorial: toy data
100 Training
examples
12 Relevant vectors
remain
0 10 20 30 40 50 60 70 80 90 100-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Results: Smoothness
0 100 200 300 400 500 600 7000
5
10
15
20
25
30
35
0 100 200 300 400 500 600 7000
5
10
15
20
25
30
35
RVM Boosting
Top Related