Background Subtraction for Temporally Irregular
Dynamic TexturesGerald Dalley, Joshua Migdal,
and W. Eric L. Grimson
Workshop on Applications of Computer Vision (WACV) 2008
8 Jan. 2008 2
Key Prior Approaches
l MoG – learn modes for all pixels & track changesl Works as long as dynamic textures are regular
l Per-pixel kernels (Mittal & Paragios)l Can pay attention to higher-level features like optical flowl How many kernels to keep?
l Spatial kernels (Sheikh & Shah)l Handles temporal irregularity in dynamic texturesl How many kernels to keep?
8 Jan. 2008 3
Results with Mittal & Paragios Traffic Clip
Key:l bboxes from
ground truthl True
positivel False
positivel False
negative
l No ground truth available
8 Jan. 2008 4
Our Goals
l Compactnessl Speedl Complexity growth
l Grow with scene complexity, not with the amount of time to see the complexity§ If a leaf blows in front of a pixel once a minute, we don’t
want to have to keep 1800 kernels
l Usable in MoG frameworksl MoG variants are very widely used (original
Stauffer & Grimson paper has 1000+ citations)
8 Jan. 2008 5
Pixels Affected by a GivenMixture ComponentOur Model
l Mixture of Gaussians (MoG)
Standard MoG
Ours
p(ci|Φ) ∝∑
j∈NiwjN (ci;µj ,Σj)
ci the observed color at pixel i
Φ the model {wj , µj ,Σj}j
Ni neighborhood of pixel i
p(ci|Φ) ∝∑
j∈NiwjN (ci;µj ,Σj)
ci the observed color at pixel i
Φ the model {wj , µj ,Σj}j
Ni neighborhood of pixel i
8 Jan. 2008 6
Foreground/Background Classification
l Find best matching background Gaussian, jl Use neighborhood
l Squared Mahalanobis Distance
-
=
input
d2ij
mixture model
Φ1 Φ2 Φ3
dij = (ci − µj)TΣ−1
j (ci − µj)dij = (ci − µj)TΣ−1
j (ci − µj)
8 Jan. 2008 7
hard Stauffer-Grimson
soft Stauffer-Grimson
best match only
update all matches
Model Update OptionsHard UpdatesSoft Updates
Loca
lR
egio
nal
8 Jan. 2008 8
Results on a Classic Dataset: Wallflower WavingTrees
Ground
Truth
Toyam
aet al.
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 9
ROC (Wallflower)
8 Jan. 2008 10
Traffic #410G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 11
Traffic #150G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 12
Pushing the Limits:Ducks
Ground
Truth
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 13
Pushing the Limits:Jug
Ground
Truth
Zhong
&
Sclaroff
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 14
Parameter Sensitivity
0 1000 2000 3000 4000 5000 6000 70000
0.2
0.4
0.6
0.8
1
min number of mistakes
freq
uenc
y
ws=7ws=5ws=3ws=1
traffic
0 500 1000 1500 20000
0.05
0.1
0.15
min number of mistakes
freq
uenc
y
ws=7ws=5ws=3ws=1
wallflower
max max
8 Jan. 2008 15
Summary
l Model representationl Same as MoG
l FG/BG Classifier Inputl Add local neighborhood loop
l Updatel Several options, including standard MoG updates
l Resultsl Reduces foreground false positivesl Reduces need for exotic input features (e.g. optical flow)l Cost window size
8 Jan. 2008 16
Also in the Paper…
l Analysisl Timing experimentsl More ROC analysis
l Implementation Detailsl MRF choicel Update formulas
8 Jan. 2008 17
Thank You
Questions...
8 Jan. 2008 18
Backup Slides
8 Jan. 2008 19
PureSoft SoftLocal PureHard HardLocal0
10
20
30
40
50
60R
elat
ive
Tim
e C
ost
Model Update Computational Costs
W=12
W=32
W=52
W=72
8 Jan. 2008 208 Jan. 2008 20
PureSoft SoftLocal PureHard HardLocal0
20
40
60
80
100R
elat
ive
Tim
e C
ost
Mahalanobis Map Computational Costs
W=12
W=32
W=52
W=72
8 Jan. 2008 21
1
2
3
4
5
6
7
8
1 2 3 4 1 2 3 4 1 2 3 41 2 3 4
Background
Model
Observed
Image
9x9
Matching
3x3
Matching
Small Windows are Still Useful
8 Jan. 2008 22
c(t−1)ic(t−1)ic(t−1)i c
(t)ic(t)ic(t)i
φ(t)jφ(t)j
z(t−1)iz(t−1)i
l(t−1)jl(t−1)jl(t−1)j l
(t)jl(t)jl(t)j
JJ
II
z(t)iz(t)i
φ(t−1)jφ(t−1)j
Generative Model
Particle locations
Particle appearances
Pixel-to-particle assignments
Pixel appearance
8 Jan. 2008 23
Independent Assignments
l(t−1)1
l(t−1)2
l(t−1)3
z(t)i = 2
l(t)1
l(t)2
l(t)3
l(t−1)1l(t−1)1l(t−1)1
l(t−1)2l(t−1)2l(t−1)2
l(t−1)3l(t−1)3l(t−1)3
z(t)i = 2z(t)i = 2z(t)i = 2
l(t)1l(t)1l(t)1
l(t)2l(t)2l(t)2
l(t)3l(t)3l(t)3
Top Related