Counting Crowded Moving Objects Vincent Rabaud and Serge Belongie Department of Computer Science and...
-
Upload
jeffery-quinn -
Category
Documents
-
view
214 -
download
0
Transcript of Counting Crowded Moving Objects Vincent Rabaud and Serge Belongie Department of Computer Science and...
Counting Crowded Moving Objects
Vincent Rabaud and Serge BelongieDepartment of Computer Science and Engineering
University of California, San Diego{vrabaud,sjb}@cs.ucsd.edu
Presentation by: Yaron KoralIDC, Herzlia, ISRAEL
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
2
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
3
Motivation
• Counting crowds of people• Counting herds of animals• Counting migrating cells
• Everything goes as long as the crowdis homogeneous!!
4
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
5
Challenges
• The problem of occlusion– Inter-object– Self occlusion
• Large number of independent motions– Dozens of erratically moving objects– Require more than two successive
frames
6
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
7
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
8
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
9
Harris Corner Detector – What are Good Features? C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
• We should easily recognize a corner by looking through a small window
• Shifting a window in any direction should give a large change in intensity
Harris Detector: Basic Idea
“flat” region:no change in all directions
“edge”:no change along
the edge direction
“corner”:significant
change in all directions
Harris Detector: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
Change of intensity for shift in [u,v] direction:
IntensityShifted intensity
Window function
orWindow function w(x,y)=
Gaussian1 in window, 0 outside
Harris Detector: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
yx vIuIyxIvyuxI ,,
v
uMvu
v
u
III
IIIyxwvu
v
uyxIyxIyxwvuE
yyx
yxx
yxyx
2
2
,
2
),(
,,,,
For small [u,v]:
We have:
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
For small shifts [u,v] we have a bilinear approximation:
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
where M is a 22 matrix computed from image derivatives:
Harris Detector: MathematicsDenotes by ei the ith eigen-vactor of
M whose eigen-value is i:
Conclusions:
0 iiTi M ee
max1, ,maxarg e vuEvu
maxmax eE
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the slowest change
direction of the fastest change
)max(-1/2
)min(-1/2
Ellipse E(u,v) = const
Harris Detector: Mathematics
1
2
“Corner”1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points
using eigenvalues of M:
Sum of Squared Differences – Tracking FeaturesTracking Features
• SSD is optimal in the sense of ML when1. Constant brightness assumption2. i.i.d. additive Gaussian noise
Ayx
tyxtvyuxvu,
2,,1,,,SSD RI
Exhaustive Search
• Loop over all parameter space• No realistic in most cases
– Computationally expensive• E.g. to search 100X100 image in 1000X1000
image using only translation~1010 operations!
– Explodes with number of parameters– Precision limited to step size
The Problem
Find (u,v) that minimizes the SSD
over region A.
Assume that (u,v) are constant over all A
Ayx
yxvyuxvuSSD,
2,,, RI
Iterative Solution
• Lucas Kanade (1981)– Use Taylor expansion of I (the optical
flow equation)
– Find
Ayx
tyxvuvu
vuvuSSD,
2
,,min,min III
0 tyx vu III
Feature Tracking with KLT(We’re back to crowd counting…)• KLT is a feature tracking algorithm• Driving Principle:
– Determine the motion parameter oflocal window W from image I to consecutive image J
– The center of the window defines the tracked feature
23
Feature Tracking with KLT
• Given a window W– the affine motion parameters A and d
are chosen to minimize the dissimilarity
24
Feature Tracking with KLT
• It is assumed that only d matters between 2 frames. Therefore a variation of SSD is used
25
• A window is accepted as a candidate feature if in the center of the window, both eigenvalues exceed a predefined threshold t
min(min(λλ11,, λ λ22) > t) > t
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
33
Increased Efficiency #1• Associating only one window with
each feature– Giving a uniform weight function that
depends on 1/(window area |w|)– Determining quality by comparing:
• Computation of different Z matrices is accelerated by “integral image”[1]
34
|| w
z
[1] Viola & Jones 2004
Increased Efficiency #2
• Run on sample training frames first– Determine parameters that lead to the
optimal windows sizes– Reduces to less than 5% of the possible
parameter set– All objects are from the same class
35
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
36
Feature Re-Spawning
• Along time, KLT looses track:– Inter-object occlusion– Self occlusion– Exit from picture– Appearance change due to perspective
and articulation
• KLT recreates features all the time– Computationally intensive– Weak features are renewed
37
Feature Re-Spawning
• Re-Spawn features only at specific locations in space and time
• Propagate them forward and backward in time– Find the biggest “holes”– Re-spawn features
in frame with theweighted average oftimes
38
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
39
Trajectory Conditioning
• KLT tracker gives a set of trajectories with poor homogeneity– Don’t begin and end at the same times– Occlusions can result in trajectory
fragmentation– Feature can lose its strength resulting in
less precise tracks
• Solution: condition the data– Spatially and temporally
40
Trajectory Conditioning
• Each trajectory is influenced by its spatial neighbors
• Apply a box to each raw trajectory• Follow all neighbor trajectories from
the time the trajectory started
41
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
42
Trajectory Clustering
• Determine number of object at time tt by clustering trajectories
• Since at time tt objects may be close, focus attention on a time interval (half-width of 200 frames)
• Build connectivity graph– At each time step, the present features form the
nodes of a connectivity graph G – Edges indicate possible membership to a common
object.
43
Trajectory Clustering• Connectivity Graph
– Bounding Box: as small as possible, able to contain every possible instance of the object
– If two features do not stay in a certain box, they do not belong to the same object.
– The 3 parameters of this box are learned from training data.
44
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
Articulation factor
Trajectory Clustering• Rigid parts merging
– Features share similar movement during whole life span, belong to a rigid part of an objectrigid part of an object, and consequently to a common objectcommon object
– RANSAC is applied to sets of trajectories • Within time window • Connected in graph G
45
Trajectory Clustering
• Agglomerative Clustering– At each iteration, the two closest sets are
considered – If all features are linked to each other in the
connectivity graph, they are merged together.
– Otherwise, the next closest sets are considered
– Proceed until all possible pairs are analyzed
46
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
47
Experimental results
• Datasets– USC: elevated view of a crowd
consisting of zero to twelve persons– LIBRARY: elevated view of a crowd of
twenty to fifty persons– CELLS: red blood cell dataset consisting
of fifty to hundred blood cells
48
Experimental results
49
Experimental results
50
Estimated
Ground Truth
Experimental results
51
Conclusion
• A new way for segmenting motions generated by multiple objects in crowd
• Enhancements to KLT tracker• Conditioning and Clustering
techniques
52
Thank You!Thank You!
53