2D Shape Matching (and Object Recognition)
description
Transcript of 2D Shape Matching (and Object Recognition)
![Page 1: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/1.jpg)
1
2D Shape Matching (and Object Recognition)
Raghuraman Gopalan
Center for Automation ResearchUniversity of Maryland, College Park
![Page 2: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/2.jpg)
2
Outline
What is a shape? Part 1: Matching/ Recognition
Shape contexts [Belongie, Malik, Puzicha – TPAMI ’02] Indexing [Biswas, Aggarwal, Chellappa – TMM ’10]
Part 2: General discussion
![Page 3: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/3.jpg)
3
Shape
Some slides were adapted from Prof. Grauman’s course at Texas Austin, and Prof. Malik’s presentation at MIT
![Page 4: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/4.jpg)
4
Where have we encountered shape before?
Edges/ Contours Silhouettes
![Page 5: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/5.jpg)
5
A definition of shape
Defn 1: A set of points that collectively represent the object We are interested in their location information
alone!!
Defn 2: Mathematically, shape is an equivalence class under a group of transformations Given a set of points X representing an object O,
and a set of transformations T, shape S={t(X)| t\in T}
Issues? – Kendall ‘84
![Page 6: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/6.jpg)
6
Applications of Shapes
Analysis of anatomical structuresFigure from Grimson & Golland
Morphologyhttp://usuarios.lycos.es/lawebdelosfosiles/iPose
Recognition, detectionFig from Opelt et al.
Characteristic featureFig from Belongie et al.
![Page 7: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/7.jpg)
7
Part 1: 2D shape matching
![Page 8: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/8.jpg)
8
Recognition using shapes – (eg. – model fitting)
[Fig from Marszalek & Schmid, 2007]
For example, the model could be a line, a circle, or an arbitrary shape.
![Page 9: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/9.jpg)
9
Example: Deformable contours
Visual Dynamics Group, Dept. Engineering Science, University of Oxford.
Traffic monitoringHuman-computer interactionAnimationSurveillanceComputer Assisted Diagnosis in medical imaging
Applications:
![Page 10: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/10.jpg)
10
Issues at stake
Representation Holistic Part-based
Matching How to compute distance between shapes?
Challenges in recognition Information loss in 3D to 2D projection Articulations Occlusion…. Invariance??? Any other issue?
![Page 11: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/11.jpg)
11
Representation [Veltkamp ’00]
Holistic Moments
Fourier descriptors Computational geometry Curvature scale-space
![Page 12: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/12.jpg)
12
![Page 13: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/13.jpg)
13
Part-based
medial axis transform – shock graphs
![Page 14: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/14.jpg)
14
Matching: How to compare shapes?
![Page 15: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/15.jpg)
15
Discussion for a set of points
Hausdorff distance
![Page 16: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/16.jpg)
16
Chamfer distance• Average distance to nearest feature
• T: template shape a set of points• I: image to search a set of points• dI(t): min distance for point t to some point in I
• Average distance to nearest feature
• T: template shape a set of points• I: image to search a set of points• dI(t): min distance for point t to some point in I
![Page 17: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/17.jpg)
17
Chamfer distance
Edge image
How is the measure different than just filtering with a mask having the shape points?
![Page 18: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/18.jpg)
18
Distance Transform
Source: Yuri Boykov
34
23
23
5 4 4
223
112
2 1 1 2 11 0 0 1 2 1
0001
2321011 0 1 2 3 3 2
101110 1
2
1 0 1 2 3 4 3 210122
Distance Transform Image features (2D)
Distance Transform is a function that for each image pixel p assigns a non-negative number corresponding to
distance from p to the nearest feature in the image I
)(D)( pD
Features could be edge points, foreground points,…
![Page 19: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/19.jpg)
19
Distance transform
original distance transformedges
Value at (x,y) tells how far that position is from the nearest edge point (or other binary mage structure)
>> help bwdist
![Page 20: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/20.jpg)
20
Chamfer distance Average distance to nearest feature
Edge image Distance transform image
![Page 21: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/21.jpg)
21
Chamfer distance
Fig from D. Gavrila, DAGM 1999
Edge image Distance transform image
![Page 22: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/22.jpg)
22
A limitation of active contours
External energy: snake does not really “see” object boundaries in the image unless it gets very close to it.
image gradientsare large only directly on the boundary
I
![Page 23: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/23.jpg)
23
What limitations might we have using only edge points to represent a shape?
How descriptive is a point?
![Page 24: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/24.jpg)
24
Comparing shapes
What points on these two sampled contours are most similar? How do you know?
![Page 25: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/25.jpg)
25
Shape context descriptor [Belongie et al ’02]
Count the number of points inside each bin, e.g.:
Count = 4
Count = 10
...
Compact representation of distribution of points relative to each point
Shape context slides from Belongie et al.
![Page 26: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/26.jpg)
26
Shape context descriptor
![Page 27: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/27.jpg)
27
Comparing shape contextsCompute matching costs using Chi Squared distance:
Recover correspondences by solving for least cost assignment, using costs Cij
(Then use a deformable template match, given the correspondences.)
![Page 28: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/28.jpg)
28
Invariance/ Robustness
Translation Scaling Rotation Modeling transformations – thin plate splines
(TPS) Generalization of cubic splines to 2D
Matching cost = f(Shape context distances, bending energy of thin plate splines) Can add appearance information too Outliers?
![Page 29: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/29.jpg)
29
An example of shape context-based matching
![Page 30: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/30.jpg)
30
Some retrieval results
![Page 31: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/31.jpg)
31
Efficient matching of shape contexts [Mori et al ’05] Fast-pruning
randomly select a set of points Detailed matching
![Page 32: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/32.jpg)
32
Efficient matching of shape contexts [Mori et al ’05] – contd’ Vector quantization
![Page 33: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/33.jpg)
33
Are things clear so far?
![Page 34: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/34.jpg)
34
Detour - Articulation [Ling, Jacobs ’07]
![Page 35: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/35.jpg)
35
Inner distance vs. (2D) geodesic distance
![Page 36: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/36.jpg)
36
The problem of junctions
Is inner distance truly invariant to articulations?
![Page 37: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/37.jpg)
37
![Page 38: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/38.jpg)
38
Non-planar articulations? [Gopalan, Turaga, Chellappa ’10]
![Page 39: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/39.jpg)
39
Indexing approach to shape matching [Biswas et al ’10] Why?
![Page 40: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/40.jpg)
40
Indexing - frameworkPair-wise Features:
Inner distance, Contour length, relative angles etc.
![Page 41: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/41.jpg)
41
![Page 42: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/42.jpg)
42
Part 2 – Shapes as equivalence classes Kendall’s shape space
Shape is all the geometric information that remains when the location, scale and rotation effects are filtered out from the object
![Page 43: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/43.jpg)
43
Pre-shape
Kendall’s Statistical Shape Theory used for the characterization of shape.
Pre-shape accounts for location and scale invariance alone.
k landmark points (X:k\times 2) Translational Invariance: Subtract mean Scale Invariance : Normalize the scale
1, 1 1 Tc k k k
CXZ where C IkC X
Some slides were adapted from Dr. Veeraraghavan’s website
![Page 44: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/44.jpg)
44
Feature extraction
Silhoutte
Landmarks
Centered Landmarks
Pre-shape vector
![Page 45: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/45.jpg)
45
[Veeraraghavan, Roy-Chowdhury, Chellappa ’05]
Shape lies on a spherical manifold.Shape distance must incorporate the non-Euclidean nature of the shape space.
![Page 46: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/46.jpg)
46
Affine subspaces [Turaga, Veeraraghavan, Chellappa ’08]
![Page 47: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/47.jpg)
47
Other examples
Space of Blur Modeling group trajectories etc..
![Page 48: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/48.jpg)
48
Conclusion
Shape as a set of points configuring the geometry of the object Representation, matching, recognition Shape contexts, Indexing, Articulation
Shape as equivalence class under a group of transformations
![Page 49: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/49.jpg)
49
Announcements
HW 5 will be posted today; On Stereo, and Shape matching Due Nov. 30 (Tuesday after Thanksgiving)
Turn in HW 4
Midterms solutions by weekend
![Page 50: 2D Shape Matching (and Object Recognition)](https://reader036.fdocuments.us/reader036/viewer/2022062222/5681683e550346895dde0ca4/html5/thumbnails/50.jpg)
50
Questions?