The Shape of Things to Come Shape Theory for Object ...md9c/presentations/UVA... · The Shape of...
Transcript of The Shape of Things to Come Shape Theory for Object ...md9c/presentations/UVA... · The Shape of...
The Shape of Things to Come*:Shape Theory for Object Recognition
Michael D. DeVore
SIE Graduate Seminar
University of Virginia
November 7, 2003
*Perhaps better put as “The Shape of Things Coming Toward You”
2
Definition of Object Recognition
• Collect measurements from a scene and decide whatobjects are present
• May also be interested in other info about the object– Dimensions
– Orientation
– Lighting conditions
– etc.
Object Recognition
Definition
Measurementand shape
ResearchLaboratory
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
SensorPlatform
InferenceSystem
ModelDatabase
HondaCivic
3
Measurement Variability
• Image variation (randomness) due to:– Lighting variation
– Surface color variation
– Shape variation
Object Recognition
Definition
Measurementand shape
ResearchLaboratory
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
4
Shape is Fundamental
• Some sensors collect shape data directly
LADAR image of a ChevroletBlazer and a Hummer
From “LADAR puts the puzzletogether” by Robert Hauge. SPIE’sOE Magazine, April 2003.
A recognition system must (implicitly or explicitly)account for shape and shape variation
Object Recognition
Definition
Measurementand shape
ResearchLaboratory
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
5
Research Laboratory
• Equipped to collect image data under computer control
• Structured-light laser to collect shape data
• Beowulf cluster for analysis
Object Recognition
Definition
Measurementand shape
ResearchLaboratory
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
6
Research Laboratory
• Collaborators at UVa– Xin Wu
– Josh Stafford
– Shankha Basu
– 4th Year Capstone GroupSaptarshi Adhikari, James Fang,
Shadie Wadie, Guenevere Lindgren
– David Luebke (Asst. Prof., CS)
Object Recognition
Definition
Measurementand shape
ResearchLaboratory
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
7
Fields: Anthropology
• In many fields, shape data are of central importance• Anthropological studies often concerned with the
evolution of physical features
Object Recognition
Need for ShapeTheory
Anthropology
Archaeology
Medicine
Others
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
1. Modern Human2. Neanderthal
3. Australopithecine4. Chimpanzee
“The Order of Man: A Biomathematical Anatomy of the Primates,” C. E. Oxnard
8
Fields: Archaeology
• Archaeological investigations often focus onevaluation of the shape of artifacts
Iron Age Brooches
Hodson, Sneath, and Doran. Biometrika 1966.
Object Recognition
Need for ShapeTheory
Anthropology
Archaeology
Medicine
Others
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
9
Fields: Medicine
• Medical studies often focus on the shape of organs
Location of the hippocampus
3D shape model of ahippocampus
From web site of Sarang Joshi,http://cis.jhu.edu/~sarang
Object Recognition
Need for ShapeTheory
Anthropology
Archaeology
Medicine
Others
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
10
Fields: Others
• Astronomy
• Biology
• Geology
• Cartography
• Manufacturing
• etc.
Object Recognition
Need for ShapeTheory
Anthropology
Archaeology
Medicine
Others
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
11
Requirements of a Theory
1. Represent the shape of an arbitrary object
2. Quantify the difference between shapes
3. Represent randomness in shape
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
Requirements
CasualDefinition
Landmarks
ProcrusteanAnalysis
Examples
12
Casual Definition
• Shape of a data set is the collection of attributes thatremain unchanged under transformations of:– Scale
– Location
– Rotation
• Two data sets have the same shape if:– can be made equal by transformations of scale, location,
and rotation
• Shape distance between two data sets:– smallest distance between sets under all possible
transformations of scale, location, and rotation
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
Requirements
CasualDefinition
Landmarks
ProcrusteanAnalysis
Examples
13
Geometrical Figures
• Data set to represent a geometric figure:– Labeled collection of landmark location
• Landmarks chosen to:– Give a partial geometric characterization
– Be identifiable across individuals
• We will use the shape of leaves as an example in 2D
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
Requirements
CasualDefinition
Landmarks
ProcrusteanAnalysis
Examples
14
Leaf Landmarks
• Landmarks will be the tips and valleys of the mainlobes
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
Requirements
CasualDefinition
Landmarks
ProcrusteanAnalysis
Examples
15
Landmark Representation
• Procrustean Analysis:– What must I do to one set of landmarks to make it match
another set?
• N landmarks represented by a matrix
X = [x1, x2, …, xN]
where xm is a column vector of length 2 for points inthe plane (3 for points in space)
• Matrix X is an element of Euclidian space ¬2N
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
16
Pre-Shape
• Location is defined as
• Scale is defined as
• By pre-shape we mean the data set with location andscale removed
†
x = 1N
xnn=1
N
Â
†
x = xn - x 2
n=1
N
Â
†
t X( ) =1x
x1 - x , x2 - x ,K , xN - x [ ] = [t1,t 2,K,t N ]
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
17
Example: Pre-Shape
Original Landmarks After Location Removal
Pre-Shape (location and scale removed)
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
18
Example: Pre-Shape Retains Orientation
Small Maple Leaf and Landmarks
Small Maple Leaf Pre-Shape Big Maple Leaf Pre-Shape
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
19
Space of Pre-Shapes
• We can think of a pre-shape t as N points in ¬2
• We can think of a pre-shape t as 1 point in ¬2N
• Each t is an element of
(A 2N-2 dimensional hyperplane containing the origin)
and an element of
(A 2N-1 dimensional hypersphere centered at the origin)
†
F 2N-2 = x1,x2,K,xN( ) Œ ¬2N : xnn=1
N
 = 0Ï Ì Ó
¸ ˝ ˛
†
S2N-1 = x1,x2,K,xN( ) Œ ¬2N : xn2
n=1
N
 =1Ï Ì Ó
¸ ˝ ˛
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
20
Space of Pre-Shapes
• The intersection, S*2N-3 = F2N-2 « S2N-1, is a 2N-3
dimensional hypersphere made by “slicing” S2N-1
• Call S*2N-3 the pre-shape space
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
21
Orientation of a Pre-Shape
• Not generally possible to “remove” orientation like wedid with location and scale
• Rotation of the original figure corresponds to a rotationof t
• Set of all possible rotations of t is called the orbit of t
• The orbit defines a set of equivalence classes
• Two pre-shapes t and n are equivalent if they have thesame orbit: O(t) = O(n)
†
O t( ) =cosq sinq
-sinq cosq
È
Î Í
˘
˚ ˙ t : 0 £ q < 2p
Ï Ì Ó
¸ ˝ ˛
à S*2N-3
†
SN = O t( ) : t Œ S*2N-3{ }
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
22
Definition of Shape
• By the shape of a labeled set of points, X = [x1, x2, …,xN] we mean the orbit of its pre-shape O(t(X))
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
23
Distance Between Pre-Shapes
• Pre-shapes lie on the unit hypersphere S*2N-3
• Great circle distance between t and n is:
d(t, n) = cos-1(t • n)
• Values between 0 and p
Recall:t • n = cos(q)
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
24
Distance Between Shapes
• Distance between shapes O(t) and O(n) is theminimum distance between pre-shapes in the orbits
• To get a closed-form solution:– Note that cos-1 is minimized when its argument is
maximized (both t and n are unit vectors)
– Differentiate the dot product with respect to q
– Set the result equal to zero
†
d O t( ),O n( )[ ] = minq1 ,q 2
dcosq1 sinq1
-sinq1 cosq1
È
Î Í
˘
˚ ˙ t ,
cosq2 sinq2
-sinq2 cosq2
È
Î Í
˘
˚ ˙ n
Ê
Ë Á
ˆ
¯ ˜
= minq
d t ,cosq sinq
-sinq cosq
È
Î Í
˘
˚ ˙ n
Ê
Ë Á
ˆ
¯ ˜
= minq
cos-1 t ,cosq sinq
-sinq cosq
È
Î Í
˘
˚ ˙ n
Ê
Ë Á
ˆ
¯ ˜
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
25
Distance Between Shapes
• Solution:
†
tan q( ) =
t n,1n n,2 - t n,2n n,1( )n=1
N
Â
t n,1n n,1 + t n,2n n,2( )n=1
N
Â
†
d O t( ),O t( )( ) = cos-1 t n,1n n,1 + t n,2n n,2( )n= 0
N
ÂÈ
Î Í
˘
˚ ˙
2
+ t n,2n n,1 - t n,1n n,2( )n= 0
N
ÂÈ
Î Í
˘
˚ ˙
2Ê
Ë
Á Á
ˆ
¯
˜ ˜
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
LandmarkRepresentation
Pre-Shape
Space of Pre-Shapes
Orientation
PreciseDefinition
Distance
Examples
Example: Leaf Shapes
Big Maple Leaf
Small Maple Leaf
Big Ivy Leaf
Small Ivy Leaf Small Tulip Poplar Leaf
Big Tulip Poplar Leaf
27
Example Leaf Shape Distances
• Distances between shapes for each pair of leaves
00.11160.77560.63960.64470.5763SmallPoplar
0.111600.71380.58970.57430.5185BigPoplar
0.77560.713800.23720.31970.3420SmallIvy
0.63960.58970.237200.38800.3609Big Ivy
0.64470.57430.31970.388000.1364SmallMaple
0.57630.51850.34200.36090.13640BigMaple
SmallPoplar
BigPoplar
SmallIvy
Big IvySmallMaple
BigMaple
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples
28
Concluding Remarks
• Randomness in the object geometryfi Randomness in the pre-shape
fi Distribution over the orbits
• Extensions from vector spaces to function spaces– Shape representations that account for the entire edge
• Eliminate landmark identification– Distance metrics not dependent on point correspondence
• Joint representation of color and shape– Random color over a random shape
Object Recognition
Need for ShapeTheory
Need from a ShapeTheory
ProcrusteanAnalysis
Examples