1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander &...
-
date post
19-Dec-2015 -
Category
Documents
-
view
230 -
download
1
Transcript of 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander &...
![Page 1: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/1.jpg)
1Numerical geometry of non-rigid shapes Partial similarity
Partial similarity
© Alexander & Michael Bronstein, 2006-2009© Michael Bronstein, 2010tosca.cs.technion.ac.il/book
048921 Advanced topics in visionProcessing and Analysis of Geometric Shapes
EE Technion, Spring 2010
![Page 2: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/2.jpg)
2Numerical geometry of non-rigid shapes Partial similarity
Greek mythology
![Page 3: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/3.jpg)
3Numerical geometry of non-rigid shapes Partial similarity
I am a centaur.Am I human? Am I equine?Yes, I’m partially human.
Yes, I’m partially equine.
Partial similarity
Partial similarity is anon-metric relation
![Page 4: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/4.jpg)
4Numerical geometry of non-rigid shapes Partial similarity
Human vision example
?
![Page 5: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/5.jpg)
5Numerical geometry of non-rigid shapes Partial similarity
Visual agnosia
Oliver Sacks
The man who mistook his wife for a hat
![Page 6: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/6.jpg)
6Numerical geometry of non-rigid shapes Partial similarity
Recognition by parts
Divide the shapes into parts
Compare each part separately using a full similarity criterion
Merge the part similarities
![Page 7: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/7.jpg)
7Numerical geometry of non-rigid shapes Partial similarity
Partial similarity
X and Y are partially similar = X and Y havesignificantsimilar parts
Illustration: Herluf Bidstrup
![Page 8: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/8.jpg)
8Numerical geometry of non-rigid shapes Partial similarity
Ingredients
Similar Dissimilar
Part = subset of the shape
Similarity = full similarity applied to parts
![Page 9: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/9.jpg)
9Numerical geometry of non-rigid shapes Partial similarity
Significance
Significance = measure of the part
Area: size of the part
Significant Insignificant
![Page 10: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/10.jpg)
10Numerical geometry of non-rigid shapes Partial similarity
Significance
Problem: how to select significant parts?
Significance of a part is a semantic definition
Different shapes may have different definition of significance
![Page 11: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/11.jpg)
11Numerical geometry of non-rigid shapes Partial similarity
Significance
Do all parts have the same importance?
It really depends on the data
High Gaussian curvature
High variation of curvature
“features”
![Page 12: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/12.jpg)
12Numerical geometry of non-rigid shapes Partial similarity
Statistical significance
A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009
syzygy in astronomy means alignment of three bodies of the solar system along a straight or nearly straight line. a planet is in syzygy with the earth and sun when it is in opposition or conjunction. the moon is in syzygy with the earth and sun when it is new or full.
syzygy in astronomy means alignment of three bodies of the solar system along a straight or nearly straight line. a planet is in syzygy with the earth and sun when it is in opposition or conjunction. the moon is in syzygy with the earth and sun when it is new or full.
in is
theor
Query q Database D
Frequent = important
Frequent = non-discriminative
syzygy
Rare = discriminativeSignificance of a term t
Term frequency Inverse documentfrequency
![Page 13: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/13.jpg)
13Numerical geometry of non-rigid shapes Partial similarity
Numerical example
A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009
![Page 14: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/14.jpg)
14Numerical geometry of non-rigid shapes Partial similarity
Multicriterion optimization
Simultaneously minimize dissimilarity and insignificance over all the
possible pairs of parts
This type of problems is called multicriterion optimization
Vector objective function
How to solve a multicriterion optimization problem?
![Page 15: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/15.jpg)
15Numerical geometry of non-rigid shapes Partial similarity
Pareto optimality
Traditional optimization Multicriterion optimization
A solution is said to be a global
optimum of an optimization problem
if there is no other such that
A solution is said to be a Pareto
optimum of a multicriterion
optimization problem
if there is no other such that
Optimum is a solution such that there is no other better solution
In multicriterion case, better = all the criteria are better
![Page 16: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/16.jpg)
16Numerical geometry of non-rigid shapes Partial similarity
Pareto frontier
Vilfredo Pareto(1848-1923)
INSIGNIFICANCE
DIS
SIM
ILA
RIT
Y
UTOPIA
Attainable set
Pareto frontier
![Page 17: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/17.jpg)
17Numerical geometry of non-rigid shapes Partial similarity
Set-valued partial similarity
The entire Pareto frontier is a set-valued distance
The dissimilarity at coincides with full similarity
![Page 18: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/18.jpg)
18Numerical geometry of non-rigid shapes Partial similarity
Scalar- vs. set-valued similarity
INSIGNIFICANCE
DIS
SIM
ILA
RIT
Y
Use intrinsic
similarity as
![Page 19: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/19.jpg)
19Numerical geometry of non-rigid shapes Partial similarity
Is a crocodile green or ?
What is better: (1,1) or (0.5,0.5)?
What is better: (1,0.5) or (0.5,1)?
Order relations
![Page 20: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/20.jpg)
20Numerical geometry of non-rigid shapes Partial similarity
Order relations
There is no total order relation between vectors
As a result, two Pareto frontiers can be non-commeasurable
INSIGNIFICANCE
DIS
SIM
ILA
RIT
Y
BLUE < RED
GREEN ? RED
![Page 21: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/21.jpg)
21Numerical geometry of non-rigid shapes Partial similarity
Scalar-valued partial similarity
In order to compare set-valued partial similarities, they should be scalarized
Fix a value of insignificance
INSIGNIFICANCE
DIS
SIM
ILA
RIT
YCan be written as
using Lagrange multiplier
Define scalar-valued partial similarity
![Page 22: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/22.jpg)
22Numerical geometry of non-rigid shapes Partial similarity
Scalar-valued partial similarity
INSIGNIFICANCE
DIS
SIM
ILA
RIT
Y
Fix a value of dissimilarity
Distance from utopia point
Area under the curve
![Page 23: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/23.jpg)
23Numerical geometry of non-rigid shapes Partial similarity
Characteristic functions
Problem: optimization over all
possible parts
A part is a subset of the shape
Can be represented using a
characteristic function
Still a problem: optimization over binary-valued
variables (combinatorial optimization)
![Page 24: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/24.jpg)
24Numerical geometry of non-rigid shapes Partial similarity
Fuzzy sets
Relax the values of the
characteristic function to the
continuous range
The value of is the
membership of the point in the
subset
is called a fuzzy set
![Page 25: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/25.jpg)
25Numerical geometry of non-rigid shapes Partial similarity
Fuzzy partial similarity
Compute partial similarity using fuzzy parts
Fuzzy insignificance
Fuzzy dissimilarity
where is the weighted distortion
![Page 26: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/26.jpg)
26Numerical geometry of non-rigid shapes Partial similarity
Numerical optimization
GMDS problem:
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 27: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/27.jpg)
27Numerical geometry of non-rigid shapes Partial similarity
Numerical optimization
Freeze parts and solve for correspondence
Freeze correspondence and solve for parts
1
2
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 28: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/28.jpg)
28Numerical geometry of non-rigid shapes Partial similarity
Example of intrinsic partial similarity
Partial similarityFull similarity Scalar
partial similarity
Human
Human-like
Horse-like
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 29: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/29.jpg)
29Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
How to make ICP compare partially similar shapes?
Introduce weights into the shape-to-shape distance to reject points with bad
correspondence
Possible selection of weights:
where is a threshold on normals
is a threshold on distance
and are the normals to shapes and
![Page 30: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/30.jpg)
30Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
Without rejection With rejection
![Page 31: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/31.jpg)
31Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
Parts obtained by matching of man and centaur using ICP with rejection for different thresholds
Problem: there is no explicit influence of the rejection thresholds and the
size of the resulting parts
Pareto framework allows to control the size of the selected parts!
![Page 32: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/32.jpg)
32Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
Correspondence stage (fixed and )
Closest point correspondence
Weighted rigid alignment
Correspondence
![Page 33: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/33.jpg)
33Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
Part selection stage (fixed and )
![Page 34: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/34.jpg)
34Numerical geometry of non-rigid shapes Partial similarity
Extrinsic partial similarity
Controllable part size by changing the value of
![Page 35: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/35.jpg)
35Numerical geometry of non-rigid shapes Partial similarity
Not only size matters
What is better?...
Many small parts……or one large part?
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 36: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/36.jpg)
36Numerical geometry of non-rigid shapes Partial similarity
Regularity
Irregular = long boundary Regular = short boundary
Shape factor (circularity)
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 37: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/37.jpg)
37Numerical geometry of non-rigid shapes Partial similarity
Regularity
Irregular = four connected components
Regular = one connected component
Equal shape factor!
Topological regularity
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 38: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/38.jpg)
38Numerical geometry of non-rigid shapes Partial similarity
Mumford-Shah functional
Salvation comes from image segmentation
[Mumford&Shah]: given image , find the segmented region
replace by a membership function
The two problems are equivalent
![Page 39: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/39.jpg)
39Numerical geometry of non-rigid shapes Partial similarity
Mumford-Shah functional
In our problem, we need only the integral along the boundary
which becomes in the fuzzy setting
![Page 40: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/40.jpg)
40Numerical geometry of non-rigid shapes Partial similarity
Numerical example
INSIGNIFICANCE
DISSIMILARITY
IRREGULARITY
A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008
![Page 41: 1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d2e5503460f94a057ad/html5/thumbnails/41.jpg)
41Numerical geometry of non-rigid shapes Partial similarity
ICP exampleSignificance
Re
gul
arit
y