EXTENDED HAUSDORFF DISTANCE AND SIMILARITY MEASURES FOR NEUTROSOPHIC REFINED SETS
Image Similarity and the Earth Mover’s Distance
description
Transcript of Image Similarity and the Earth Mover’s Distance
![Page 1: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/1.jpg)
Image Similarity and the Earth Mover’s Distance
Empirical Evaluation of Dissimilarity Measures for Color and TextureY. Rubner, J. Puzicha, C. Tomasi and T.M. Buhmann
The Earth Mover’s Distance as a Metric for Image Retrieval
Y. Rubner, C. Tomasi and J.J. GuibasThe Earth Mover’s Distance is the Mallows Distance: Some Insights from Statistics
E. Levina and P.J. Bickel
Learning-Based Methods in Vision - Spring 2007Frederik Heger
(with graphics from last year’s slides)
1 February 2007
![Page 3: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/3.jpg)
3 LBMV Spring 2007 - Frederik Heger [email protected]
Similarity is Important for …• Image classification
• Is there a penguin in this picture?• This is a picture of a penguin.
• Image retrieval• Find pictures with a penguin in them.• Image as search query
• Find more images like this one.• Image segmentation
• Something that looked like this was called penguin before.
![Page 4: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/4.jpg)
4 LBMV Spring 2007 - Frederik Heger [email protected]
Space Shuttle Cargo Bay
Image Representations: Histograms
Normal histogram Cumulative histogram•Generalize to arbitrary dimensions•Represent distribution of features
• Color, texture, depth, …
Images from Dave Kauchak
![Page 5: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/5.jpg)
5 LBMV Spring 2007 - Frederik Heger [email protected]
Image Representations: Histograms
Joint histogram• Requires lots of data• Loss of resolution to
avoid empty bins
Images from Dave Kauchak
Marginal histogram• Requires independent features• More data/bin than
joint histogram
![Page 6: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/6.jpg)
6 LBMV Spring 2007 - Frederik Heger [email protected]
Space Shuttle Cargo Bay
Image Representations: Histograms
Adaptive binning• Better data/bin distribution, fewer empty bins• Can adapt available resolution to relative feature importance
Images from Dave Kauchak
![Page 7: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/7.jpg)
7 LBMV Spring 2007 - Frederik Heger [email protected]
EASE Truss Assembly
Space Shuttle Cargo Bay
Image Representations: Histograms
Clusters / Signatures• “super-adaptive” binning• Does not require discretization along any fixed axis
Images from Dave Kauchak
![Page 8: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/8.jpg)
8 LBMV Spring 2007 - Frederik Heger [email protected]
Distance Metrics
-
-
-
= Euclidian distance of 5 units
= Grayvalue distance of 50 values
= ?
x
y
x
y
![Page 9: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/9.jpg)
9 LBMV Spring 2007 - Frederik Heger [email protected]
Issue: How to Compare Histograms?
Bin-by-bin comparisonSensitive to bin size. Could use wider bins …
… but at a loss of resolution
Cross-bin comparisonHow much cross-bin influence is necessary/sufficient?
![Page 10: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/10.jpg)
10 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresHeuristic Histogram Distance:
Minkowski-form distance (Lp)
Special Cases:L1 Mahattan distanceL2 Euclidian DistanceL Maximum value distance
![Page 11: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/11.jpg)
11 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresHeuristic Histogram Distance:
Weighted-Mean-Variance (WMV)
Info:• Per-feature similarity measure• Based on Gabor filter image representation• Shown to outperform several parametric models
for texture-based image retrieval
![Page 12: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/12.jpg)
12 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresNonparametric Test Statistic:
Kolmogorov-Smirnov distance (KS)
Info:• Defined for only one dimension• Maximum discrepancy between cumulative
distributions• Invariant to arbitrary monotonic feature
transformations
![Page 13: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/13.jpg)
13 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresNonparametric Test Statistic:
Cramer/von Mises type statistic (CvM)
Info:• Squared Euclidian distance between distributions• Defined for single dimension
![Page 14: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/14.jpg)
14 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresNonparametric Test Statistic:
2
Info:• Very commonly used
![Page 15: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/15.jpg)
15 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresInformation-theory Divergence:
Kullback-Leibler divergence (KL)
Info:• Code one histogram using the other as true
distribution• How inefficient would it be?• Also widely used.
![Page 16: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/16.jpg)
16 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresInformation-theory Divergence:
Jeffrey-divergence (JD)
Info:• Similar to KL divergence• But symmetric and numerically stable
![Page 17: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/17.jpg)
17 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresGround Distance Measure:
Quadratic Form (QF)
Info:• Heuristic approach• Matrix A incorporates cross-bin information
![Page 18: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/18.jpg)
18 LBMV Spring 2007 - Frederik Heger [email protected]
Overview: Similarity MeasuresGround Distance Measure
Earth Mover’s Distance (EMD)
Info:• Based on solution of linear optimization problem
(transportation problem)• Minimal cost to transform one distribution to the
other• Total cost = sum of costs for individual features
![Page 23: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/23.jpg)
23 LBMV Spring 2007 - Frederik Heger [email protected]
Earth Mover’s Distance
=
(amount moved) * (distance moved)
![Page 24: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/24.jpg)
24 LBMV Spring 2007 - Frederik Heger [email protected]
How EMD Works
All movements
(distance moved) * (amount moved)
(distance moved) * (amount moved)
* (amount moved)
n clusters
Q
Pm clusters
![Page 25: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/25.jpg)
25 LBMV Spring 2007 - Frederik Heger [email protected]
How EMD Works
Move earth only from P to Q
P’
Q’n clusters
Q
Pm clusters
![Page 26: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/26.jpg)
26 LBMV Spring 2007 - Frederik Heger [email protected]
How EMD Works
n clusters
Q
Pm clusters
P cannot send more earth than there is
P’
Q’
![Page 27: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/27.jpg)
27 LBMV Spring 2007 - Frederik Heger [email protected]
How EMD Works
n clusters
Q
Pm clusters
Q cannot receive more earth than it can hold
P’
Q’
![Page 28: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/28.jpg)
28 LBMV Spring 2007 - Frederik Heger [email protected]
How EMD Works
n clusters
Q
Pm clusters
As much earth as possiblemust be moved
P’
Q’
![Page 29: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/29.jpg)
29 LBMV Spring 2007 - Frederik Heger [email protected]
Color-based Image Retrieval
Jeffrey divergence
Quadratic form distance
Earth Mover Distance
χ2 statistics
L1 distance
![Page 32: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/32.jpg)
32 LBMV Spring 2007 - Frederik Heger [email protected]
EMD with Position Encoding
without position
with position
![Page 33: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/33.jpg)
33 LBMV Spring 2007 - Frederik Heger [email protected]
Issues with EMD• High computational complexity
• Prohibitive for texture segmentation• Features ordering needs to be known
• Open eyes / closed eyes example• Distance can be set by very few features.
• E.g. with partial match of uneven distribution weight
EMD = 0, no matter how many features follow
![Page 34: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/34.jpg)
34 LBMV Spring 2007 - Frederik Heger [email protected]
Help From Statisticians• For even-mass distributions,
EMD is equivalent to Mallows distance• (for uneven mass distributions,
the two distances behave differently)• Trick to compute Mallows distance
• 1-D marginals give better classification results than joint distributions (experimental results)
• Get marginals from empirical distribution by sorting feature vectors
![Page 35: Image Similarity and the Earth Mover’s Distance](https://reader036.fdocuments.us/reader036/viewer/2022062410/5681595d550346895dc69c3f/html5/thumbnails/35.jpg)
35 LBMV Spring 2007 - Frederik Heger [email protected]
EMD Summary / Conclusions• Ground distance metric for image similarity• Uses signatures for best adaptive binning and
to lessen impact of prohibitive complexity• Can deal with partial matches• Good performance for color/texture
classification• Statistical grounding