Survey of Algorithms to Query Image Databases COMP 290-72:Computational Geometry Benjamin Lok...
-
Upload
collin-andrews -
Category
Documents
-
view
221 -
download
1
Transcript of Survey of Algorithms to Query Image Databases COMP 290-72:Computational Geometry Benjamin Lok...
Survey of Algorithms to Query Image Databases
COMP 290-72:Computational Geometry
Benjamin Lok 11/2/98
Image from Kodak’sPhotoQuilt
Outline of Talk
Overview of the problem Three methods
• Color based• Shape based• Vision based
Conclusions
Image from Microsoft Clip Gallery
Problem
Query an image database
What does a “match” mean?• Application dependent
• Notion of subjectivity
• Sensitivity to noise
Problem
Semantic similarity is still not possible• ex. “All images with cats”
To determine similarity,
we need to a new:• Metric• Space
Images from Microsoft Clip Gallery and website
Ambiguity
Guy
Girl
Guy
Images from “Shrine to Long Haired Men” and “Videos of Women Getting Their Heads Shaved” websites
Yossi Rubner, Leonidas Guibas, Carlos Tomasi (1997)
Stanford Vision Laboratory
The Earth Mover’s Distance, Multi-Dimensional Scaling,
and Color-Based Image Retrieval
Image from Microsoft Clip Gallery
Color Signatures
Utilize the CIE-LAB color space• Based on human perception of color
Map each pixel to a point in color space• Common color values increase weight of point
Group clusters into points (8-12 per image)
Rubner, Guibas, and Tomasi
Earth Mover’s Distance
To compare two images, compute the “work” needed to move the cluster points from one image to the other
Rubner, Guibas, and Tomasi
Earth Mover’s Distance (cont)
Solving a linear programming problem:
Given two signatures:p = {(p1,wp1),…,(pm,wpm)} and
q = {(q1,wq1),…,(qn,wqn)}
Find C where Cij is the amount of weight pi matched qj
),min(1 1
qp
m
i
n
jjiij
ww
qpC
Applications
Visualize Databases (Queries and Results) Scale the multiple dimensions into 2D using
MDS and minimize STRESS
Rubner, Guibas, and Tomasi
Algorithm Recap
Map pixels to 3D color space points Locate and compress “clusters” of points 8 to 12 points determine the color signature Calculate the Earth Mover’s Distance to
determine “distance” between two images
Image from YenPen Stationary Website
Advantages
Based on human perception of color
Some invariance to small change in viewpoint and lighting
Meaningful metric Relatively fast Can embed multiple
metrics
Disadvantages
Depending on application, query format might be not be intuitive
Not much use for non-color images
False positives a real possibility depending on working domain
Shaped-based Image Retrieval Using Geometric Hashing
Scott D. Cohen and Leonidas J. Guibas
1997
Stanford Vision Laboratory
Image from Microsoft Clip Gallery
Overview
Implementation• Search through 500
Chinese characters
Goals• Provide invariance to scale, rotation, and
translation• speed and accuracy
Cohen and Guibas
Generating a Illustration
Illustration - set of curves that summarize an image• Edgel detection• Medial Axis determination
Cohen and Guibas
Approximating with Polylines
Convert medial axis representation to polylines
Tradeoff between speed and accuracy
Cohen and Guibas
Geometric Hashing
Geometric Hashing - method used to compare two point sets under some transformation group
Take each point and use it as the origin of a coordinate system
},1:{)(
},1:{)(
jlnlqqQI
ikmkppPI
jlj
iki
Cohen and Guibas
Geometric Hashing (cont)
If translating P by qj - pi produces a good match Ii(P) and Ij(Q) will match.
This property can be generalized to other transformation groups.
Each line segment is a basis of a coordinate system• Translation, Rotation, and Orientation defined• I(P) = transform all other segments into new CS
Cohen and Guibas
Notes on GH
Each segment will be transformed to 2m Coordinate systems
I(P) stores O(m2) segments Can be done as preprocessing step Expressing the different possible
transformations using each segment as a basis
Cohen and Guibas
Querying the Database
Query image undergoes the
feature extraction process For each query feature, a nearest-neighbor
query is applied and the k closest or within some j
Similarity score increases if database image has a feature that is “close” to the query feature
Cohen and Guibas
“Closeness”
How do you describe the closeness of two lines?
Transform to a 4D space made of (l,,a,b) With two (l,,a,b) descriptions for lines, can
compute distance Divide by standard deviation
over sample of database
features
Details
Closeness is relative to database contents Nearest-neighbor algorithm by Arya,
Mount, et. al (1994). Query time for k nearest features is O(k log n)
Cohen and Guibas
Advantages
Fast• Queries database of
500 characters in 1 second on SGI Indy
Queries based on important features
Disadvantages
Working domain currently limited
Could get too expensive as complexity in images increases
Cohen and Guibas
A Multi-Resolution Technique for Comparing Images Using the Hausdorff Distance
Daniel Huttenlocher and William Rucklidge
1992 Cornell University
Huttenlocher and Rucklidge
Directed Hausdorff Distance
Given A={a1, … , am} and B={b1, … , bm}
Identifies the point in A farthest from any point in B
Measures the degree of mismatch between between two sets.
baBAh
ABhBAhBAH
BbAa
minmax),(
)),(),,(max(),(
Properties of Hausdorff Distances
Not symmetric h(A,B) != h(B,A) Compute kth maximum
• Notion of rank• Reduces sensitivity• Fraction of A within
h(A,B) of B• Obscured portions
h(A,B) = hypothesis
h(B,A) = test
Transformations t( ) =
Given A is an image, B is the model Without Orientation, if A is in B then A
undergoes transformation t.fB(t)=H(t(B),A) t=(tx,ty,sx,sy) forward
fA(t)=H(A,t(B)) reverse
),(min)),(( yyyxxxAaBb
K tbstbsathKABth
Bidirectional Hausdorff Distance
Solve for which values of t,
the following holds:
Results in searching a four
dimensional space
))(,(
))),(()),(,((max))(,(
BtAH
ABthBtAhBtAH
LK
KLLK
Restricting Search Space
Slope of f(t)=HLK(A,t(B))
is linear Divide space into cells Calculate HLK(A,tc(B))
Determine a maximum delta per cell • Based on limit in scale and translation• Allows for quick rejection and acceptance
Label cells as interesting or disregard
Restricting Search Space
Create smaller cells from interesting cells Bounds based on
transformations Quickly narrow
down to areas that
could possibly be
within of A
Subtleties
Discretization useful if working in computer vision domain (integers)
Can compare partially obscured images Optimizations
• Early rejection/acceptance
Pretty slow 200 to 250 seconds
Website on submarines
Advantages
Accurate Geared towards
image processing and vision
Partially obscured images
Searches similar to humans
Disadvantages
Slow No Orientation Database must be
specialized Potential problems in
generating queries
Recap
Three Algorithms• Color Based
• Color Signatures
• Earth Mover’s Distance
• Shaped Based• Polylines
• Transform Invariant Sets
• Vision Based• Hausdorff Distance
• Subdivision of Transformation Space
www.sportsmanscaps.com
Final Thoughts
Algorithms work well in various domains Query construction not formalized Other methods:
• wavelet-based• texture-based• object-based
Took 5 minutes to find “Shrine to Men with Long Hair” and “Videos of Women Getting Their Head Shaved”
www.jerryspringer.com
All other images generated by author using Paint Shop Pro