Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Deformation Invariant Image Matching

Haibin Ling and David W. Jacobs

Center for Automation ResearchComputer Science Department

University of Maryland, College Park

Oct, 20, 2005, ICCV

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Outline

Introduction

Deformation Invariant Framework

Experiments

Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

General Deformation

• One-to-one, continuous mapping.• Intensity values are deformation invariant.

– (their positions may change)

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Our Solution

• A deformation invariant framework

– Embed images as surfaces in 3D

– Geodesic distance is made deformation invariant by adjusting an embedding parameter

– Build deformation invariant descriptors using geodesic distances

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Related Work• Embedding and geodesics

– Beltrami framework [Sochen&etal98]– Bending invariant [Elad&Kimmel03]– Articulation invariant [Ling&Jacobs05]

• Histogram-based descriptors– Shape context [Belongie&etal02]– SIFT [Lowe04]– Spin Image [Lazebnik&etal05, Johnson&Hebert99]

• Invariant descriptors– Scale invariant descriptors [Lindeberg98, Lowe04]– Affine invariant [Mikolajczyk&Schmid04, Kadir04,

Petrou&Kadyrov04]– MSER [Matas&etal02]

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Outline

Introduction

Deformation Invariant Framework Intuition through 1D images2D images

Experiments

Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

1D Image Embedding

1D Image I(x)

EMBEDDINGI(x) ( (1-α)x, αI )αI(1-α)x

Aspect weight α : measures the importance of the intensity

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Geodesic Distance

αI

(1-α)x

p qg(p,q)

• Length of the shortest path along surface

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Geodesic Distance and α

I1 I2

Geodesic distance becomes deformation invariant

for α close to 1

embed embed

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Image Embedding & Curve Lengths

]1,0[:),( 2 RyxI

dtIyxl ttt 222222 )1()1(

))('),('),('()( tztytxt

Depends only on intensity I Deformation Invariant

IzyyxxI ',)1(',)1('),(

dtI t

21

Image I

Embedded Surface

Curve on

Length of

Take limit

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Geodesic Distance for 2D Images

• Computation– Geodesic level curves – Fast marching [Sethian96]

is the marching speed 2/122222)1(

yx IIF

• Geodesic distance– Shortest path– Deformation invariant

F

T is the geodesic distance

T=1T=2

T=3

T=4

p

q1|| FT

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Deformation Invariant Sampling

Geodesic Sampling1. Fast marching: get

geodesic level curves with sampling interval Δ

2. Sampling along level curves with Δ

p

sparsedense

Δ

ΔΔ

Δ

Δ

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Deformation Invariant Descriptor

p qp q

Geodesic-Intensity Histogram (GIH)

geodesic distance

inte

nsity

geodesic distance

inte

nsity

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Real Example

pq

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Deformation Invariant Framework

Image Embedding ( close to 1)

Deformation Invariant SamplingGeodesic Sampling

Build Deformation Invariant Descriptors(GIH)

),(),( IyxI

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Practical Issues

• Lighting change– Affine lighting model– Normalize the intensity

• Interest-Point– No special interest-point is required– Extreme point (LoG, MSER etc.) is more

reliable and effective

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Invariant vs. Descriminative

0

10

1

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Outline

Introduction

Deformation Invariance for Images

ExperimentsInterest-point matching

Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Data Sets

Synthetic Deformation & Lighting Change (8 pairs) Real Deformation (3 pairs)

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Interest-Points

* Courtesy of Mikolajczyk, http://www.robots.ox.ac.uk/~vgg/research/affine/

Interest-point Matching

• Harris-affine points [Mikolajczyk&Schmid04] *

• Affine invariant support regions• Not required by GIH• 200 points per image

• Ground-truth labeling• Automatically for synthetic image pairs• Manually for real image pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Descriptors and Performance Evaluation

Descriptors• We compared GIH with following descriptors:

Steerable filter [Freeman&Adelson91], SIFT [Lowe04], moments [VanGool&etal96], complex filter [Schaffalitzky&Zisserman02], spin image [Lazebnik&etal05] *

•

Performance Evaluation• ROC curve: detection rate among top N matches. • Detection rate

matches possible#

matchescorrect #r

* Courtesy of Mikolajczyk, http://www.robots.ox.ac.uk/~vgg/research/affine/

98.0

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Synthetic Image Pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Real Image Pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Outline

Introduction

Deformation Invariance for Images

Experiments

Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Conclusion and Future Work

Conclusion A new deformation invariant framework Deformation invariant descriptor (GIH)

Future Work Understanding how to effectively vary α Noise & Occlusion Fast algorithm Real application …

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Acknowledgement

• Krystian Mikolajczyk and Cordelia Schmid for the feature extraction code.

• Paolo Favaro and Kevin S. Zhou for discussion.

• NSF (ITR- 03258670325867).

• The Horvitz Assistantship.

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005

Thank You!