1Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Invariant shape similarity

© Alexander & Michael Bronstein, 2006-2009© Michael Bronstein, 2010tosca.cs.technion.ac.il/book

048921 Advanced topics in visionProcessing and Analysis of Geometric Shapes

EE Technion, Spring 2010

2Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Invariant similarity

SIMILARITY

TRANSFORMATION

3Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

Equal

Congruent Isometric

4Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

Equivalence is a binary relation on the space of shapes

which

for all satisfies

Reflexivity:

Symmetry:

Transitivity: Can be expressed as a binary function

if and only if

Quotient space is the space of equivalence classes

5Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

6Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

All deformations of the

human shape are “the same”

7Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Similarity

Shapes are rarely truly equivalent (e.g., due to acquisition noise or

since

most shapes are rigid)

We want to account for “almost equivalence” or similarity

-similar = -isometric (w.r.t. some metric)

Define a distance on the shape space quantifying the degree of

dissimilarity of shapes

8Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Similarity

A monkey shape is

more similar to a

deformation of a

monkey shape…

…than to a

human shape

9Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Isometry-invariant distance

Non-negative function satisfying for all

Corollary: is a metric on the quotient space

Similarity: and are -isometric;

and are -isometric

(In particular, if and only if )

Symmetry:

Triangle inequality:

Given discretized shapes and

sampled with radius

Consistency to sampling:

10Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Canonical forms distance

Compute Hausdorff distance over all isometries in

Minimum-distortion

embedding

Minimum-distortion

embedding

No fixed embedding space will give distortion-less canonical forms

11Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Gromov-Hausdorff distance

Isometric

embedding

Isometric

embedding

Mikhail Gromov

Gromov-Hausdorff distance: include

into minimization

12Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Properties of Gromov-Hausdorff distance

Metric on the quotient space of isometries of shapes

Similarity: and are -isometric;

and are -isometric

Consistent to sampling: given discretized shapes

and sampled with radius

Generalization of Hausdorff distance:

Hausdorff distance between subsets of a metric space

Gromov-Hausdorff distance between metric spacesGromov, 1981

13Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition I (metric coupling)

is the disjoint union of and

the (semi-) metric satisfies and

where

Mémoli, 2008

14Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Optimization over translates into finding the values of

Mémoli, 2008

A lot of constraints!

Alternative definition I (metric coupling)

15Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence

A subset is called a correspondence between and

if for every there exists at least one such that

and similarly for every there exists such that

Particular case: given and

16Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence distortion

The distortion of correspondence is defined as

17Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

1. Show that for any there exists with

Proof sketch

Since , by definition of , and are subspaces of

some such that

Let

By triangle inequality, for

18Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

2. Show that for any

Let

It is sufficient to show that there is a (semi-)metric on the disjoint union

such that , , and

Construct the metric as follows

(in particular, for ).

19Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

First,

For each

Since for ,

Second, we need to show that is a (semi-)metric on

On and , it is straightforward

We only need to show metric properties hold on

20Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition III

measures how much is distorted by when embedded into

21Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

measures how much is distorted by when embedded into

Alternative definition III

22Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

measures how far is from being the inverse of

Alternative definition III

23Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Generalized MDS

A. Bronstein, M. Bronstein & R. Kimmel, 2006

24Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Difficulties

How to represent points on ?

Global parametrization is not always available.

Some local representation is required in general case.

No more closed-form expression for .

Metric needs to be approximated.

Minimization algorithm.

25Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Local representation

is sampled at and represented as a

triangular mesh .

Any point falls into one of the triangles .

Within the triangle, it can be represented as convex combination

of triangle vertices ,

Barycentric coordinates .

We will need to handle discrete indices in minimization algorithm.

26Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distances

Distance terms can be precomputed, since are

fixed.

How to compute distance terms ?

No more closed-form expression.

Cannot be precomputed, since are minimization variables.

can fall anywhere on the mesh.

Precompute for all .

Approximate

for any .

27Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

Approximation from .

First order accurate:

Consistent with data:

Symmetric:

Smoothness: is and a closed-form expression

for its derivatives is available to minimization algorithm.

Might be only at some points or along some lines.

Efficiently computed: constant complexity independent of .

28Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

Compute for .

falls into triangle and is represented as

Particular case:

Hence, we can precompute distances

How to compute from ?

29Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

We have already encountered this problem in fast marching.

Wavefront arrives at triangle vertex at time .

When does it arrive to ?

Adopt planar wavefront model.

Distance map is linear in the triangle (hence, linear in )

Solve for coefficients and obtain a linear interpolant

30Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

General case: falls into triangle and is

represented

as

Apply previous steps in triangle to obtain

Apply once again in triangle to obtain

31Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

A four-step dance

32Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Minimization algorithm

How to minimize the generalized stress?

Particular case: L2 stress

Fix all and all except for some .

Stress as a function of only becomes quadratic

33Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Quadratic stress

Quadratic stress

is positive semi-definite.

is convex in (but not necessarily in

together).

34Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Quadratic stress

Closed-form solution for minimizer of

Problem: solution might be outside the

triangle.

Solution: find constrained minimizer

Closed-form solution still exists.

35Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Minimization algorithm

Initialize

For each

Fix and compute gradient

Select corresponding to maximum .

Compute minimizer

If constraints are active

translate to adjacent triangle.

Iterate until convergence…

36Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

How to move to adjacent triangles?

Three cases

All : inside triangle.

: on edge opposite to .

: on vertex .

inside on edge on vertex

37Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Point on edge

on edge opposite to .

If edge is not shared by any other triangle

we are on the boundary – no translation.

Otherwise, express the point as in triangle .

contains same values as .

May be permuted due to different vertex

ordering in .

Complication: is not on the edge.

Evaluate gradient in .

If points inside triangle, update to .

38Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Point on vertex

on vertex .

For each triangle sharing vertex

Express point as in .

Evaluate gradient in .

Reject triangles with pointing

outside.

Select triangle with maximum

.

Update to .

39Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

MDS vs GMDS

Stress

Generalized MDSMDS

Generalized stress

Analytic expression for

Nonconvex problem

Variables: Euclidean

coordinates

of the points

must be interpolated

Nonconvex problem

Variables: points on in

barycentric coordinates

40Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution

Stress is non convex – many small local minima.

Straightforward minimization gives poor results.

How to initialize GMDS?

Multiresolution:

Create a hierarchy of grids in ,

Each grid comprises

Sampling:

Geodesic distance matrix:

41Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution

Initialize at the coarsest resolution in .

For

Starting at initialization , solve the GMDS problem

Interpolate solution to next resolution level

Return .

42Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

GMDS

Interpolation

GMDS

43Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution encore

So far, we created a hierarchy of embedded spaces .

One step further: create a hierarchy of embedding spaces

.

44Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Labeling problem

Torresani, Kolmogorov, Rother 2008Wang, B 2010

Build a graph with vertices and edges

Label each vertex

Minimum distortion correspondence = graph labeling problem

Efficient solvers with good global convergence properties

Complexity:

Hierarchical solution complexity can be lowered to

45Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

MATLAB® intermezzoGMDS

46Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Discrete Gromov-Hausdorff distance

Two coupled GMDS problems

Can be cast as a constrained problem

Bronstein, Bronstein & Kimmel, 2006

47Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Numerical example

Canonical forms (MDS based on 500 points)

Gromov-Hausdorff distance(GMDS based on 50 points)

Bronstein, Bronstein & Kimmel, 2006

48Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Extrinsic similarity using Gromov-Hausdorff distance

Mémoli (2008)

Connection between Euclidean GH and ICP distances:

Congruence Euclidean isometry

EXTRINSIC SIMILARITY

ICP distance: GH distance with Euclidean

metric:

Mémoli, 2008

49Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Connection to canonical form distance

50Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence again

A different representation for correspondence using indicator functions

defines a valid correspondence if

51Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Lp Gromov-Hausdorff distance

We can give an alternative formulation of the Gromov-Hausdorff distance

Can we define an Lp version of the Gromov-Hausdorff distance by

relaxing the above definition?

52Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Measure coupling

Let be probability measures defined on and

The measure can be considered as a relaxed version of the indicator

function or as fuzzy correspondence

A measure on is a coupling of and if

for all measurable sets

Mémoli, 2007

(a metric space with measure is called a metric measure or mm space)

53Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Gromov-Wasserstein distance

The relaxed version of the Gromov-Hausdorff distance is given by

and is referred to as Gromov-Wasserstein distance [Memoli 2007]

Mémoli, 2007

54Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Earth Mover’s distance

Let be a metric space, and measures supported

on

EMD as optimal mass transport:

mass transported from to

distance traveled

Mémoli, 2007

The Wasserstein or Earth Mover’s distance (EMD) is given by

Define the coupling of

55Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

The analogy

Hausdorff

Mémoli, 2007

Distance between subsets

of a metric space .

Gromov-Hausdorff

Distance between metric spaces

Wasserstein

Distance between subsets

of a metric measure

space .

Gromov-Wasserstein

Distance between metric

measure spaces