Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors

Rotation Invariant Spherical Harmonic Representation of

3D Shape DescriptorsMichael KazhdanMichael Kazhdan

ThThoomas Funkhousermas FunkhouserSzymon RusinkiewiczSzymon RusinkiewiczPrinceton UniversityPrinceton University

Motivation

Large databases of 3D modelsLarge databases of 3D models

Mechanical CAD(National Design Repository)

Molecular Biology(Audrey Sanderson)

Computer Graphics(Princeton 3D Search Engine)

Retrieval Approach

3D Model ShapeDescriptor

Model Database

Nearest Neighbor

Shape Unchanged by Rotation

=

Problem

Many shape descriptors are functions that Many shape descriptors are functions that rotate with the shaperotate with the shape

Extended Gaussian Image[Horn ’84]

Spherical Attribute Image[Ikeuchi ’95]

Shape Histogram [Ankerst ’99]

Spherical Extent Function[Vranic ’00]

Reflective Symmetry Descriptor [Kazhdan ’02]

Gaussian EDT[Funkhouser ’03]

Goal

Compute similarity of shape descriptors Compute similarity of shape descriptors independent of rotationindependent of rotation

- = ?

Brute Force Approach

-

-

-

-

min (rotation)- =

Impractical for databasesImpractical for databases

Normalization

Use PCA to place models into a canonical Use PCA to place models into a canonical coordinate framecoordinate frame

Covariance MatrixComputation

Principal Axis Alignment

Normalization

Doesn’t always workDoesn’t always work• Only second order informationOnly second order information

Our Approach

Eliminate rotation dependence in spherical Eliminate rotation dependence in spherical and 3D descriptorsand 3D descriptors

Shape Descriptor

EGI [Horn ’84]SAI [Ikeuchi ’95]EXT [Vranic ’00]RSD [Kazhdan ’02]EDT [Funkhouser ’03]etc.

Shape Descriptor

Our Approach

Eliminate rotation dependence in spherical Eliminate rotation dependence in spherical and 3D descriptorsand 3D descriptors

Rotation Invariant

Representation

Shape Descriptor

Outline

IntroductionIntroductionBackgroundBackgroundHarmonic RepresentationHarmonic RepresentationPropertiesPropertiesExperimental ResultsExperimental ResultsConclusion and Future WorkConclusion and Future Work

Key Idea

Obtain rotation invariant representation by Obtain rotation invariant representation by storing amplitude and eliminating phasestoring amplitude and eliminating phase

+ + + +…=

[Lo 1989]

[Burel 1995]

Fourier Descriptors

CircularFunction

Fourier Descriptors

+ + += + …

Cosine/Sine Decomposition

CircularFunction

Fourier Descriptors

=

+ + +

Constant

= + …

Frequency Decomposition

CircularFunction

Fourier Descriptors

=

+ + +

+

Constant 1st Order

= + …+


CircularFunction

Fourier Descriptors

=

+ + +

+ +

Constant 1st Order 2nd Order

= + …+


CircularFunction

Fourier Descriptors

=

+ + +

+ + +

Constant 1st Order 2nd Order 3rd Order

= + …

+ …

+


CircularFunction

+ + + + …+

Fourier Descriptors

= + + +


+ …


=Amplitudes invariant

to rotationCircularFunction

Harmonic Representation

SphericalFunction


=

SphericalFunction

+ + + + …

Harmonic Decomposition


=

SphericalFunction

+ + + + …

+ + + +…=



+ + + +…=

Norms Invariantto Rotation

Store “how much” Store “how much” (L(L22-norm) of the -norm) of the shape resides in shape resides in each frequencyeach frequency

3D Function (Voxel Grid)

Restrict to concentric spheresRestrict to concentric spheres


=

=

=

+

+

+ +

+

+ +

+

+

+

+

+

Compute harmonic representation of each Compute harmonic representation of each sphere independentlysphere independently


Combine harmonic representationsCombine harmonic representations

Radius

Frequency

Matching

LL22-difference of harmonic representations…-difference of harmonic representations…

Harmonic Representation Harmonic Representation

-2

Matching

min (rotations)

--2 2

… … bounds proximity of descriptors over all bounds proximity of descriptors over all rotationsrotations

Outline


Advantages

The harmonic representations is:The harmonic representations is:•Rotation invariantRotation invariant•Multi-resolutionMulti-resolution•CompactCompact•DiscriminatingDiscriminating

Compact

…

……

Compact

……

……

Information Loss

•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss•Cross-radial information lossCross-radial information loss

Information Loss (Spherical Descriptor)

•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss

Information Loss (Spherical Descriptor)

•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss

+

+

22.5o90o

=

=

Information Loss (3D Descriptor)

•Cross-radial information lossCross-radial information loss

Outline


Shape DescriptorsExtended Gaussian Image

Horn 1984

Spherical Extent Function Vranic 2000

Shape Histogram Ankerst 1999

Gaussian EDT Funkhouser 2003

Experimental Database

Viewpoint “household” databaseViewpoint “household” database1,890 models, 85 classes1,890 models, 85 classes

153 dining chairs 25 livingroom chairs 16 beds 12 dining tables

8 chests 28 bottles 39 vases 36 end tables

Gaussian EDT ResultsPCA-Normalized Results

Harmonic Representation Results

Query

11 22 33 44 55

66 77 88 99 1010

11 22 33 44 55

66 77 88 99 1010

Gaussian EDT Results

Precision vs. RecallPrecision vs. Recall

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

HarmonicsPCA

Retrieval ResultsHarmonicsPCA

HarmonicsPCA

HarmonicsPCA

HarmonicsPCA

EGI

EDTEXT

SECT

50% 100%

100%

0%

50%

RecallPr

ecis

ion

0%

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

•EGI: Extended Gaussian Image

•SECT: Shape Histogram (Sectors)

•EXT: Spherical Extent Function

•EDT: Gaussian Euclidean Distance Transform

Retrieval ResultsHarmonicsPCA

EGIHarmonicsPCA

SECT

50% 100%

100%

0%

50%

RecallPr

ecis

ion

0%

HarmonicsPCA

EDT

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

HarmonicsPCA

EXT

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

50% 100%

100%

0%

50%

Recall

Prec

isio

n

0%

•EGI: Extended Gaussian Image

•SECT: Shape Histogram (Sectors)

•EXT: Spherical Extent Function

•EDT: Gaussian Euclidean Distance Transform

Exhaustive Gaussian EDT Results

HarmonicPCA

min L2

100%

50%

0%0% 50% 100%

Recall

Prec

isio

n

Gaussian EDT -

-

-

-

min (rotation)

Summary and Conclusion

Provide a rotation invariant representation of Provide a rotation invariant representation of shape descriptors that:shape descriptors that:• Eliminates PCA dependenceEliminates PCA dependence• Gives better matching performanceGives better matching performance• Is more compactIs more compact• Is a multi-resolution representationIs a multi-resolution representation

Future Work

Managing Information LossManaging Information Loss• Obtain cross radial information for 3D descriptorsObtain cross radial information for 3D descriptors• Obtain cross frequency informationObtain cross frequency information• Get finer resolution of rotation invariance within Get finer resolution of rotation invariance within

frequenciesfrequencies

More GenerallyMore Generally• Consider new shape descriptorsConsider new shape descriptors

Thank YouFundingFunding

National Science FoundationNational Science Foundation

Sloan FoundationSloan Foundation

Spherical HarmonicsSpherical HarmonicsDan Rockmore and Peter KostelecDan Rockmore and Peter Kostelec

SpharmonicKit:SpharmonicKit:

http://www.cs.dartmouth.edu/~geelong/spherehttp://www.cs.dartmouth.edu/~geelong/sphere3D Shape Matching3D Shape Matching

Patrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce ChenPatrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce Chen

Princeton 3D Model Search Engine:Princeton 3D Model Search Engine:

http://shape.cs.princeton.eduhttp://shape.cs.princeton.edu

Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors

Documents

Transcript of Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors