Working with Implicit Surfaces and Point Clouds

Working with Implicit Surfaces and Point Clouds

Guillermo SapiroGuillermo SapiroElectrical and Computer EngineeringElectrical and Computer Engineering

University of MinnesotaUniversity of Minnesotaguille@ece.umn.eduguille@ece.umn.edu

Supported by NSF, ONR, PECASE, CAREER, NIH

OverviewOverview

•• MotivationMotivation

•• Geodesics and generalized geodesicsGeodesics and generalized geodesics

•• Comparing point cloudsComparing point clouds

•• MeshlessMeshless geometric subdivisiongeometric subdivision

•• The future and concluding remarksThe future and concluding remarks

MotivationMotivation

•• Implicit surfacesImplicit surfaces• Facilitate fundamental computations

• Natural representation for many algorithms (e.g., medical imaging)

• Part of the computation very often (distance functions)

•• Point cloudsPoint clouds• Natural representation for 3D scanners

• Natural representation for manifold learning

•• Dimensionality independent Dimensionality independent •• Pure geometry (no artificial meshes, etc)Pure geometry (no artificial meshes, etc)

Geodesics and Generalized GeodesicsGeodesics and Generalized Geodesics

Joint with Joint with FacundoFacundo MemoliMemoli

Motivation: A Few ExamplesMotivation: A Few Examples

Motivation: A Few Examples (cont.)Motivation: A Few Examples (cont.)

noisy cleaned

Motivation: What is a Geodesic?Motivation: What is a Geodesic?

====x

p

CgS dsCgxpd )(inf),(

Motivation: What is a Geodesic?Motivation: What is a Geodesic?

====x

p

CgS dsCgxpd )(inf),(

•• Complexity: Complexity: O(n log n)O(n log n)

•• Advantage:Advantage: Works in any Works in any dimension and with any dimension and with any geometry (graphs)geometry (graphs)

•• Problems:Problems:• Not consistent

• Unorganized points?

• Noise?

• Implicit surfaces?

Background: Distance and Geodesic Computation via DijkstraBackground: Distance and Geodesic Computation via Dijkstra

•• Complexity: Complexity: O(n log n)O(n log n)

•• Advantage:Advantage: Works in any Works in any dimension and with any dimension and with any geometry (graphs)geometry (graphs)

•• Problems:Problems:• Not consistent

• Unorganized points?

• Noise?

• Implicit surfaces?

Background: Distance and Geodesic Computation via DijkstraBackground: Distance and Geodesic Computation via Dijkstra

•• Complexity: Complexity: O(n log n)O(n log n)

•• Advantage:Advantage: Works in any Works in any dimension and with any dimension and with any geometry (graphs)geometry (graphs)

•• Problems:Problems:• Not consistent

• Unorganized points?

• Noise?

• Implicit surfaces?

Background: Distance and Geodesic Computation via DijkstraBackground: Distance and Geodesic Computation via Dijkstra

•• Complexity: Complexity: O(n log n)O(n log n)

•• Advantage:Advantage: Works in any Works in any dimension and with any dimension and with any geometry (graphs)geometry (graphs)

•• Problems:Problems:• Not consistent

• Unorganized points?

• Noise?

• Implicit surfaces?

Background: Distance and Geodesic Computation via DijkstraBackground: Distance and Geodesic Computation via Dijkstra

•• Complexity: Complexity: O(n log n)O(n log n)

•• Advantage:Advantage: Works in any Works in any dimension and with any dimension and with any geometry (graphs)geometry (graphs)

•• Problems:Problems:• Not consistent

• Unorganized points?

• Noise?

• Implicit surfaces?

Background: Distance and Geodesic Computation via DijkstraBackground: Distance and Geodesic Computation via Dijkstra

Background: Distance Functions as Hamilton-Jacobi EquationsBackground: Distance Functions as Hamilton-Jacobi Equations

•• g g = weight on the hyper= weight on the hyper --surfacesurface

•• The The gg--weighted distance function between weighted distance function between two points two points pp and and xx on the hyperon the hyper --surface surface SSis: is:

gxpd gSS ====∇∇∇∇ ),(

gxpd gSS ====∇∇∇∇ ),(

d

Slope = 1

Background: Computing Distance Functions as Hamilton-JacobiEquations

Background: Computing Distance Functions as Hamilton-JacobiEquations•• Solved in O(n log n) by Solved in O(n log n) by TsitsiklisTsitsiklis , by , by

SethianSethian , and by , and by HelmsenHelmsen , , onlyonly for for Euclidean spaces and Cartesian gridsEuclidean spaces and Cartesian grids

•• Solved Solved onlyonly for acute 3D triangulations by for acute 3D triangulations by Kimmel and Kimmel and SethianSethian

gxpdg

====∇∇∇∇ ),(

A real time exampleA real time example

The ProblemThe Problem

•• How to compute intrinsic distances and How to compute intrinsic distances and geodesics forgeodesics for• General dimensions

• Implicit surfaces

• Unorganized noisy points (hyper-surfaces just given by examples)

Our ApproachOur Approach

•• We have to solve We have to solve

gxpd gSS ====∇∇∇∇ ),(

Basic IdeaBasic Idea

Basic IdeaBasic Idea

Basic IdeaBasic Idea

Theorem (Memoli-Sapiro): 0→→→→−−−−g

S

gdd

Basic ideaBasic idea

>>>>

→→→→−−−−

metric smart"" ,

analytic local

general /

1

21

γγγγγγγγh

h

h

dd gS

g

Why is this good?Why is this good?

gxpd gSS ====∇∇∇∇ ),(

gxpdg

====∇∇∇∇ ),(

Implicit Form RepresentationImplicit Form Representation

)(:: ofset levelS 0====ΨΨΨΨ====→→→→ΨΨΨΨ−−−−==== xxRRn

Figure from G. Turk

0====∇∇∇∇⋅⋅⋅⋅∇∇∇∇++++∂∂∂∂∂∂∂∂

)( )(tI Isign

0)(:M =Ψ= xx

Data extensionData extension

•• Embed M:Embed M:

•• Extend I outside M:Extend I outside M:

RM:I →→→→ M II

II

II

ExamplesExamples

ExamplesExamples

ExamplesExamples

ExamplesExamples

ExamplesExamples

Unorganized pointsUnorganized points

Unorganized points (cont.)Unorganized points (cont.)

Unorganized pointsUnorganized points

Randomly sampled manifolds(with noise)Randomly sampled manifolds(with noise)

Theorem (Memoli - S. 2002):

Examples (VRML)Examples (VRML)

ExamplesExamples

Intrinsic Voronoi of Point CloudsIntrinsic Voronoi of Point Clouds

Intermezzo: de Silva, Tenenbaum, et al...Intermezzo: de Silva, Tenenbaum, et al...

Intermezzo: Tenenbaum, de Silva, et al...Intermezzo: Tenenbaum, de Silva, et al...

•• Main Problem:Main Problem:• Doesn’t address noisy examples/measurements:

Much less robust to noise!

Error increases with the number of samples!

Intermezzo: de Silva, Tenenbaum, et al...Intermezzo: de Silva, Tenenbaum, et al...

•• Problems:Problems:• Doesn’t address noisy examples/measurements:

Much less robust to noise!

• Only convex surfaces

• Uses Dijkestra (back to non consistency)

• Doesn’t work for implicit surface representations

Is this a geodesic?Is this a geodesic?

noisy cleaned

Generalized geodesics: Harmonic mapsGeneralized geodesics: Harmonic maps

•• Find a smooth map from two manifolds Find a smooth map from two manifolds (M,g)(M,g) and and (N,h) (N,h) such thatsuch that

M

p

MNMC dvolC min : ΩΩΩΩ

→→→→ ∇∇∇∇

0>=>=>=>=∇∇∇∇∇∇∇∇<<<<++++∆∆∆∆

==== CCCACtC

MMNM ,)(

∂∂∂∂∂∂∂∂

ExamplesExamples

•• M is an Euclidean space and N the real lineM is an Euclidean space and N the real line

•• M = [0,1], M = [0,1], geodesics!geodesics!

0====∆∆∆∆ C

02

2

>=>=>=>=∇∇∇∇∇∇∇∇<<<<++++∂∂∂∂∂∂∂∂

CCCAt

CMMN ,)(

Color Image Enhancement (with B. Tang and V. Caselles)Color Image Enhancement (with B. Tang and V. Caselles)

Implicit surfacesImplicit surfaces

•• Domain and target are implicitly Domain and target are implicitly represented: Simple Cartesian represented: Simple Cartesian numericsnumerics

(((( )))) xC

,xC

HCPdivtC

k kk

ΦΦΦΦ∇∇∇∇

∂∂∂∂∂∂∂∂

∂∂∂∂∂∂∂∂++++∇∇∇∇====

∂∂∂∂∂∂∂∂

ΦΦΦΦ∇∇∇∇

Example: Chroma denoising on a surface (with Bertalmio, Cheng, Osher)

Example: Chroma denoising on a surface (with Bertalmio, Cheng, Osher)

original

noisy enhanced

Example: Direction denoising(with Bertalmio, Cheng, Osher)

Example: Direction denoising(with Bertalmio, Cheng, Osher)

noisy cleaned

Application (with G. Gorla and V. Interrante)Application (with G. Gorla and V. Interrante)

Texture mapping denoisingTexture mapping denoising

Texture mapping denoisingTexture mapping denoising

Examples(with Betalmio, Cheng, Osher)Examples(with Betalmio, Cheng, Osher)

Vector field visualization (e.g., principal directions)(with Bertalmio, Cheng, Osher)

Vector field visualization (e.g., principal directions)(with Bertalmio, Cheng, Osher)

Concluding remarksConcluding remarks

•• A general computational framework for A general computational framework for distance functions, geodesics, and distance functions, geodesics, and generalized geodesicsgeneralized geodesics

•• Implicit hyperImplicit hyper --surfaces and unsurfaces and un --organized organized pointspoints

Comparing Point CloudsComparing Point Clouds

Joint with Joint with FacundoFacundo MemoliMemoli

What is and MotivationWhat is and Motivation

•• Comparing point Comparing point cloudsclouds• Dimension independent

• Geometric

• Bending (isometric) invariant

• Supported by theory and computational framework

The Gromov-Hausdorff DistanceThe Gromov-Hausdorff Distance

•• HausdorffHausdorff distancedistance

•• GromovGromov --HausdorffHausdorff distancedistance

Key questionKey question

•• How to estimate the How to estimate the GromovGromov --HousdorffHousdorffdistance from noisy distance from noisy samples of the metric samples of the metric spacespace

First step: Working with point cloudsFirst step: Working with point clouds

•• Consequence:Consequence: Working with point clouds Working with point clouds “is possible”“is possible”

How we compute the distance?How we compute the distance?

•• Consequence:Consequence: If we see a small If we see a small pairwisepairwisedistance, the objects are isometricdistance, the objects are isometric

The need for a probabilistic frameworkThe need for a probabilistic framework

The need for a probabilistic framework (cont.)The need for a probabilistic framework (cont.)

•• The problem is well posedThe problem is well posed

•• No reason for the No reason for the yy’s’s to be given:to be given:

•• We need probabilistic bounds!We need probabilistic bounds!

The probabilistic frameworkThe probabilistic framework

•• Bottleneck distanceBottleneck distance between two samples of the same between two samples of the same space:space:

•• Using concepts from intrinsic Using concepts from intrinsic VoronoiVoronoi diagrams and diagrams and coupon collector theorem we have:coupon collector theorem we have:

The probabilistic framework (cont.)The probabilistic framework (cont.)

The probabilistic framework (cont.)The probabilistic framework (cont.)

Computational considerationsComputational considerations

•• Bounds on the number of sample points Bounds on the number of sample points neededneeded

•• Covers of Y found using Covers of Y found using farthest point farthest point sampling.sampling.

•• Geodesic distances for points on X and YGeodesic distances for points on X and Y

•• Select matching points of X and Y Select matching points of X and Y following our theoryfollowing our theory

ExamplesExamples

Meshless Geometric SubdivisionMeshless Geometric Subdivision

Joint with Joint with

CarstenCarsten MoenningMoenningFacundoFacundo MemoliMemoliNiraNira DynDynN. N. DodgsonDodgson

What is and MotivationWhat is and Motivation

•• Mesh based subdivisionMesh based subdivision• Refinement (add points and edges)

• Averaging

•• Mesh not really geometricMesh not really geometric

What is and Motivation (cont.)What is and Motivation (cont.)

•• Point clouds are natural for 3D scannersPoint clouds are natural for 3D scanners•• Point clouds are the “true” geometryPoint clouds are the “true” geometry•• Point clouds are dimensionality independentPoint clouds are dimensionality independent•• All operations are geometricAll operations are geometric

Main StepsMain Steps

•• Intrinsic point cloud simplificationIntrinsic point cloud simplification

•• Intrinsic proximity informationIntrinsic proximity information

•• Geodesic Geodesic centroidcentroid computationcomputation

•• Intrinsic subdivision schemeIntrinsic subdivision scheme

Intrinsic point cloud simplificationIntrinsic point cloud simplification

•• Follows Follows MeonningMeonning & & DodgsonDodgson

•• Based on progressive Based on progressive farthest point samplingfarthest point sampling

•• Computed based on Computed based on intrinsic intrinsic VoronoiVoronoidiagram (uses distance diagram (uses distance on point clouds)on point clouds)

•• Guaranteed bounds on Guaranteed bounds on distance between distance between samplessamples

Intrinsic proximity informationIntrinsic proximity information

•• ““ Replaces” (nonReplaces” (non --geometric) connectivity in geometric) connectivity in mesh techniquesmesh techniques

•• Given by neighbors from Given by neighbors from the intrinsic the intrinsic VoronoiVoronoi

•• Easily updated when the Easily updated when the point cloud is refined point cloud is refined (using geodesics on point (using geodesics on point clouds)clouds)

Geodesic centroid computationGeodesic centroid computation

Geodesic centroid computation (cont.)Geodesic centroid computation (cont.)

Intrinsic subdivision schemeIntrinsic subdivision scheme

•• Geometric averaging rule:Geometric averaging rule: Replace the Replace the point by the geodesic point by the geodesic centroidcentroid of its of its intrinsic neighborhoodintrinsic neighborhood

•• Refinement rule: Refinement rule: For each neighbor, insert For each neighbor, insert the geodesic the geodesic centroidcentroid of the joint of the joint neighborhoodneighborhood

ExampleExample

Quantitative study in the paper

ExampleExample

ConclusionsConclusions

•• Work with implicit surfaces and point clouds!!!Work with implicit surfaces and point clouds!!!

ThanksThanks

F. F. MemoliMemoli and G. Sapiro, “Fast computation of weighted distanc e functionsand G. Sapiro, “Fast computation of weighted distanc e functions and geodesics on implicit hyperand geodesics on implicit hyper --surfaces,” surfaces,” Journal of Computational PhysicsJournal of Computational Physics 173:2, pp. 730173:2, pp. 730 --764, November 2001.764, November 2001.

B. TangB. Tang , G. Sapiro, and V. , G. Sapiro, and V. CasellesCaselles , “Color image enhancement via chromaticity diffusio n,” , “Color image enhancement via chromaticity diffusio n,” IEEE Trans. Image IEEE Trans. Image Processing Processing 10, pp. 70110, pp. 701 --707, May 2001.707, May 2001.

B. TangB. Tang , G. Sapiro, and V. , G. Sapiro, and V. CasellesCaselles , “Diffusion of general data on non, “Diffusion of general data on non --flat manifolds via harmonic maps theory: The flat manifolds via harmonic maps theory: The direction diffusion case,” direction diffusion case,” Int. Journal Computer VisionInt. Journal Computer Vision 36:2, pp. 14936:2, pp. 149 --161, February 2000.161, February 2000.

M. M. BertalmioBertalmio , L. T. Cheng, L. T. Cheng , S. , S. OsherOsher , and G. Sapiro, “, and G. Sapiro, “ VariationalVariational problems and partial differential equations on impl icit problems and partial differential equations on impl icit surfaces,’’ surfaces,’’ Journal of Computational PhysicsJournal of Computational Physics 174:2, pp. 759174:2, pp. 759 --780, 2001.780, 2001.

A. A. BartesaghiBartesaghi and G. Sapiro, “A system for the generation of curve s on 3D braand G. Sapiro, “A system for the generation of curve s on 3D bra in images,” in images,” Human Brain MappingHuman Brain Mapping 14:1, 14:1, pp. 1pp. 1 --15, 2001.15, 2001.

F. F. MemoliMemoli , G. Sapiro, and S. , G. Sapiro, and S. OsherOsher , “Solving , “Solving PDEsPDEs and and variationalvariational problems onto general target surfaces,” problems onto general target surfaces,” Journal of Journal of Computational Physics, 2004.Computational Physics, 2004.

F. F. MemoliMemoli , G. Sapiro, and P. Thompson, “Implicit brain imagin g,” , G. Sapiro, and P. Thompson, “Implicit brain imagin g,” Brain Imaging, Brain Imaging, to appear.to appear.

F. F. MemoliMemoli and G. Sapiro, “Comparing point clouds,” IMA TR, Apri l 2004 (and G. Sapiro, “Comparing point clouds,” IMA TR, Apri l 2004 (www.ima.umn.eduwww.ima.umn.edu ))

F. F. MemoliMemoli and G. Sapiro, “Geodesics on point clouds,” IMA TR, Dec. 2002/and G. Sapiro, “Geodesics on point clouds,” IMA TR, Dec. 2002/April 2003 (April 2003 ( www.ima.umn.eduwww.ima.umn.edu ))

C. C. MoenningMoenning , F. , F. MemoliMemoli , et al., “, et al., “ MeshelessMesheless geometric subdivision,” IMA TR, April 2004 (geometric subdivision,” IMA TR, April 2004 ( www.ima.umn.eduwww.ima.umn.edu ))

Sulcii extraction on meshes(with A. Bartesaghi)

Sulcii extraction on meshes(with A. Bartesaghi)

Follows Kimmel-Sethian