EFFICIENT VARIANTS OF THE ICP ALGORITHM Szymon Rusinkiewicz Marc Levoy.

EFFICIENT VARIANTS OF EFFICIENT VARIANTS OF THE ICP ALGORITHMTHE ICP ALGORITHM

Szymon RusinkiewiczSzymon Rusinkiewicz

Marc LevoyMarc Levoy

IntroductionIntroduction

Problem of aligning Problem of aligning 3D models, based on 3D models, based on geometry or color of geometry or color of meshesmeshes

ICP is the chief ICP is the chief algorithm usedalgorithm used

Used to register Used to register output of 3D scannersoutput of 3D scanners

[1]

ICPICP

Starting point: Two meshes and an initial Starting point: Two meshes and an initial guess for a relative rigid-body transformguess for a relative rigid-body transform

Iteratively refines the transformIteratively refines the transform

Generates pairs of corresponding points Generates pairs of corresponding points on the meshon the mesh

Minimizes an error metricMinimizes an error metric

RepeatsRepeats

Initial alignmentInitial alignment

Tracking scanner position…Tracking scanner position…

Indexing surface features…Indexing surface features…

Spin image signatures…Spin image signatures…

Exhaustive search…Exhaustive search…

User Input……User Input……

[2]

ConstraintsConstraints

Assume a rough initial alignment is Assume a rough initial alignment is availableavailable

Focus only on a single of meshesFocus only on a single of meshes

Global registration problem not addressedGlobal registration problem not addressed

Stages of the ICPStages of the ICP

SelectionSelection of the set of points of the set of points

MatchingMatching the points to the samples the points to the samples

WeightingWeighting corresponding pairs corresponding pairs

RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers

AssigningAssigning an error metric an error metric

MinimizingMinimizing the error metric the error metric

FocusFocus

SpeedSpeedAccuracyAccuracyPerformance in tough scenesPerformance in tough scenesIntroducing test scenesIntroducing test scenesDiscuss combinationsDiscuss combinationsNormal-space directed samplingNormal-space directed samplingConvergence performanceConvergence performanceOptimal combinationOptimal combination

Comparison Methodology Comparison Methodology

Baseline Algorithm: [Pulli 99]Baseline Algorithm: [Pulli 99]1.1. Random sampling on both meshesRandom sampling on both meshes2.2. Matching to a point where the Matching to a point where the normal is < 45 degrees from the sourcenormal is < 45 degrees from the source3.3. Uniform weightingUniform weighting4.4. Rejection of edge vertices pairsRejection of edge vertices pairs5.5. Point-to-plane error metricPoint-to-plane error metric6.6. “ “Select-match-minimize” iterationSelect-match-minimize” iteration

AssumptionsAssumptions

2000 source points and100,000 samples2000 source points and100,000 samples

Simple perspective range images Simple perspective range images

Surface normal is based on the four Surface normal is based on the four nearest neighborsnearest neighbors

Only geometry (color, intensity excluded)Only geometry (color, intensity excluded)

Test ScenesTest Scenes

a) Wave Scene

b) Fractal Landscape

c) Incised Plane

Shamelessly stolen from [3]

Sample scanning application

Representative of different kinds of surfaces

• Low frequency

• All frequency

• High Frequency

Smooth statues

Unfinished statues

Fragments

More shameless lifts from [3]

Comparison StagesComparison Stages







Selection of point pairsSelection of point pairs

Use all available pointsUse all available points

Uniform sub-samplingUniform sub-sampling

Random samplingRandom sampling

Pick points with high intensity gradientPick points with high intensity gradient

Pick from one or both meshesPick from one or both meshes

Select points where the distribution of the Select points where the distribution of the normal between these points is as large as normal between these points is as large as possiblepossible

Normal SamplingNormal Sampling

Small features may play a critical roleSmall features may play a critical role

Distribute the spread of the points across Distribute the spread of the points across the position of the normalsthe position of the normals

SimpleSimple Low-costLow-costLow robustnessLow robustness

Comparison of performanceComparison of performance

• Uniform sub-sampling

• Random sampling

• normal-space sampling

Comparison of performanceComparison of performance

Incised Plane: Only the normal-space sampling converges

Why?Why?Samples outside Samples outside the grooves: 1 the grooves: 1 translation, 2 translation, 2 rotationsrotationsInside the grooves: Inside the grooves: 2 translations, 1 2 translations, 1 rotationrotationFewer samples + Fewer samples + noise + distortionnoise + distortion= bad results= bad results

Sampling DirectionSampling DirectionPoints from one mesh vs. points from both Points from one mesh vs. points from both meshesmeshesDifference is minimal, as algorithm is Difference is minimal, as algorithm is symmetricsymmetric

Sampling directionSampling direction

Asymmetric algorithmAsymmetric algorithm

Two meshes is betterTwo meshes is better

If overlap is small, two If overlap is small, two meshes is bettermeshes is better

Matching PointsMatching Points

Match a sample point with the closest in Match a sample point with the closest in the other meshthe other mesh

Normal shootingNormal shooting

Reverse calibrationReverse calibration

Project source point onto destination Project source point onto destination mesh; search in destination range imagemesh; search in destination range image

Match points compatible with source Match points compatible with source points points

Variants comparedVariants compared

Closest pointClosest point Closest compatible pointClosest compatible point Normal shootingNormal shooting Normal shooting to a Normal shooting to a compatible pointcompatible point ProjectionProjection Projection followed by a search : uses Projection followed by a search : uses steepest-descent neighbor-neighbor walksteepest-descent neighbor-neighbor walk

k-d tree

Fractal SceneFractal Scene

Best: normal shooting

Worst: closest-point

Incised PlaneIncised Plane

Closest point converges: most robust

ErrorError

Error as a function Error as a function of running timeof running time

Applications that Applications that need quick running need quick running of the ICP should of the ICP should choose algorithms choose algorithms with the fastest with the fastest performanceperformance

Best: Projection algorithm

AlgorithmsAlgorithms

Constant weightConstant weight

Lower weights for points with higher point-Lower weights for points with higher point-point distancespoint distances

Weight = 1 – [Dist(pWeight = 1 – [Dist(p11, p, p22)/Dist )/Dist maxmax]]

Weight based on normal compatibilityWeight based on normal compatibility

Weight = nWeight = n11* n* n22

Weight based on the effect of noise on Weight based on the effect of noise on uncertaintyuncertainty

Wave SceneWave Scene

Rejecting PairsRejecting Pairs

Pairs of points more than a given distance Pairs of points more than a given distance apartapart

Worst Worst n%n% pairs, based on a metric (n=10) pairs, based on a metric (n=10)

Pairs whose point-point distance is > Pairs whose point-point distance is > multiple multiple mm of the standard deviation of of the standard deviation of distances (m = 2.5)distances (m = 2.5)

Rejecting PairsRejecting Pairs

Pairs that are not consistent with neighboring Pairs that are not consistent with neighboring pairspairs

Two pairs are inconsistent iffTwo pairs are inconsistent iff

| Dist(p| Dist(p11,p,p22) – Dist(q) – Dist(q11,q,q22) |) |

Threshold: Threshold:

0.1 * max(Dist(p0.1 * max(Dist(p11,p,p22) – Dist(q) – Dist(q11,q,q22) )) )

Pairs containing points on mesh boundariesPairs containing points on mesh boundaries

Points on mesh boundariesPoints on mesh boundaries

• Incomplete overlap:

Low cost

Fewer disadvantages

Rejection on the wave sceneRejection on the wave scene

•Rejection of outliers does not help with initial convergence

•Does not improve convergence speed

Error metricsError metrics

Sum of squared distances between Sum of squared distances between corresponding pointscorresponding points

1) SVD1) SVD

2) Quaternions2) Quaternions

3) Orthonormal Matrices3) Orthonormal Matrices

4) Dual Quaternions4) Dual Quaternions

Error metricsError metrics

Point-to-point metric, taking into account Point-to-point metric, taking into account distance and color differencedistance and color difference

Point-to-plane methodPoint-to-plane method

The least-squares equations can be The least-squares equations can be solved either by using a non-linear method solved either by using a non-linear method or by linearizing the problemor by linearizing the problem

Search for the alignmentSearch for the alignment

Generate a set of points Generate a set of points Find a new transformation that minimizes the Find a new transformation that minimizes the error metricerror metricCombine with extrapolationCombine with extrapolationIterative minimization, with perturbations initially, Iterative minimization, with perturbations initially, then selecting the best resultthen selecting the best resultUse random subsets of points, select the optimal Use random subsets of points, select the optimal using a robust metricusing a robust metricUse simulated annealing and perform a Use simulated annealing and perform a stochastic search for the best transformstochastic search for the best transform

Extrapolation algorithmExtrapolation algorithm

Besl and McKay’s algorithmBesl and McKay’s algorithm

For a downward parabola, the largest x-For a downward parabola, the largest x-intercept is usedintercept is used

The extrapolation is multiplied by a The extrapolation is multiplied by a dampening factordampening factor

Increases stabilityIncreases stability

Reduces overshootReduces overshoot

Fractal SceneFractal Scene

Best: Point-to-plane error metric


Point-to-point cannot reach the right solution

High-Speed VariantsHigh-Speed Variants

Applications of ICP in real time:Applications of ICP in real time:

1) Involving a user in a scanning process 1) Involving a user in a scanning process for alignmentfor alignment

““Next-best-view” problemNext-best-view” problem

““Given a set of range images, to determine Given a set of range images, to determine the position/orientation of the range scanner to the position/orientation of the range scanner to scan all visible surfaces of an unknown scenescan all visible surfaces of an unknown scene” ” [4][4]

2) Model-based tracking of a rigid object2) Model-based tracking of a rigid object

Optimal AlgorithmOptimal Algorithm

Projection-based algorithm to generate Projection-based algorithm to generate point correspondencespoint correspondencesPoint-to-plane error metricPoint-to-plane error metric““Select-match-minimize” ICP iterationSelect-match-minimize” ICP iterationRandom samplingRandom samplingConstant weightingConstant weightingDistance threshold for pair rejectionDistance threshold for pair rejectionNo extrapolation of transforms (Overshoot)No extrapolation of transforms (Overshoot)

Optimal ImplementationOptimal Implementation

Former implementation using point-to-point metric

Point-to-plane is much faster

ConclusionConclusion

Compared ICP variantsCompared ICP variants

Introduced a new sampling methodIntroduced a new sampling method

Optimized ICP algorithmOptimized ICP algorithm

Future WorkFuture Work

Focus on stability and robustnessFocus on stability and robustness

Effects of noise and distortionEffects of noise and distortion

Algorithms that switch between variants Algorithms that switch between variants would increase robustnesswould increase robustness

ReferencesReferences

[1] [1] http://foto.hut.fi/opetus/ 260/luennot/9/9.htmlhttp://foto.hut.fi/opetus/ 260/luennot/9/9.html[2] [2] http://www.sztaki.hu/news/2001_07/maszk_allthrhttp://www.sztaki.hu/news/2001_07/maszk_allthree.jpgee.jpg[3][3]

http://graphics.stanford.edu/projects/mich/http://graphics.stanford.edu/projects/mich/[4] [4] http://www.cs.unc.edu/~sud/courses/comp258/fihttp://www.cs.unc.edu/~sud/courses/comp258/final_pres.ppt#257,2,Problem Statementnal_pres.ppt#257,2,Problem Statement

EFFICIENT VARIANTS OF THE ICP ALGORITHM Szymon Rusinkiewicz Marc Levoy.

Documents

Transcript of EFFICIENT VARIANTS OF THE ICP ALGORITHM Szymon Rusinkiewicz Marc Levoy.