Algorithmic Techniques for Parametric Model Recovery

1/38

Dr.-Ing. Sung Joon Ahn

CurvSurf, Inc.

Algorithmic Techniques for Parametric Model Recovery

2/38

• Dr. Ahn, Sung Joon

• BS: Seoul Nat’l University (1985)

MS: KAIST (1987)

Dr.-Ing.: University of Stuttgart (2004)

• LG Electronics (1987–1990)

Fraunhofer IPA (1990–2004)

Sungkyunkwan University (2005–2013)

CurvSurf, Inc. (2013– )

• Circular coded target

Image measurement errors

Stripe-projection method

Orthogonal distance fitting (this talk)

Prolog (1)

Production Eng.

Robotics & Autom.

Mechanical Eng.

Mechanical design

Tech. info. proc.

Computer Vision &

Graphics

Meas. automation

Photogrammetry

3-D measurement

Numerical Math.

Edu

Job

Pub

Who I am

3/38

• Parametric model recovery

• Automation of parametric model recovery

= The Holy Grail.

Prolog (2)

4/38

Prolog (3)

• Tools of parametric model recovery

• Hough Transform (Hough 1959)

– Discrete parameter space (bin space)

– Applicable to small 𝑝 and not small 𝑛 (Length of bin−axis−array)𝑝

Resolution of parameters

∝ 𝑛 (should be large enough)

• RANSAC (Fischler 1981)

– Discrete parameter space (pick-points)

– Applicable to small 𝑝 and not small 𝑛 𝑛𝑝

𝑚𝑝

=𝑛 𝑛−1 … 𝑛−𝑝+1

𝑚 𝑚−1 … 𝑚−𝑝+1= Hit−rate of picking p points belong to the target feature

• Least Squares Method (Gauss 1795)

– Continuous parameter space

– Applicable to a wide range of curves and surfaces ( 𝑝 ≤ 𝑛 )

– Needs robust algorithms for segmentation & model fitting.

𝑝 = DOF of parameters

= Dim. of bin space

= No. of pick-points

Plane: 3

Cylinder: 2+3

Sphere: 4

Cone: 6

Torus: 7

𝑚 = No. of data points

𝑛 = No. of target points

5/38

• Parametric model recovery, directly from point cloud or via meshing?

• Via meshing (with Rev. Eng. SW)

– Why meshing?: Point normal and connectivity.

But, meshes from point cloud are arbitrary

– Information loss: Point decimation and noise

reduction are obligatory before meshing,

to hold inter-point distance ≫ variations

– Model meshes visualization: Due

Noisy points meshes model: Nonsense!

• Directly from point cloud (noisy points model)

– Online processing is possible

– Wide range of applications

– Points are individually processed as measured.

Prolog (4)

1

2

1

3

3

4

4

Top

Front

Case 1 Case 2

Var.

6/38

Content

1. Prolog

2. Error Model for Point Cloud Point & points

3. Orthogonal Distance Fitting Best math. tool of model fitting

4. Competent Parameterization Math. model description

5. Local Curvature Analysis Solution to initial value problem

6. Region Growing Segmentation & model updating

7. Automatic Feature Extraction The Holy Grail

8. Closing Now & future

7/38

Error Model for Point Cloud (1)

• Point error model

• Cartesian measuring device: Distance-distance-distance errors

Spherical measuring device: Angle - angle - distance errors

• Assume a Gaussian error ellipsoid

• Maximum likelihood estimation

– Maximize the probability density

The measurement point substitutes an unknown object point.

The nature of

coordinate metrology

8/38

Error Model for Point Cloud (2)

• Curve/surface error model

• Gaussian error band (= Superposition of Gaussian error ellipsoids)

– Measurement uncertainty is observable only in normal direction (= The lateral size of object is unmeasurable by probing in normal direction)

• Maximum likelihood estimation

– Maximize the ‘sum’ of probability density

Model feature substitutes an unknown object feature, minimizing

the sum of the squared error distances to measurement points.

The nature of

coordinate metrology

9/38

Least Squares Curve/Surface Fitting

• Best error measure:

The minimum (shortest/orthogonal/normal) distance

between measurement point and model curve/surface

(Invariant to coordinate transformation)

• Twofold nonlinear minimization problem

• Find the minimum distance points on model curve/surface

• Minimize the sum of the squared minimum distances

through adjusting the model parameters.

Best math. tool of

model fitting

10/38

Moment Method for Line/Plane

• Alias: Eigenvalue decomposition, total least squares,

principal component analysis (cf. linear regression)

• Line/plane fitting:

• Minimize the sum of the squared normal distances between

the measurement points and the model line/plane

• Linear problem: Solved in closed form (Pearson 1901).

Invariant to

coordinate

transformation

Best math. tool of

model fitting

11/38

Orthogonal Distance Fitting (1)

• Alias: Best fitting, Euclidean fitting, geometric fitting

• Curve/surface fitting:

• Minimize the sum of the squared shortest distances between

the measurement points and the model curve/surface

• Nonlinear problem: Solved through iteration (Boggs 1987)

• S.J. Ahn, doctoral diss., LNCS 3151, Springer-Verlag, 2004.

Best math. tool of

model fitting

12/38


• Information flow with ODF of implicit curve/surface

Best math. tool of

model fitting

13/38


• Infomat’n flow with ODF of parametric curve/surface

Best math. tool of

model fitting

14/38

Doctoral dissertation

University of Stuttgart

2004

Orthogonal Distance Fitting (4) Best math. tool of

model fitting

15/38

Content

1. Prolog

2. Error Model for Point Cloud

3. Orthogonal Distance Fitting

4. Competent Parameterization

5. Local Curvature Analysis

6. Region Growing

7. Automatic Feature Extraction

8. Closing

16/38

• Math. description of ODF line

* The ODF line runs through 𝐗 ( ). While 𝜅 and 𝐗 are relatively insensitive to small changes

in measurement points, 𝑏, 𝑐, and 𝑑 change significantly their values because of the

leverage effect centered at 𝐗 . Thus, the best description of ODF line is using 𝜅 and 𝐗 .

Furthermore, 𝜅 and 𝐗 are invariant to the coordinate transformation of measurement

points, reproducing perfectly the rotation and translation of measurement points.

Competent parameterization (1) Curves and surfaces

in space

rcos𝜅sin𝜅

,

17/38

Competent parameterization (2)

• Curve/surface in space

• Conic section: Which a curve is this, ellipse?

𝑎𝑋2 + 𝑏𝑌2 + 𝑐𝑋𝑌 + 𝑑𝑋 + 𝑒𝑌 + 𝑓 = 0.

If so, axis lengths? Center? Pose angle?

• Quadratic surface: ? ? ? ? !

𝑎𝑋2 + 𝑏𝑌2 + 𝑐𝑍2 + 𝑑𝑋𝑌 + 𝑒𝑌𝑍 + 𝑓𝑍𝑋 + 𝑔𝑋 + ℎ𝑌 + 𝑖𝑍 + 𝑗 = 0

• Cubic, quartic, catenary, 3-D circle, helix, sine curve, … … … .

• Recall the ODF line: 𝑋𝑌

= 𝑢cos𝜅sin𝜅

+ 𝑋

𝑌 , decomposed to

𝑋𝑌

=cos𝜅 −sin𝜅sin𝜅 cos𝜅

𝑥𝑦 + 𝑋

𝑌 Coordinate transformation:

𝐗=𝐑−1𝐱+𝐗

and 𝑦 = 0 or 𝑥𝑦 =

𝑢0

Canonical form:

𝑓 𝑥,𝑦 =0 or 𝑥(𝑢)𝑦(𝑢)

.

Curves and surfaces

in space

18/38


• Parameter decomposition (cf. camera parameters in Photogrammetry)

• Canonical form: Specific to model curve/surface 𝐗 = 𝐑−1𝐱 + 𝐗o: Common to all curves and surfaces in space

• Intrinsic: Shape parameters (size, radius, vertex angle, etc.)

Extrinsic: Position/rotation parameters (rigid body motion).

Canonical form of circle

(𝑋 − 𝑋o)2+(𝑌 − 𝑌o)

2−𝑟2 = 0.

+

Curves and surfaces

in space

Don’t

combine!

19/38


• ODF with parameter decomposition

Curves and surfaces

in space

Canonical form

20/38


• Examples of canonical form (Implementation of ISO 10360-6)

• 2-D line : 𝑦 = 0

• 3-D line : (0, 0, 𝑢)

• Plane : 𝑧 = 0

• 2-D circle : 𝑥2 + 𝑦2 − 𝑟2 = 0

• 3-D circle : (𝑟cos𝑢, 𝑟sin𝑢, 0)

• Sphere : 𝑥2 + 𝑦2 + 𝑧2 − 𝑟2 = 0

• Cylinder : 𝑥2 + 𝑦2 − 𝑟2 = 0

• Cone : 𝑥2 + 𝑦2 − (𝑟 − 𝑧tan(𝜓/2))2= 0

• Torus : 𝑥4 + 𝑦4 + 𝑧4 + 2 𝑥2𝑦2 + 𝑦2𝑧2 + 𝑧2𝑥2 −2 𝑅2 + 𝑟2 𝑥2 + 𝑦2 + 2 𝑅2 − 𝑟2 𝑧2 + 𝑅2 − 𝑟2 2 = 0.

2-D/3-D curve: Ellipse, hyperbola, parabola, sine curve, catenary, helix, ...

Surface:

Ellipsoid, hyperboloid, paraboloid, …

Curves and surfaces

in space

21/38


• Position of ODF cylinder/cone (cf. position of ODF line: 𝐗 )

• A point on axis

• The competent one is the closest point to the mass center 𝐗

of data points, resulting in minimum variances because the

ODF feature is adhering to data points.

Curves and surfaces

in space

22/38

Content

1. Prolog





6. Region Growing


8. Closing

23/38

Local Curvature Analysis (1)

• Provide ODF with initial values

• Type of feature: Plane, sphere, cylinder, cone, or torus,

determined by comparing the signed curvature radii

• Feature parameters: Radius, position, and axis direction,

derived from the principal curvatures and axes.

Plane Sphere (torus) Cylinder (cone) Torus

* k1, k2 : Principal curvatures

Differential geometry

24/38

• Initial values for ODF

• Radius of sphere/cylinder = |𝑟1| Position of sphere/cylinder = 𝐗f − 𝑟1𝐧 = 𝐗o

Axis direction of cylinder = 𝐯

• Cone and torus:

Beyond this talk’s

scope.


𝐗f : Surface point

𝐧 : Surface normal

𝐮, 𝐯 : Principal axes

𝑟1, 𝑟2 : Curvature radii

= 1 𝑘1 , 1 𝑘2 .

|𝑟1| ≤ |𝑟2|.


25/38


• Automatic feature extraction

1. Pick a point in point cloud

2. Cut out a small patch around the pick-point

3. ODF plane to the patch

4. ODF quadratic to the patch, starting from the ODF plane

5. Calculate the principal curvatures and axes

6. Determine the type and initial values of ODF feature

7. Refine the ODF feature by means of region growing.


26/38

Region Growing (1)

• Loop of segmentation and model updating (fitting)

1. Initial model feature is given

2. Collect inlier candidates around the current model feature

3. Eliminate/clear outliers from inlier candidates

4. Update the model feature through ODF to inliers

5. if (any break condition is invoked) break;

else go to 2.;

• Criteria of continuing, discarding, or accepting

• Rms error of inliers

• Parameter reliability (covariance)

• Density of inliers, etc.

27/38

Region Growing (2)

• Collecting inlier candidates

• Activity level (passive to aggressive) of region growing is

controlled in two mutually orthogonal directions:

– Radial expansion: 2~3 times the current rms error

– Lateral extension: Increment of current feature length/width.

Definition of search domain

hull volume for inlier

candidates around the

current model cylinder

28/38

Content

1. Prolog





6. Region Growing


8. Closing

29/38

• Fully Automatic Feature Extraction

– Combine all the techniques presented

– The pick-point is randomly selected

– Not ripe enough yet for industrial applications

– To do: Point cloud + image.

Automatic Feature Extraction (1)

30/38


“Fully automatic feature extraction from point cloud,”

http://www.youtube.com/watch?v=42ZKbkjpBbY

31/38

• Semi-automation (1)

• Keeping the person in the loop

– Agent of feature extraction:

Diligent worker inside point cloud

User:

Intelligent decision maker giving instructions to agent

(Type of target feature and pick-point)

– The agent works in the way and manner the user expected

Boosting user’s confidence & productivity!


32/38

• Semi-automation (2)

• Low cost of producing high-quality deliverables

– Low rate of Type I and Type II errors

Person in the loop

– Best quality of feature extraction (ISO 10360-6)

The nature of ODF

– Short lap time per feature

Single-point clicking

No clipping

Zooming & panning if necessary

• DLL modules (Win32, x64)

– Agent of feature extraction

– Application neutral.


33/38

• DLL modules of CurvSurf, Inc.

• Extracting:

– Plane, sphere, cylinder, cone, or torus

• No. of pick-points:

– 1: Model features of usual form & size

– 2: Long strip plane,

long rod cylinder

– 3: Large plane,

large sphere,

thin disk cylinder,

thin disk cone,

thin ring torus.


34/38

• DLL parameters

• Point cloud

– Point accuracy

– Point density

• Patch around the pick-point

– Touch radius (size of patch)

• Region growing

– Radial Expansion: 0–10

– Lateral Extension: 0–10

• Switching from cone to cylinder

– Threshold value of vertex angle

• Callback to caller program

– Informs the caller of the current states of feature extraction.

Set DLL parameters


Select a DLL module

Pick point(s)

Press ‘F3’

Last again

35/38


“Feature Extraction from 2 Mio. Points”

http://www.youtube.com/watch?v=OdJSIYidEXg

36/38

• Best error measure

• Orthogonal distance fitting

• Competent parameterization

• Local curvature analysis

• Region growing

• Automatic feature extraction

• Long way to go.

You have learned

37/38

• The Status Quo

• Optical 3-D measuring device & data processing machine

– 10 times more versatile & powerful in every 4–5 years

• IT Giants moved in 3-D

– Apple (PrimeSense), Google (Tango), Intel (RealSense),

Microsoft (Kinect)

• The One Problem

– Automatic feature extraction

• Outlook

• ODF: Tool of processing dimensional information

• Competent parameterization:

Rule of how to handle curves and surfaces in space

• Information fusion (multi-sensor, multi-source).

Closing

38/38

• Further discussions are welcomed

Closing

Thank you!

Algorithmic Techniques for Parametric Model Recovery

Data & Analytics

Transcript of Algorithmic Techniques for Parametric Model Recovery