From noisy object surface scans to conformal unstructured grids of multiple materials for physical...

uib.no

U N I V E R S I T Y O F B E R G E N

From noisy object surface scans to conformal

unstructured grids of multiple materials for physical

finite element analysis (FEA)

Christian Kehl, University of Bergen / Uni Research AS

supervisor: Sophie Viseur, CEREGE/AMU

uib.no

Who am I ?

• Christian Kehl, M.Eng. cum laude (2012)

• home institution: Univ. Appl. Sciences Wismar

• internship semester A’dam (NL, 2009)

• ext. masters: Aalborg University (DK, 2011)

• MSc thesis: TU Delft (NL, 2012)

• Research: TU Deflt (NL, 2012-2014)

• Ph.D. candidacy: Uni Bergen (NO, 2014-2017)

• Ph.D. research visit: CEREGE / AMU (F, 2016/17)

• research interest: real-world scans to physical volume

models

uib.no

Outline

uib.no

Motivation

• 3D data acquisition gets easier, cheaper and more

accessible

• Range of equipment allows acquisition for any budget,

adapted to quality demands

• volumetric analysis of the physical world at the core of

many science disciplines

– medicine and biomechanics; geology (nat. res.);

archaeology; planetary studies; climate change;

natural disasters; urban heat management;

mechanics and materials; aerospace engineering;

energy science;

uib.no

Motivation

• great potential and interest in simulations vs. geometric

knowledge of domain experts

• Requirements and constraints:

– high simulation accuracy

– large data (extent & resolution; dimensionality and

time-dependency)

– limited computational resources (field studies;

desktop simulations)

– budget ...

• transfer geometric knowledge into tangible software &

algorithms

uib.no

Outline

uib.no

Point Set Scanning: LiDAR

• issue: any material (gas) between scanner and object attenuates I

• problems: – scattered reflection

(metals)

– refracting materials (gems, fluids, glass ...)

• noise: attenuation, scattering, refracted light

• uniform density, irregular positioning

uib.no

Point Set Scanning: Structured Light

structured pattern can also be projected in non-visible wavelengths

(=> Xbox Kinect)

• range imaging (disparity)

• projection of structured pattern

• problems: – scattered reflection

(metals)

– refracting materials (gems, fluids, glass ...)

– wave interferences

• uniform density; regular lattice positioning

uib.no

Point Set Scanning: SfM

• relies on: – image quality

– # images; temporal correlation

– lighting & materials (light scattering, specular highlights, ...)

– point correlation acc.

– numerical optimisation algorithm

• flat, blank object (areas) no “features” scan holes

• advantage: underwater scan

• non-uniform density; irregular positioning

uib.no

Examples: Point Set Scanning

LiDAR

Structure-from-

Motion structured light

uib.no

Point Set Scanning

• LiDAR: the accuracy references; issue: $$$

• structured light: cheap alternative indoors; outdoors

(Kinect) -> IR interference: the sun

• Structure-from-Motion (SfM): versatile & cheap;

demands knowledge & patience

• Alternative: Stereo Imaging

• More info: ISPRS & Photogrammetric Record

uib.no

Outline

uib.no

Mono-material reconstruction

• geometric demands:

– coherently-oriented surface elements

– hole-free

– topologically correct shape reconstruction

– density adaptive (optional, but advantageous)

– closed C2 surface (i.e. tight envelope)

– non-manifold (for Delaunay Tetrahedralisation)

– high-quality volume elements

– minimal volume element count (simulation

convergence)

uib.no


• Problem: irregular, noisy scans vs. exact-geometry

reconstruction

• Exact-geometry schemes often fail:

– Delaunay Triangulation (e.g. Shewchuck 1996, Alliez

et al. 2011)

– Cocone (Dey & Goswami 2003, Dey and Levine 2009)

– PowerCrust (Amenta et al. 2001)

– Alpha Shapes (Edelsbrunner & Mücke 1992)

– Ball Pivoting (Bernardini et al. 1999)

• varying point density & holes insufficient samples for

surface reconstruction

uib.no


• RMLS (Fleishman et al. 2005) and Poisson surfaces

(Khazdan et al. 2006) account for varying sample density

• Tetrahedralisation for FEA without surface geometry

changes possible [George et al. 1991]

• persisting issues:

– surface: self-intersection, manifolds, triangle count

– overly smooth surface approximation; crease angles

– new vertex set instead of triangulating the original

– lack of theoretical guarantees of shape approximation

uib.no


uib.no


• specific application: estimate mineral volumes

– current volume estimation means very expensive

– currently lab experimental estimation optical scans

as cheap & less labour-intense alternative

uib.no

Outline

uib.no

Segmentation of surfaces

• Until now:

– scan + surface & volume reconstruction of 1 object

– treating as homogeneous surface / material / object

• Physical reality:

– object consists of sub-entities

– entities are heterogeneous (from one another as

potentially in itself)

multiple materials

labelling, semantics and segmentation

uib.no


• Goal: segment a given (surface) geometry into distinct

regions, based on its intrinsic properties

• specific application case/ motivation for our studies:

Interactive segmentation of outcrop surfaces into its

composing elements, on mobile devices

• PhD research: “Visual Techniques for Geological

Fieldwork Using Mobile Devices” [Kehl et etl. 2015/2016]

uib.no


• application:

segmentation outdoors,

on tablets

• conditions:

– domain expert influence

– fast computation, limited

hardware performance

– input: anisotropic,

irregular surface meshes

– no change of underlying

surface structure

– noise-resilient

uib.no


• algorithmic space: parametric vs. geometric

• general segmentation:

– part-aware [Liu et al. 2009]

– geometric intrinsic (e.g. curvature-based [van Kaick

et al. 2014])

– interactive ([Zhang et al. 2011])

• Good reviews: Shamir 2008, Benhabiles et al. 2010,

Theologou et al. 2015

• Our approach: interactive; combine geometry,

morphology & statistical optimisation

uib.no


• Key algorithmic components:

– combinatorial expansion, by curvature & extrema

– morphology operations (erode,dilate,open,close)

– stochastic optimisation simulated annealing (SA)

• alg. components are known, but morphology and SA on

unstructured, irregular meshes undefined

principal shapes for geometric

classification [Kudelski et al. 2011]

morphological classification, including topo-

logical guarantees [Williams & Rossignac 2004] stochastic active contours by

simulated annealing [Horritt 1999]

uib.no

Segmentation of surfaces – combinatorics & curvature

• Starting point: Interactive initialisation via lines on

surface

• surface integration: flag by line intersection

uib.no


• Next: expand initial area to boundary curve

• boundary conditions: direction- weighted geometry-

intrinsic (i.e. curvature, extrema)

• iterative refinement: • track boundary vertices

• check inside-validity criterion

• include vertex

• make its neighbours boundary

candidates

• repeat until equilibrium

else

nnvNvvvkvkF

else

vNvvkvk

vk

F

bnnnn

bnnifl

:0

1),(),())(sgn())(sgn(:1:cond. changeSign

:0

)())(sgn())(sgn(:1

0)(:1

:cond.point Infliction

12

sgn2121

0)(1)()0)((, sgn vFvFvFVvSv iflifl

uib.no

Segmentation of surfaces: process

uib.no


• EXAMPLE 1:

EXPANSION WITH

ISLE

• EXAMPLE 2:

BASELINE

uib.no


• Pro:

– versatile boundary condition

– VERY FAST optimisation compared to alternatives

(e.g. particle systems, active contours)

• Contra:

– isle occurrences

– thin bridges

– sharp (i.e. zig-zag) contour for irregular meshes

uib.no


• Pro:

– versatile boundary condition

– VERY FAST optimisation compared to alternatives

(e.g. particle systems, active contours)

• Contra:

– isle occurrences

– thin bridges

– sharp (i.e. zig-zag) contour for irregular meshes

=> MORPHOLOGY

=> CURVE STRAIN ENERGY

=> FILTERING

uib.no

Segmentation of surfaces - morphology

• Morphology, Curve Strain Energy and Filtering

Tightening [Williams & Rossignac 2005]

• mapping to discrete space: pixel cluster = triangle

StrtCStSDilate s

,|:

StrtCStSErode s

,|:

SSRSSRSR rrrrr

)(,)(:)( Closing

SSFSSFSF rrrrr

)(,)(:)( Opening

)(,,)( Anticore

)()(Mortar

)()( Core

SMStTtTSA

SFSM

SRSX

AAAA

r

r

uib.no

Segmentation of surfaces – simulated annealing opt.

• Williams & Rossignac 2005: Fast Marching curve

evolution, curvature speed func. [Osher & Sethian 1988]

• Fast Marching in discrete space undefined => stat.

optimisation via simulated annealing (SA)

• advantage: morphology defines topology => no checks

• Open problems to address:

– r-constrain on the surface

– convexity-based energy function

M

j

K

k

inouti

i

i

kjicjicc

ceSE

ceSE

0 0

1

1

1

,,,

1)'(

1)(

:functionsEnergy

]2.1,8.0[

:0

)()'(:1

),'(),(

:yprobabilit Acceptance

1

else

SESEc

eSESEP

i

uib.no


uib.no


SA

optimisation;

morph.

r-constrain

0.10

Input

SA

optimisation;

morph.

r-constrain

0.13

Output

uib.no

Outline

uib.no

Multi-material volume reconstruction

• Given a segmented volume image, how do we construct

minimal, accurate, conformal FEM meshes?

• Known approaches:

– (weighted) Delaunay based on interfaces [Boltcheva

et al. 2009[a]]

– Lattice Cleaving [Bronson et al. 2014]

– BioMesh3D [Meyer et al. 2007/2008]

=> starting point

uib.no


• Specific application scenario: patient-specific prostheses

– specific case: femur replacement

– patient-specific prosthesis design leads to less complications after operation

– adapting design to “normal stress” of the patient: accurate FEA and FEA models

– details:

• MSc Thesis Christian Kehl (2012) “Conformal multi-material mesh generation from labelled medical volumes”

• PhD Thesis Daniel F. Malan (2015) “Pinning down loosened prostheses: imaging and planning of percutaneous hip refixation”

uib.no


orthopaedic workflow hip replacement: CT scanning of the patient (a); segmenting

scan into regions of different objects (semantics), respectively: different materials (b);

constructing high-quality, minimal-element FEM volume mesh (c); FEA stress

simulation in bones (d);

FINAL part (not depicted): refine prosthesis (central, grey, ‘banana-shaped’ object)

design and positioning based on stress simulation; repair or insert prosthesis.

uib.no


• conformal meshes: meshes sharing vertex- and edge

configurations on their interfaces

• minimal mesh: describing an object’s shape / volume

with a minimal set geometric primitives accurately

=> inter-vertex distances of interface surfaces can vary

significantly

uib.no


• minimal inter-vertex distance => minimum radius of

curvature (rmin) expressed in the local feature size (λ)

• sampling theorem: ε-sampling [Amenta et al. 1998[b],

Boissonnat and Oudot 2005, Meyer et al. 2005,

Shewchuck 2008]

uib.no


• needs: 3D skeleton / medial axis transform

• approximation; finding the minimum medial axis is NP-

hard [Coeurjolly et al. 2008]

• approach Meyer et al. 2007: high-density proxy surfaces

at interfaces

• MAT from implicit representation of proxy surfaces (see

[Hesselink & Roerdink 2008, Coeurjolly & Montanvert

2007])

uib.no


• next: construct distance field: surface MAT

• original, segmented voxel centres (as vertices, vo) are

“on” or “close-to” the proxy surface (Tp)

0 < d(vo,Tp) < λ(Tp)

• sample λ(vo) from the distance field

• result: maximal distance at vo to guarantee distance to

adjacent vertices

uib.no


• Next: Particle system [Witkin & Heckbert 1994, Meyer et

al. 2005]; voxel centres are particles (seeds)

• move particles on proxy surface: max. inter-particle

spacing, min. energy:

speedparticlematidentIgradientfuncimplFposparticle vp ii

.;.;...;

E = energy function to minimize

(functions from Meyer et al. 2005)

uib.no


Seed particles with surface normals and

distance constraint (arrow colour), moving

on proxy surface

segmented seed

particles

reconstructed

surface & MAT

vert.

uib.no


• Final step: construct final surface(s) [per material] from

optimised particle set via Delaunay Triangulation

=> mathematically well defined iff particle satisfies ε-

criterion [Meyer et al. 2007, Amenta et al. 1998[b]]

• preserve segmentation / material label via initial,

segmented volume image

uib.no


• Issue: ε-criterion (for Delaunay) violated in domains of

crease angles

• Our extension: meshing by injection

– optimise vertex positions via particle system

– iteratively inject particles in proxy mesh

– remove original proxy vertices iteratively

– enforces edge consistency by edge flipping

• algorithm for crease-angle domains and sparse samples,

as not bound by ε-criterion

uib.no


• publication of the algorithm currently on halt; expand and

resubmit end 2017/ begin 2018

initial workshop paper project website

Meyer et al., 2008,

high sample

Meyer et al., 2008 Kehl et al., 2015

uib.no

Outline

uib.no

Future Vision - Conceptual

• Multi-material meshing: extension to segmented point

set meshing

– combine volume meshing from optical scans with

multi-material meshing

– proxy surfaces also derivable for point sets

(theoretically)

– particle system sampling optimises final vertex

positions and drastically reduces tetrahedral element

count

– address some interesting applications ...

uib.no

Future Vision - Conceptual

• Multi-material meshing: influence of surface sample

schemes

– particle optimisation computationally very expensive;

demands proxy surface

– with noisy point sets from scanning: disadvantageous

– great study on different vertex samples: Pilleboue et

al. 2015

– idea:

• evaluate given noisy sample for suitable sampling

scheme representative

• chose reconstruction method based on given sample

uib.no

Future Vision- Applications

• Volume Reconstruction in geosciences (lab)

– currently being done: volume mesh from sketched

surfaces (Rapid Reservoir Modelling;

www.rapidreservoir.org; Jacquemyn et al. 2017)

– goal: seamlessly integrate natural observation (digital

outcrop) in conceptual models (e.g. Caumon et al.

2004, Jackson et al. 2015,)

– physical stress simulation, directly on outcrop scan

uib.no

Future Vision - Applications

• Volume Reconstruction in geosciences (field)

– scanning hand samples in the field

– integrate volume & simulation result back on field

tablet

– approximate stress/cleavage simulations on small-

scale samples on the tablet via GPU Computing

uib.no

References

[Alliez 2011] P. Alliez, L. Saboret & G. Guennebaud, “Surface Reconstruction from Point Sets”, 2001

[Amenta1998b] N. Amenta, M. Bern & D. Eppstein, “The crust and the beta-Skeleton: combinatorial curve reconstruction”, Graph. Models and

Image Proc., vol. 60, p. 125-135, 1998

[Amenta2001] N.Amenta, S. Choi & R.K. Kolluri, “The Power Crust”, 6th Symposium on Solid Modelling and Applications, p. 249-266, 2001

[Benhabiles2010] H. Benhabiles, J.-P. Vandeborre, G. Lavoué & M. Daoudi, “A comparitive study of existing metrics for 3D-mesh

segmentation evaluation”, The Visual Computer, vol. 26 no. 12, p. 1451-1466, 2010

[Bernardini2010] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin, “The Ball-Pivoting Algorithm for Surface Reconstruction”,

IEEE Trans. Vis. Comp. Graph., vol. 5, p. 349-359, 1999

[Boissonnat2005] J.-D. Boissonnat & S. Oudot, “Provably good sampling and meshing of surfaces”, Graphical Models, vol. 67 no. 5, p. 405-451,

2005

[Boltcheva2009] D. Boltcheva, M. Yvinec & J.-D. Boissonnat, “Feature preserving Delaunay mesh generation from 3D multi-material images”,

Computer Graphics Forum, vol. 28 no. 5, 2009

[Bronson2014] J. Bronson, J.A. Levine & R.T. Whitaker, “Lattice Cleaving: A Multimaterial Tetrahedral Meshing Algorithm with Guarantees”,

IEEE Trans. Vis. Comp. Graph., vol. 20 no. 2, p. 223-237, 2014

[Coeurjolly2008] D. Coeurjolly, J. Hulin & I. Sivignon, “Finding a minimum medial axis of a discrete shape is NP-hard”, Theoretical Comp. Sci.,

vol. 406 no. 1, p. 72-79, 2008

[Coeurjolly2007] D. Coeurjolly & A. Montanvert, “Optimal Separable Algorithms to Compute the Reverse Euclidean Distance Transformation and

Discrete Medial Axis in Arbitrary Dimension”, IEEE PAMI, vol. 29 no. 3, p. 437-448, 2007

[Dey2003] T.K. Dey & S. Goswami, “Tight Cocone: A water tight surface reconstuctor”, 8th Symposium on Solid Modelling and

Applications, 2003

[Dey2009] T.K. Dey & J.A. Levine, “Delaunay Meshing of Piecewise Smooth Complexes without Expensive Predicates”, J. Algorithms, vol.

2 no. 4, 1327-1349, 2009

[Edelsbrunner1992] H. Edelsbrunner & E.P. Mücke, “Three-dimensional alpha shapes”, Proc. ACM Volume Visualization, p. 75-82, 1992

[Fleishman2005] S. Fleishman, D. Cohen-Or & C. Silva, “Robust moving least-squares fitting with sharp features”, ACM TOG, vol. 24, p. 544-

552, 2005

[George1991] P.L. George, F. Hecht & E. Saltel, “Automatic mesh generator with specified boundary”, Computer Methods in Applied

Mechanics and Engineering, vol. 92 no. 3, p. 269-288, 1991

[Hesselink2008] W.H. Hesselink & J.B.T.M. Roerdink, “Euclidean Skeletons of Digital Image and Volume Data in Linear Time by the Integer

Medial Axis Transform”, IEEE PAMI, vol. 30 no 12, p. 2204-2217, 2008

[Jackson2015] M.D. Jackson, J. Percival, P. Mostaghimi, B. Tollit, D. Pavlidis, C. Pain, J. Gomes, A.H. Elsheikh, P. Salinas & A. Muggeridge,

“Reservoir modeling for flow simulation by use of surfaces, adaptive unstructured meshes, and an overlapping-control-volume

finite-element method”, SPE Reservoir Evaluation & Engineering, vol. 18 no. 2, p. 115-132, 2015

uib.no

References [Jacquemyn2017] C. Jacquemyn, Y. Melnikova, M.D. Jackson & G.J. Hampson, “Surface-Based Modelling of Subsurface Reservoirs Using

Parametric NURBS Surfaces”, 79th EAGE Conference and Exhibition, 2017

[VanKaick2014] O. van Kaick, N. Fish, Y. Kleiman, S. Asafi & D. Cohen-Or, “Shape Segmentation by Approximate Convexity Analysis”, ACM

TOG, vol. 34 no. 4, p. 1-11, 2014

[Kazhdan2006] M. Kazhdan, M. Bolitho & H. Hoppe, “Poisson surface reconstruction”, Proc. Symp. Geometry Processing, p. 61-70, 2006

[Kehl2016] C. Kehl, S. J. Buckley, R. L. Gawthorpe, I. Viola & J. A. Howell, “DIRECT IMAGE-TO-GEOMETRY REGISTRATION USING

MOBILE SENSOR DATA”, ISPRS Annals, 2016

[Kehl2017] C. Kehl, S.J. Buckley, S. Viseur, R.L. Gawthorpe & J.A. Howell, “Automatic Illumination-Invariant Image-to-Geometry

Registration in Outdoor Environments”, The Photogrammetric Record, vol. 32 no. 158, 2017

[Kudelski2011] D. Kudelski, S. Viseur, G. Scrofani & J.L. Mari, “FEATURE LINE EXTRACTION ON MESHES THROUGH VERTEX MARKING

AND 2D TOPOLOGICAL OPERATORS”, Int. J. Image and Graphics, vol. 11 no. 4, p. 531-548, 2011

[Liu2009] R. Liu, H. Zhang, A. Shamir & D. Cohen-Or, “A Part-aware Surface Metric for Shape Analysis”, Computer Graphics Forum, vol.

28 no. 2, p. 397-406, 2009

[Meyer2005] M.D. Meyer, P. Georgel & R.T. Whitaker, “Robust particle systems for curvature dependent sampling of implicit surfaces”, Proc.

Shape Modelling and Appl., 2005

[Meyer2007] M.D. Meyer, R.M. Kirby & R.T. Whitaker, “Topology, Accuracy, and Quality of Isosurface Meshes Using Dynamic Particles”,

IEEE Trans. Vis. Comp. Graph., vol. 13 no. 6, p. 1704-1711, 2007

[Meyer2008] M.D. Meyer, R.T. Whitaker, R.M. Kirby, C. Ledergerber & H.-P. Pfister, “Particle-based Sampling and Meshing of Surfaces in

Multimaterial Volumes”, IEEE Trans. Vis. Comp. Graph., vol. 14 no. 6, p. 1539-1546, 2008

[Osher1988] S. Osher & J.A. Sethian, “Fronts Propagating with Curvature Dependent Speed: Algorithms Based on Hamilton-Jacobi

Formulations”, J. Comp. Physics, vol. 79, p. 12-49, 1988

[Pilleboue2015] A. Pilleboue, G. Singh, D. Coeurjolly, M. Kazhdan & V. Ostromoukhov, “Variance Analysis for Monte Carlo Integration”, ACM

TOG, vol. 34 no. 4, 2015

[Shamir2008] A. Shamir, “A survey on Mesh Segmentation Techniques”, Computer Graphics Forum, vol. 26 no. 6, p. 1539-1556, 2008

[Shewchuk1996] J.R. Shewchuk, “Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator”, LNCS Applied Computational

Geometry, p. 203-222, 1996

[Shewchuk2008] J.R. Shewchuk, “General-dimensional constrained Delaunay and constrained regular triangulations, I: Combinatorial

properties”, Discrete & Computational Geometry, vol. 39, no. 1, p. 580-637, 2008

[Shewchuk2014] J.R. Shewchuk & H. Si, “Higher-Quality Tetrahedral Mesh Generation for Domains with Small Angles by Constrained Delaunay

Refinement”, Proc. 13th Symp. Computational Geometry, p. 290-299, 2014

{Theologou2015] P. Theologou and I. Pratikakis and T. Theoharis, “A comprehensive overview of methodologies and performance evaluation

frameworks in 3D mesh segmentation “, Computer Vision and Understanding, vol- 135, p. 49-82, 2015

[Williams2005] J. Williams & J. Rossignac, “Tightening: Curvature-limiting morphological simplification”, 6th Symposium on Solid and Physical

Modelling, 2005

[Witkin1994] A. Wikin and P.S. Heckbert, “Using Particles to Sample and Control Implicit Surfaces”, 1984

From noisy object surface scans to conformal unstructured grids of multiple materials for physical...

Technology

Transcript of From noisy object surface scans to conformal unstructured grids of multiple materials for physical...