Marc Levoy Marc Levoy 11

Szymon Szymon Rusinkiewicz Rusinkiewicz 11

Matt Ginzton Matt Ginzton 11

Jeremy Ginsberg Jeremy Ginsberg 11

Kari Pulli Kari Pulli 11

David Koller David Koller 11

Sean Anderson Sean Anderson 11

Jonathan Shade Jonathan Shade 22

Brian Curless Brian Curless 22

Lucas Pereira Lucas Pereira 11

James Davis James Davis 11

Duane Fulk Duane Fulk 33

11Computer Science Computer Science Department Stanford Department Stanford

UniversityUniversity22Department of Department of

Computer Science and Computer Science and Engineering University of Engineering University of

WashingtonWashington33Cyberware Inc.Cyberware Inc.

Presented Presented by : Abhinav by : Abhinav DayalDayal

Overview Overview

MotivationsMotivations• push 3D scanning technology push 3D scanning technology • tool for art historianstool for art historians• lasting archivelasting archive

Technical goalsTechnical goals• scan a big statue scan a big statue • capture chisel markscapture chisel marks• capture reflectancecapture reflectance

5 meters

1/4 mm1/4 mm

20,000:1

20,0002

1 billion

Why capture chisel Why capture chisel marks?marks?

Atlas (Accademia)

ugnetto

?

Day (Medici Chapel)

2 mm

single scan of St. Matthew 1 mm

Issues InvolvedIssues Involved

ScanningScanning Scanner designScanner design Scanning ProcedureScanning Procedure

Post ProcessingPost Processing Range ProcessingRange Processing Color ProcessingColor Processing

Handling large data setsHandling large data sets

Scanner DesignScanner Design

Laser stripe triangulation systemLaser stripe triangulation system Resolution and field of view Resolution and field of view

Should capture chisel marksShould capture chisel marks Reasonable size of resulting datasetReasonable size of resulting dataset

Standoff and baselineStandoff and baseline Longer standoff Longer standoff access to deeper access to deeper

recesses + safe distance from statuerecesses + safe distance from statue Longer standoff Longer standoff longer baselinelonger baseline

prone to miscalibrationprone to miscalibration

Color AcquisitionColor Acquisition

Single passSingle pass 1D luminaire and 1D color sensor1D luminaire and 1D color sensor

Cross talk b/w luminaire & sensorCross talk b/w luminaire & sensor poor fidelity poor fidelity RGB lasersRGB lasers

Large and complexLarge and complex

Separate passSeparate pass They used broadband luminaire and They used broadband luminaire and

separate sensor (Digital Camera – 1520 x separate sensor (Digital Camera – 1520 x 1144 pix res)1144 pix res)

Camera Resolution and Camera Resolution and fieldfield

For color to range calibration For color to range calibration match respective standoffsmatch respective standoffs

One color per pixel resolution One color per pixel resolution (per (per range data)range data)

Controlled illuminationControlled illumination Depth of fieldDepth of field

DOF > FOV(Z) of range-cameraDOF > FOV(Z) of range-camera

Gantry:Geometric Gantry:Geometric designdesign

4 motorized axes

laser, range camera,white light, and color camera

truss extensionsfor tall statues

Working volume of the Working volume of the scannerscanner

calibrated motionscalibrated motions pitch pitch (yellow)(yellow) pan pan (blue)(blue) horizontal translation horizontal translation (orange)(orange)

• uncalibrated motions– vertical translation– remounting the scan head– moving the entire gantry

Scanning the DavidScanning the David

height of gantry: 7.5 height of gantry: 7.5 metersmeters

weight of gantry: 800 weight of gantry: 800 kilogramskilograms

CalibrationCalibrationRange CalibrationRange Calibration

Color CalibrationColor Calibration

Their calibration was complex and did moderately Their calibration was complex and did moderately wellwell

• Used a planar target with feature points to Used a planar target with feature points to calculate camera’s intrinsic parameters and calculate camera’s intrinsic parameters and to build a per pixel intensity correction to build a per pixel intensity correction tabletable

Scanning procedureScanning procedure

Range ScanningRange Scanning Typical range involves several Typical range involves several

horizontally adjacent vertical sweeps or horizontally adjacent vertical sweeps or vice versavice versa

Overlap adjacent sweeps by 40% Overlap adjacent sweeps by 40% Overlap adjacent shells by 15% Overlap adjacent shells by 15%

Color ScanningColor Scanning Image with spotlight – Image without Image with spotlight – Image without

spotlight = Image with only spotlightspotlight = Image with only spotlight

Safety ConcernsSafety Concerns

energy depositionenergy deposition Low and not a problemLow and not a problem

avoiding collisionsavoiding collisions manual motion controlsmanual motion controls automatic cutoff switchesautomatic cutoff switches one person serves as spotterone person serves as spotter

surviving collisionssurviving collisions pad the scan headpad the scan head

Range processing Range processing pipelinepipeline stepssteps

manual initial alignmentmanual initial alignment ICP to one existing scanICP to one existing scan automatic ICP of all automatic ICP of all

overlapping pairsoverlapping pairs global relaxation to spread out global relaxation to spread out

errorerror merging using volumetric merging using volumetric

methodmethod(space carving)(space carving)

problemsproblems should have tracked the gantry should have tracked the gantry

locationlocation ICP is unstable on smooth ICP is unstable on smooth

surfacessurfaces

Color processing Color processing pipelinepipeline

stepssteps compensate for ambient compensate for ambient

illuminationillumination discard shadowed or specular pixelsdiscard shadowed or specular pixels map onto vertices – one color per map onto vertices – one color per

vertexvertex correct for irradiance correct for irradiance diffuse diffuse

reflectancereflectance

limitationslimitations ignored interreflectionsignored interreflections ignored subsurface scatteringignored subsurface scattering treated diffuse as Lambertiantreated diffuse as Lambertian used aggregate surface normalsused aggregate surface normals

Handling large Handling large datasetsdatasets

range images instead of polygon range images instead of polygon meshesmeshes r(u,v) (special case of displacement r(u,v) (special case of displacement

map) yields 18:1 lossless map) yields 18:1 lossless compression (run-length encoding)compression (run-length encoding)

multiresolution using (range) multiresolution using (range) image pyramidimage pyramid

lazy evaluationlazy evaluation viewer based on point rendering viewer based on point rendering

(Qsplat)(Qsplat)

Range image Range image pyramidspyramids

Range image at 1x resolutionRange image at 1x resolutionRange image at 1/2x resolutionRange image at 1/2x resolution

Range image at 1/4x resolutionRange image at 1/4x resolution

Samples with any of Samples with any of its four children at the its four children at the next finer pyramid next finer pyramid level missing deletedlevel missing deleted

Redden parents of Redden parents of missing children in missing children in proportion to the proportion to the fraction of its missing fraction of its missing childrenchildren

Can see holes at Can see holes at coarser coarser resolutionsresolutions

Problems facedProblems faced

Approximated marble as truly lambertianApproximated marble as truly lambertian Many unavoidable holes in the scanMany unavoidable holes in the scan Expensive and bulky gantryExpensive and bulky gantry Inadequate calibrationInadequate calibration Manual view planning Manual view planning prone to errors prone to errors Manual alignment of successive scansManual alignment of successive scans

How optically cooperative is How optically cooperative is marble?marble?

• systematic bias of 40 micronssystematic bias of 40 microns

• noise of 150 – 250 micronsnoise of 150 – 250 microns

– worse at oblique angles of incidenceworse at oblique angles of incidence

– worse for polished statuesworse for polished statues

Removing the Removing the holesholes Space carving techniqueSpace carving technique

Create a maximum region of space Create a maximum region of space consistent with the scansconsistent with the scans

Creates a watertight surface Creates a watertight surface may lead to surfaces that are less may lead to surfaces that are less

plausible than smoothly extending plausible than smoothly extending the observed surfacesthe observed surfaces.

Recent work based upon Recent work based upon Volumetric diffusion (Volumetric diffusion (Video))

Reference: Filling holes in complex surfaces using volumetric diffusion: James Davis, Steven R. Marschner, Matt Garr, Marc Levoy, Computer Science Department, Stanford University (August, 2001)

ConclusionConclusion provides remarkable background for provides remarkable background for

future developmentsfuture developments Provides massive mesh modelsProvides massive mesh models Discusses basic issues involved in 3D Discusses basic issues involved in 3D

scanningscanning Many lessons learntMany lessons learnt Encouraging Encouraging many new applicationsmany new applications ((in areas in areas

such as: reverse engineering; industrial design; repair, such as: reverse engineering; industrial design; repair, reproduction, and improvement of machinery; medical reproduction, and improvement of machinery; medical diagnostics, analysis and simulation; 3D photography; diagnostics, analysis and simulation; 3D photography; and building rich virtual environmentsand building rich virtual environments))

This is just a beginningThis is just a beginning

QUESTIONS?QUESTIONS?