SmartBoxes for Interactive Urban Reconstruction

Liangliang Nan1, Andrei Sharf1, Hao Zhang2, Daniel Cohen-Or3, Baoquan Chen1

1 Shenzhen Institutes of Advanced Technology (SIAT), China2 Simon Fraser University, Canada

3University of Tel Aviv, Israel

Virtual BerlinVirtual Philadelphia

3D Cities

Acquisition of Urban Environments

Cameras/videos

Remote Sensing Systems

3D Digital CityAuto-mounted LIDAR

Airborne LIDAR

3D LiDAR scanner

• Street-level• 60km/h• 180 pitch• [100-300m] range • 100K points/second• 5cm XY accuracy

Outdoor Urban Scanning

Imperfect Scans - Occlusions

• Point cloud contains holes due to various occlusions (“shadows”)

Imperfect Scans – Angle & Range

• Oblique scanning angle• Laser energy attenuation

on range

Urban Building Characteristics

• Repetitions, intra symmetry and regularity• Axis-aligned basic primitives• Dominant planes

SmartBoxes

• Box-up and Smart!

SmartBoxes

• Box prior shape fitting • Smart context awareness

Both

Context

Data

Live Demo

9

Related Work

• Procedural modeling of buildings and facades [Wonka2003;Muller2007]

• Automatic 3D reconstruction from 2D images [Zisserman2002;Xiao2009;Furukawa2009]

• Interactive modeling of architectural structures [Debevec1996;Schindler2003; Xiao2008;Sinha2008;Jiang2009]

• Primitive fitting to data [Gal2007;Schnabel 2009]

Preprocessing

• Automatic detection of planes and edges assuming dominant orthogonal axes– RANSAC planes– Line sweep edges

Snapping a Box

• 2D rubber band ROI• Collect planes, edges, corners• Find the best fitting box using

data fitting force D(B,P)

Data fitting D(B,P) snap

Facet Edge

• Data fitting force

Data quality(Confidence + Density)

Distance

Grouping

• Simple SmartBox Compound SmartBox• Align to remove gaps and intersections

– cluster and align close to co-linear edges

Drag-and-drop context C(Bi-1, Bi)

• The context of Bi

••• Bi-1Bi-2Bi-3

Interval Alignment Scale

Context

Bi

Drag-and-drop context C(Bi-1, Bi)

• The context of Bi

– Interval term

••• Bi-1Bi-2Bi-3Bi

Drag-and-drop context C(Bi-1, Bi)

• The context of Bi

– Alignment term

••• BiBi-1Bi-2Bi-3

Bi

Drag-and-drop context C(Bi-1, Bi)

• The context of Bi

– Scale term

••• BiBi-1Bi-2Bi-3

Discrete objective minimization

• Find linear transformation T(excluding rotation) to minimize:

Data fitting force

Contextual force

Discrete objective minimization

••• Bi-1Bi-2Bi-3

• Find linear transformation T(excluding rotation) to minimize:

Data fitting force

Contextual force

Balance between two forces

Emerging city of Shenzhen, China

Results: textured buildings

Manchester Civil Justice Centre (Manchester, UK)Habitat 67 (Montreal, Canada)The Crooked House (Sopot, Poland)

(Thank You)!