Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu...
Transcript of Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu...
![Page 1: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/1.jpg)
Dense image matching and surface reconstruction when photogrammetry meets computational geometry
Florent Lafarge
Sophia Antipolis - France
![Page 2: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/2.jpg)
Problem statement
June 2014
Given (at least) two (calibrated) images observing
the same scene, what type of 3D information can
we extract ?
1
![Page 3: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/3.jpg)
Problem statement
June 2014
Given (at least) two (calibrated) images observing
the same scene, what type of 3D information can
we extract ?
1
=2 : Stereo Matching
>2 : Multi-View Stereo (MVS)
![Page 4: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/4.jpg)
Problem statement
June 2014
Given (at least) two (calibrated) images observing
the same scene, what type of 3D information can
we extract ?
1
Intrinsics
(focal length, radial distortion..)
Extrinsics
(optical center, camera orentation)
![Page 5: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/5.jpg)
Overview
June 2014
Stereo matching
Multi-View Stereo
Beyond free-form surface reconstruction
2
![Page 6: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/6.jpg)
June 2014
1. Stereo matching
![Page 7: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/7.jpg)
P
optical center
left camera
F
Principle
optical center
right camera
In the 3D world,
June 2014 3
![Page 8: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/8.jpg)
A
P
optical center
left camera
F
Principle
optical center
right camera
In the 3D world,
June 2014 3
![Page 9: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/9.jpg)
A
P
Principle
optical center
left camera
optical center
right camera
In the 3D world,
June 2014 3
![Page 10: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/10.jpg)
A
P
optical center
left camera
optical center
right camera
In the 3D world,
Principle
June 2014 3
![Page 11: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/11.jpg)
A
P
optical center
left camera
optical center
right camera
In the 3D world,
Principle
epipolar plane
June 2014 3
![Page 12: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/12.jpg)
A
P
optical center
left camera
optical center
right camera
In the 3D world,
Principle
epipolar plane
epipolar lines
June 2014 3
![Page 13: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/13.jpg)
A
?
optical center
left camera
optical center
right camera
In the 3D world,
Principle
P
June 2014 3
A
![Page 14: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/14.jpg)
A
?
optical center
left camera
optical center
right camera
In the 3D world,
Principle
P
June 2014 3
![Page 15: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/15.jpg)
A
?
optical center
left camera
optical center
right camera
In the 3D world,
Principle
matching
cost
≠
June 2014 3
![Page 16: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/16.jpg)
A
?
optical center
left camera
optical center
right camera
In the 3D world,
Principle
≠
matching
cost
June 2014 3
![Page 17: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/17.jpg)
A
?
optical center
left camera
optical center
right camera
In the 3D world,
Principle
≠
matching
cost
June 2014 3
![Page 18: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/18.jpg)
A
optical center
left camera
optical center
right camera
In the 3D world,
Principle
matching
cost
June 2014 3
![Page 19: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/19.jpg)
A
P
optical center
left camera
optical center
right camera
In the 3D world,
Principle
matching
cost
June 2014 3
![Page 20: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/20.jpg)
Principle Z
June 2014 4
![Page 21: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/21.jpg)
Principle Z
June 2014 4
![Page 22: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/22.jpg)
Principle Z
June 2014 4
Depth map
![Page 23: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/23.jpg)
Principle
left image
(reference) right image
Base B Z
depth z
focal
length f
June 2014 4
![Page 24: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/24.jpg)
Principle Z
left image
(reference) right image
June 2014 4
![Page 25: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/25.jpg)
Z
disparity d
Principle
left image
(reference) right image
June 2014 4
![Page 26: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/26.jpg)
Z
disparity d
Base B
depth z
focal
length f
d
fBz
Principle
left image
(reference) right image
June 2014 4
![Page 27: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/27.jpg)
June 2014
What matching cost ?
C( , ) 0 and C( , ) > 0
![Page 28: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/28.jpg)
June 2014
What matching cost ?
C( , ) 0 and C( , ) > 0
Sum of Absolute Distances (SAD)
Sum of Square Differences (SSD)
Normalized Cross Correlation (NCC)
Mutual Information (MI)
… potentially object-based, eg line segments
[Hirschmuller and Scharstein, Evaluation of cost functions for stereo
matching, CVPR 2007]
![Page 29: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/29.jpg)
June 2014
What matching constraints ?
Uniqueness
Ordering
Spatial coherence (or smoothness)
Sharp feature preservation
…
7
![Page 30: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/30.jpg)
June 2014
What mechanism ?
Local
greedy local matching, eg pixel-per-pixel matching
Semi-global
spatial coherence along multiple 1D paths, eg SGM
Global
spatial coherence in 2D domain, eg MRF-based
8
![Page 31: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/31.jpg)
June 2014
http://vision.middlebury.edu/stereo/
Middlebury benchmark
150 algorithms evaluated
9
![Page 32: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/32.jpg)
June 2014
2. Multi-View Stereo
![Page 33: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/33.jpg)
Principle
Stereo Matching • 2 input images
• 2.5D output representation
(depth maps)
June 2014 10
![Page 34: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/34.jpg)
Principle
Stereo Matching • 2 input images
• 2.5D output representation
(depth maps)
Multi-View Stereo (MVS) • N input images
• 3D output representation
(eg meshes)
June 2014 10
![Page 35: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/35.jpg)
Principle
Stereo Matching • 2 input images
• 2.5D output representation
(depth maps)
Multi-View Stereo (MVS) • N input images
• 3D output representation
(eg meshes)
How to reconstruct the 3D shape of
an object from N calibrated images?
June 2014 10
![Page 36: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/36.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 37: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/37.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 38: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/38.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 39: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/39.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 40: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/40.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 41: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/41.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 42: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/42.jpg)
Depth map fusion
A natural extension of Stereo Matching
June 2014 11
![Page 43: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/43.jpg)
Depth map fusion
A natural extension of Stereo Matching
time-consuming
occlusions & conflicts
sharp features
June 2014 11
![Page 44: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/44.jpg)
Point cloud generation
Why not using 3D points instead of depth maps ?
June 2014 12
![Page 45: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/45.jpg)
Point cloud generation
Why not using 3D points instead of depth maps ?
For each pair of images,
Keypoint extraction
(SIFT, Harris corners..)
June 2014 12
![Page 46: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/46.jpg)
Point cloud generation
Why not using 3D points instead of depth maps ?
For each pair of images,
Keypoint extraction
(SIFT, Harris corners..) Keypoint descriptor
matching
June 2014 12
![Page 47: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/47.jpg)
Point cloud generation
Why not using 3D points instead of depth maps ?
For each pair of images,
Keypoint extraction
(SIFT, Harris corners..) Keypoint descriptor
matching
June 2014 12
![Page 48: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/48.jpg)
Point cloud generation
Why not using 3D points instead of depth maps ?
For each pair of images,
natural unit element in 3D
fast
capture well sharp features
Keypoint extraction
(SIFT, Harris corners..) Keypoint descriptor
matching
June 2014 12
![Page 49: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/49.jpg)
Point cloud generation
At the scale of a facade
June 2014 13
Input images Keypoints detected in one image
3D point cloud
![Page 50: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/50.jpg)
Point cloud generation
At the scale of a facade
June 2014 13
Input images Keypoints detected in one image
3D point cloud
Noise &
outliers !
![Page 51: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/51.jpg)
Point cloud generation
At the scale of a city
June 2014 14
![Page 52: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/52.jpg)
Surface reconstruction
From (defect-laden) point cloud to surface/volume representations
June 2014 15
Data-structure : mesh with triangular facets
with borders
watertight
![Page 53: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/53.jpg)
Surface reconstruction
An old topic in computer graphics
June 2014 16
![Page 54: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/54.jpg)
Surface reconstruction
An old topic in computer graphics
but without considering photo-consistency
[Berger et al, State of the art in surface reconstruction
from point clouds, Eurographics 2014]
June 2014 16
![Page 55: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/55.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
June 2014 17
![Page 56: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/56.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
1. Point cloud generation
2. Visibility consistent mesh extraction
3. Variationnal refinement with photo-consistency
June 2014 17
![Page 57: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/57.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
June 2014 18
![Page 58: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/58.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
June 2014 18
![Page 59: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/59.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
Label each Delaunay cell as inside or outside the observed
objects using visibility
+1 +1
+1 +1 -1
Before the vertex: +1 (outside)
After the vertex: -1 (inside)
June 2014 18
![Page 60: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/60.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
Label each Delaunay cell as inside or outside the observed
objects using visibility
out in out
out out
out
in
June 2014 18
![Page 61: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/61.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
Label each Delaunay cell as inside or outside the observed
objects using visibility
out in out
out out
out
in
June 2014 18
![Page 62: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/62.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
2. Visibility consistent mesh extraction
Partition the space with a 3D Delaunay triangulation
Label each Delaunay cell as inside or outside the observed
objects using visibility
June 2014 18
![Page 63: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/63.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
3. Variationnal refinement with photo-consistency
June 2014 19
![Page 64: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/64.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
3. Variationnal refinement with photo-consistency
June 2014 19
![Page 65: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/65.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
3. Variationnal refinement with photo-consistency
Minimize the reprojection error of image 1 into image 2
according to the surface S
June 2014 19
![Page 66: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/66.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
June 2014 20
![Page 67: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/67.jpg)
Focus on [Vu et al, High accuracy and visibility-consistent
dense multiview Stereo, PAMI 2012]
Surface reconstruction
now a company
June 2014 21
![Page 68: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/68.jpg)
June 2014
3. Beyond free-form surface
reconstruction
1
![Page 69: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/69.jpg)
What can we do next ?
June 2014 22
![Page 70: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/70.jpg)
What can we do next ?
June 2014
More compact and meaningful mesh
22
![Page 71: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/71.jpg)
What can we do next ?
June 2014
More compact and meaningful mesh
High model complexity
11M facets
22
![Page 72: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/72.jpg)
What can we do next ?
June 2014
More compact and meaningful mesh
Free of geometric structure
22
planar surface
![Page 73: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/73.jpg)
What can we do next ?
June 2014
More compact and meaningful mesh
Free of geometric structure
Free of semantics
roof
facade
22
![Page 74: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/74.jpg)
1st idea: Hybrid surfaces
June 2014
High model complexity
No structure
Free-form reconstruction
(Poisson algorithm)
23
![Page 75: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/75.jpg)
June 2014
If you can detect the right
set of geometric primitives
23
1st idea: Hybrid surfaces
![Page 76: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/76.jpg)
June 2014
and the right primitive
adjacencies
23
1st idea: Hybrid surfaces
![Page 77: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/77.jpg)
June 2014
low model complexity
structure-aware
23
1st idea: Hybrid surfaces
![Page 78: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/78.jpg)
June 2014
low model complexity
structure-aware
But No guarantee of finding the right primitives and right adjacency graph
23
1st idea: Hybrid surfaces
![Page 79: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/79.jpg)
June 2014
low model complexity
structure-aware
But No guarantee of finding the right primitives and right adjacency graph
No guarantee that the object can be entirely explained by primitives
23
1st idea: Hybrid surfaces
![Page 80: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/80.jpg)
June 2014
We need both structure and free-form!
Structure: primitive assembling for describing regular components
Free-form: smooth mesh for preserving the details
23
1st idea: Hybrid surfaces
![Page 81: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/81.jpg)
Hybrid surfaces (by point set structuring)
June 2014 24
![Page 82: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/82.jpg)
June 2014
Hybrid surfaces (by point set structuring)
24
![Page 83: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/83.jpg)
June 2014
Hybrid surfaces (by point set structuring)
24
![Page 84: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/84.jpg)
June 2014
Texte courant dunt am iriure commolut eumsandio odolor sectem
Hybrid surfaces (by point set structuring)
24
![Page 85: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/85.jpg)
25
Hybrid surfaces (by point set structuring)
![Page 86: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/86.jpg)
highly freeform (no structure)
high accuracy
high running time
Hybrid surfaces (by point set structuring)
25
![Page 87: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/87.jpg)
structure ↑ (freeform ↓ )
accuracy ↓
running time ↓
Hybrid surfaces (by point set structuring)
25
![Page 88: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/88.jpg)
fully structured
low accuracy
low running time
Hybrid surfaces (by point set structuring)
25
![Page 89: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/89.jpg)
26
Hybrid surfaces (by point set structuring)
Reduction of the structure complexity while preserving details
![Page 90: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/90.jpg)
June 2014
Hybrid surfaces (by shape sampling)
27
MVS images and a rough initial surface
![Page 91: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/91.jpg)
June 2014
Hybrid surfaces (by shape sampling)
27
MVS images and a rough initial surface
Goal: refine the surface while sampling geometric primitives (planes,
cylinders…) by Jump-Diffusion
![Page 92: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/92.jpg)
June 2014
Hybrid surfaces (by shape sampling)
27
MVS images and a rough initial surface
Goal: refine the surface while sampling geometric primitives (planes,
cylinders…) by Jump-Diffusion
![Page 93: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/93.jpg)
June 2014
Hybrid surfaces (by shape sampling)
28
![Page 94: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/94.jpg)
June 2014
Hybrid surfaces (by shape sampling)
29
![Page 95: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/95.jpg)
Hybrid surfaces (by planimetric map)
June 2014 30
3D Primitive detection
![Page 96: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/96.jpg)
Hybrid surfaces (by planimetric map)
June 2014 30
3D Primitive detection 2.5D planimetric map
![Page 97: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/97.jpg)
Hybrid surfaces (by planimetric map)
June 2014 30
3D Primitive detection 2.5D planimetric map
hybrid surface
![Page 98: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/98.jpg)
June 2014
3D-model
Aerial image (from Google Maps)
Hybrid surfaces (by planimetric map)
31
![Page 99: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/99.jpg)
June 2014
Hybrid surfaces (by planimetric map)
32
![Page 100: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/100.jpg)
2nd idea: 3D-Block assembling
June 2014
A building = an assemblage of
elementary urban 3D-models
33
![Page 101: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/101.jpg)
2nd idea: 3D-Block assembling
June 2014
A building = an assemblage of
elementary urban 3D-models
33
Elevation map (DEM) Polygonalization of building
footprints by point process
![Page 102: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/102.jpg)
2nd idea: 3D-Block assembling
June 2014
A building = an assemblage of
elementary urban 3D-models
33
Elevation map (DEM) Polygonalization of building
footprints by point process
![Page 103: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/103.jpg)
2nd idea: 3D-Block assembling
June 2014
A building = an assemblage of
elementary urban 3D-models
33
Elevation map (DEM) Polygonalization of building
footprints by point process
Building reconstruction by
3D-block sampling
![Page 104: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/104.jpg)
2nd idea: 3D-Block assembling
June 2014
A building = an assemblage of
elementary urban 3D-models
33
Elevation map (DEM) Polygonalization of building
footprints by point process
Building reconstruction by
3D-block sampling
![Page 105: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/105.jpg)
June 2014
Pixel resolution: 0.70 m Pixel resolution: 0.10 m
34
3D-Block assembling
![Page 106: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/106.jpg)
Urban 3D models with different Levels Of Details
June 2014 35
3rd idea: LOD generation
![Page 107: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/107.jpg)
Urban 3D models with different Levels Of Details
3rd idea: LOD generation
June 2014 35
![Page 108: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/108.jpg)
1. Semantical segmentation of dense mesh into 4 classes of interest
LOD generation
June 2014
Classification Abstraction Reconstruction
roof
facade
tree
ground
36
![Page 109: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/109.jpg)
1. Semantical segmentation of dense mesh into 4 classes of interest
Geometric descriptors
LOD generation
June 2014 36
![Page 110: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/110.jpg)
1. Semantical segmentation of dense mesh into 4 classes of interest
Geometric descriptors
Markov Random Fields with feature-preserving potential
LOD generation
June 2014 36
![Page 111: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/111.jpg)
1. Semantical segmentation of dense mesh into 4 classes of interest
Geometric descriptors
Markov Random Fields with feature-preserving potential
LOD generation
June 2014 36
![Page 112: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/112.jpg)
LOD generation
June 2014
2. Abstraction of urban objects
Planar hypotheses
globally consistent
Tree localization
37
![Page 113: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/113.jpg)
LOD generation
June 2014
2. Abstraction of urban objects
Detection of planar primitives with global regularities
(parallelism, orthogonality, Z-symmetry, coplanarity)
37
![Page 114: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/114.jpg)
LOD generation
June 2014
2. Abstraction of urban objects
Detection of planar primitives with global regularities
Iconization of trees and superstructures
37
![Page 115: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/115.jpg)
3. 3D modeling at various LOD
LOD generation
June 2014
Classification Abstraction Reconstruction
38
LOD0
LOD1
LOD2
LOD1
![Page 116: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/116.jpg)
3. 3D modeling at various LOD
3D arrangement of planar primitives filtered at a given LOD
LOD generation
June 2014
Classification Abstraction Reconstruction
38
![Page 117: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/117.jpg)
3. 3D modeling at various LOD
3D arrangement of planar primitives filtered at a given LOD
Min-cut formulation with a discrete cost estimation
LOD generation
June 2014
Classification Abstraction Reconstruction
38
![Page 118: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/118.jpg)
June 2014
Citer mes papiers avec une photo
LOD generation
39
![Page 119: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/119.jpg)
June 2014
LOD generation
40
1 km² of Paris
input mesh: 11M facets
output: 175K facets (LOD2)
![Page 120: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/120.jpg)
References
June 2014 41
Hybrid surfaces by point set structuring Surface reconstruction through point set structuring, Eurographics’13
Hybrid surfaces by shape sampling A hybrid multi-view stereo algorithm for modeling urban scenes, IEEE PAMI 2013
Hybrid surfaces by planimetric map Creating large-scale city models from 3D-point clouds: A robust approach with hybrid
representation, IJCV 2012
3D-block assembling Structural approach for building reconstruction from a single DSM, IEEE PAMI 2010
LOD generation Recently submitted..
![Page 121: Dense image matching and surface reconstructionisprs-commission5.fbk.eu › sites › isprs5.fbk.eu › files › ...Partition the space with a 3D Delaunay triangulation June 2014](https://reader036.fdocuments.us/reader036/viewer/2022081522/5f11f5f5b7de9074a651d8d7/html5/thumbnails/121.jpg)
June 2014
Thank you!