1
3D reconstruction3D reconstruction from un from uncalibrated imagescalibrated images
Young Ki Baik
CV Lab
2
ContentsContents
Introduce Basic geometrical theory Overview – 3D reconstruction
Conditions for 3D reconstruction and Solution Correspondence Camera parameters and motion
Results Experimental results and demonstration
Future Works
3
Introduction(1)Introduction(1)
3D point
Camera
3D object
Camera
Mapping to images
mapping
Image plane
Camera system for obtaining images
4
Introduction(2)Introduction(2)
3D point
Camera
3D object
Camera
3D reconstruction from images Point correspondence
Camera parameter and motion
3D reconstruction system to make 3D object
5
3D reconstruction3D reconstruction from uncalibrated i from uncalibrated imagesmages Overview
Image Sequence
Feature Extraction/ Matching
Relating Image
Projective Reconstructi
on
Auto-Calibration
Dense Matching
3D Model Building
6
Conditions for Conditions for 3D 3D reconstructionreconstruction Correspondence
Feature extraction Harris corner method SIFT method
Scale Invariant Feature Transform
Initial feature matching Template matching (Image base descriptor) Descriptor (SIFT-d, PCA-d, SIFT-d+PCA-d, …)
Feature matching RANdom SAmple Consensus
To eliminate outlier
7
Conditions for Conditions for 3D 3D reconstructionreconstruction Correspondence
Guide matching To get more correspondence Using previous features and Geometry information
xxxxFxFx ,,,Cost SADdd T
Difference Absolute of Sum:) (
distanceEuclidean :) (
pair Matching:
matrix lFundamenta :
constant:,
SAD
d
xx,
F
Geometry based distance value
using fundamental
matrix
Correlation based cost
value
About 2 times more correspondence
8
Conditions for Conditions for 3D 3D reconstructionreconstruction Camera parameter and motion (Using Self-calibration)
Dual Absolute Conic Hartley ’94 / Hartley ’99, David Nistér IJCV 2004 ( + cheirality sol
ution )
Dual Absolute Quadric Triggs’97 M.Pollefeys et al. PAMI’98, ECCV 2002, IJCV 2004
Dual Absolute Quadric
M. Pollefeys
9
Conditions for Conditions for 3D 3D reconstructionreconstruction Constraints for self-calibration
Constant internal parameter Fixed camera
K1 = K2 = …
Known internal parameter Rectangular pixel : s = 0 Square pixel : s = 0, fx = fy
Principle point known : ( ux , uy ) = image center
1yy
xx
uf
usf
K
10
Experiments and resultsExperiments and results
Result using rig Rig
Calibration using vanishing point
DAQ (using weighted linear equation)
100
0.652765.700
5.187650.725 766.67
100
0.8635-752.2830
13.6090 752.283
100
0763.2190
00763.219
Using the calibration rig information
Using the manual
vanishing points input
Self-calibration result using rig correspondence
only
Self-calibration result is similar to the method using calibration rig.
11
Experiments and resultsExperiments and results
Real scene test Assuming that self-calibration works well
12
Experiments and resultsExperiments and results
Manual input to check self-calibration results Points : Correspondence information Line : Connection information
13
Experiments and resultsExperiments and results Test 1 (Pinball machine : 3 images)
Fig.1 Fig.2 Fig.3
5476 5609 8530
Fig.1-2 Fig.2-3 Fig.1-2-3
Initial
match
146 196 41
RANSAC 104 124
Guide
match
230 160 67
RANSAC 281 202
Key points
Match
14
Experiments and resultsExperiments and results Test 2 (Mask : 3 images)
Fig.1 Fig.2 Fig.3
1837 1420 1888
Fig.1-2 Fig.2-3 Fig.1-2-3
Initial
match
102 158 4
RANSAC 35 100
Guide
match
150 258 23
RANSAC 78 186
Key points
Match
15
Experiments and resultsExperiments and results Test 3 (Building : 6 images)
Fig.1 Fig.2 Fig.6
959 1064 1177
Fig.1-2 Fig.2-3 Fig.1~6
Initial
match
386 377 30
RANSAC 227 254
Guide
match
465 484 35
RANSAC 308 309
Key points
Match
16
Experiments and resultsExperiments and results Test 4 (House : 5 images)
Fig.1 Fig.2 Fig.5
3013 3084 2873
Fig.1-2 Fig.2-3 Fig.1~5
Initial
match
1023 973 15
RANSAC 656 716
Guide
match
1186 1216 54
RANSAC 911 909
Key points
Match
17
Future worksFuture works
Quasi-Dense matching technique and reconstruction To get more reliable results
Full side 3D reconstruction Using attaching algorithm
Bundle adjustment algorithm To reduce error
Full 3D reconstruction system Dense matching and 3D modeling
Top Related