Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane...
Transcript of Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane...
![Page 1: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/1.jpg)
Lecture 6 -Silvio Savarese 5-Feb-14
Professor Silvio Savarese
Computational Vision and Geometry Lab
Lecture 6Stereo SystemsMulti-view geometry
![Page 2: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/2.jpg)
Lecture 5 -Silvio Savarese 5-Feb-14
• Stereo systems• Rectification• Correspondence problem
• Multi-view geometry• The SFM problem• Affine SFM
Reading: [AZ] Chapter: 4 “Estimation – 2D perspective transformationsChapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation”Chapter: 11 “Computation of the Fundamental Matrix F”
[FP] Chapters: 10 “The geometry of multiple views”
Lecture 6Stereo SystemsMulti-view geometry
![Page 3: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/3.jpg)
• Epipolar Plane • Epipoles e1, e2
• Epipolar Lines
• Baseline
Epipolar geometry
O1O2
x2
X
x1
e1 e2
= intersections of baseline with image planes
= projections of the other camera center
For details see CS131A
Lecture 9
![Page 4: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/4.jpg)
OO’
pp’
P
Epipolar Constraint
0pEpT
E = Essential Matrix(Longuet-Higgins, 1981)
RTE
![Page 5: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/5.jpg)
OO’
pp’
P
Epipolar Constraint
0pFpT
F = Fundamental Matrix(Faugeras and Luong, 1992)
1
KRTKFT
![Page 6: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/6.jpg)
O1O2
X
e2
x1 x2
e2
Example: Parallel image planes
K K
x
y
z
• Epipolar lines are horizontal
• Epipoles go to infinity
• y-coordinates are equal
1
1
1 y
x
x
1
2
2 y
x
x
![Page 7: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/7.jpg)
Rectification: making two images “parallel”
H
Courtesy figure S. Lazebnik
For details on how to rectify two views see CS131A Lecture 9
![Page 8: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/8.jpg)
Why are parallel images useful?
• Makes the correspondence problem easier• Makes triangulation easy• Enables schemes for image interpolation
![Page 9: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/9.jpg)
Application: view morphingS. M. Seitz and C. R. Dyer, Proc. SIGGRAPH 96, 1996, 21-30
![Page 10: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/10.jpg)
Morphing without using geometry
![Page 11: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/11.jpg)
Rectification
x
x
x
y
y
y
î1
î0
îS
I0
I1
Is
P
C1
Cs
C0
![Page 13: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/13.jpg)
![Page 14: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/14.jpg)
![Page 15: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/15.jpg)
![Page 17: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/17.jpg)
Why are parallel images useful?
• Makes the correspondence problem easier• Makes triangulation easy• Enables schemes for image interpolation
![Page 18: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/18.jpg)
Point triangulation
O O’
P
e2
p p’e2
K Kx
y
z
u
v
u
v
![Page 19: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/19.jpg)
f
u u’
Baseline B
z
O O’
P
f
z
fBuu
'
Note: Disparity is inversely proportional to depth
Computing depth
= disparity
![Page 20: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/20.jpg)
Disparity maps
http://vision.middlebury.edu/stereo/
Stereo pair
Disparity map / depth map Disparity map with occlusions
u u’
z
fBuu
'
![Page 21: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/21.jpg)
Why are parallel images useful?
• Makes the correspondence problem easier• Makes triangulation easy• Enables schemes for image interpolation
![Page 22: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/22.jpg)
Correspondence problem
p’
P
p
O1O2
Given a point in 3d, discover corresponding observations
in left and right images [also called binocular fusion problem]
![Page 23: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/23.jpg)
Correspondence problem
p’
P
p
O1O2
Given a point in 3d, discover corresponding observations
in left and right images [also called binocular fusion problem]
![Page 24: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/24.jpg)
•A Cooperative Model (Marr and Poggio, 1976)
•Correlation Methods (1970--)
•Multi-Scale Edge Matching (Marr, Poggio and Grimson, 1979-81)
Correspondence problem
[FP] Chapters: 11
![Page 25: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/25.jpg)
Correlation Methods (1970--)
210
p p’
195 150 210
![Page 26: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/26.jpg)
Correlation Methods (1970--)
• Pick up a window around p(u,v)
p
![Page 27: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/27.jpg)
Correlation Methods (1970--)
• Pick up a window around p(u,v)
• Build vector w
• Slide the window along v line in image 2 and compute w’
• Keep sliding until w∙w’ is maximized.
![Page 28: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/28.jpg)
Correlation Methods (1970--)
Normalized Correlation; minimize:)ww)(ww(
)ww)(ww(
![Page 29: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/29.jpg)
Left Right
scanline
Correlation methods
Norm. corrCredit slide S. Lazebnik
![Page 30: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/30.jpg)
– Smaller window
- More detail
- More noise
– Larger window
- Smoother disparity maps
- Less prone to noise
Window size = 3 Window size = 20
Correlation methods
Credit slide S. Lazebnik
![Page 31: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/31.jpg)
Issues
•Fore shortening effect
•Occlusions
OO’
O O’
![Page 32: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/32.jpg)
O O’
Issues
• Small error in
measurements
implies large error
in estimating
depth
• It is desirable
to have small
B/z ratio!
![Page 33: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/33.jpg)
•Homogeneous regions
Issues
Hard to match pixels in these regions
![Page 34: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/34.jpg)
•Repetitive patterns
Issues
![Page 35: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/35.jpg)
Correspondence problem is difficult!
- Occlusions
- Fore shortening
- Baseline trade-off
- Homogeneous regions
- Repetitive patterns
Apply non-local constraints to help enforce the correspondences
![Page 36: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/36.jpg)
Results with window search
Data
Ground truth
Credit slide S. Lazebnik
Window-based matching
![Page 37: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/37.jpg)
Lecture 5 -Silvio Savarese 5-Feb-14
• Stereo systems• Rectification• Correspondence problem
• Multi-view geometry• The SFM problem• Affine SFM
Lecture 6Stereo SystemsMulti-view geometry
![Page 38: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/38.jpg)
Courtesy of Oxford Visual Geometry Group
Structure from motion problem
![Page 39: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/39.jpg)
Structure from motion problem
x1j
x2j
xmj
Xj
M1
M2
Mm
Given m images of n fixed 3D points
•xij = Mi Xj , i = 1, … , m, j = 1, … , n
![Page 40: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/40.jpg)
From the mxn correspondences xij, estimate:
•m projection matrices Mi
•n 3D points Xj
x1j
x2j
xmj
Xj
motion
structure
M1
M2
Mm
Structure from motion problem
![Page 41: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/41.jpg)
Affine structure from motion(simpler problem)
ImageWorld
Image
From the mxn correspondences xij, estimate:
•m projection matrices Mi (affine cameras)
•n 3D points Xj
![Page 42: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/42.jpg)
Finite cameras
p
q
r
R
Q
P
O
XTRKx
M
![Page 43: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/43.jpg)
??10
0100
0010
0001
TR
Question:
![Page 44: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/44.jpg)
Finite cameras
p
q
r
R
Q
P
O
XTRKx
10
TR
0100
0010
0001
KM 3x3
Canonical perspective projection matrix
Affine homography
(in 3D)
Affine
Homography
(in 2D)
100
y0
xs
K oy
ox
![Page 45: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/45.jpg)
p
q
r
R
Q
O
P
Affine cameras
100
yα0
xsα
oy
ox
K
10
TR
1000
0010
0001
KM
Canonical affine projection matrix
(points at infinity are mapped as points at infinity)
R’
Q’
P’
What’s the main difference???
![Page 46: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/46.jpg)
Transformation in 2D
Affinities:
1
y
x
H
1
y
x
10
tA
1
'y
'x
a
![Page 47: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/47.jpg)
Affine cameras
XTRKx
100
00
00
y
x
K
10
TR
1000
0010
0001
KM
10
bA
1000
]affine44[
1000
0010
0001
]affine33[ 2232221
1131211
baaa
baaa
M
12
1
232221
131211 XbAXx EucM
b
b
Z
Y
X
aaa
aaa
y
x
[Homogeneous]
[non-homogeneous
image coordinates] bAMMEuc
;1
PEucM
![Page 48: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/48.jpg)
Pp
p’
;1
PbAPp M
v
u bAM
Affine cameras
M = camera matrix
To recap: from now on we define M as the camera matrix for the affine case
![Page 49: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/49.jpg)
The Affine Structure-from-Motion Problem
Given m images of n fixed points Pj (=Xi) we can write
Problem: estimate the m 24 matrices Mi and
the n positions Pj from the mn correspondences pij .
2m n equations in 8m+3n unknowns
Two approaches:- Algebraic approach (affine epipolar geometry; estimate F; cameras; points)
- Factorization method
How many equations and how many unknown?
N of cameras N of points
![Page 50: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/50.jpg)
A factorization method –Tomasi & Kanade algorithm
C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization
method. IJCV, 9(2):137-154, November 1992.
• Centering the data
• Factorization
![Page 51: Lecture 6 - Silvio Savarese · Lecture 6 Stereo Systems Multi-view geometry •Epipolar Plane •Epipoles e 1, e 2 •Epipolar Lines •Baseline Epipolar geometry O 1 O 2 x 2 X x](https://reader030.fdocuments.us/reader030/viewer/2022041101/5eda99edb814eb156475a580/html5/thumbnails/51.jpg)
Next lecture
Multiple view geometry