Computer Vision Multiple View Geometry Marc Pollefeys COMP 256.
More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry...
Transcript of More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry...
More on single-view geometry class 10
Multiple View GeometryComp 290-089Marc Pollefeys
Multiple View Geometry course schedule
(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16 (no class) Projective 2D Geometry
Jan. 21, 23 Projective 3D Geometry (no class)
Jan. 28, 30 Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13 Camera Calibration Single View Geometry
Feb. 18, 20 Epipolar Geometry 3D reconstruction
Feb. 25, 27 Fund. Matrix Comp. Structure Comp.
Mar. 4, 6 Planes & Homographies Trifocal Tensor
Mar. 18, 20 Three View Reconstruction Multiple View Geometry
Mar. 25, 27 MultipleView Reconstruction Bundle adjustment
Apr. 1, 3 Auto-Calibration Papers
Apr. 8, 10 Dynamic SfM Papers
Apr. 15, 17 Cheirality Papers
Apr. 22, 24 Duality Project Demos
Single view geometry
Camera model
Camera calibration
Single view geom.
Gold Standard algorithmObjective
Given n≥6 2D to 2D point correspondences {Xi
↔xi
’}, determine the Maximum Likelyhood Estimation of P
Algorithm(i)
Linear solution:(a)
Normalization: (b)
DLT
(ii)
Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
(iii)
Denormalization:
ii UXX~ = ii Txx~ =
UP~TP -1=
~ ~~
More Single-View Geometry
•
Projective cameras and planes, lines, conics and quadrics.
•
Camera calibration and vanishing points, calibrating conic and the IAC
** CPPQ =T
coneQCPP =T
Action of projective camera on planes
[ ] [ ]⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡==
1ppp
10ppppPXx 4214321 Y
XYX
The most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation
(affine camera-affine transformation)
Action of projective camera on lines
forward projection
( ) μbaμPBPAμB)P(AμX +=+=+=
back-projection
lPT=Π
PXlX TT =Π
Action of projective camera on conics
back-projection to cone
CPPQ Tco =
[ ] ⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡= 00
0CKK0|KC0KQ
TT
T
co
example:
Images of smooth surfaces
The contour generator Γ
is the set of points X on S at which rays are tangent to the surface. The corresponding apparent contour γ
is the set of points x which are the image of X, i.e. γ
is the image of Γ
The contour generator Γ
depends only on position of projection center, γ
depends also on rest of P
Action of projective camera on quadrics
back-projection to cone
TPPQC ** = 0lPPQlQ T*T*T ==ΠΠ
The plane of Γ
for a quadric Q is camera center C is given by Π=QC (follows from pole-polar relation)
The cone with vertex V and tangent to the quadric Q is the degenerate Quadric: TT
CO (QV)(QV)-QV)QV(Q = 0VQCO =
The importance of the camera center
]C~|[IR'K'P'],C~|KR[IP −=−=
( ) PKRR'K'P' -1=
( ) ( ) xKRR'K'PXKRR'K'XP'x' -1-1 ===
( )-1KRR'K'HHx with x' ==
Moving the image plane (zooming)
( ) xKK'0]X|[IK'x'0]X|K[Ix
1-===
( ) ⎥⎦⎤
⎢⎣⎡ −
==10
x~k)(1kIKK'H T01-
⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡=
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡ −
=⎥⎦⎤
⎢⎣⎡ −
=
100kIK
10x~kA
10x~A
10x~k)(1kIK
10x~k)(1kIK'
TT0
T0
T0
T0
'/ ffk =
Camera rotation
xKRK0]X|K[Rx'0]X|K[Ix
1-===
-1KRKH =
conjugate rotation
{ }θθ ii ee −μ,μμ,
Synthetic view
(i)
Compute the homography that warps some a rectangle to the correct aspect ratio
(ii)
warp the image
Planar homography mosaicing
close-up: interlacingcan be important problem!
Planar homography mosaicingmore examples
Projective (reduced) notation
T4
T3
T2
T1 )1,0,0,0(X,)0,1,0,0(X,)0,0,1,0(X,)0,0,0,1(X ====
T4
T3
T2
T1 )1,1,1(x,)1,0,0(x,)0,1,0(x,)0,0,1(x ====
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−−
=dcdbda
000000
P
Tdcba ),,,(C 1111 −−−−=
Moving the camera center
motion parallax
epipolar line
What does calibration give?
xKd 1−=
⎥⎦⎤
⎢⎣⎡= 0d0]|K[Ix
( )( ) ( )( )21-T-T
211-T-T
1
2-1-TT
1
2T
21T
1
2T
1
)xK(Kx)xK(Kx
)xK(Kx
dddd
ddcos ==θ
An image l defines a plane through the camera center with normal n=KTl measured in the camera’s Euclidean frame
The image of the absolute conic
KRd0d]C~|KR[IPXx =⎟⎠⎞⎜
⎝⎛−== ∞
mapping between π∞
to an image is given by the planar homogaphy x=Hd, with H=KR
image of the absolute conic (IAC)
( ) 1-T-1T KKKKω ==− ( )1TCHHC −−a
(i)
IAC depends only on intrinsics(ii)
angle between two rays(iii)
DIAC=ω*=KKT
(iv)
ω ⇔ K (cholesky factorisation)(v)
image of circular points
( )( )2T
21T
1
2T
1
ωxxωxx
ωxxcos =θ
A simple calibration device
(i)
compute H for each square (corners (0,0),(1,0),(0,1),(1,1))
(ii)
compute the imaged circular points H(1,±i,0)T
(iii)
fit a conic to 6 circular points(iv)
compute K from ω
through cholesky factorization
(= Zhang’s calibration method)
Orthogonality = pole-polar w.r.t. IAC
The calibrating conic
1T K1
11
KC −−
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
−=
Vanishing points( ) ( ) λKdaλPDPAλPXλx +=+==
( ) ( ) KdλKda limλ xlimvλλ
=+==∞→∞→
KdPXv == ∞
ML estimate of a vanishing point from imaged parallel scene lines
Vanishing lines
Orthogonality relation
( )( )2T
21T
1
2T
1
ωvvωvv
ωvvcos =θ
Five constraints gives us five equations and can determine w
Calibration from vanishing points and lines
Assumes zero skew, square pixels and 3 orthogonal
vanishing points
Principal point is the orthocenter of the trinagle made of 3 orthogonol vanishing lines
Assume zero skew, square pixels, calibrating conic is a circle;How to find it, so that we can get K?
Assume zero skew , square pixels, and principal point is at the image centerThen IAC is diagonal{1/f^2, 1/f^2,1) i.e. one degree of freedom need one more Constraint to determine f, the focal length two vanishing points corresponding To orthogonal directions.
Next class: Two-view geometry Epipolar geometry