CMPUT 412 3D Computer Vision
Presented by Azad Shademan
Feb. 13-15, 2007
Acknowledgement
• Marc Pollefeys’ slides (University of North Carolina), comp290-89 Spring 2003
http://www.cs.unc.edu/~marc/mvg/slides.html
Motivation
• Images form one of the most important feedback signals in robotics.
• Vision-based control:– Robotic Visual Servoing– Mobile Navigation
Eye-In-Hand Visual Servoing
More motivation
• Course project:– Two cameras looking at one scene– Need to “correspond” similar points in these two
imagesLaser projection on the wall
Image planeLeft camera
Image planeRight camera
Camera Model
Mostly pinhole camera model
Pinhole Camera Model
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
101
01
01
1Z
Y
X
f
f
Z
fY
fX
0|I)1,,(diagP ff
Pinhole Camera Model
Tyx
T pZfYpZfXZYX )/,/(),,(
principal pointT
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
Principle Point Offset
camX0|IKx
1y
x
pf
pf
K calibration matrix
Principle Point Offset
C~
-X~
RX~
cam
X10
RCR
1
10
C~
RRXcam
Z
Y
X
camX0|IKx XC~
|IKRx
t|RKP C~
Rt PXx
Camera Rotation and Orientation
X.xλor
1
10
t
1
1
1
11
λ3
PR
Z
Y
X
pf
pf
y
x
y
x
T
Hierarchy of transformations
vTv
tAProjective15dof
Affine12dof
Similarity7dof
Euclidean6dof
Intersection and tangency
Parallellism of planes,Volume ratios, centroids
Volume
10
tAT
10
tRT
s
10
tRT
Camera Calibration
• Procedure that finds the camera matrix, i.e.,– Intrinsic camera parameters– Extrinsic camera parameters
• Rotation• Orientation
• There is a lot to be done with calibrated cameras, BUT we are more interested in uncalibrated cameras (more challenging)
Some linear algebra to get started
• Singular value decomposition (SVD)– Decomposes a matrix as follows:
TVUΣA
Tnnnmmmnm VΣUA
IUU T
021 n
IVV T
nm
XXVT XVΣ T XVUΣ T
TTTnnn VUVUVUA 222111
000
00
00
00
Σ
2
1
n
Singular Value Decomposition
Singular Value Decomposition
• Homogeneous least-squares
• Span and null-space
• Closest rank r approximation
• Pseudo inverse
1X AXmin subject to nVX solution
nrrdiag ,,,,,, 121 0 ,, 0 ,,,,~
21 rdiag TVΣ~
UA~
TVUΣA
0000
0000
000
000
Σ 2
1
4321 UU;UU LL NS 4321 VV;VV RR NS
TUVΣA 0 ,, 0 ,,,, 112
11 rdiag
TVUΣA
Parameter estimation
• 2D homographyGiven a set of (xi,xi’), compute H (xi’=Hxi)
• 3D to 2D camera projectionGiven a set of (Xi,xi), compute P (xi=PXi)
• Fundamental matrixGiven a set of (xi,xi’), compute F (xi’TFxi=0)
Number of measurements required
• At least as many independent equations as degrees of freedom required
• Example:
Hxx'
11
λ
333231
232221
131211
y
x
hhh
hhh
hhh
y
x
2 independent equations / point8 degrees of freedom
4x2≥8
Approximate solutions
• Minimal solution4 points yield an exact solution for H
• More points– No exact solution, because
measurements are inexact (“noise”)– Search for “best” according to some cost
function– Algebraic or geometric/statistical cost
Direct Linear Transformation(DLT)
ii Hxx 0Hxx ii
i
i
i
i
xh
xh
xh
Hx3
2
1
T
T
T
iiii
iiii
iiii
ii
yx
xw
wy
xhxh
xhxh
xhxh
Hxx12
31
23
TT
TT
TT
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
Tiiii wyx ,,x
0hA i
Direct Linear Transformation(DLT)
• Equations are linear in h
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
0AAA 321 iiiiii wyx
0hA i
• Only 2 out of 3 are linearly independent
(indeed, 2 eq/pt)
0
h
h
h
x0x
xx0
3
2
1
TTT
TTT
iiii
iiii
xw
yw
(only drop third row if wi’≠0)• Holds for any homogeneous
representation, e.g. (xi’,yi’,1)
Direct Linear Transformation(DLT)
• Solving for H
0Ah 0h
A
A
A
A
4
3
2
1
size A is 8x9 or 12x9, but rank 8
Trivial solution is h=09T is not interesting
1-D null-space yields solution of interestpick for example the one with 1h
Direct Linear Transformation(DLT)
• Over-determined solution
No exact solution because of inexact measurementi.e. “noise”
0Ah 0h
A
A
A
n
2
1
Find approximate solution- Additional constraint needed to avoid 0, e.g.
- not possible, so minimize
1h Ah0Ah
DLT algorithmObjective
Given n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi
Algorithm
(i) For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed.
(ii) Assemble n 2x9 matrices Ai into a single 2nx9 matrix A
(iii) Obtain SVD of A. Solution for h is last column of V
(iv) Determine H from h
Normalized DLT algorithmObjective
Given n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi
Algorithm
(i) Normalize points
(ii) Apply DLT algorithm to
(iii) Denormalize solution
,x~x~ ii inormiinormi xTx~,xTx~
norm-1
norm TH~
TH
Reprojection error
221- Hx,xxHx, dd
22 x̂,xx̂x, dd
OpenCV
• cvFindHomography()
• cvFindFundamentalMat()
• …
cvFindHomographyFinds perspective transformation between two planes void cvFindHomography( const CvMat* src_points, const CvMat* dst_points,
CvMat* homography ); src_points
Point coordinates in the original plane, 2xN, Nx2, 3xN or Nx3 array (the latter two are for representation in homogeneous coordinates), where N is the number of points.
dst_pointsPoint coordinates in the destination plane, 2xN, Nx2, 3xN or Nx3 array (the latter two are for
representation in homogeneous coordinates) homography
Output 3x3 homography matrix. The function cvFindHomography finds perspective transformation H=||hij|| between the
source and the destination planes: [x',,i,,] [x,,i,,] s,,i,,[y',,i,,]~H*[y,,i,,] [1 ] [ 1] So that the back-projection error is minimized: sum_i((x',,i,,-(h11*x,,i,, + h12*y,,i,, + h13)/(h31*x,,i,, + h32*y,,i,, + h33))^2^+ (y',,i,,-
(h21*x,,i,, + h22*y,,i,, + h23)/(h31*x,,i,, + h32*y,,i,, + h33))^2^) -> min The function is used to find initial intrinsic and extrinsic matrices. Homography matrix is determined up to a scale, thus it is normalized to make h33=1.
Top Related