Post on 29-Jan-2016
description
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
3-D Computer Vision3-D Computer VisionCSc 83020CSc 83020
Camera CalibrationCamera Calibration
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera CalibrationCamera Calibration Problem: Estimate camera’s extrinsic & Problem: Estimate camera’s extrinsic &
intrinsic parameters.intrinsic parameters. Method: Use image(s) of known scene.Method: Use image(s) of known scene. Tools:Tools:
Geometric camera models.Geometric camera models. SVD and constrained least-squares.SVD and constrained least-squares. Line extraction methods.Line extraction methods.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera CalibrationCamera Calibration
FeatureExtraction
PerspectiveEquations
From Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Perspective Projection, Perspective Projection, Remember?Remember?
fZ Z
Xfx
XO
x
From Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Intrinsic Camera ParametersIntrinsic Camera Parameters
Determine the intrinsic parameters Determine the intrinsic parameters of a camera (with lens)of a camera (with lens)
What are What are Intrinsic ParametersIntrinsic Parameters??
(can you name 7?)(can you name 7?)
From Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Intrinsic ParametersIntrinsic Parameters
fZ
XO
yx oo ,center image
yx ss , size pixel
flength focal
Z
Xfx
21, distortion lens kk
From Sebastian Thrun and Jana Kosecka
Image center(ox, oy)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Intrinsic Camera ParametersIntrinsic Camera Parameters IntrinsicIntrinsic Parameters: Parameters:
Focal Length Focal Length ff Pixel size Pixel size ssx x ,, ssyy
Image center Image center oox x ,, ooyy
(Nonlinear radial distortion coefficients (Nonlinear radial distortion coefficients kk1 1 ,, kk22…)…) Calibration = Determine the intrinsic Calibration = Determine the intrinsic
parameters of a cameraparameters of a cameraFrom Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Coordinate FramesCoordinate Frames
Xw
YwZw
World Coordinate Frame
Xc
Yc
Zc
Camera Coordinate Frame
Image Coordinate Frame
Pixel Coordinates
IntrinsicParameters
ExtrinsicParameters
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Why Calibrate?Why Calibrate?
Image
Image point
Calibration: relates points in the image to rays in the scene
Scene ray
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Why Calibrate?Why Calibrate?
Calibration: relates points in the image to rays in the scene
Image
Image point x
y
xy
Scene ray
z
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Example Calibration PatternExample Calibration Pattern
Calibration Pattern: Object with features of known size/geometryFrom Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Harris Corner DetectorHarris Corner Detector
From Sebastian Thrun and Jana Kosecka
Perspective CameraPerspective Camera
(X,Y,Z)
(x,y,z)
Center ofProjection
r r’
r =(x,y,z)
r’=(X,Y,Z)
r/f=r’/Z
f: effective focal length:distance of image plane from O.
x=f * X/Zy=f * Y/Zz=f
Extrinsic Parameters (2Extrinsic Parameters (2nd nd
formulation)formulation)
Pc=R Pw +TRotation followed by translation
R same as beforeT different
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
The Rotation MatrixThe Rotation Matrix
R * R = R * R = I =>R = ROrthonormal MatrixDegrees of freedom?
1 0 0I= 0 1 0
0 0 1
T T
T-1
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Perspective Camera ModelPerspective Camera Model Step 1: Transform into camera Step 1: Transform into camera
coordinatescoordinates
Step 2: Transform into image Step 2: Transform into image coordinatescoordinates
yc
c
yim
xc
c
xim
oZ
Y
s
fy
oZ
X
s
fx
~
~
~
~
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
~
~
~
From Sebastian Thrun and Jana Kosecka
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Intrinsic ParametersIntrinsic Parameters
YZ
fy
XZ
fx
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Image and Camera FramesImage and Camera Frames
Ximage
Yimage
Xcamera
Ycamera
(xim, yim)
(ox,oy)
Zcamera
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Geometric ModelGeometric Model
•Transformation fromWorld to Camera Frame.•Perspective projection (f, R, T)
Point in Camera Frame
•Transformation from Imageto Camera Frame.
(ox,oy,sx,sy)•No distortion!
3D Point inCameraCoordinateFrame
x=
y=
XcT
ZcTYcT
ZcT
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration: IssuesCamera Calibration: Issues
Which parameters need to be Which parameters need to be estimated.estimated. Focal length, image center, aspect ratioFocal length, image center, aspect ratio Radial distortionsRadial distortions
What kind of accuracy is needed.What kind of accuracy is needed. Application dependentApplication dependent
What kind of calibration object is used.What kind of calibration object is used. One plane, many planesOne plane, many planes Complicated three dimensional objectComplicated three dimensional object
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera CalibrationCamera Calibration
Calibration object Extracted features
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera CalibrationCamera Calibration
Extract centers of circles
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
yc
c
yim
xc
c
xim
oZ
Y
s
fy
oZ
X
s
fx
zwwwc
ywwwc
xwwwc
TZrYrXrZ
TZrYrXrY
TZrYrXrX
333231
232221
131211
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
yc
c
y
xc
c
x
oZ
Yfy
oZ
Xfx
zwwwc
ywwwc
xwwwc
TZrYrXrZ
TZrYrXrY
TZrYrXrX
333231
232221
131211
In the remaining slidesx means xim and y means yim
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
yc
cx
xc
c
x
oZ
Yfy
oZ
Xfx
zwwwc
ywwwc
xwwwc
TZrYrXrZ
TZrYrXrY
TZrYrXrX
333231
232221
131211
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
Extrinsic Parameters 1) Rotation matrix R (3x3) 2) Translation vector T (3x1)
Intrinsic Parameters 1) fx=f/sx, length in effective horizontal pixel size units.2) α=sy/sx, aspect ratio.3) (ox,oy), image center coordinates.4) Radial distortion coefficients.
Total number of parameters (excluding distortion): ?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
zwww
ywww
yy
zwww
xwww
xx
TZrYrXr
TZrYrXrfoy
TZrYrXr
TZrYrXrfox
333231
232221
333231
131211
1) Assume that image center is known.2) Solve for the remaining parameters.3) Use N image points and their corresponding N world points
),( ii yx
Twi
wi
wi ZYX ],,[
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
zwww
ywww
y
zwww
xwww
x
TZrYrXr
TZrYrXrfy
TZrYrXr
TZrYrXrfx
333231
232221
333231
131211
1) Assume that image center is known.2) Solve for the remaining parameters.3) Use N image points and their corresponding N world points
),( ii yx
Twi
wi
wi ZYX ],,[
(1)
Here x means xim - ox and y means yim - oy
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
)(
)(
131211
232221
xwi
wi
wixi
ywi
wi
wiyi
TZrYrXrfy
TZrYrXrfx
1) Assume that image center is known.2) Solve for the remaining parameters.3) Use N image points and their corresponding N world points
),( ii yx
Twi
wi
wi ZYX ],,[
(2)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
xy
iwii
wii
wii
iwii
wii
wii
TvTv
rvrv
rvrv
rvrv
vyvZyvYyvXy
vxvZxvYxvXx
84
137233
126222
115211
8765
4321
,
,
,
,
0 (3)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
xy
iwii
wii
wii
iwii
wii
wii
TvTv
rvrv
rvrv
rvrv
vyvZyvYyvXy
vxvZxvYxvXx
84
137233
126222
115211
8765
4321
,
,
,
,
0 (3)
0vA
How would we solve this system?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Basic EquationsBasic Equations
(3)0vA
NwNN
wNN
wNNN
wNN
wNN
wNN
wwwwww
wwwwww
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
A
........
........22222222222222
11111111111111
How would we solve this system?
Rank of matrix A?
Solution up to a scale factor.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Singular Value Singular Value DecompositionDecomposition
TUDVA Appendix A.6 (Trucco)
A: m x nU: m x m, columns orthogonal unit vectors.V: n x n , -//-D: m x n , diagonal. The diagonal elements
σi are the singular values σ1>= σ2>= … >= σn >= 0
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Singular Value Singular Value DecompositionDecompositionAppendix A.6 (Trucco)
1. Square A non-singular iff σi != 02. For square A C=σ1/σN is the condition number3. For rectangular A # of non-zero σi is the rank4. For square non-singular A:5. For square A, pseudoinverse:6. Singular values of A = square roots of
eigenvalues of and7. Columns of U, V Eigenvectors of 8. Frobenius norm of a matrix
TUVDA 11 TUVDA 1
0
TAA AAT
TAA AAT
TUDVA
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Singular Value Singular Value DecompositionDecompositionAppendix A.6 (Trucco)
0vA
UDVA T
If rank(A)=n-1 (7 in our case) then
the solution is the eigenvector which corresponds to the ONLY zero eigenvalue.
Solution up to a scale factor.
Solving for vSolving for v
(3)0vA
NwNN
wNN
wNNN
wNN
wNN
wNN
wwwwww
wwwwww
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
A
........
........22222222222222
11111111111111
How would we solve this system: SVD. Solution:
Uknown scale factor γ=?
Aspect ratio α=?
),,,,,,,( 131211232221 xy TrrrTrrr v
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Solving for Tz and fx?Solving for Tz and fx?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Solving for Tz and fx?Solving for Tz and fx?
)(
)(
131211
333231
xw
iw
iw
ix
zw
iw
iw
ii
TZrYrXrf
TZrYrXrx
b
x
z
f
TA
How would we solve this system?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Solving for Tz and fx?Solving for Tz and fx?
)(
)(
131211
333231
xw
iw
iw
ix
zw
iw
iw
ii
TZrYrXrf
TZrYrXrx
b
x
z
f
TA
How would we solve this system?
bTT
x
z AA)Af
T 1^
^
(
Solution in the least squares sense.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Models (linear Camera Models (linear versions)versions)
•Transformation fromWorld to Camera Frame.•Perspective projection (f, R, T)
Point in Camera Frame
•Transformation from Imageto Camera Frame.
(ox,oy,sx,sy)•No distortion!
3D Point inWorld CoordinateFrame
x=
y=
Camera Models (linear Camera Models (linear versions)versions)
World Point(Xw, Yw,Zw)
Measured Pixel(xim, yim)
Elegant decomposition.No distortion!
?
HomogeneousCoordinates
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration – Other Camera Calibration – Other methodmethod
Extracted features
1
Z
Y
X
P
w
v
u
w
vy
w
ux
Step 1: Estimate PStep 2: Decompose P into internal and external parameters R,T,C
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
1
Z
Y
X
P
w
v
u
w
vy
w
ux
vwy
uwx
2423222134333231
1413121134333231
pZpYpXppZpYpXpy
pZpYpXppZpYpXpx
Each point (x,y) gives us two equations
w
uv
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
2423222134333231
1413121134333231
pZpYpXppZpYpXpy
pZpYpXppZpYpXpx
0
0
1
0000
0
1
000
34
33
32
31
24
23
22
21
14
13
12
11
p
p
p
p
p
p
p
p
p
p
p
p
y
x
yZ
xZ
yY
xY
yX
xX
ZYX
ZYX
Each corner (x,y) gives us two equations
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
0
0
0
0
1
0000
0
1
000
1
0000
0
1
000
34
33
32
31
24
23
22
21
14
13
12
11
p
p
p
p
p
p
p
p
p
p
p
p
y
x
yZ
xZ
yY
xY
yX
xX
ZYX
ZYX
y
x
yZ
xZ
yY
xY
yX
xX
ZYX
ZYX
2n
n points gives us 2n equations
A
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
0
0
0
0
1
0000
0
1
000
1
0000
0
1
000
34
33
32
31
24
23
22
21
14
13
12
11
p
p
p
p
p
p
p
p
p
p
p
p
y
x
yZ
xZ
yY
xY
yX
xX
ZYX
ZYX
y
x
yZ
xZ
yY
xY
yX
xX
ZYX
ZYX
2n
A
pp
Amin0pAWe need to solve
In the presence of noise we need to solve
The solution is given by the eigenvector with the smallest eigenvalue of AAT
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
i i
ii
i
ii w
vy
w
ux
22
minp
1i
i
i
i
i
i
Z
Y
X
P
w
v
u
The result can be improved throughnon-linear minimization.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Camera Calibration: Step 1Camera Calibration: Step 1
Extracted features
The result can be improved throughnon-linear minimization.
i i
ii
i
ii w
vy
w
ux
22
minp
1i
i
i
i
i
i
Z
Y
X
P
w
v
u
Minimize the distance between thepredicted and detected features.