Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer...
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
1
Transcript of Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer...
![Page 1: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/1.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Stanford CS223B Computer Vision, Winter 2007
Lecture 4 Camera Calibration
Professors Sebastian Thrun and Jana Kosecka
CAs: Vaibhav Vaish and David Stavens
![Page 2: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/2.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software
![Page 3: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/3.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Camera Calibration
FeatureExtraction
PerspectiveEquations
![Page 4: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/4.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Perspective Projection, Remember?
fZ Z
Xfx
XO
x
![Page 5: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/5.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Intrinsic Camera Parameters
• Determine the intrinsic parameters of a camera (with lens)
• What are Intrinsic Parameters?
(can you name 7?)
![Page 6: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/6.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Intrinsic Parameters
fZ
XO yx oo ,center image
yx ss , size pixel
flength focal
Z
Xfx
21, distortion lens kk
![Page 7: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/7.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Intrinsic Camera Parameters
• Intrinsic Parameters:– Focal Length f
– Pixel size sx , sy
– Image center ox , oy
– (Nonlinear radial distortion coefficients k1 , k2…)
• Calibration = Determine the intrinsic parameters of a camera
![Page 8: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/8.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Why Intrinsic Parameters Matter
xx
oZ
X
s
fx
Z
Xfx
yy
oZ
Y
s
fy
Z
Yfy
![Page 9: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/9.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Questions
• Can we determine the intrinsic parameters by exposing the camera to many known objects?
• If so, – How often do we have to see the object?– How many features on the object do we need?
![Page 10: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/10.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Example Calibration Pattern
Calibration Pattern: Object with features of known size/geometry
![Page 11: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/11.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Harris Corner Detector(see Assignment 2)
![Page 12: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/12.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Why Tilt the Board?
![Page 13: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/13.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Experiment 1: Parallel Board
![Page 14: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/14.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
30cm10cm 20cm
Projective Perspective of Parallel Board
Z
Xfx
![Page 15: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/15.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Experiment 2: Tilted Board
![Page 16: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/16.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
30cm10cm 20cm
500cm50cm 100cm
Projective Perspective of Tilted Board
![Page 17: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/17.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Intrinsics and Extrinsics
• Intrinsics: – Focal Length f
– Pixel size sx , sy
– Image center ox , oy
• Extrinsics:– Location and orientation of k-th calib. pattern:
T,,,
![Page 18: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/18.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Perspective Camera Model
• Step 1: Transform into camera coordinates
• Step 2: Transform into image coordinates
),,,,(~
~
~
T
Z
Y
X
f
Z
Y
X
W
W
W
C
C
C
yc
c
yim
xc
c
xim
oZ
Y
s
fy
oZ
X
s
fx
~
~
~
~
![Page 19: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/19.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Perspective Camera Model
• Step 1: Transform into camera coordinates
• Step 2: Transform into image coordinates
yc
c
yim
xc
c
xim
oZ
Y
s
fy
oZ
X
s
fx
~
~
~
~
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
~
~
~
![Page 20: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/20.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
y
zW
W
W
YW
W
W
y
x
zW
W
W
XW
W
W
x
im
im
o
T
Z
Y
X
T
Z
Y
X
s
f
o
T
Z
Y
X
T
Z
Y
X
s
f
y
x
cossin0
sincos0
001
cos0sin
010
sin0cos
100
cossin0
sincos0
001
cos0sin
010
sin0cos
0cossin
cossin0
sincos0
001
cos0sin
010
sin0cos
100
cossin0
sincos0
001
cos0sin
010
sin0cos
0sincos
The Full Perspective Camera Model
![Page 21: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/21.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
The Calibration Problem
• Given – Calibration pattern with N corners– K views of this calibration pattern
• Recover the intrinsic parameters– We’ll also recover the extrinsics, but we won’t
care about them
![Page 22: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/22.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration Questions
• Can we determine the intrinsic parameters by exposing the camera to many known objects?
• If so, – How often do we have to see the object?– How many features on the object do we need?– Do we need to see object at angle? Yes.
![Page 23: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/23.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software
![Page 24: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/24.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration constraints
• Step 1: Transform into camera coordinates
• Step 2: Transform into image coordinates
yc
c
yim
xc
c
xim
okiZ
kiY
s
fkiy
okiZ
kiX
s
fkix
],[~
],[~
],[
],[~
],[~
],[
][
][
][
][
][
][
][cos][sin0
][sin][cos0
001
][cos0][sin
010
][sin0][cos
100
0][cos][sin
0][sin][cos
],[~
],[~
],[~
kT
kT
kT
iZ
iY
iX
kk
kk
kk
kk
kk
kk
kiZ
kiY
kiX
Z
Y
X
W
W
W
C
C
C
image
feature
k
i
![Page 25: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/25.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
min
][
][
][
][
][cos][sin0
][sin][cos0
001
][cos0][sin
010
][sin0][cos
100
][
][
][
][
][cos][sin0
][sin][cos0
001
][cos0][sin
010
][sin0][cos
0][cos][sin
][
][
][
][
][cos][sin0
][sin][cos0
001
][cos0][sin
010
][sin0][cos
100
][
][
][
][
][cos][sin0
][sin][cos0
001
][cos0][sin
010
][sin0][cos
0][sin][cos
],[
],[
,
2
ki
y
zW
W
W
YW
W
W
y
x
zW
W
W
XW
W
W
x
im
im
o
kT
iZ
iY
iX
kk
kk
kk
kk
kT
iZ
iY
iX
kk
kk
kk
kk
kk
s
f
o
kT
iZ
iY
iX
kk
kk
kk
kk
kT
iZ
iY
iX
kk
kk
kk
kk
kk
s
f
kiy
kix
Camera Calibration
![Page 26: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/26.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration by nonlinear Least Squares
• Least Mean Square
• Gradient descent:
J
},,,,]},[{]},[{]},[{]},[{{ yxyx oossfkTkkkX
0X
0X
J
)(1.0 11
kkk XX
JXX
min),,,,],[],[],[],[,
][
][
][
(],[
],[
,
2
kiyxyx
W
W
W
im
im oossfkTkkk
iZ
iY
iX
gkiy
kixJ
![Page 27: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/27.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
The Calibration Problem Quiz
• Given – Calibration pattern with N corners– K views of this calibration pattern
• How large would N and K have to be?
• Can we recover all intrinsic parameters?
![Page 28: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/28.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Intrinsic Parameters, Degeneracy
fZ
XO
yx ss , size pixel
flength focal
![Page 29: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/29.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Summary Parameters, Revisited
• Extrinsic
– Rotation
– Translation
• Intrinsic
– Focal length
– Pixel size
– Image center coordinates
][],[],[ kkk ][kT
f
),( yx oo
),( yx ssFocal length, in pixel units
Aspect ratioy
x
s
s xs
f
![Page 30: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/30.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
The Calibration Problem Quiz
• Given – Calibration pattern with N corners– K views of this calibration pattern
• How large would N and K have to be?
• Can we recover all intrinsic parameters?
N 1 3 1 3 4 4 6
K 1 1 3 3 3 4 6
![Page 31: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/31.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Constraints
• N points
• K images 2NK constraints
• 4 intrinsics (distortion: +2)
• 6K extrinsics
need 2NK ≥ 6K+4
(N-3)K ≥ 2
Hint: may not be co-linear
![Page 32: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/32.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
The Calibration Problem Quiz
N 1 3 1 3 4 4 6
K 1 1 3 3 3 4 6
No No No No Yes Yes Yes
need (N-3)K ≥ 2
Hint: may not be co-linear
![Page 33: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/33.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Problem with Least Squares
• Many parameters (=slow)
• Many local minima! (=slower)
![Page 34: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/34.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software
![Page 35: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/35.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Perspective Camera Model
• Step 1: Transform into camera coordinates
• Step 2: Transform into image coordinates
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
~
~
~
yc
c
yim
xc
c
xim
oZ
Yfy
oZ
Xfx
~
~
~
~
![Page 36: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/36.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration Model (extrinsic)
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
01
sin0cos
100
0cossin
0sincos
~
~
~
C
C
CC
C
Y
X
Z
f
Y
X~
~
~
(Homogeneous Coordinates)
(nonlinear perspective projection)
![Page 37: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/37.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Affine Problem Relaxation
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
rrr
rrr
rrr
Z
Y
X
333231
232221
131211
~
~
~
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
~
~
~
![Page 38: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/38.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Affine Problem Relaxation
ZWWWC
YWWWC
XWWWC
TZrYrXrZ
TZrYrXrY
TZrYrXrX
333231
232221
131211
~
~
~
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
rrr
rrr
rrr
Z
Y
X
333231
232221
131211
~
~
~
yc
c
yim
xc
c
xim
oZ
Yfy
oZ
Xfx
~
~
~
~
![Page 39: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/39.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration via SVD [see Trucco/Verri]
zWWW
xWWW
xx TZrYrXr
TZrYrXrfox
333231
131211
333231
232221
131211
rrr
rrr
rrr
R
zWWW
yWWW
yy TZrYrXr
TZrYrXrfoy
333231
232221
z
y
x
T
T
T
T
![Page 40: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/40.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration via SVD
N>=7 points, not coplanar
zWi
WW
xWi
WW
xi TZrYrXr
TZrYrXrfx
ii
ii
333231
131211
zWi
WW
yWi
WW
yi TZrYrXr
TZrYrXrfy
ii
ii
333231
232221
)()( 131211232221 xWi
WWxiy
Wi
WWyi TZrYrXrfyTZrYrXrfx iiii
![Page 41: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/41.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration via SVD
)()( 131211232221 xWi
WWxiy
Wi
WWyi TZrYrXrfyTZrYrXrfx iiii
087654321 vyvZyvYyvXyvxvZxvYxvXx iW
iW
iW
iiW
iW
iW
i iiiiii
y
x
f
f
TxvTv
rvrv
rvrv
rvrv
y
84
137233
126222
115211
![Page 42: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/42.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration via SVD087654321 vyvZyvYyvXyvxvZxvYxvXx i
Wi
Wi
Wii
Wi
Wi
Wi iiiiii
NWNN
WNN
WNN
WNN
WN
WNN
WWWWWW
WWWWWW
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
yZyYyXyxZxYxXx
A
NN
2222222222222
1111111111
2
1111
0Av A has rank 7 (without proof)
ion)Decomposit Value(Singular SVD viaTUDVA
![Page 43: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/43.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration via SVD• Remaining Problem:
• See book
matrixrotation a givet doesn' V
TxvTv
rvrv
rvrv
rvrv
y
84
137233
126222
115211
![Page 44: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/44.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Summary, SVD Solution
• Replace rotation matrix by arbitrary matrix
• Transform into linear set of equations
• Solve via SVD
• Enforce rotation matrix (see book)
• Solve for remaining parameters (see book)
![Page 45: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/45.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Comparison
Nonlinear least squares• Gaussian image
noise• Many local minima• Iterative• Can incorporate non-
linear distortion
Singular Value Decomp.• Gaussian parameter
noise (algebraic)• No local minima• “Closed” form• No distortion
![Page 46: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/46.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software
![Page 47: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/47.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Homogeneous Coordinates
• Idea: In homogeneous coordinates most operations become linear!
• Extract Image Coordinates by Z-normalization
C
C
CC
C
Y
X
ZY
X~
~
~1
C
C
C
Z
Y
X
~
~
~
1
12
10
000,1
000,12
000,10
5.1
18
15
1
12
10
![Page 48: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/48.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software
![Page 49: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/49.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Advanced Calibration:Nonlinear Distortions
• Barrel and Pincushion
• Tangential
![Page 50: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/50.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Barrel and Pincushion Distortion
telewideangle
![Page 51: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/51.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Models of Radial Distortion
)1( 42
21 rkrk
y
x
y
x
d
d
distance from center
![Page 52: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/52.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Tangential Distortion
cheapglue
cheap CMOS chipcheap lens image
cheap camera
![Page 53: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/53.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Image Rectification (to be continued)
![Page 54: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/54.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Distorted Camera Calibration
• Set k1k2, solve for undistorted case
• Find optimal k1k2via nonlinear least squares
• Iterate
Tends to generate good calibrations
![Page 55: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/55.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Today’s Goals
• Calibration: Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software
![Page 56: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/56.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration Software: Matlab
![Page 57: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/57.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Calibration Software: OpenCV
![Page 58: Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.](https://reader035.fdocuments.us/reader035/viewer/2022062313/56649d5e5503460f94a3d4f2/html5/thumbnails/58.jpg)
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007
Summary
• Calibration: Problem definition• Solution by nonlinear Least Squares • Solution via Singular Value Decomposition• Homogeneous Coordinates• Distortion• Calibration Software