geometry magnetic fields simulation trigger calibration reconstruction planning
Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28...
Transcript of Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28...
![Page 1: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/1.jpg)
Camera models and calibration
Read tutorial chapter 2 and 3.1
http://www.cs.unc.edu/~marc/tutorial/
Szeliski’s book pp.29-73
![Page 2: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/2.jpg)
Schedule (tentative)
2
# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry 5 Oct.19 Model fitting (RANSAC, EM, …) 6 Oct.26 Stereo Matching 7 Nov.2 Segmentation 8 Nov.9 Shape from Silhouettes 9 Nov.16 Optical Flow 10 Nov.23 Structure from motion 11 Dec.1 Tracking (Kalman, particle filter) 12 Dec.8 Object recognition 13 Dec.15 Object category recognition 14 Dec.22 Applications and demos
![Page 3: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/3.jpg)
Brief geometry reminder
3
0=xlTl'×l=x x'×x=l
2D line-point coincidence relation: Point from lines:
2D Ideal points ( )T0,, 21 xx2D line at infinity ( )T1,0,0=l∞
0=XπT
0πXXX
3
2
1
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
T
T
T
0Xπππ
3
2
1
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
T
T
T3D plane-point coincidence relation: Point from planes: Plane from points:
Line from points:
3D line representation: (as two planes or two points) [ ] 2x2
PA B 0
Q⎡ ⎤
=⎢ ⎥⎣ ⎦
T
T
3D Ideal points ( )1 2 3, , ,0X X X T
3D plane at infinity ( )0,0,0,1∞Π =T
![Page 4: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/4.jpg)
Conics and quadrics
4
lTC* l = 0xTCx = 0
l=Cx
l
x C
1-* CC =
0=QXXT 0=πQπ *T
Conics
Quadrics
![Page 5: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/5.jpg)
Homographies
5
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
3
2
1
333231
232221
131211
3
2
1
'''
xxx
hhhhhhhhh
xxx
Transformation for lines
l=l' -THTransformation for conics
-1-TCHHC ='Transformation for dual conics
THHCC ** ='
x=x' HFor a point transformation
![Page 6: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/6.jpg)
Fixed points and lines
e=e λH
(eigenvectors H =fixed points)
l lλ=H-T
(eigenvectors H-T =fixed lines)
(λ1=λ2 ⇒ pointwise fixed line)
![Page 7: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/7.jpg)
Hierarchy of 2D transformations
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1002221
1211
y
x
taataa
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1002221
1211
y
x
tsrsrtsrsr
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
333231
232221
131211
hhhhhhhhh
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1002221
1211
y
x
trrtrr
Projective 8dof
Affine 6dof
Similarity 4dof
Euclidean 3dof
Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio
Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞
Ratios of lengths, angles. The circular points I,J
lengths, areas.
invariants transformed squares
![Page 8: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/8.jpg)
The line at infinity
00
l l 0 lt 1
1A
−−
∞ ∞ ∞−
⎛ ⎞⎡ ⎤ ⎜ ⎟ʹ′ = = =⎢ ⎥ ⎜ ⎟−⎣ ⎦ ⎜ ⎟
⎝ ⎠
AH
A
TT
T T
The line at infinity l∞ is a fixed line under a projective transformation H if and only if H is an affinity
Note: not fixed pointwise
![Page 9: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/9.jpg)
Affine properties from images
projection rectification
APA
lllHH⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
321
010001
[ ] 0≠,l 3321∞ llll T=
![Page 10: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/10.jpg)
Affine rectification v1 v2
l1
l2 l4
l3
l∞
21∞ vvl ×=
211 llv ×=
432 l×l=v
![Page 11: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/11.jpg)
The circular points
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
=
0
1I i
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
−=
0
1J i
I0
1
0
1
100cossinsincos
II =⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−==ʹ′ iseitsstss
iy
x
Sθθθ
θθ
H
The circular points I, J are fixed points under the projective transformation H iff H is a similarity
![Page 12: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/12.jpg)
The circular points
“circular points”
0=++++ 233231
22
21 fxxexxdxxx 0=+ 2
221 xx
l∞
( )( )T
T
0,-,1J
0,,1I
i
i
=
=
( ) ( )TT 0,1,0+0,0,1=I i
Algebraically, encodes orthogonal directions
03 =x
![Page 13: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/13.jpg)
Conic dual to the circular points
TT JIIJ*∞ +=C
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
000010001
*∞C
TSS HCHC *
∞*∞ =
The dual conic is fixed conic under the projective transformation H iff H is a similarity
*∞C
Note: has 4DOF l∞ is the nullvector
*∞C
l∞
I
J
![Page 14: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/14.jpg)
Angles
( )( )2221
22
21
2211
+++
=cosmmll
mlmlθ
( )T321 ,,=l lll ( )T321 ,,=m mmm
Euclidean:
Projective: ( )( )mmll
mlcos**
*
∞∞
∞=CC
CTT
T
θ
0=ml *∞C
T
(orthogonal)
![Page 15: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/15.jpg)
Transformation of 3D points, planes and quadrics
Transformation for planes
( )l=l' -TH
Transformation for quadrics ( )-1-TCHHC ='
Transformation for dual quadrics
( )THHCC ** ='
( )x=x' HFor a point transformation
X=X' H
π=π' -TH
-1-TQHH=Q'
THHQ=Q' **
(cfr. 2D equivalent)
![Page 16: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/16.jpg)
Hierarchy of 3D transformations
⎥⎦
⎤⎢⎣
⎡
vTvtAProjective
15dof
Affine 12dof
Similarity 7dof
Euclidean 6dof
Intersection and tangency
Parallellism of planes, Volume ratios, centroids, The plane at infinity π∞
Angles, ratios of length The absolute conic Ω∞
Volume
⎥⎦
⎤⎢⎣
⎡
10tA
T
⎥⎦
⎤⎢⎣
⎡
10tR
T
s
⎥⎦
⎤⎢⎣
⎡
10tR
T
![Page 17: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/17.jpg)
The plane at infinity
000
π π π0- t 11
A
−−
∞ ∞ ∞−
⎛ ⎞⎜ ⎟⎡ ⎤ ⎜ ⎟ʹ′ = = =⎢ ⎥ ⎜ ⎟⎣ ⎦⎜ ⎟⎝ ⎠
AH
A
TT
T T
The plane at infinity π∞ is a fixed plane under a projective transformation H iff H is an affinity
1. canonical position 2. contains directions 3. two planes are parallel ⇔ line of intersection in π∞ 4. line // line (or plane) ⇔ point of intersection in π∞
( )T1,0,0,0=π∞ ( )T0,,,=D 321 XXX
![Page 18: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/18.jpg)
The absolute conic
The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity
04
23
22
21 =
⎭⎬⎫++
XXXX
The absolute conic Ω∞ is a (point) conic on π∞. In a metric frame:
( ) ( )T321321 ,,I,, XXXXXXor conic for directions: (with no real points)
1. Ω∞ is only fixed as a set 2. Circle intersect Ω∞ in two circular points 3. Spheres intersect π∞ in Ω∞
![Page 19: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/19.jpg)
The absolute dual quadric
⎥⎦
⎤⎢⎣
⎡=Ω
000I*
∞ T
The absolute dual quadric Ω*∞ is a fixed conic under
the projective transformation H iff H is a similarity
1. 8 dof (to describe location in projective space) 2. plane at infinity π∞ is the nullvector of Ω∞ 3. Angles:
( )( )2*∞21
*∞1
2*∞1
πΩππΩππΩπ
=cosTT
T
θ
![Page 20: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/20.jpg)
Camera model
Relation between pixels and rays in space
?
![Page 21: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/21.jpg)
Pinhole camera
Gemma Frisius, 1544
![Page 22: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/22.jpg)
22
Distant objects appear smaller
![Page 23: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/23.jpg)
23
Parallel lines meet
• vanishing point
![Page 24: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/24.jpg)
24
Vanishing points
VPL VPR H
VP1 VP2
VP3
To different directions correspond different vanishing points
![Page 25: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/25.jpg)
25
Geometric properties of projection
• Points go to points • Lines go to lines • Planes go to whole image
or half-plane • Polygons go to polygons
• Degenerate cases: – line through focal point yields point – plane through focal point yields line
![Page 26: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/26.jpg)
Pinhole camera model
TT ZfYZfXZYX )/,/(),,(
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
10100
1ZYX
ff
ZfYfX
ZYX
linear projection in homogeneous coordinates!
![Page 27: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/27.jpg)
Pinhole camera model
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
10100
ZYX
ff
ZfYfX
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
1010101
1ZYX
ff
ZfYfX
PXx =
[ ]0|I)1,,(diagP ff=
![Page 28: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/28.jpg)
Principal point offset
Tyx
T pZfYpZfXZYX )+/,+/(),,(
principal point Tyx pp ),(
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
+
+
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
10100
1ZYX
pfpf
ZZpfYZpfX
ZYX
y
x
x
x
![Page 29: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/29.jpg)
Principal point offset
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
+
+
10100
ZYX
pfpf
ZZpfYZpfX
y
x
x
x
[ ] camX0|IKx =
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1y
x
pfpf
K calibration matrix
![Page 30: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/30.jpg)
Camera rotation and translation
( )C~-X~R=X~cam
X10RC-R
110C~R-RXcam ⎥
⎦
⎤⎢⎣
⎡=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎥⎦
⎤⎢⎣
⎡=
ZYX
[ ] camX0|IKx = [ ]XC~-|IKRx =
[ ]t|RKP = C~R-t =PX=x
~
![Page 31: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/31.jpg)
CCD camera
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1yy
xx
pp
K α
α
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
11y
x
y
x
pfpf
mm
K
![Page 32: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/32.jpg)
General projective camera
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1yx
xx
pps
K α
α
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1yx
xx
pp
K α
α
[ ]C~|IKRP =
non-singular
11 dof (5+3+3)
[ ]t|RKP =
intrinsic camera parameters extrinsic camera parameters
![Page 33: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/33.jpg)
Radial distortion
• Due to spherical lenses (cheap) • Model:
R
2 2 2 2 21 2( , ) (1 ( ) ( ) ...)
xx y K x y K x y
y⎡ ⎤
= + + + + + ⎢ ⎥⎣ ⎦
R
http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpg straight lines are not straight anymore
![Page 34: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/34.jpg)
Camera model
Relation between pixels and rays in space
?
![Page 35: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/35.jpg)
Projector model
Relation between pixels and rays in space
(dual of camera) (main geometric difference is vertical principal point
offset to reduce keystone effect)
?
![Page 36: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/36.jpg)
Meydenbauer camera
vertical lens shift to allow direct ortho-photographs
![Page 37: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/37.jpg)
Affine cameras
![Page 38: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/38.jpg)
Action of projective camera on points and lines
forward projection of line
( ) µbaµPBPAµB)P(AµX +=+=+=
back-projection of line
lPT=Π
PXlX TT =Π ( )PX x0;xlT ==
PXx =projection of point
![Page 39: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/39.jpg)
Action of projective camera on conics and quadrics back-projection to cone
CPPQ Tco = 0CPXPXCxx TTT ==
( )PXx =
projection of quadric
TPPQC ** = 0lPPQlQ T*T*T ==ΠΠ
( )lPT=Π
![Page 40: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/40.jpg)
ii xX ↔ ? P
Resectioning
![Page 41: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/41.jpg)
Direct Linear Transform (DLT)
ii PXx = [ ] ii PXx ×
rank-2 matrix
![Page 42: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/42.jpg)
Direct Linear Transform (DLT)
Minimal solution
Over-determined solution
⇒ 5½ correspondences needed (say 6) P has 11 dof, 2 independent eq./points
n ≥ 6 points
Apminimize subject to constraint
1p =
use SVD
![Page 43: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/43.jpg)
Degenerate configurations
(i) Points lie on plane or single line passing
through projection center
(ii) Camera and points on a twisted cubic
![Page 44: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/44.jpg)
Scale data to values of order 1 1. move center of mass to origin 2. scale to yield order 1 values
Data normalization
D3σD2σ
![Page 45: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/45.jpg)
Line correspondences
Extend DLT to lines
ilPT=Π
ii 1TPXl
(back-project line)
ii 2TPXl (2 independent eq.)
![Page 46: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/46.jpg)
Geometric error
![Page 47: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/47.jpg)
Gold Standard algorithm
Objective Given n≥6 2D to 3D point correspondences {Xi↔xi’}, determine the Maximum Likelyhood Estimation of P
Algorithm (i) Linear solution:
(a) Normalization: (b) DLT
(ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
(iii) Denormalization:
ii UXX~ = ii Txx~ =
UP~TP -1=
~ ~ ~
![Page 48: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/48.jpg)
Calibration example
(i) Canny edge detection (ii) Straight line fitting to the detected edges (iii) Intersecting the lines to obtain the images corners
typically precision <1/10 (H&Z rule of thumb: 5n constraints for n unknowns)
![Page 49: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/49.jpg)
Errors in the world
Errors in the image and in the world
ii XPx
=
iX
Errors in the image
iPXx̂ =i
(standard case)
![Page 50: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/50.jpg)
Restricted camera estimation
Minimize geometric error → impose constraint through parametrization
Find best fit that satisfies • skew s is zero • pixels are square • principal point is known • complete camera matrix K is known
Minimize algebraic error → assume map from param q → P=K[R|-RC], i.e. p=g(q) → minimize ||Ag(q)||
![Page 51: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/51.jpg)
Restricted camera estimation
Initialization • Use general DLT • Clamp values to desired values, e.g. s=0, αx= αy Note: can sometimes cause big jump in error Alternative initialization • Use general DLT • Impose soft constraints
• gradually increase weights
![Page 52: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/52.jpg)
![Page 53: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/53.jpg)
Image of absolute conic
![Page 54: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/54.jpg)
A simple calibration device
(i) compute H for each square (corners → (0,0),(1,0),(0,1),(1,1))
(ii) compute the imaged circular points H(1,±i,0)T
(iii) fit a conic to 6 circular points (iv) compute K from ω through cholesky factorization
(≈ Zhang’s calibration method)
![Page 55: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/55.jpg)
Some typical calibration algorithms Tsai calibration
Zhangs calibration
http://research.microsoft.com/~zhang/calib/
Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.
Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666-673, September 1999.
http://www.vision.caltech.edu/bouguetj/calib_doc/
![Page 56: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/56.jpg)
80 from Szeliski’s book
![Page 57: Camera models and calibration - CVG...# date topic 1 Sep.21 Introduction and geometry 2 Sep.28 Invariant features 3 Oct.5 Camera models and calibration 4 Oct.12 Multiple-view geometry](https://reader034.fdocuments.us/reader034/viewer/2022050200/5f5379217a0c303eb401c1eb/html5/thumbnails/57.jpg)
81
Next week: Multiple-view geometry