Cameras, lenses, and calibration
description
Transcript of Cameras, lenses, and calibration
![Page 1: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/1.jpg)
Cameras, lenses, and calibration
• Camera models• Projection equations
Images are projections of the 3-D world onto a 2-D plane…
![Page 2: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/2.jpg)
Pinhole camera only allows rays from one point in the scene to strike each point of the paper.
Light rays from many different parts of the scene strike the same point on the paper.
Forsyth&Ponce
![Page 3: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/3.jpg)
Pinhole camera geometry:Distant objects are smaller
![Page 4: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/4.jpg)
Perspective projection
y
y’
f
z
camera world
Cartesian coordinates:We have, by similar triangles, that
(x, y, z) -> (f x/z, f y/z, -f)Ignore the third coordinate, and get
(x, y,z) ( fx
z, f
y
z)
![Page 5: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/5.jpg)
• Points go to points• Lines go to lines• Planes go to whole image or half-planes.• Polygons go to polygons
• Degenerate cases– line through focal point to point– plane through focal point to line
Geometric properties of projection
![Page 6: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/6.jpg)
ctztz
btyty
atxtx
0
0
0
)(
)(
)(
x'(t) fx
z
f (x0 at)
z0 ct
y'(t) fy
z f (y0 bt)
z0 ct
Line in 3-spacePerspective projection
of that line
x'(t)fa
c
y'(t) fb
c
In the limit as we have (for ):
This tells us that any set of parallel lines (same a, b, c parameters) project to the same point (called the vanishing point).
t0c
![Page 7: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/7.jpg)
http://www.ider.herts.ac.uk/school/courseware/graphics/two_point_perspective.html
![Page 8: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/8.jpg)
Vanishing points
• Each set of parallel lines (=direction) meets at a different point– The vanishing point for this direction
• Sets of parallel lines on the same plane lead to collinear vanishing points. – The line is called the horizon for that plane
![Page 9: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/9.jpg)
What if you photograph a brick wall head-on?
x
y
![Page 10: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/10.jpg)
0
0
0
)(
)(
)(
ztz
yty
atxtx
x'(t) f (x0 at)
z0
y'(t) f y0
z0
Brick wall line in 3-space Perspective projection of that line
All bricks have same z0. Those in same row have same y0
Thus, a brick wall, photographed head-on, gets rendered as set of parallel lines in the image plane.
![Page 11: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/11.jpg)
Other projection models: Orthographic projection
),(),,( yxzyx
![Page 12: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/12.jpg)
Other projection models: Weak perspective
• Issue– perspective effects, but not
over the scale of individual objects
– collect points into a group at about the same depth, then divide each point by the depth of its group
– Adv: easy– Disadv: only approximate
00
,),,(z
fy
z
fxzyx
![Page 13: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/13.jpg)
Three camera projections
(1) Perspective:
(2) Weak perspective:
(3) Orthographic: ),(),,(
,),,(
,),,(
00
yxzyx
z
fy
z
fxzyx
z
fy
z
fxzyx
3-d point 2-d image position
![Page 14: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/14.jpg)
Homogeneous coordinates• Is this a linear transformation?
Trick: add one more coordinate:
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
• no—division by z is nonlinear
Slide by Steve Seitz
![Page 15: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/15.jpg)
Perspective Projection• Projection is a matrix multiply using homogeneous
coordinates:
This is known as perspective projection• The matrix is the projection matrix
Slide by Steve Seitz
1 0 0 0
0 1 0 0
0 0 1/ f 0
x
y
z
1
x
y
z / f
fx
z, f
y
z
![Page 16: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/16.jpg)
Perspective ProjectionHow does scaling the projection matrix change the transformation?
Slide by Steve Seitz
1 0 0 0
0 1 0 0
0 0 1/ f 0
x
y
z
1
x
y
z / f
fx
z, f
y
z
f 0 0 0
0 f 0 0
0 0 1 0
x
y
z
1
fx
fy
z
fx
z, f
y
z
![Page 17: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/17.jpg)
Orthographic Projection
Special case of perspective projection• Orthography is an approximate model for long focal length (telephoto)
lenses and objects whose depth is shallow relative to their distance to the camera
• Also called “parallel projection” . What’s the projection matrix?
Image World
Slide by Steve Seitz
?
![Page 18: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/18.jpg)
Orthographic Projection
Special case of perspective projection
• Also called “parallel projection”• What’s the projection matrix?
Image World
Slide by Steve Seitz
![Page 19: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/19.jpg)
![Page 20: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/20.jpg)
Homogeneous coordinates
2D Points:
p x
y
p'x
y
1
p'x'
y'
w'
p x ' /w'
y ' /w'
2D Lines:
a b c x
y
1
0
ax by c 0
l a b c nx ny d
d(nx, ny)
![Page 21: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/21.jpg)
Homogeneous coordinates
Intersection between two lines:
a1x b1y c1 0
a2x b2y c2 0
x12
l1 a1 b1 c1
l2 a2 b2 c2
x12 l1 l2
![Page 22: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/22.jpg)
Homogeneous coordinates
Line joining two points:
ax by c 0
p1
p1 x1 y1 1
p2 x2 y2 1
l p1 p2
p2
![Page 23: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/23.jpg)
2D Transformations
![Page 24: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/24.jpg)
2D Transformations
tx
ty= +
Example: translation
![Page 25: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/25.jpg)
2D Transformations
tx
ty= +
1
=1 0 tx
0 1 ty.
Example: translation
![Page 26: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/26.jpg)
2D Transformations
tx
ty= +
1
=1 0 tx
0 1 ty.
=1 0 tx
0 1 ty
0 0 1
.
Example: translation
Now we can chain transformations
![Page 27: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/27.jpg)
Translation and rotation, written in each set of coordinates
Non-homogeneous coordinates
B r p A
BR Ar p A
Br t
ABC
A
B R
|
AB r t
|0 0 0 1
B r p A
BC Ar p
Homogeneous coordinates
where
![Page 28: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/28.jpg)
Translation and rotation
Let’s write
as a single matrix equation:
B r p A
BR Ar p A
Br t
B pxB pyB pz
1
A
B R
|
AB t
|0 0 0 1
A pxA pyA pz
1
Ai
Ak
Aj
A px
A py
A pz
AB r t
“as described in the coordinates of frame B”
r p
![Page 29: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/29.jpg)
Camera calibration
Use the camera to tell you things about the world:– Relationship between coordinates in the world
and coordinates in the image: geometric camera calibration, see Szeliski, section 5.2, 5.3 for references
– (Relationship between intensities in the world and intensities in the image: photometric image formation, see Szeliski, sect. 2.2.)
![Page 30: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/30.jpg)
Intrinsic parameters: from idealized world coordinates to pixel values
Forsyth&Ponce
z
yfv
z
xfu
Perspective projection
![Page 31: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/31.jpg)
Intrinsic parameters
z
yv
z
xu
But “pixels” are in some arbitrary spatial units
![Page 32: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/32.jpg)
Intrinsic parameters
z
yv
z
xu
Maybe pixels are not square
![Page 33: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/33.jpg)
Intrinsic parameters
0
0
vz
yv
uz
xu
We don’t know the origin of our camera pixel coordinates
![Page 34: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/34.jpg)
Intrinsic parameters
0
0
)sin(
)cot(
vz
yv
uz
y
z
xu
May be skew between camera pixel axes
v
u
v
u
vuvuu
vv
)cot()cos(
)sin(
![Page 35: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/35.jpg)
r p K C
r p
Intrinsic parameters, homogeneous coordinates
0
0
)sin(
)cot(
vz
yv
uz
y
z
xu
u
v
1
cot() u0
0
sin()v0
0 0 1
0
0
0
x
y
z
1
Using homogenous coordinates,we can write this as:
or:
In camera-based coords
In pixels
![Page 36: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/36.jpg)
Extrinsic parameters: translation and rotation of camera frame
C r p W
CR W r p W
Cr t Non-homogeneous
coordinates
Homogeneous coordinates
C r p
W
C R
|
WC r
t
|0 0 0 1
W r p
![Page 37: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/37.jpg)
Combining extrinsic and intrinsic calibration parameters, in homogeneous coordinates
Forsyth&Ponce
r p K W
C R WC r
t W r p
r p K Cr
p
r p M W r
p
Intrinsic
Extrinsic
C r p
W
C R
|
WC r
t
|0 0 0 1
W r p
World coordinatesCamera coordinates
pixels
0 0 0 1
![Page 38: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/38.jpg)
Other ways to write the same equation
u
v
1
. m1T . .
. m2T . .
. m3T . .
W pxW pyW pz
1
r p M W r
p
Pm
Pmv
Pm
Pmu
3
2
3
1
pixel coordinates
world coordinates
Conversion back from homogeneous coordinates leads to:
![Page 39: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/39.jpg)
Projection equation
• The projection matrix models the cumulative effect of all parameters• Useful to decompose into a series of operations
ΠXx
1****
****
****
Z
Y
X
s
sy
sx
fsx 0 x 'c
0 fsy y 'c0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
R3x 3
03x1
01x 3 1
I
3x 3T3x1
01x 3 1
projectionintrinsics rotation translation
identity matrix
Camera parametersA camera is described by several parameters
• Translation T of the optical center from the origin of world coords• Rotation R of the image plane• focal length f, principle point (x’c, y’c), pixel size (sx, sy)• blue parameters are called “extrinsics,” red are “intrinsics”
• The definitions of these parameters are not completely standardized– especially intrinsics—varies from one book to another
![Page 40: Cameras, lenses, and calibration](https://reader035.fdocuments.us/reader035/viewer/2022062517/5681334b550346895d9a4e32/html5/thumbnails/40.jpg)
Projection matrices for Orhographic and scaled orthographic projections
Orthographic projection
Scaled orthographic projection
10rr
P 21T
11T
tt
ktt
/10rr
P 21T
11T
(5dof)
(6dof)