Lecture 8 Camera Models - Stanford...
59
Lecture 8 - Silvio Savarese 15-Oct-14 Professor Silvio Savarese Computational Vision and Geometry Lab Lecture 8 Camera Models
Transcript of Lecture 8 Camera Models - Stanford...
Lecture 8 -Silvio Savarese 15-Oct-14
Professor Silvio Savarese
Computational Vision and Geometry Lab
Lecture 8Camera Models
Lecture 8 -Silvio Savarese 15-Oct-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras• Other camera models
Lecture 8Camera Models
Reading: [FP] Chapter 1 “Cameras” [FP] Chapter 2 “Geometric Camera Models”[HZ] Chapter 6 “Camera Models”
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
How do we see the world?
• Let’s design a camera– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
Some history…
Photography (Niepce, “La
Table Servie,” 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
Some history…
Photography (Niepce, “La
Table Servie,” 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
• Daguerréotypes (1839)
• Photographic Film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers,
1908)
Some history…
Let’s also not forget…
Motzu
(468-376 BC)
Aristotle
(384-322 BC)
Also: Plato, Euclid
Al-Kindi (c. 801–873)
Ibn al-Haitham
(965-1040)Oldest existent
book on geometry
in China
Pinhole perspective projectionPinhole cameraf
f = focal length
o = aperture = pinhole = center of the camera
o
z
yf'y
z
xf'x
y
xP
z
y
x
P
Pinhole camera
Derived using similar triangles
O
P = [x, z]
P’=[x’, f ]
f
z
x
f
x
i
k
Pinhole camera
Common to draw image plane in front of
the focal point. What’s the transformation between these 2 planes?
f f
Pinhole camera
z
yf'y
z
xf'x
Kate lazuka ©
Pinhole camera
Is the size of the aperture important?
Shrinking
aperture
size
- Rays are mixed up
Adding lenses!
-Why the aperture cannot be too small?
-Less light passes through
-Diffraction effect
Cameras & Lenses
• A lens focuses light onto the film
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
“circle of
confusion”
Cameras & Lenses
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
Cameras & Lenses
• A lens focuses light onto the film
– All parallel rays converge to one point on a plane
located at the focal length f
– Rays passing through the center are not deviated
focal point
f
Cameras & Lenses
f
Cameras & Lenses
z
y'z'y
z
x'z'x
ozf'z
)1n(2
Rf
zo-z
Z’
From Snell’s law:
Thin Lenses
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1 n2 2
n1 = n (lens)
n1 = 1 (air)
zo
z
y'z'y
z
x'z'x
ozf'z
)1n(2
Rf
Focal length
No distortion
Pin cushion
Barrel (fisheye lens)
Issues with lenses: Radial Distortion
– Deviations are most noticeable for rays that pass through
the edge of the lens
Image magnification
decreases with distance
from the optical axis
Lecture 8 -Silvio Savarese 15-Oct-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras
• Intrinsic• Extrinsic
• Other camera models
Lecture 2Camera Models
Pinhole perspective projectionPinhole cameraf
f = focal length
o = center of the camera
o
)z
yf,
z
xf()z,y,x(
2E
3
From retina plane to images
Pixels, bottom-left coordinate systems
Coordinate systems
xc
yc
Converting to pixels
x
y
xc
yc
C=[cx, cy]
)cz
yf,c
z
xf()z,y,x( yx
1. Off set
Converting to pixels
)cz
ylf,c
z
xkf()z,y,x( yx
1. Off set
2. From metric to pixels
x
y
xc
yc
C=[cx, cy] Units: k,l : pixel/m
f : m: pixel
, Non-square pixels
• Matrix form?
A related question:
• Is this a linear transformation?x
y
xc
yc
C=[cx, cy]
)cz
y,c
z
x()z,y,x( yx
Converting to pixels
Is this a linear transformation?
No — division by z is nonlinear
How to make it linear?
)z
yf,
z
xf()z,y,x(
Homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
• Converting from homogeneous
coordinates
Camera Matrix
)cz
y,c
z
x()z,y,x( yx
1
z
y
x
0100
0c0
0c0
z
zcy
zcx
X y
x
y
x
x
y
xc
yc
C=[cx, cy]
Perspective Projection
Transformation
1
z
y
x
0100
00f0
000f
z
yf
xf
X
z
yf
z
xf
iX
XMX
M
3H
4
XMX
X0IK
Camera Matrix
1
z
y
x
0100
0c0
0c0
X y
x
Camera
matrix K
Finite projective cameras
1
z
y
x
0100
0c0
0cs
X y
x
Skew parameter
x
y
xc
yc
C=[cx, cy]K has 5 degrees of freedom!
Lecture 8 -Silvio Savarese 15-Oct-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras
• Intrinsic• Extrinsic
• Other camera models
Lecture 2Camera Models
World reference system
Ow
iw
kw
jwR,T
•The mapping so far is defined within the camera
reference system
• What if an object is represented in the world
reference system
3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
x
Y’
p’
x’
y
z
wXTRK
World reference system
Ow
iw
kw
jwR,T
wXTR
X
4410
In 4D homogeneous coordinates:
Internal parameters External parameters
XIKX 0' wXTR
IK
4410
0
M
X
X’
wx XMX 4313
14w4333 XTRK
Projective cameras
Ow
iw
kw
jwR,T
How many degrees of freedom?
5 + 3 + 3 =11!
100
c0
cs
K y
x
X
X’
Camera calibration
Estimate intrinsic and extrinsic parameters from 1 or multiple images
wx XMX 4313
14w4333 XTRK
How many degrees of freedom?
5 + 3 + 3 =11!
100
c0
cs
K y
x
More details in CS231A
Projective cameras
Ow
iw
kw
jwR,T
w13 XMX
14w4333 XTRK
3
2
1
m
m
m
M
W
W
W
W
m
m
m
m
m
m
X
X
X
X
3
2
1
3
2
1
),(3
2
3
1
w
w
w
w
X
X
X
X
m
m
m
m
E
X
X’
Properties of Projection• Points project to points
• Lines project to lines
• Distant objects look smaller
•Angles are not preserved
•Parallel lines meet!
Parallel lines in the world
intersect in the image at a
“vanishing point”
Properties of Projection
Horizon line (vanishing line)
horizon
l
Horizon line (vanishing line)
One-point perspective• Masaccio, Trinity,
Santa Maria
Novella, Florence,
1425-28
Credit slide S. Lazebnik
Lecture 8 -Silvio Savarese 15-Oct-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras
• Intrinsic• Extrinsic
• Other camera models
Lecture 2Camera Models
Projective camera
p’
q’
r’O
f’
R’
Q’
P’
Weak perspective projection
R’
Q’
P’
O
f’
When the relative scene depth is small compared to its distance from the camera
zo
p’
q’
r’ Q
P
R
Weak perspective projection
O
f’
When the relative scene depth is small compared to its distance from the camera
zo
p’
q’
r’ Q
P
R
yz
fy
xz
fx
yz
fy
xz
fx
0
0
''
''
''
''
Magnification m
R’
Q’
P’
Weak perspective projection
10
bAM
wPMP
TRKM
1v
bA
Instead of
O
f’ zo
p’
q’
r’ Q
P
R
R’
Q’
P’
1000
2
1
3
2
1
m
m
m
m
m
1
P
P
P W2
W1
W
3
2
1
m
m
m
m
m
)P,P( 21 ww mm
E
wPMP
10
bAM
magnification
3
2
1
m
m
m
wPMP
1v
bAM
W3
W2
W1
W
3
2
1
P
P
P
P
m
m
m
m
m
m
)P
P,
P
P(
3
2
3
1
w
w
w
w
m
m
m
mE
Perspective
Weak perspective
Orthographic (affine) projection
Distance from center of projection to image plane is infinite
yy
xx
yz
fy
xz
fx
'
'
''
''
Pros and Cons of These Models
• Weak perspective much simpler math.– Accurate when object is small and distant.
– Most useful for recognition.
• Pinhole perspective much more accurate for scenes.– Used in structure from motion.
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Weak perspective projection
Weak perspective projection
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Things to remember
