Lecture 13: Camera Projection II - Penn State College of ...rtc12/CSE486/lecture13.pdfLecture 13:...
Transcript of Lecture 13: Camera Projection II - Penn State College of ...rtc12/CSE486/lecture13.pdfLecture 13:...
CSE486, Penn StateRobert Collins
Recall: Imaging Geometry
VU
W
Object of Interestin World CoordinateSystem (U,V,W)
CSE486, Penn StateRobert Collins
Imaging Geometry
Z
f
Camera Coordinate System (X,Y,Z).
• Z is optic axis• Image plane located f units
out along optic axis• f is called focal length
X
Y
CSE486, Penn StateRobert Collins
Imaging Geometry
VU
W
Z
Forward Projection onto image plane.3D (X,Y,Z) projected to 2D (x,y)
y
xX
Y
CSE486, Penn StateRobert Collins
Imaging Geometry
VU
W
Zy
Our image gets digitizedinto pixel coordinates (u,v)
xX
Y
u
v
CSE486, Penn StateRobert Collins
Imaging Geometry
VU
W
Zy
World Coordinates
CameraCoordinates
Image (film)Coordinates
PixelCoordinates
u
v
xX
Y
CSE486, Penn StateRobert Collins
Forward Projection
UVW
XYZ
xy
uv
WorldCoords
CameraCoords
FilmCoords
PixelCoords
We want a mathematical model to describehow 3D World points get projected into 2DPixel coordinates.
Our goal: describe this sequence of transformations by a big matrix equation!
CSE486, Penn StateRobert Collins
Intrinsic Camera Parameters
UVW
XYZ
xy
uv
WorldCoords
CameraCoords
FilmCoords
PixelCoords
Affine Transformation
CSE486, Penn StateRobert Collins
Intrinsic parameters
• Describes coordinate transformation between film coordinates (projected image) and pixel array
• Film cameras: scanning/digitization
• CCD cameras: grid of photosensors
still in T&V section 2.4
CSE486, Penn StateRobert Collins
Intrinsic parameters (offsets)
film plane(projected image)
x
y
(0,0)
u (col)
v (row)
(0,0)
pixel array
yo
xo
xouZ
Xf yov
Z
Yf
ox and oy called image center or principle point
CSE486, Penn StateRobert Collins
Intrinsic parameters
film plane
x
y
(0,0)
u (col)
v (row)
(0,0)
pixel array
yo
xo
sometimes one or more coordinate axes are flipped (e.g. T&V section 2.4)
xouZ
Xf yov
Z
Yf
CSE486, Penn StateRobert Collins
analog
Intrinsic parameters (scales)
pixel array
sampling determines how many rows/cols in the image
C cols x R rows
film scanning resolution
CCD
resample
CSE486, Penn StateRobert Collins
Effective Scales: sx and sy
O.Camps, PSU
xouZ
Xf yov
Z
Yf
1
sx
1
sy
Note, since we have different scale factors in x and y,we don’t necessarily have square pixels!
Aspect ratio is sy / sx
CSE486, Penn StateRobert Collins
Perspective projection matrix
10
0
0
10
/
0
0
0
/
'
'
'
Z
Y
X
o
o
sf
sf
z
y
x
y
x
y
x
O.Camps, PSU
Adding the intrinsic parameters into theperspective projection matrix:
xouZ
Xf
1
sxyov
Z
Yf
1
sy
uz’
x’
vz’
y’
To verify:
CSE486, Penn StateRobert Collins
Note:
Sometimes, the image and the camera coordinate systemsSometimes, the image and the camera coordinate systemshave opposite orientations: [the book does it this way] have opposite orientations: [the book does it this way]
10
0
0
10
/
0
0
0
/
'
'
'
Z
Y
X
o
o
sf
sf
z
y
x
y
x
y
x
yy
xx
sovZ
Yf
souZ
Xf
)(
)(
CSE486, Penn StateRobert Collins
Note 2
10
0
0
1
0
0
0
0
0
0
'
'
'
Z
Y
X
f
f
z
y
x
100
a13a12a11
a23a22a21
w’
v’
u’
In general, I like to think of the conversion asa separate 2D affine transformation from filmcoords (x,y) to pixel coordinates (u,v):
u = Mint PC = Maff Mproj PC
MprojMaff
CSE486, Penn StateRobert Collins
Huh?
Did he just say it was “a fine” transformation?
No, it was “affine” transformation, a type of2D to 2D mapping defined by 6 parameters.
More on this in a moment...
CSE486, Penn StateRobert Collins
Summary : Forward Projection
UVW
XYZ
xy
uv
WorldCoords
CameraCoords
FilmCoords
PixelCoords
Mext Mproj Maff
MUVW
uv
34
24
14
33
23
13
31
22
12
31
21
11
m
m
m
m
m
m
m
m
m
m
m
m
MintUVW
XYZ
uv
Mext
CSE486, Penn StateRobert Collins
Summary: Projection Equation
World to cameraPerspectiveprojection
Film planeto pixels
MextMprojMaff
Mint
M
CSE486, Penn StateRobert Collins
Geometric Image Mappings
image transformed imageGeometric
transformation
(x,y)
(x’,y’)
x’ = f(x, y, {parameters})y’ = g(x, y, {parameters})
CSE486, Penn StateRobert Collins
Linear Transformations
image transformed imageGeometric
transformation
(x,y)
(x’,y’)
x’y’1
xy1
(Can be written as matrices)
M(params)=
CSE486, Penn StateRobert Collins
Euclidean (Rigid)
0 1
1
0x
y
x’
y’
equations matrix form
transform )
tx
ty
CSE486, Penn StateRobert Collins
Partitioned Matrices
http://planetmath.org/encyclopedia/PartitionedMatrix.html
CSE486, Penn StateRobert Collins
Partitioned Matrices
2x1 2x2 2x12x1
1x1 1x2 1x11x1
equation form
matrix form
CSE486, Penn StateRobert Collins
Another Example (from last time)
1
W
V
U
1
Z
Y
X tx
1000
r13r12r11
r23r22r21
r33r32r31
ty
tz
PC = R PW + T
PC
1=
R T
0 1
PW
1
3x1
1x1
3x3 3x1
1x3 1x1 1x1
3x1
CSE486, Penn StateRobert Collins
Similarity (scaled Euclidean)
0 1
1
0x
y
x’
y’
equations matrix form
transform )
tx
ty
S