(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet Planar ...
Transcript of (c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet Planar ...
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
Planar Homographies Nov. 15, ’04
References:
• Your lecture notes on coordinate frames
• Your lecture notes on epipolar geometry
• http://www.robots.ox.ac.uk/˜vgg/projects/SingleView/
Planar Homographies 1
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
A quick review of projection
We need to determine the parameters that govern the projection
from points in the world to points in an image, a quick review of
coordinate frames and transformations is in order.
Homogeneous Coordinates:
Let ~x = (x1, x2, x3)T be a 3-D point in some reference
frame, we can express ~x in Homogeneous coordinates as
~xH = (x1, x2, x3, 1)T .
A general 3D rigid transformation can be written as a 4 by 4
matrix multiplication using Homogeneous coordinates:
M3D−trans =
(
R3,3~d3,1
01,3 1
)
(1)
Where R is a 3 by 3 rotation matrix, and d = (dx, dy, dz)T ,
specifies translation components for the x, y, and z coordinates.
The rotation matrix in turn can be decomposed into three
matrices, each specifying a rotation around each of the coordinate
axes.
A transformation of the type described above is used to convert
coordinates between different coordinate frames. The first set
A quick review of projection 2
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
of parameters that governs the mapping of points in the world
to points in the image is given by a transformation from world
coordinates to camera coordinates:
Mext =(
R3,3 −R3,3~d3,1
)
(2)
Where R is the rotation from world to camera coordinates, and~d contains the world coordinates of the camera’s nodal point.
Notice that Mext is a 3 by 4 matrix with 12 D.O.F., R and ~d are
known as the extrinsic camera parameters.
A quick review of projection 3
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
A quick review of projection
The second set of parameters that govern the mapping
from world points to image points is given by the specific
characteristics of the projection of light rays onto the image.
Figure 1 illustrates the projection process in a pinhole camera.
In this figure, C is the center of projection, X, Y , and Z are
the coordinate axes in camera coordinates, u and v are the
image coordinate axes, f is the focal length of the camera, and
a point ~xcam in camera coordinates projects to point ~xim in
image coordinates.
A quick review of projection 4
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
The transformation that maps from camera coordinates to
image coordinates has 2 components, the first component
converts camera coordinates to image coordinates. If ~xcam =
(x, y, z)T , then ~xim = (u, v) = (fx/z, fy/z). This
operation can be written as the 3 by 3 perspective projection
matrix:
Mproj =
f 0 0
0 f 0
0 0 1
(3)
The second component accounts for the size and shape of pixels,
and for the image coordinates. These effects are specified with
a 3 by 3 matrix:
Mim =
1/l1 0 pc,1
0 1/l2 pc,2
0 0 1
(4)
Where l1 is the width of the pixels, l2 is the height of the
pixels, and pc = (pc,1, pc,2) is the point (in image coordinates)
where the optical axis of the camera intersects the image plane.
The parameters in the previous 2 matrices are known as the
camera’s intrinsic parameters. The complete mapping from a
point W in (homogeneous) world coordinates to point Pim in
A quick review of projection 5
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
image coordinates is:
Pim = Mim · Mproj · Mext ·~W T (5)
A quick review of projection 6
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
2-D Homographies
2-D homographies are projective transformations that map
points from one plane to another plane (for example the
transformation mapping points in a planar surface in the world
to the image plane). Figure 2 illustrates the geometry involved
in this process.
2-D Homographies 7
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
2-D Homographies
Consider the problem of determining the homography that
maps points in one image to the corresponding points in a
second image.
Assuming that we can identify corresponding points in both
images (let’s say, by detecting and matching interest points),
such a homography exists and can be computed, consider the
homography Hl mapping points on a plane to image points on
the left side image:
~xl,i = αHl · ~qi, α > 0 (6)
2-D Homographies 8
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
Where ~qi = (u, v, 1)T is a point on some plane π, and ~xl,i =
(u′
l, v′
l, 1)T its projection onto the left image. In a similar
fashion, consider the homography Hr that maps points on π
to image points on the right side image:
~xr,i = βHr · ~qi, β > 0 (7)
Solving for ~qi in Eq. and substituting in Eq. we have:
~xl,i =α
βHl · H
−1r · ~xr,i = γH~xr,i (8)
Where H = Hl ·H−1r is the homography that maps points on
the right side image to points on the left side image. Notice that
we can do this without ever knowing the location of the points
~qi. Figure 5 shows the result of applying the homography to
the left side image, and a composite of both images showing
the correspondence.
2-D Homographies 9
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
Range: [20.6, 254] Dims: [384, 256]
Range: [−173, 215] Dims: [384, 256]
2-D Homographies 10
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
2-D Homographies
A 2-D homography is defined as a 3 by 3 homogeneous matrix
such that for any point ~xi = (γu, γv, γ) on π, and its
corresponding point ~xi′ on π′:
~xi′ = H · ~xi (9)
2-D homographies have 8 D.O.F. (9 entries in the H matrix, but
the common scale factor is not relevant), hence, to determine
the homography we require 4 pairs of corresponding points.
Notice though that 3 collinear points in either plane result in
a configuration with no unique solution.
Homographies can be applied to many problems in computer
vision including stereo reconstruction, image mosaics, and
applications using perspective geometry. Figure 3 shows an
example of using 2-D planar homographies to rectify an image
region (remove distortion due to perspective projection).
2-D Homographies 11
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
In the above example, the 4 corners of the window are
mapped to the corners of a rectangular polygon, however, any
rectangular polygon will offer 4 correspondence pairs. The
aspect ratio for the rectangular polygon can not be determined
from one view of the 4 corners of the window.
2-D Homographies 12
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
Perspective Structure
Finally, let’s take a look at an image taken under perspective
projection and observe some of its characteristics.
One important property of perspective projection is that
parallel lines converge to some point in the image (though
the point of convergence may be at infinity as is the case for
horizontal and vertical lines in the image above).
The image location of these intersection points (usually called
vanishing points) determine the orientation in 3D of the set of
parallel lines that converge to it. It is also possible to determine
Perspective Structure 13
(c) 2004 F.J. Estrada & A.D. Jepson & D. Fleet
from pairs of vanishing points the 3D orientation of planar
surfaces bounded by parallel lines converging to either of the
vanishing points. This enables us to do take the above image
and turn it into a 3D model as shown below.
..
This type of reconstruction relies heavily on planar
homographies to determine the correct mapping between image
pixels and reconstructed 3D surfaces. For more information on
this type of reconstruction, visit the link shown in the first
slide.
Perspective Structure 14