Geometric Image Manipulation: Part 2cs510/yr2015sp/more...© Bruce A. Draper & J. Ross Beveridge,...
Transcript of Geometric Image Manipulation: Part 2cs510/yr2015sp/more...© Bruce A. Draper & J. Ross Beveridge,...
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Geometric Image Manipulation: Part 2
Lecture #2 January 26, 2015
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Translation Applied to Images
Translate 20 in x Translate -20 in x
€
1 0 200 1 00 0 1
"
#
$ $ $
%
&
' ' '
€
1 0 −200 1 00 0 1
#
$
% % %
&
'
( ( (
Why Black?
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Scale Applied to Images
Scale Uniformly by 2 Scale Uniformly by 0.5
€
2 0 00 2 00 0 1
"
#
$ $ $
%
&
' ' '
€
0.5 0 00 0.5 00 0 1
"
#
$ $ $
%
&
' ' '
Note the
origin
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Rotation Applied to Images
Rotate by 15° Rotate by -15°
Note that a positive rotation rotates the positive X axis toward the positive Y axis
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Remember Composition of Matrices To rotate by θ around a point (x,y):
1 0 x0 1 y0 0 1
!
"
###
$
%
&&&
cos θ( ) −sin θ( ) 0
sin θ( ) cos θ( ) 0
0 0 1
!
"
####
$
%
&&&&
1 0 −x0 1 −y0 0 1
!
"
###
$
%
&&&=
1 0 x0 1 y0 0 1
!
"
###
$
%
&&&
cos θ( ) −sin θ( ) ysin θ( )− xcos θ( )sin θ( ) cos θ( ) −xsin θ( )− ycos θ( )0 0 1
!
"
####
$
%
&&&&
=
cos θ( ) −sin θ( ) ysin θ( )− xcos θ( )+ xsin θ( ) cos θ( ) −xsin θ( )− ycos θ( )+ y0 0 1
!
"
####
$
%
&&&&
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Similarity & Affine Transformations • All the similarity transforms can be combined
into one generic matrix:
• But! This matrix does more. What?
– hint: 2 more transformations. – hint: 6 degrees of freedom.
• How can you specify this matrix?
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
11001yx
fedcba
vu Hint: diagonal
terms are not equal, and b ≠ -d.
Equivalent to adding 2 shear parameters, or unequal scaling & 1 shear parameter.
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Affine Examples: Shear
€
1 0.5 00 1 00 0 1
"
#
$ $ $
%
&
' ' '
€
1 1 00 1 00 0 1
"
#
$ $ $
%
&
' ' '
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Similarity vs. Affine Matrices • Similarity : 4 DOF
• Affine : 6 DOF
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
11001yx
fedcba
vu
uv1
!
"
###
$
%
&&&=
a b c−b a d0 0 1
!
"
###
$
%
&&&
xy1
!
"
###
$
%
&&&
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Specifying Affine Transformations
• There are six unknowns in the matrix (a through f) • If you specify one point in the source image and a
corresponding point in the target image, that yields two equations:
• So providing three point-to-point correspondences specifies an affine matrix
feydxvcbyaxu
iii
iii
++=
++=
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Affine Specification: Example
There is one affine transformation that will map the green point on the right to the green point on the left, and align the red and blue points also.
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving Affine Transformations These linear equations can be easily solved:
– WLOG, assume x1=y1=0 – then u1= c and v1 = f – so:
( )
( )
( )( ) ( )
2323
123232
32
23
122
313
122
3133
2
23
3132
2123
2
212
1333
1222
xyyxuuxuux
yxyx
uuxxuu
b
uuxx
uubyxyx
byuux
byuuxxbyuu
a
ubyaxuubyaxu
−−
−−−=
−−
−−−=
−−−=""#
$%%&
'−
−
−−=−−
−−=
++=
++=
Calculation of a, b & c is independent of calculation of e, f & g.
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving Affine (cont.)
• This can be substituted in to solve for a • The same process with y’s solves for d,e,f • About the WLOG:
– It was true because you can translate the original coordinate system by (-x1, -y1)
– So what do you do to compensate?
• Alternatively, set up a system of linear equations and solve... – Will show this for a harder case shortly….
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Perspective Transformations
• We can go beyond affine transformations. • We can do any perspective transformation of a
one 3D view of a plane to another view. • Therefore, we can model an image as a plane
in space, and project it onto any other image. – How does this differ from the perspective
projection pipeline in CS410?
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Perspective Matrix
• Why does element [3,3] = 1? • How many points are needed to specify this matrix?
€
u'v 'w
"
#
$ $ $
%
&
' ' '
=
a b cd e fg h 1
"
#
$ $ $
%
&
' ' '
xy1
"
#
$ $ $
%
&
' ' '
u = u'w , v = v'
w
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving for Perspective
• Four corresponding points produce eight equations, eight unknowns --- but we can’t observe w
1'
1'
++
++==
++
++==
ii
ii
i
ii
ii
ii
i
ii
hygxfeydx
wvv
hygxcbyax
wuu
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving (cont.)
• Multiply to get rid of the fraction…
• Now, remember that the u’s, v’s, x’s & y’s are known; group the unknown terms
( )( ) feydxhygxv
cbyaxhygxu
iiiii
iiiii
++=++
++=++
11
iiiiiii
iiiiiii
vhyvgxfeydxvuhyugxcbyaxu
−−++=
−−++=
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving (III)
• And express the result as a system of linear equations
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
−−
−−
−−
−−
−−
−−
−−
−−
=
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
hgfedcba
vyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyx
vuvuvuvu
444444
444444
333333
333333
222222
222222
111111
111111
4
4
3
3
2
2
1
1
10000001100000011000000110000001
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving (IV)
• Finally, invert the constant matrix and solve
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
=
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
−−
−−
−−
−−
−−
−−
−−
−−−
hgfedcba
vuvuvuvu
vyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyx
4
4
3
3
2
2
1
11
444444
444444
333333
333333
222222
222222
111111
111111
10000001100000011000000110000001
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Solving (V) : Questions
• Is there always a solution? • Under what conditions is the matrix invertible? • Is the solution always unique?
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Perspective Image Transforms (Intuition)
• What does the following matrix do?
!!!
"
#
$$$
%
& −
100022022
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Matrix Decomposition
• Note that such decompositions are: – not unique (why?) – difficult to intuit
!!!!!!
"
#
$$$$$$
%
& −
!!!
"
#
$$$
%
&
=!!!
"
#
$$$
%
& −
100
021
21
021
21
100020002
100022022
original Scale by 2 Rotation
by 45
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
More Intuitions
• What will the following matrix do?
• More specifically, what will it do to the giraffe image?
1 0 00 1 01 −1 1
"
#
$$$
%
&
'''
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Check Your Intuitions
• What’s going on here?
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
More Intuition Checking
• Part of what you are seeing is a scale effect – positive terms in the bottom row create larger w values, and
therefore smaller u, v values
• Something much weirder is also going on: – What happens when y = x+1? – How do you interpret this geometrically? – Isn’t the perspective transform linear?
• So how do you select transform matrices?
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Perspective Transform of 2D Planes Continued.
• Recall the basic equation for the perspective transform
• The only practical way to specify an image
transform is by providing four point correspondences
€
u'v 'w
"
#
$ $ $
%
&
' ' '
=
a b cd e fg h 1
"
#
$ $ $
%
&
' ' '
xy1
"
#
$ $ $
%
&
' ' '
u = u'w , v = v'
w
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Computing Transformations
• Remember how to build a transformation from four point correspondences….
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
−−
−−
−−
−−
−−
−−
−−
−−
=
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
hgfedcba
vyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyxvyvxyxuyuxyx
vuvuvuvu
444444
444444
333333
333333
222222
222222
111111
111111
4
4
3
3
2
2
1
1
10000001100000011000000110000001
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Computing... • So if we want the following mapping:
(0,0)→(0,0), (0,144)→(0,144), (152,0)→(152,50), (152,144)→(152,94)
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
−−
−−
−
−
−
13536142881144152000218882310400011441520760010152000023104000101522073601144000000000114400010000000000100
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
…More Computing...
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
=
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
=
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
!!!!!!!!!!!
"
#
$$$$$$$$$$$
%
&
−−
−−
−−−
−−−−
−
−−−−
001497.01077.100274.3
9415250152144000
0000.0000.0000.0000.000.000.000.000.000.000.000.000.000000100007.0007.007.007.007.007.007.002.014.002.007.002.014.002.0000000100000007.0007.022.007.022.014.023.007.023.014.
hgfedcba
M-1 u&v
vector
What does This say about x?
How does This alter it? Remember the earlier
WLOG? c = u1,…
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
…yields !!!
"
#
$$$
%
&
1001497.01077.100274.3
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Geometric Transformation : Review
• Goal: Ii(u,v,w) = Ij(G�[x,y,1]T) • Transformation classes:
– Similarity (4 DoF : translation, rotation, scale) – Affine (6 DoF : adds shear & non-uniform scale) – Perspective (8 DoF : 2D plane to plane)
• Specified via point correspondences
© Bruce A. Draper & J. Ross Beveridge, January 23, 2015
Still To Do….
• Photometric registration • Interpolation
– Transformations are not integer to integer
• Then we can tackle matching under various transformation classes