Geometric Transformation - University of...
Transcript of Geometric Transformation - University of...
1
Geometric TransformationGeometric Transformation
• Operations depend on pixel’s Coordinates.
• Context free.
• Independent of pixel values.
( )( ) ',
',
yyxfy
xyxfx
y
x
=→
=→( ) ( )( )yxfyxfIyxI yx ,,,'),( =
(x’,y’)(x,y)
I(x,y) I’(x’,y’)
• Example: Translation
( )( ) 1,
3,
−==′
+==′
yyxfy
xyxfx
y
x
( ) ),(1,3' yxIyxI =−+
(x,y)
I(x,y) I’(x’,y’)
(x’,y’)
Forward MappingForward Mapping
• Forward mapping:
Forward
Mapping
Source Target
( )( ) yyxfy
xyxfx
y
x
′=→
′=→
,
,
• Problems with forward mapping due to sampling:
– Holes (some target pixels are not populated)
– Overlaps (some target pixels assigned few colors)
Forward MappingForward Mapping
Source Target
x=1 x=2 x=1 x=2 x=3 x=4
( ) yyxfy y==′ ,( ) xyxfx x
==′ 2,
TargetSource
x=1 x=2 x=1 x=2
?
( ) yyxfy y==′ ,( ) xyxfx x
==′ 7.0,
• Inverse mapping:
• Each target pixel assigned a single color.
• Color Interpolation is required.
Inverse
Mapping
( )( ) yyxfy
xyxfx
y
x
=′′→′
=′′→′−
−
,
,
1
1
TargetSource
Inverse MappingInverse Mapping
• Example: Scaling along X
– Forward mapping:
– Inverse mapping:
(0,0)
(0,0)
Source Target
Source Target
yyxx =′=′ ;2
yyxx ′=′= ;2/
InterpolationInterpolation
• What happens when a mapping function calculates a
fractional pixel location?
• Interpolation: generates a new pixel by analyzing the
surrounding pixels.
• Good interpolation techniques attempt to find an optimal
balance between three undesirable artifacts: aliasing,
blurring, and edge halos.
9
InterpolationInterpolation
x4 scaling
aliasing blurring halos
Nearest Neighbor InterpolationNearest Neighbor Interpolation
• The assign value is taken from the pixel closest to the
generated location:
• Advantage:
– Fast
• Disadvantage:
– Jagged results
– Aliasing near edges
( ) ( ){ } ( ){ }( )yxfroundyxfroundIyxI yx′′′′=′′′ −− ,,,, 11
Original ImageNearest N.
Interpolation
Original Image
Nearest N.
Interpolation
Bilinear InterpolationBilinear Interpolation
• The assign value is a weighted sum of the four
nearest pixels.
• Each weight is proportional to the distance from each
existing pixel.
Linear InterpolationLinear Interpolation
• Isolating v in the above equation:
xw xe
vw
ve
x
v
we
w
we
w
vv
vv
xx
xx
−−
=−−
( ) we vvv αα −+= 1we
w
xx
xxwhere
−−
=α
NW
NE
SESW
x∆
S
N
V
Bilinear InterpolationBilinear Interpolation
( )xSWxSES ∆−⋅+∆⋅= 1
( )xNWxNEN ∆−⋅+∆⋅= 1
( )ySyNV ∆−⋅+∆⋅= 1
y∆
NW
NE
SESW
x∆
S
N
V
Bilinear InterpolationBilinear Interpolation
y∆
• The bilinear interpolation is the best fit low-degree
polynomial of the form:
• The pixel’s boundaries are C0 continuous
(continuous values across boundaries).
ji1
0ji
ij yxayxv ∆∆=∆∆ ∑=,
),(
Bilinear example
z=15 z=7
z=2 z=3
∆x
∆y
1
1.5
2
1
1.5
20
5
10
15
v
∆x ∆y
0
10
Nearest N.Interpolation
Bilinear Interpolation
Nearest N.Interpolation
Bilinear Interpolation
Bicubic InterpolationBicubic Interpolation
• The assign value is a weighted sum of the 4x4
nearest pixels:
ji3
0ji
ij yxayxv ∆∆=∆∆ ∑=,
),(
How can we find the right coefficients?
• Denote the pixel values Vpq {p,q=0..3}
• The unknown coefficients are aij {i,j=0..3}
21
],[,},..{,,
21yx30qpforyxav ji3
0ji
ijpq −∈∆∆=∆∆=∑=
s
t
• We have a linear system of 16
equations with 16 coefficients.
• The pixel’s boundaries are C1
continuous (continuous
derivatives across boundaries).
N.N Bilinear Bicubic
N.N
Bilinear
Bicubic
Applying the TransformationApplying the Transformation
T = …… % 2x2 transformation matrix[r,c] = size(img)
% create array of destination x,y coordinates[X,Y]=meshgrid(1:c,1:r);
% calculate source coordinatessourceCoor = inv(T) * [X(:) Y(:) ] ‘ ;
% calculate nearest neighbor interpolationXs = round(sourceCoor(1,:));Ys = round(sourceCoor(2,:));
indx=find(Xs<1 | Xs>r); %out of range pixelsXs(indx)=1; Ys(indx)=1;
indy=find(Ys<1 | Ys>c); %out of range pixelsXs(indy)=1; Ys(indy)=1;
% calculate new imagenewImage = img((Xs-1).*r+Ys);newImage(indx)=0; newImage(indx)=0; newImage = reshape(newImage,r,c);
Types of linear 2D transformationsTypes of linear 2D transformations
• Rigid (Euclidean) transformation:
– Translation + Rotation (distance preserving).
• Similarity transformation:
– Translation + Rotation + Uniform Scale (angle preserving).
• Affine transformation:
– Translation + Rotation + Scale + Shear (parallelism preserving).
• Projective transformation
– Cross-ratio preserving
• All above transformations are groups where Rigid ⊂
Similarity ⊂ Affine ⊂ Projective
Types of linear 2D transformationsTypes of linear 2D transformations
• All above transformations are groups where Rigid ⊂
Similarity ⊂ Affine ⊂ Projective
Matrix NotationMatrix Notation
• Every location (x,y) is treated as a column vector:
• Coordinate transformation is obtained by
multiplying with a 2x2 matrix?
y
x
+
+=
dycx
byax
y
x
dc
ba
Matrix Notation - ScaleMatrix Notation - Scale
=
y
x
y
x
b
a
'
'
0
0
1
1
• Scale(a,b): (x,y) (ax,by)
• If a or b are negative, we get reflection.
• Inverse: S-1(a,b)=S(1/a,1/b)
=
by
ax
y
x
b
a
0
0
Matrix Notation - ShearMatrix Notation - Shear
• Shear(a,b): (x,y) (x+ay,y+bx)
+
+=
bxy
ayx
y
x
b
a
1
1
a=0.5, b=0
Matrix Notation - RotationMatrix Notation - Rotation
• Rotate(θ):
(x,y) (xcosθ+ysinθ , -xsinθ + ycosθ)
• Inverse: R-1(θ)=RT(θ)=R(-θ)
+−
+=
− θθ
θθθθθθ
cossin
sincos
cossin
sincos
yx
yx
y
x
Matrix Notation - TranslationMatrix Notation - Translation
• Translation(a,b):
• Cannot represent translation using
2x2 matrices.
• Inverse:
+
+→
by
ax
y
x
−′
−′→
′
′
by
ax
y
x
Homogeneous CoordinatesHomogeneous Coordinates
• Homogeneous Coordinates is a mapping from Rn to Rn+1:
• Note: (tx,ty,t) all correspond to the same non-
homogeneous point (x,y). E.g. (2,3,1)≡(6,9,3) ≡(4,6,2).
• Inverse mapping:
( ) ( )yx,,,, =
→W
Y
W
XWYX
),,(),,(),( ttytxWYXyx ≡→
Homogeneous CoordinatesHomogeneous Coordinates
(x,y)
1
(tx,ty,t)
=(x,y,1)
Some 2D TransformationsSome 2D Transformations
• Translation :
• Affine transformation:
• Projective transformation:
+
+
=
=
′
′
′
11100
10
01
y
x
y
x
ty
tx
y
x
t
t
W
Y
X
=
′
′
′
1100
y
x
tdc
tba
W
Y
X
y
x
=
′
′
′
11
y
x
fe
tdc
tba
W
Y
X
y
x
Affine
Hierarchy of Linear 2D TransformationsHierarchy of Linear 2D Transformations
Global Transformations – Image RectificationGlobal Transformations – Image Rectification
Global Transformations – Global WarpingGlobal Transformations – Global Warping
=
′
′
1
y
x
A
1
y
x
Global Transformations – Global WarpingGlobal Transformations – Global Warping
points pi points qimatch
=
111
ppp
ppp
A
111
qqq
qqqY
3
y
2
y
1
x3
x2
x1
y
3
y
2
y
1
x3
x2
x1
...
...
...
...
Global Transformations – Global WarpingGlobal Transformations – Global Warping
points pi points qimatch
=
−
111
qqq
qqq
A
111
ppp
pppy
3
y
2
y
1
x3
x2
x1
1Y
3
y
2
y
1
x3
x2
x1
...
...
...
...
Inverse Mapping:
Global Transformations – Global WarpingGlobal Transformations – Global Warping
Solve for A-1 in terms of the least mean square.
i.e. find A-1 which minimizes:
∑ −−i
ii qAp 2
1 ||||
Global Transformations – Global WarpingGlobal Transformations – Global Warping
solution:
( ) 1)(
−= TT
XXXXpinv
=−
111111
321
321
321
321
1...
...
...
...yyy
xxx
Yyy
xxx
qqq
qqq
pinvppp
ppp
A
Global Transformations – Global WarpingGlobal Transformations – Global Warping
Alternative representation:
=
′
′
f
e
d
c
b
a
yx
yx
y
x
1000
0100
=
=
′
′
110011
y
x
fdc
eba
y
x
Ay
x
Rearrange:
Global Transformations – Global WarpingGlobal Transformations – Global Warping
=
fedcba
p
p
yx
yx
y
x
1000
010011
11
1
1
Global Transformations – Global WarpingGlobal Transformations – Global Warping
=
fedcba
p
p
p
p
p
p
yx
yx
yx
yx
yx
yx
y
x
y
x
y
x
1000
0100
1000
0100
1000
0100
33
33
22
22
11
11
3
3
2
2
1
1
Global Transformations – Global WarpingGlobal Transformations – Global Warping
=
p
p
p
p
p
p
y
x
y
x
y
x
yx
yx
yx
yx
yx
yx
pinv
f
e
d
c
b
a
3
3
2
2
1
1
33
33
22
22
11
11
1000
0100
1000
0100
1000
0100
'
( ) TTQQQQpinv
1)('
−=
solution:
Global Transformations – Global WarpingGlobal Transformations – Global Warping
What about Projective Transformations?
=
′
′
111
y
x
hg
fdc
eba
y
x
Homogeneity must be preserved!
1';
1'
++
++=
++
++=
hygx
fdycxy
hygx
ebyaxx
Global Transformations – Global WarpingGlobal Transformations – Global Warping
What about Projective Transformations?
1';
1'
++
++=
++
++=
hygx
fdycxy
hygx
ebyaxx
( )
−−=′
h
g
f
e
d
c
b
a
xyxxyxx ''0100
And similarly for y’
Global Transformations – Image RectificationGlobal Transformations – Image Rectification
So who ARE we?
Local Transformations – Image WarpingLocal Transformations – Image Warping
Demoalex
Local Transformations – Image WarpingLocal Transformations – Image Warping
ps
pd
Area of influence
ps = source pointpd = destination point
ps
pd
Area of influence
Local Transformations – Image WarpingLocal Transformations – Image Warping
ps
pd
Area of influence
ps = source pointpd = destination point
source
destination
ps pd
pd
ps
Image Morphing (Image Metamorphosis)Image Morphing (Image Metamorphosis)
Demo bw
Cross Dissolve (pixel operations)Cross Dissolve (pixel operations)
DestinationImage
SourceImage
( ) ( )]1,0[
1
∈
⋅+⋅−=
t
TtSttI
t
cross dissolve
warp + dissolve
Warping + Cross Dissolve Warping + Cross Dissolve
• Warp source image towards intermediate image.
• Warp destination image towards intermediate image.
• Cross-dissolve the two images by taking the weighted average at each pixel.
time
Cross-dissolve
warping images
source
destination
warp
warp
Cross-dissolve
Cross-dissolve
Image MetamorphosisImage Metamorphosis
• Let S,T be the source and the target images
• Let G(p) be the transformation from S towards T, where
G(0)=I (the identity)
• Let t∈[0,1] the time step to be synthesized
Algorithm:
1. Warp S towards T:
2. Warp T toward S:
3. Cross dissolve:
( ) ( ){ }SptGtS ⋅=
( ) ( )( ) { }TptGtT1
1−⋅−=
( ) ( ) ( ) ( )tTttSttI ⋅+⋅−= 1
t
sourse
target
S(t)=G(t⋅p){S}
T(t)=G((1-t)⋅p)-1{T}
I(t)=(1-t)⋅S(t)+t ⋅(T(t))
Feature Based MorphingFeature Based Morphing
• Morph one shape into another shape
• Use local features to define the geometric warping
P
Q
P’
Q’
P
Q
P’
Q’
P
Q
P’
Q’
P
Q
P’
Q’
P
Q
P’
Q’
P
Q
P’
Q’
• α∈[0,1] is the relative position along the segment (P’,Q’).
• β is the actual perpendicular distance to the segment.
• (u’,v’) is the local coordinates of the segment (P’,Q’):
– u’ is a unit vector parallel to Q’-P’
– v’ is the unit vector perpendicular to Q’-P’
α
P
QRβ
α
P’
Q’
R’β
Source Image Dest Image
uv
u’ v’
( )PQ
PQu
′−′′−′
=′
′−
′=′=⊥′
x
y
u
uuv
One Segment WarpingOne Segment Warping
α
P’
Q’
R’β
u’ v’
( ) ( ) vPRPQ
uPR′⋅′−′=
′−′′⋅′−′
= βα ;
• The point R’ is mapped into (α,β) :
where
'''''' vuPQPR βα +−+=
αP
QRβ
α
P’
Q’
R’β
Source Image Dest Image
uv
u’ v’
Inverse Mapping:
where (u,v) is the local coordinates of the segment (P,Q):
vuPQPR βαβα +−+=),(
( )PQ
PQu
−−
=
−==⊥
x
y
u
uuv
Multiple Segment WarpingMultiple Segment Warping
• In multiple segment warping the point R’ is influenced by multiple segments.
• The influence strength of each segments is proportional to:
– Segment length
– The distance from the point R’
P1
Q1R1
β1
P2
Q2β 2
P1’
Q1’
R’β’1
β’2
P2’
Q2’
R2
• The influence of each segments is:
• The value p∈[0,1] controls the influence of the line length.
• The value a is a small number avoiding division by zero.
• The value b determines how the relative weight diminish as the β
increases
• The final mapping is:
b
i
p
ii
ia
PQW
β+
−=
∑∑
=
k
k
k
kk
W
RW
R
Example:Example:
For more details see:
Thaddeus Beier & Shawn Neely / Feature-Based Image Metamorphosis Siggraph '92
http://www.hammerhead.com/thad/morph.html
Example images from:
http://www.cc.gatech.edu/classes/AY2001/cs4451_spring/projects/Seven/
73
Another Example:Another Example:
Mesh WarpingMesh Warping
From:
http://www.cs.utk.edu/~huangj/CS594F01/imageMorph.ppt
2-Pass Mesh Warping Algorithm2-Pass Mesh Warping Algorithm
The first pass warps the rows of the image:
For each column of the mesh determine the x-coordinates at
which the mesh column crosses each image row.
2-Pass Mesh Warping Algorithm2-Pass Mesh Warping Algorithm
Then, each row of the image is warped individually bylinearly interpolating each segment between the x-coordinates defined by the source mesh to the size of the corresponding segment defined by the x-coordinates of the destination Mesh.
The second pass performs the exact same procedure on
the columns of the image by interpolating the y-coordinates of the meshes.
VidMorph
Fun Morph