ANALYSIS OF A FOOTBALL PUNT David Bannard TCM Conference NCSSM 2005.
Image Alignment by Image Averaging David Hong NCSSM, IE364 2008.
-
Upload
horatio-ferdinand-barton -
Category
Documents
-
view
215 -
download
0
Transcript of Image Alignment by Image Averaging David Hong NCSSM, IE364 2008.
Image Alignment byImage Averaging
David Hong
NCSSM, IE364 2008
Example Problem 1
In:
Out: 4 0 0s s sX Y
Example Problem 2
In:
Out: 0 4 0s s sX Y
Example Problem 3
In:
Out: 0 0 6s s sX Y
Problem (Formal Statement)
In: , n nu v R
Out: , , such that
translated by and and
rotated by is
s s s
s s
s
X Y
v X Y
u
Motivation
Many Applications:– Special Effects (Movie)– Video Compression– Pattern Recognition– Image Stabilization (Digital Cameras)– Dead-reckoning (Mobile Robotics)
State of the Art
Optical Flow
Lucas-Kanade (1985)
Optical Flow with Smoothness Constraint
Horn-Schunck (1980)
Phase Correlation
X
Y
u(x,y,t)
U(X,Y)
xy
Lucas-Kanade
Lucas-Kanade
0 0, , ,U X Y u x y t
Then the floor-coordinate is (X0,Y0) and the sensor-coordinate is (x,y) at time t.
Let us consider a point on the plane.
From there, we can see:
Lucas-Kanade
0 0, , ,
0
U X Y u x y t
d dU u
dt dtdx
u u udtdyx y t
dt
Differentiating on time gives us:
Lucas-Kanade
0
0
cos sin
sin coss s s
s s s
X Xx
Y Yy
Expressing (x,y) in terms of (X0,Y0) and the sensor position (Xs,Ys,Θs) gives us:
Lucas-Kanade
s
s
s
dX
dtdYu u u u u
x yx y y x dt t
d
dt
Putting the two together, we get:
This is underdetermined!
Algorithm
1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's
i j i j i j i j i j i j i j i j s i j i j
s
X
u u u u u u i u u j Y u u
uu’
Improvement by Iteration
u
u’
Iteration 1:
3
2
0
s
s
s
X
Y
u’’
Iteration 2:
1
1
0
s
s
s
X
Y
Overall Estimate:
4
1
0
s
s
s
X
Y
u’’’
Improvement by Iteration
u’
u
u’’
u’(x’,y’)
u’’(x’’,y’’)
Improvement by Iteration
Coordinate Transformation:
cos sin' ''
sin cos' ''ss s
s s s
Xx x
y y Y
Improvement by Iteration
u’
Places u’ is defined
Place we need to evaluate u’
(x’, y’)
(x’0, y’0) (x’1, y’0)
(x’0, y’1) (x’1, y’1)
Improvement by Iteration
1 10 0
0 1 0 1
1 00 1
0 1 1 0
0 11 0
1 0 0 1
0 0
1
Lagrange Interpolation:
' ' ' ''( ', ') ' ' , '
' ' ' '
' ' ' ' ' ' , '
' ' ' '
' ' ' ' ' ' , '
' ' ' '
' ' ' '
'
x x y yu x y u x y
x x y y
x x y yu x y
x x y y
x x y yu x y
x x y y
x x y y
x x
1 10 1 0
' ' , '' ' '
u x yy y
Improvement by Iteration
u’
u
u’’
u’ not defined!
Improvement by Iteration
u’
u’’
u’’ was not evaluated here
Valid Region
Improvement by Iteration
Places we need to evaluate u’
(i-1,0’, j-1,0’ )
(i0,-1’, j0,-1’ )
(i0,1’, j0,1’ )
(i1,0’, j1,0’ )
Performance of Algorithm
Good Surface: Bad Surface:
Algorithm Fails!
Performance of Algorithm
Surface:
Performance of AlgorithmSurface:
Performance of AlgorithmSurface:
Assumptions Made
• The Error Function is locally quadratic
• The floor is linear
Weaknesses
• Many Iterations– Inherent to Technique
• “Fooled” by symmetry (Aliasing problem)– Inherent to Problem
Strengths
• Accurate
• Improvement by Iteration
• Finds Error Function Root by Newton’s
yu
v(xv,yv)
u(xu,yu)
x v
y v
Phase Correlation
xu
Phase Correlation
We consider the image to be like a 2-D wave.
Then, displacement is simply a “phase shift”
Rotation can similarly be found
So, we “correlate” the “phases”
Phase Correlation
Discrete Fourier Transform:
1
1
( )
ijij
ijij
ij ij
Fn
Gn
FG
Phase CorrelationKey Theorem:
,where is the motion by displacement and
is the desired output.
d d
d
d
F W F
d
W
' d dFz F z W Fz
Phase Correlation
1
2
Key Theorem:
' : such that '
:
( ')( ) ( )( )
m
i vd
Let z z R x d z x
Then
F z v e F z Rv
R R
Phase Correlation
Corollary:
( ) ( )
'( ) ' ( )
:
'
Let u v F z v
Let u v F z v
Then
u v u Rv
Weaknesses
• Inaccurate on first iteration
• Boundary Problem (Repetion Assumption)
• High complexity– FFT is “O(nlogn)”
Strengths
• Elegant
• Makes a big leap
• Works well on images with pattern
• Separates displacement and rotation (DFT)
yu
v(xv,yv)
u(xu,yu)
x v
y v
Image Averaging
xu
Image Averaging
• Find a “Center-of-Mass” of each image
• Track the motion of the center-of-mass
w x xx
w x
Weaknesses
• Boundary Problem (Average Point Moves)
• Average is affected by small discretization issues
Strengths
• Elegant• Makes a big leap• Very fast– Complexity of O(n)
• Yields itself well to Improvement by Iteration– Using same technique as in Lucas-Kanade
Handling the Weaknesses
• Here we decide to take an alternative approach
• Separate displacements from rotation
• Do this using FFT (as in Phase Correlation)
Handling the Weaknesses
• We handle rotation first– Post-FFT, only rotation remains
Handling the Weaknesses
Weaknesses
• We introduce an FFT ( O(nlogn) operation)
• However, only requires 2– Phase correlation requires up to 3 or 4
Future Work
• Make Image Alignment Rigorous– Use complex numbers to notate displacement
• Smoothness Constraint• Pre-processing the image• Condition for Convergence
Thank You!