8/11/2019 Convolution Divide and Conquer
1/110
CS 473ug: Algorithms
Chandra [email protected]
3228 Siebel Center
University of Illinois, Urbana-Champaign
Fall 2007
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
2/110
PolynomialsConvolutions
FFT
Part I
Polynomials, Convolutions and FFT
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
3/110
PolynomialsConvolutions
FFTComputing with Polynomials
Polynomials
Definition
Apolynomialis a function of one variable built from additions,subtractions and multiplications (but no divisions).
p(x) =n1j=0
ajxj
The numbers a0, a1, . . . , an are thecoefficientsof the polynomial.
Thedegreeis the highest power ofxwith a non-zero coefficient.
Example
p(x) = 3 4x+ 5x3
a0 = 3, a1 =4, a2 = 0, a3 = 5 and deg(p)=3Chekuri CS473ug
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
4/110
PolynomialsConvolutions
FFTComputing with Polynomials
Polynomials
Definition
Apolynomialis a function of one variable built from additions,subtractions and multiplications (but no divisions).
p(x) =n1j=0
ajxj
The numbers a0, a1, . . . , an are thecoefficientsof the polynomial.
Thedegreeis the highest power ofxwith a non-zero coefficient.
Representation
Polynomials represented by vector a= (a0, a1, . . . an1) ofcoefficients.
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
5/110
8/11/2019 Convolution Divide and Conquer
6/110
PolynomialsConvolutions
FFTComputing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial pand a value x, compute p(x)
Add Given polynomials p, q, compute polynomial p+q
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
7/110
PolynomialsConvolutions
FFTComputing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial pand a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute(reprsentation of) polynomial p+q
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
8/110
PolynomialsConvolutions
FFTComputing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial pand a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute(reprsentation of) polynomial p+q
Multiply Given (representation of) polynomials p, q, compute(representation of) polynomial p q.
Chekuri CS473ug
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
9/110
PolynomialsConvolutions
FFTComputing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial pand a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute(reprsentation of) polynomial p+q
Multiply Given (representation of) polynomials p, q, compute(representation of) polynomial p q.
Roots Given pfind all rootsofp.
Chekuri CS473ug
P l i l
http://find/8/11/2019 Convolution Divide and Conquer
10/110
PolynomialsConvolutions
FFTComputing with Polynomials
Evaluation
Compute value of polynomial a= (a0, a1, . . . an1) at x
Chekuri CS473ug
P l i l
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
11/110
PolynomialsConvolutions
FFTComputing with Polynomials
Evaluation
Compute value of polynomial a= (a0, a1, . . . an1) at x
power = 1
value = 0
f o r j = 0 t o n1
// invariant: power = xj
value = value + aj power
power = power x
end for
return value
Uses 2n multiplication and n additions
Chekuri CS473ug
Polynomials
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
12/110
PolynomialsConvolutions
FFTComputing with Polynomials
Evaluation
Compute value of polynomial a= (a0, a1, . . . an1) at x
power = 1
value = 0
f o r j = 0 t o n1
// invariant: power = xj
value = value + aj power
power = power x
end for
return value
Uses 2n multiplication and n additionsHorners rulecan be used to cut the multiplications in half
a(x) =a0+x(a1+x(a2+ +xan1))
Chekuri CS473ug
Polynomials
http://find/8/11/2019 Convolution Divide and Conquer
13/110
PolynomialsConvolutions
FFTComputing with Polynomials
Evaluation: Numerical Issues
Question
How long does evaluation really take? O(n) time?
Chekuri CS473ug
Polynomials
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
14/110
PolynomialsConvolutions
FFTComputing with Polynomials
Evaluation: Numerical Issues
Question
How long does evaluation really take? O(n) time?
Size ofxn in terms of bits is n log xwhile size ofx is only log x.Thus, need to pay attention to size of numbers and multiplicationcomplexity.
Ignore this issue for now. Can get around it for applications ofinterest.
Chekuri CS473ug
Polynomials
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
15/110
PolynomialsConvolutions
FFTComputing with Polynomials
Addition
Compute the sum of polynomialsa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
Chekuri CS473ug
Polynomials
http://find/8/11/2019 Convolution Divide and Conquer
16/110
yConvolutions
FFTComputing with Polynomials
Addition
Compute the sum of polynomialsa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)a+b= (a0+b0, a1+b1, . . . an1+bn1). Takes O(n) time.
Chekuri CS473ug
Polynomials
http://find/8/11/2019 Convolution Divide and Conquer
17/110
yConvolutions
FFTComputing with Polynomials
Multiplication
Compute the product of polynomialsa= (a0, a1, . . . an) and b= (b0, b1, . . . bm)
Chekuri CS473ug
Polynomials
http://find/8/11/2019 Convolution Divide and Conquer
18/110
ConvolutionsFFT
Computing with Polynomials
Multiplication
Compute the product of polynomialsa= (a0, a1, . . . an) and b= (b0, b1, . . . bm)Recall a b= (c0, c1, . . . cn+m) where
ck=
i,j:i+j=k
ai bj
for j = 0 to n+m cj= 0f o r j = 0 t o n
f o r k = 0 t o mcj+k= cj+k+aj bk
return (c0, c1, . . . cn+m)
Takes O(n2) time.
Chekuri CS473ug
PolynomialsC l i C i i h P l i l
http://find/8/11/2019 Convolution Divide and Conquer
19/110
ConvolutionsFFT
Computing with Polynomials
Multiplication
Compute the product of polynomialsa= (a0, a1, . . . an) and b= (b0, b1, . . . bm)Recall a b= (c0, c1, . . . cn+m) where
ck=
i,j:i+j=k
ai bj
for j = 0 to n+m cj= 0f o r j = 0 t o n
f o r k = 0 t o mcj+k= cj+k+aj bk
return (c0, c1, . . . cn+m)
Takes O(n2) time. We will give a better algorithm!
Chekuri CS473ug
PolynomialsC l ti
Definition
http://find/8/11/2019 Convolution Divide and Conquer
20/110
ConvolutionsFFT
DefinitionApplications
Convolutions
Definition
Theconvolutionof vectors a= (a0, a1, . . . an) and
b= (b0, b1, . . . bm) is the vector c= (c0, c1, . . . cn+m) where
ck=
i,j:i+j=k
ai bj
Convolution of vectors a and bis denoted by a
b.
Chekuri CS473ug
PolynomialsConvolutions
Definition
http://find/8/11/2019 Convolution Divide and Conquer
21/110
ConvolutionsFFT
Applications
Convolutions
Definition
Theconvolutionof vectors a= (a0, a1, . . . an) and
b= (b0, b1, . . . bm) is the vector c= (c0, c1, . . . cn+m) where
ck=
i,j:i+j=k
ai bj
Convolution of vectors a and bis denoted by a
b. In other words,the convolution is the coefficients of the product of the twopolynomials
Chekuri CS473ug
PolynomialsConvolutions
Definition
http://find/8/11/2019 Convolution Divide and Conquer
22/110
ConvolutionsFFT
Applications
Applications: Signal Processing
Let a= (a0, a1, . . . an1) be a sequence of measurements overtime.
Chekuri CS473ug
PolynomialsConvolutions
Definition
http://find/8/11/2019 Convolution Divide and Conquer
23/110
ConvolutionsFFT
Applications
Applications: Signal Processing
Let a= (a0, a1, . . . an1) be a sequence of measurements overtime.
To account for measurement errors or random fluctuations,measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, inGaussiansmoothingai is replaced by
ai= 1
Z
i+kj=ik
aje(ji)2
Chekuri CS473ug
PolynomialsConvolutions
DefinitionA li i
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
24/110
ConvolutionsFFT
Applications
Applications: Signal Processing
Let a= (a0, a1, . . . an1) be a sequence of measurements overtime.
To account for measurement errors or random fluctuations,measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, inGaussiansmoothingai is replaced by
ai= 1
Z
i+kj=ik
aje(ji)2
Smoothing can be thought of as vector of weightsw= (wk, w(k1), . . . , w1, w0, w1. . . wk) used to average
each entry as ai =k
s=kwsai+s
Chekuri CS473ug
PolynomialsConvolutions
DefinitionA li ti
http://find/8/11/2019 Convolution Divide and Conquer
25/110
FFTApplications
Applications: Signal Processing
Let a= (a0, a1, . . . an1) be a sequence of measurements overtime.
To account for measurement errors or random fluctuations,measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, inGaussiansmoothingai is replaced by
ai= 1
Z
i+kj=ik
aje(ji)2
Smoothing can be thought of as vector of weightsw= (wk, w(k1), . . . , w1, w0, w1. . . wk) used to average
each entry as ai =k
s=kwsai+sTaking b= (b0, b1, . . . b2k) to be bj=wkj, a
=a
b
Chekuri CS473ug
PolynomialsConvolutions
DefinitionApplications
http://find/8/11/2019 Convolution Divide and Conquer
26/110
FFTApplications
Historical Applications
Gauss used convolutions in the 1800s to compute the path ofasteroids from a finite number of equi-spaced observations.
Chekuri CS473ug
PolynomialsConvolutions
DefinitionApplications
http://find/8/11/2019 Convolution Divide and Conquer
27/110
FFTApplications
Historical Applications
Gauss used convolutions in the 1800s to compute the path ofasteroids from a finite number of equi-spaced observations.
In the mid-60s, Cooley and Tukey used it to detect Sovietnuclear tests by interpolating off-shore seismic readings
Chekuri CS473ug
PolynomialsConvolutions
DefinitionApplications
http://find/8/11/2019 Convolution Divide and Conquer
28/110
FFTApplications
Many Applications
To mention a few:
Signal and image processing (radar, MRI, astronomy,compression, ...)
Statistics
Multiplication of numbers
Pattern matching
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
29/110
FFTComplex Roots of UnityFFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a= (a0, a1, . . . an1) ofcoefficients.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of Unity
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
30/110
FFTp y
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a= (a0, a1, . . . an1) ofcoefficients.
Question
Are there other ways to represent polynomials?
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of Unity
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
31/110
FFTp y
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a= (a0, a1, . . . an1) ofcoefficients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r) = 0. Ifr1, r2, . . . , rn1are roots then p(x) =an1(x
r1)(x
r2) . . . (x
rn1).
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
32/110
FFTFFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a= (a0, a1, . . . an1) ofcoefficients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r) = 0. Ifr1, r2, . . . , rn1are roots then p(x) =an1(x
r1)(x
r2) . . . (x
rn1).
Theorem (Fundamental Theorem of Algebra)
Every polynomial p(x) of degree d has exactly d roots r1, r2, . . . , rdwhere the roots can be complex numbers and can be repeated.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Al i h
http://find/8/11/2019 Convolution Divide and Conquer
33/110
FFTFFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor an1 and rootsr1, r2, . . . , rn1.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Al ith
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
34/110
FFTFFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor an1 and rootsr1, r2, . . . , rn1.
Evaluating pat a given x is easy. Why?
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
35/110
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor an1 and rootsr1, r2, . . . , rn1.
Evaluating pat a given x is easy. Why?
Multiplication: given p, qwith roots r1, . . . , rn1 ands1, . . . , sm1 the product p qhas rootsr1, . . . , rn1, s1, . . . , sm1. Easy!
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
36/110
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor an1 and rootsr1, r2, . . . , rn1.
Evaluating pat a given x is easy. Why?
Multiplication: given p, qwith roots r1, . . . , rn1 ands1, . . . , sm1 the product p qhas rootsr1, . . . , rn1, s1, . . . , sm1. Easy!
Addition: requires O(n2) time??
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
37/110
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor an1 and rootsr1, r2, . . . , rn1.
Evaluating pat a given x is easy. Why?
Multiplication: given p, qwith roots r1, . . . , rn1 ands1, . . . , sm1 the product p qhas rootsr1, . . . , rn1, s1, . . . , sm1. Easy!
Addition: requires O(n2) time??Given coefficient representation, how do we go to rootrepresentation?? No finite algorithm because of potential forirrational roots.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
38/110
FFT Algorithm
Representing Polynomials by Samples
Let pbe a polynomial of degree n 1.Pick n distinct samples x0, x1, x2, . . . , xn1
Let y0 =p(x0), y1 =p(x1), . . . , yn1 =p(xn1).
Representation
Polynomials represented by (x0, y0), (x1, y1), . . . , (xn1, yn1).
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
39/110
go t
Representing Polynomials by Samples
Let pbe a polynomial of degree n 1.Pick n distinct samples x0, x1, x2, . . . , xn1
Let y0 =p(x0), y1 =p(x1), . . . , yn1 =p(xn1).
Representation
Polynomials represented by (x0, y0), (x1, y1), . . . , (xn1, yn1).
Is the above a valid representation?
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
40/110
g
Representing Polynomials by Samples
Let pbe a polynomial of degree n 1.Pick n distinct samples x0, x1, x2, . . . , xn1
Let y0 =p(x0), y1 =p(x1), . . . , yn1 =p(xn1).
Representation
Polynomials represented by (x0, y0), (x1, y1), . . . , (xn1, yn1).
Is the above a valid representation? Why do we use 2n numbersinstead ofn numbers for coefficient and root representation?
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
41/110
Sample Representation
Theorem
Given a list{(x0, y0), (x1, y1), . . . (xn1, yn1)} there is exactly onepolynomial p of degree n 1 such that p(xj) =yj forj= 0, 1, . . . , n 1.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
42/110
Sample Representation
Theorem
Given a list{(x0, y0), (x1, y1), . . . (xn1, yn1)} there is exactly onepolynomial p of degree n 1 such that p(xj) =yj forj= 0, 1, . . . , n 1.So representation is valid.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
43/110
Sample Representation
Theorem
Given a list{(x0, y0), (x1, y1), . . . (xn1, yn1)} there is exactly onepolynomial p of degree n 1 such that p(xj) =yj forj= 0, 1, . . . , n 1.So representation is valid.Can use same x0, x1, . . . , xn1 for all polynomials of degree n 1!No need to store them explicitly! Need only n numbersy0, y1, . . . , yn1.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
44/110
Sample Representation
Theorem
Given a list{(x0, y0), (x1, y1), . . . (xn1, yn1)} there is exactly onepolynomial p of degree n 1 such that p(xj) =yj forj= 0, 1, . . . , n 1.So representation is valid.Can use same x0, x1, . . . , xn1 for all polynomials of degree n 1!No need to store them explicitly! Need only n numbersy0, y1, . . . , yn1.
Lagrange interpolation formula:Given (x
0,y
0), . . . , (xn1,
yn1) thefollowing polynomial psatisfies p(xj) =yj forj= 0, 1, 2, . . . , n 1.
p(x) =n1j=0
yj
k=j(xj xk)k=j
(x xk)
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
45/110
Lagrange Interpolation
For n= 3
p(x) =y0(x x1)(x x2)
(x0 x1)(x0 x2)+y1
(x x0)(x x2)
(x1 x0)(x1 x2)+y2
(x x0)(x x1)
(x2 x0)(x2 x1)Easy to verify that p(xj) =yj! Thus there exists one polynomial ofdegree n 1 that interpolates the values (x0, y0), . . . , (xn1, yn1).
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
46/110
Lagrange Interpolation
For n= 3
p(x) =y0(x x1)(x x2)
(x0 x1)(x0 x2)+y1
(x x0)(x x2)
(x1 x0)(x1 x2)+y2
(x x0)(x x1)
(x2 x0)(x2 x1)Easy to verify that p(xj) =yj! Thus there exists one polynomial ofdegree n 1 that interpolates the values (x0, y0), . . . , (xn1, yn1).
Can there be two distinct polynomials?
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
47/110
Lagrange Interpolation
For n= 3
p(x) =y0(x x1)(x x2)
(x
0x
1)(x
0x
2)
+y1(x x0)(x x2)
(x
1x
0)(x
1x
2)
+y2(x x0)(x x1)
(x
2x
0)(x
2x
1)Easy to verify that p(xj) =yj! Thus there exists one polynomial ofdegree n 1 that interpolates the values (x0, y0), . . . , (xn1, yn1).
Can there be two distinct polynomials?
No! Use Fundamental Theorem of Algebra to prove it exercise.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
48/110
Addition and Multiplication with Sample Representation
Let{(x0, y0), (x1, y1), . . . (xn1, yn1)} and{(x0, y0), (x1, y1), . . . (xn1, yn1)} be representations of twopolynomials of degree n 1
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
49/110
Addition and Multiplication with Sample Representation
Let{(x0, y0), (x1, y1), . . . (xn1, yn1)} and{(x0, y0), (x1, y1), . . . (xn1, yn1)} be representations of twopolynomials of degree n 1a+bcan be represented by{(x0, (y0+y0)), (x1, (y1+y1)), . . . (xn1, (yn1+yn1))}
Thus, can be computed in O(n) time
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
50/110
Addition and Multiplication with Sample Representation
Let{(x0, y0), (x1, y1), . . . (xn1, yn1)} and{(x0, y0), (x1, y1), . . . (xn1, yn1)} be representations of twopolynomials of degree n 1a+bcan be represented by{(x0, (y0+y0)), (x1, (y1+y1)), . . . (xn1, (yn1+yn1))}
Thus, can be computed in O(n) time
a bcan be evaluated at n samples
{(x0, (y0
y0)), (x1, (y1
y1)), . . . (xn1, (yn1
yn1))
}Can be computed in O(n) time!
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
51/110
Addition and Multiplication with Sample Representation
Let{(x0, y0), (x1, y1), . . . (xn1, yn1)} and{(x0, y0), (x1, y1), . . . (xn1, yn1)} be representations of twopolynomials of degree n 1a+bcan be represented by{(x0, (y0+y0)), (x1, (y1+y1)), . . . (xn1, (yn1+yn1))}
Thus, can be computed in O(n) time
a bcan be evaluated at n samples
{(x0, (y0
y0)), (x1, (y1
y1)), . . . (xn1, (yn1
yn1))
}Can be computed in O(n) time!But what ifp, qare given in coefficient form? Convolution requiresp, qto be in coefficient form.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
52/110
Coefficient representation to Sample representation
Given pas (a0, a1, . . . , an1) can we obtain a samplerepresentation (x0, y0), . . . , (xn1, yn1) quickly? Also can weinvertthe representation quickly?
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
53/110
Coefficient representation to Sample representation
Given pas (a0, a1, . . . , an1) can we obtain a samplerepresentation (x0, y0), . . . , (xn1, yn1) quickly? Also can weinvertthe representation quickly?
Suppose we choose x0, x1, . . . , xn1 arbitrarily.
Take O(n) time to evaluate yj=p(xj) given (a0, . . . , an1).
Total time is (n2)!
Inversion via Lagrange interpolation also (n2).
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
54/110
Key Idea
Can choose x0, x1, . . . , xn1 carefully!
Total time to evaluate p(x0), p(x1), . . . , p(xn1) should be betterthan evaluating each separately.
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
http://find/8/11/2019 Convolution Divide and Conquer
55/110
Key Idea
Can choose x0, x1, . . . , xn1 carefully!
Total time to evaluate p(x0), p(x1), . . . , p(xn1) should be betterthan evaluating each separately.
How do we choose x0, x1, . . . , xn1 to save work?
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
S S
http://find/8/11/2019 Convolution Divide and Conquer
56/110
A Simple Start
a(x) =a0+a1x+a2x2 +a3x
3 +. . .+an1xn1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)2j =x2j. Can we exploit this?
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
A Si l S
http://find/8/11/2019 Convolution Divide and Conquer
57/110
A Simple Start
a(x) =a0+a1x+a2x2 +a3x
3 +. . .+an1xn1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)2j =x2j. Can we exploit this?Example
3 + 4x+ 6x2
+ 2x3
+ x4
+ 10x5
= (3 + 6x2
+ x4
) + x(4+2x2
+ 10x4
)
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
A Si l S
http://find/8/11/2019 Convolution Divide and Conquer
58/110
A Simple Start
a(x) =a0+a1x+a2x2 +a3x
3 +. . .+an1xn1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)2j =x2j. Can we exploit this?Example
3 + 4x+ 6x2
+ 2x3
+ x4
+ 10x5
= (3 + 6x2
+ x4
) + x(4+2x2
+ 10x4
)If we have a(x) then easy to also compute a(x)!
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Odd d E D iti
http://find/8/11/2019 Convolution Divide and Conquer
59/110
Odd and Even Decomposition
Let a= (a0, a1, . . . an1) be a polynomial.
Let aodd = (a1, a3, a5, . . .) be the n/2 degree polynomialdefined by the odd coefficients; so
aodd(x) =a1+a3x+a5x2
+ Let aeven= (a0, a2, a4, . . .) be the n/2 degree polynomialdefined by the even coefficients.
Observe
a(x) =aeven(x2) +xaodd(x2)
Thus, evaluating a at xcan be reduced to evaluating lowerdegree polynomials plus constantly many arithmeticoperations.
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
E l iti Odd E D iti
http://find/8/11/2019 Convolution Divide and Conquer
60/110
Exploiting Odd-Even Decomposition
a(x) =aeven(x2) +xaodd(x
2)
Choose n samples
x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
E l iti Odd E D siti
http://find/8/11/2019 Convolution Divide and Conquer
61/110
Exploiting Odd-Even Decomposition
a(x) =aeven(x2) +xaodd(x
2)
Choose n samples
x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21Sufficient to evaluate aeven and aodd at x
20 , x
21 , x
22 , . . . , x
2n/21!
Pluse O(n) work gives a(x) for all n samples!
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Exploiting Odd Even Decomposition
http://find/8/11/2019 Convolution Divide and Conquer
62/110
Exploiting Odd-Even Decomposition
a(x) =aeven(x2) +xaodd(x
2)
Choose n samples
x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21Sufficient to evaluate aeven and aodd at x
20 , x
21 , x
22 , . . . , x
2n/21!
Pluse O(n) work gives a(x) for all n samples!
Suppose we can make this work recursively. Then
T(n) = 2T(n/2) +O(n) which implies T(n) =O(n log n)
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Can we recurse?
http://find/8/11/2019 Convolution Divide and Conquer
63/110
Can we recurse?
n samples x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21Next step in recursion x20 , x
21 , . . . , x
2n/21
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Can we recurse?
http://find/8/11/2019 Convolution Divide and Conquer
64/110
Can we recurse?
n samples x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21Next step in recursion x20 , x
21 , . . . , x
2n/21
To continue recursion, we need
{x20 , x21 , . . . , x2n/21}={z0, z1, . . . , zn/41,z0,z1, . . . ,zn/41}
Chekuri CS473ug
http://find/8/11/2019 Convolution Divide and Conquer
65/110
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Can we recurse?
8/11/2019 Convolution Divide and Conquer
66/110
Can we recurse?
n samples x0, x1, x2, . . . , xn/21,x0,x1, . . . ,xn/21Next step in recursion x20 , x
21 , . . . , x
2n/21
To continue recursion, we need
{x20 , x21 , . . . , x2n/21}={z0, z1, . . . , zn/41,z0,z1, . . . ,zn/41}
Ifz0 =x20 and sayz0 =x2j then x0 =
1xj! That isx
0 =ix
j where i
is the imaginary number.Can continue recursion but need to go to complexnumbers.
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
67/110
Complex Numbers
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Chekuri CS473ug
PolynomialsConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
68/110
Complex Numbers
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Cartesian a+ib
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
69/110
Complex Numbers
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Cartesian a+ib
Polar rei
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
70/110
Complex Numbers
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Cartesian a+ib=
a2 +b2e(arctan(b/a))i
Polar rei
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
71/110
C p b
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Cartesian a+ib=
a2 +b2e(arctan(b/a))i
Polar rei =r(cos +isin )
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Numbers
http://find/8/11/2019 Convolution Divide and Conquer
72/110
p
Notation
For the rest of lecture, istands for1
DefinitionComplex numbersare points lying in the complex planerepresented as
Cartesian a+ib=
a2 +b2e(arctan(b/a))i
Polar rei =r(cos +isin )Thus, ei =1 and e2i = 1.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Power Series for Functions
http://find/8/11/2019 Convolution Divide and Conquer
73/110
What is ez when z is a real number? When z is a complexnumber?
ez = 1 +z/1! +z2/2! +. . .+zj/j! +. . .
Therefore
ei = 1 +i/1! + (i)2/2! + (i)3/3! +. . .
= (1
2/2! +4/4!
. . . +) +i(
3/3! +. . . +)
= cos +isin
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
74/110
p y
What are the roots of the polynomial xk 1?Clearly 1 is a root.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Roots of Unity
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
75/110
What are the roots of the polynomial xk 1?Clearly 1 is a root.
Supposerei is a root then rkeki = 1 which implies thatr= 1 and k= 2 since eki = cos(k) +isin(k) = 1
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Complex Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
76/110
What are the roots of the polynomial xk 1?Clearly 1 is a root.
Supposerei is a root then rkeki = 1 which implies thatr= 1 and k= 2 since eki = cos(k) +isin(k) = 1
Let k=e2i/k. The roots are 1 =0k,
2k, . . . ,
k1k where
wjk=e
2ji/k.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
More on the Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
77/110
Observations
jk=jmod kk
k=jjk; thus, every kth root is also a jkth root.k1
s=0 (jk)
s = (1 +jk+2jk +. . .+
j(k1)k ) = 0 for j= 0
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
More on the Roots of Unity
http://find/8/11/2019 Convolution Divide and Conquer
78/110
Observations
jk=jmod kk
k=jjk; thus, every kth root is also a jkth root.k1
s=0 (jk)
s = (1 +jk+2jk +. . .+
j(k1)k ) = 0 for j= 0
jk is root ofxk 1 = (x 1)(xk1 +xk2 +. . .+ 1)
Thus, j
k is root of (xk1 +xk2 +. . .+ 1)
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Back to Recursive Idea
http://find/8/11/2019 Convolution Divide and Conquer
79/110
Let a= (a0, a1, . . . an1) be a polynomial. Assume n is apower of 2.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Back to Recursive Idea
http://find/8/11/2019 Convolution Divide and Conquer
80/110
Let a= (a0, a1, . . . an1) be a polynomial. Assume n is apower of 2.
Recalla(x) =aeven(x
2) +xaodd(x2)
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Back to Recursive Idea
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
81/110
Let a= (a0, a1, . . . an1) be a polynomial. Assume n is apower of 2.
Recalla(x) =aeven(x
2) +xaodd(x2)
Choose x0, x1, . . . , xn1 to be nth roots of unity. That isxj=
jn.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Evaluating at roots of unity
http://find/8/11/2019 Convolution Divide and Conquer
82/110
Problem: Evaluate degree n 1 polynomiala= (a0, a1, . . . an1) on the nth roots of unity
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Evaluating at roots of unity
http://find/8/11/2019 Convolution Divide and Conquer
83/110
Problem: Evaluate degree n 1 polynomiala= (a0, a1, . . . an1) on the nth roots of unity
Recalla(jn) =aeven((
jn)
2) +jnaodd((jn)
2)
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Evaluating at roots of unity
http://find/8/11/2019 Convolution Divide and Conquer
84/110
Problem: Evaluate degree n 1 polynomiala= (a0, a1, . . . an1) on the nth roots of unity
Recalla(jn) =aeven((
jn)2) +
jnaodd((
jn)2)
Observe that (jn)2 =j
n/2
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Evaluating at roots of unity
http://find/8/11/2019 Convolution Divide and Conquer
85/110
Problem: Evaluate degree n 1 polynomiala= (a0, a1, . . . an1) on the nth roots of unity
Recalla(jn) =aeven((
jn)2) +
jnaodd((
jn)2)
Observe that (jn)2 =j
n/2
Thus, evaluating a on the nth roots of unity, can beaccomplished by evaluating aeven and aodd on the n/2 roots ofunity!
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Divide and Conquer Evaluation
http://find/8/11/2019 Convolution Divide and Conquer
86/110
Evaluation Problem
Evaluate n 1-degree polynomial on nth roots of unityConstruct polynomials aeven and aoddRecurisvely evaluate aeven and aodd on the n/2th roots of unity
Compute a(jn) as aeven(j
n/2) +jnaodd(
j
n/2)
Let T(n) denote the running time of evaluating an n 1-degreepolynomial on the nth roots of unity. Then,
T(n)2T(n/2) +O(n) =O(n log n)
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial Representations
Basic IdeasComplex Roots of UnityFFT Algorithm
Discrete Fourier Transform
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
87/110
Definition
Given a polynomial a= (a0, a1, . . . , an1) the Discrete FourierTransform ofa is the vector a = (a0, a
1, . . . , a
n1) where
aj=a(jn) for 0j
8/11/2019 Convolution Divide and Conquer
88/110
Convolutions
Compute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1
Compute values ofa and bat some n sample points.2 Compute sample representation of product. That is
c = (a0b0, a
1b
1, . . . , a
n1b
n1).
3 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Back to Convolutions and Polynomial Multiplication
http://find/8/11/2019 Convolution Divide and Conquer
89/110
Convolutions
Compute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1
Compute values ofa and bat the nth roots of unity.2 Compute sample representation of product. That is
c = (a0b0, a
1b
1, . . . , a
n1b
n1).
3 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
How can be compute c from c? We only have n sample pointsand c has 2n 1 coefficients!
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Convolutions and Polynomial Multiplication
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
90/110
ConvolutionsCompute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1 Pada with n zeroes to make it a (2n 1) degree polynomiala= (a0, a1, . . . , an1, an, an+1, . . . , a2n1). Similarly for b.
2 Compute values ofa and bat some 2n sample points.
3 Compute sample representation of product. That isc = (a0b
0, a
1b
1, . . . , a
n1b
n1, . . . , a
2n1b
2n1).
4 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Convolutions and Polynomial Multiplication
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
91/110
ConvolutionsCompute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1 Pada with n zeroes to make it a (2n 1) degree polynomiala= (a0, a1, . . . , an1, an, an+1, . . . , a2n1). Similarly for b.
2 Compute values ofa and bat the 2nth roots of unity.
3 Compute sample representation of product. That isc = (a0b
0, a
1b
1, . . . , a
n1b
n1, . . . , a
2n1b
2n1).
4 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Convolutions and Polynomial Multiplication
http://find/8/11/2019 Convolution Divide and Conquer
92/110
ConvolutionsCompute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1 Pada with n zeroes to make it a (2n 1) degree polynomiala= (a0, a1, . . . , an1, an, an+1, . . . , a2n1). Similarly for b.
2 Compute values ofa and bat the 2nth roots of unity.
3 Compute sample representation of product. That isc = (a0b
0, a
1b
1, . . . , a
n1b
n1, . . . , a
2n1b
2n1).
4 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Step 1 takes O(n log n) using divide and conquer algorithm
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Convolutions and Polynomial Multiplication
http://find/8/11/2019 Convolution Divide and Conquer
93/110
ConvolutionsCompute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1 Pada with n zeroes to make it a (2n 1) degree polynomiala= (a0, a1, . . . , an1, an, an+1, . . . , a2n1). Similarly for b.
2 Compute values ofa and bat the 2nth roots of unity.
3 Compute sample representation of product. That isc = (a0b
0, a
1b
1, . . . , a
n1b
n1, . . . , a
2n1b
2n1).
4 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Step 1 takes O(n log n) using divide and conquer algorithmStep 2 takes O(n) time
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Convolutions and Polynomial Multiplication
http://find/8/11/2019 Convolution Divide and Conquer
94/110
ConvolutionsCompute convolution c= (c0, c1, . . . , c2n2) ofa= (a0, a1, . . . an1) and b= (b0, b1, . . . bn1)
1 Pada with n zeroes to make it a (2n 1) degree polynomiala= (a0, a1, . . . , an1, an, an+1, . . . , a2n1). Similarly for b.
2 Compute values ofa and bat the 2nth roots of unity.
3 Compute sample representation of product. That isc = (a0b
0, a
1b
1, . . . , a
n1b
n1, . . . , a
2n1b
2n1).
4 Compute coefficients of unique polynomial associated withsample representation of product. That is compute c from c.
Step 1 takes O(n log n) using divide and conquer algorithmStep 2 takes O(n) timeStep 3??
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Inverse Fourier Transform
http://find/8/11/2019 Convolution Divide and Conquer
95/110
Input Given the evaluation of a 2n 1-degree polynomial con the 2nth roots of unity
Goal Compute the coefficients ofc
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
Inverse Fourier Transform
http://find/8/11/2019 Convolution Divide and Conquer
96/110
Input Given the evaluation of a 2n 1-degree polynomial con the 2nth roots of unity
Goal Compute the coefficients ofc
We saw that a can be computed from a in O(n log n) time. Canwe compute a from a in O(n log n) time?
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
A Matrix Point of View
http://find/8/11/2019 Convolution Divide and Conquer
97/110
a0 =a(x0), a1 =a(x1), . . . , a
n1 =a(xn1) where xj=
jn.
Let 1n =e2/n =.
1 x0 x20 . . . x
n10
1 x1 x21 . . . xn11...
... ...
. . . ...
1 xj x2j . . . x
n1j
... ...
... . . .
...
1 xn1 x2n1 . . . xn1n1
a0
a1...
aj...
an1
=
a0
a1...
aj...
an1
Ch k i CS473
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic IdeasComplex Roots of UnityFFT Algorithm
A Matrix Point of View
http://find/8/11/2019 Convolution Divide and Conquer
98/110
a0 =a(x0), a1 =a(x1), . . . , a
n1 =a(xn1) where xj=
jn.
Let 1n =e2/n =.
1 1 1 . . . 11 2 . . . n1
... ...
... . . .
...
1 j 2j . . . j(n1)
.
..
.
..
.
.. .
. .
.
..1 n1 2(n1) . . . (n1)(n1)
a0a1...
aj.
..an1
=
a
0a1...
aj.
..an1
Ch k i CS473
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Inverting the Matrix
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
99/110
a0a1...
aj...
an1
=
1 1 1 . . . 11 2 . . . n1
... ...
... . . . ...
1 j 2j . . . j(n1)
... ...
... . . .
...
1
n1
2(n1)
. . .
(n1)(n1)
1
a0a
1...aj...
an1
Ch k i CS473
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Inverting the Matrix
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
100/110
2666666666664
1 1 1 . . . 1
1 2 . . . n1
.
.
.
.
.
.
.
.
.. . .
.
.
.
1 j 2j . . . j(n1)
.
.
.
.
.
.
.
.
.. .
....
1 n1 2(n1) . . . (n1)(n1)
3777777777775
1
=1
n
2666666666664
1 1 1 . . . 1
1 1 2 . . . (n1)
.
.
.
.
.
.
.
.
.. . .
.
.
.
1
j
2j
. . .
j(n1)
.
.
.
.
.
.
.
.
.. . .
.
.
.
1 (n1) 2(n1) . . . (n1)(n1)
3777777777775
Replace by1 which is also a root of unity!
Inverse matrix is simply a permutation of the original matrixmodulo scale factor 1/n.
Ch k i CS473
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Why does it work?
http://find/8/11/2019 Convolution Divide and Conquer
101/110
Can check using simple algebra VV1 =I where V is the orignalmatrix and I is the n n identity matrix.
(1,
j
,
2j
, . . . ,
j(n1)
)(1, k
,
2k
, . . . ,
k(n1)
) =
n1
s=0
(jk)s
Note that jk is a nth root of unity. If j=k then sum is n,otherwise by previous observation sum is 0.
Rows of matrix V(and hence also those ofV1) are orthogonal.Thus a =Vacan be thought of a transforming the vector a into anew Fourier basis with basis vectors corresponding to rows ofV.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Inverse Fourier Transform
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
102/110
Input Given the evaluation of a 2n 1-degree polynomial con the 2nth roots of unity
Goal Compute the coefficients ofc
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Inverse Fourier Transform
http://find/8/11/2019 Convolution Divide and Conquer
103/110
Input Given the evaluation of a 2n 1-degree polynomial con the 2nth roots of unity
Goal Compute the coefficients ofc
We saw that a can be computed from a in O(n log n) time. Canwe compute a from a in O(n log n) time?
Yes! a=V1
a which is simply a permuted and scaled version ofDFT. Hence can be computed in O(n log n) time.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Convolutions Once More
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
104/110
Convolutions
Compute convolution ofa= (a0, a1, . . . an1) andb= (b0, b1, . . . bn1)
1 Compute values ofa and bat the 2nth roots of unity
2 Compute sample representation c of product c=a b3 Compute c from c using inverse Fourier transform.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Convolutions Once More
http://find/8/11/2019 Convolution Divide and Conquer
105/110
Convolutions
Compute convolution ofa= (a0, a1, . . . an1) andb= (b0, b1, . . . bn1)
1 Compute values ofa and bat the 2nth roots of unity
2 Compute sample representation c of product c=a b3 Compute c from c using inverse Fourier transform.
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Convolutions Once More
http://find/8/11/2019 Convolution Divide and Conquer
106/110
Convolutions
Compute convolution ofa= (a0, a1, . . . an1) andb= (b0, b1, . . . bn1)
1 Compute values ofa and bat the 2nth roots of unity
2 Compute sample representation c of product c=a b3 Compute c from c using inverse Fourier transform.
Step 1 takes O(n log n) using two FFTs
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Convolutions Once More
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
107/110
Convolutions
Compute convolution ofa= (a0, a1, . . . an1) andb= (b0, b1, . . . bn1)
1 Compute values ofa and bat the 2nth roots of unity
2 Compute sample representation c of product c=a b3 Compute c from c using inverse Fourier transform.
Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Chekuri CS473ug
Polynomials
ConvolutionsFFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Convolutions Once More
http://find/8/11/2019 Convolution Divide and Conquer
108/110
Convolutions
Compute convolution ofa= (a0, a1, . . . an1) andb= (b0, b1, . . . bn1)
1 Compute values ofa and bat the 2nth roots of unity
2 Compute sample representation c of product c=a b3 Compute c from c using inverse Fourier transform.
Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) timeStep 3 takes O(n log n) using one FFT
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
FFT Circuit
http://find/http://goback/8/11/2019 Convolution Divide and Conquer
109/110
FFT(n/2)
FFT(n/2)
P P*
U U*
V V*
bit reversal permutation butterfly network
000
001
010
011
100
101
110
111
000
001
010
011
100
101
110
111
000
001
010
011
100
101
110
111
The recursive structure of the FFT algorithm.
Chekuri CS473ug
PolynomialsConvolutions
FFT
Polynomial RepresentationsBasic Ideas
Complex Roots of UnityFFT Algorithm
Numerical Issues
http://find/8/11/2019 Convolution Divide and Conquer
110/110
As noted earlier evaluating a polynomial a at a point x makesnumbers big
Are we cheating when we say O(n log n) algorithm forconvolution?
Can get around numerical issues work in finite fields andavoid numbers growing too big.
Outside the scope of class.
Chekuri CS473ug
http://find/Top Related