EE123 Digital Signal Processingee123/sp18/Notes/Lecture3C.pdfBased on Course Notes by J.M Kahn Fall...

M. Lustig, EECS UC Berkeley

EE123Digital Signal Processing

Lecture 3CFast Convolutions

The FFT

based on slides by J.M. Kahn


GPIO18GPIO4

0-200MHz


PiFM/QRPi

• PI FM: http://www.icrobotics.co.uk/wiki/index.php/Turning_the_Raspberry_Pi_Into_an_FM_Transmitter#Steps_to_play_sound:

• RPiTX:https://github.com/F5OEO/rpitx

• WsprryPi:https://github.com/JamesP6000/WsprryPi

• QRPi:http://rfsparkling.com/qrpi/

• qtcsdr:https://github.com/ha7ilm/qtcsdr


Fourier Series of a Square Wave

x(t) =4

⇡

1X

k=1

sin(2⇡(2k � 1)ft)

2k � 1

https://commons.wikimedia.org/wiki/File:Spectrum_square_oscillation.jpg#filelinks

spectrum of 1KHz square wave


QRPi• Bandpass analog filter around 14MHz

– Some isolation and gain (20dbm = 0.1W)


stations that around the world that spotted a beacon from a raspberry pi at my home in a period of 24 hours. The beacon transmits a 2min message every 10min putting out only 0.1 Watt of power!


Last Time

• Discrete Fourier Transform– Linear convolution through circular– Block convolutions through DFT

• Overlap and add• Overlap and save

• Today– Finish block convolutions– The Fast Fourier Transform

• Lab 1 is out Due Next Friday-- Will talk about it on Monday


Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]

Input Signal, Length 33

0 10 20 30

-0.5

0

0.5

n

h[n]

Impulse Response, Length P = 6

0 10 20 30

-0.5

0

0.5

n

y[n]

Linear Convolution, Length 38

Miki Lustig UCB. Based on Course Notes by J.M Kahn Fall 2012, EE123 Digital Signal Processing

Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]


0 10 20 30

-0.5

0

0.5

n

h[n]


0 10 20 30

-0.5

0

0.5

n

y[n]



Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]


0 10 20 30

-0.5

0

0.5

n

h[n]


0 10 20 30

-0.5

0

0.5

n

y[n]



h[n] Impulse response, Length P=6

x[n] Input Signal, Length P=33 y[n] Output Signal, Length P=38

Block Convolution

Example:


Overlap-Add Method

• Decompose into non-overlapping segmentsOverlap-Add Method

0 10 20 30

-0.5

0

0.5

n

x 0[n]

Overlap-Add, Input Segments, Length L = 11

0 10 20 30

-0.5

0

0.5

n

y 0[n]

Overlap-Add, Output Segments, Length L+P-1 = 16

0 10 20 30

-0.5

0

0.5

n

x 1[n]

0 10 20 30

-0.5

0

0.5

n

y 1[n]

0 10 20 30

-0.5

0

0.5

n

x 2[n]

0 10 20 30

-0.5

0

0.5

n

y 2[n]

0 10 20 30

-0.5

0

0.5

n

x[n]

Overlap-Add, Sum of Input Segments

0 10 20 30

-0.5

0

0.5

n

y[n]

Overlap-Add, Sum of Output Segments


Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]


0 10 20 30

-0.5

0

0.5

nh[n]


0 10 20 30

-0.5

0

0.5

n

y[n]



x0[n]

x1[n]

x2[n]

x[n] = x0[n]+x1[n]+x2[n]

xr[n] =

⇢x[n] rL n < (r + 1)L

0 otherwise

• The input signal is the sum of segments

x[n] =1X

r=0

xr[n]


Overlap-Add Method

• The output is:

• Each output segment xr[n]*h[n] is length N=L+P-1

x[n] =1X

r=0

xr[n]

y[n] = x[n] ⇤ h[n] =1X

r=0

xr[n] ⇤ h[n]


Overlap-Add Method

• We can compute xr[n]*h[n] using linear convolution

• Using the DFT:– Zero-pad xr[n] to length N– Zero-pad h[n] to length N and compute DFTN{hzp[n]} (only once) WHY?

– Compute

• Neighboring outputs overlap by P-1– Add overlaps to get final sequence

xr[n] ⇤ h[n] = DFT�1 {DFT{xr,zp[n]} ·DFT{hzp[n]}}

Overlap-Add Method

0 10 20 30

-0.5

0

0.5

n

x 0[n]

Overlap-Add, Input Segments, Length L = 11

0 10 20 30

-0.5

0

0.5

n

y 0[n]

Overlap-Add, Output Segments, Length L+P-1 = 16

0 10 20 30

-0.5

0

0.5

n

x 1[n]

0 10 20 30

-0.5

0

0.5

n

y 1[n]

0 10 20 30

-0.5

0

0.5

n

x 2[n]

0 10 20 30

-0.5

0

0.5

n

y 2[n]

0 10 20 30

-0.5

0

0.5

n

x[n]

Overlap-Add, Sum of Input Segments

0 10 20 30

-0.5

0

0.5

n

y[n]

Overlap-Add, Sum of Output Segments


Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]


0 10 20 30

-0.5

0

0.5

n

h[n]


0 10 20 30

-0.5

0

0.5

n

y[n]



Block Convolution

Example:

0 10 20 30

-0.5

0

0.5

n

x[n]


0 10 20 30

-0.5

0

0.5

n

h[n]


0 10 20 30

-0.5

0

0.5

n

y[n]



x0[n]

x1[n]

x2[n]

x[n] = x0[n]+x1[n]+x2[n] y[n] = y0[n]+y1[n]+y2[n]

x0[n]

x1[n]

x2[n]

Example of overlap and add:Change to Ys!


Overlap-Save Method

• Basic idea:• Split input into (P-1) overlapping segments

with length L+P-1

• Perform circular convolution in each segment, and keep the L sample portion which is a valid linear convolution

xr[n] =

⇢x[n] rL n < (r + 1)L+ P

0 otherwise


Recall:x1[n]

x2[n]

n0 1 2 3 4

1

n0 1 2

1

5 6

3 4 5 6

n0 1 2 3 4

2

5 6

4Valid linear convolution!

Overlap-Save Method

0 10 20 30

-0.5

0

0.5

n

y[n]

Overlap-Save, Concatenation of Usable Output Segments

0 10 20 30

-0.5

0

0.5

n

x 0[n]

Overlap-Save, Input Segments, Length L = 16

0 10 20 30

-0.5

0

0.5

n

y 0p[n

]

Overlap-Save, Output Segments, Usable Length L - P + 1

Usable (y0[n])

Unusable

0 10 20 30

-0.5

0

0.5

n

x 1[n]

0 10 20 30

-0.5

0

0.5

n

y 1p[n

]

Usable (y1[n])

Unusable

0 10 20 30

-0.5

0

0.5

n

x 2[n)

0 10 20 30

-0.5

0

0.5

n

y 2p[n

]

Usable (y2[n])

Unusable


Example of overlap and save:

Circular Convolution as Matrix Operation

Circular convolution:

h[n]�N x [n] =

2

6664

h[0] h[N � 1] · · · h[1]h[1] h[0] h[2]

...h[N � 1] h[N � 2] h[0]

3

7775

2

6664

x [0]x [1]...

x [N]

3

7775

= Hc

x

Hc

is a circulant matrixThe columns of the DFT matrix are Eigen vectors of circulantmatrices.Eigen vectors are DFT coe�cients. How can you show?


[N-1]

Proof in HWvalues

Circular Convolution as Matrix Operation

Diagonalize:

WN

Hc

W�1

n

=

2

64H[0] 0 · · · 00 H[1] · · · 0... 0 H[N � 1]

3

75

Right-multiply by WN

WN

Hc

=

2

64H[0] 0 · · · 00 H[1] · · · 0... 0 H[N � 1]

3

75WN

Multiply both sides by x

WN

Hc

x =

2

64H[0] 0 · · · 00 H[1] · · · 0... 0 H[N � 1]

3

75WN

x


N

Fast Fourier Transform Algorithms

We are interested in e�cient computing methods for the DFTand inverse DFT:

X [k] =N�1X

n=0

x [n]W kn

N

, k = 0, . . . ,N � 1

x [n] =N�1X

k=0

X [k]W�kn

N

, n = 0, . . . ,N � 1

whereW

N

= e�j( 2⇡N

).


Recall that we can use the DFT to compute the inverse DFT:

DFT �1{X [k]} =1

N(DFT {X ⇤[k]})⇤

Hence, we can just focus on e�cient computation of the DFT.

Straightforward computation of an N-point DFT (or inverseDFT) requires N2 complex multiplications.


Fast Fourier transform algorithms enable computation of anN-point DFT (or inverse DFT) with the order of justN · log

2

N complex multiplications.This can represent a huge reduction in computational load,especially for large N.

N N2 N · log2

N N

2

N·log2

N

16 256 64 4.0128 16,384 896 18.31,024 1,048,576 10,240 102.48,192 67,108,864 106,496 630.26⇥ 106 36⇥ 1012 135⇥ 106 2.67⇥ 105

* 6Mp image size


Most FFT algorithms exploit the following properties of W kn

N

:

Conjugate Symmetry

W k(N�n)

N

= W�kn

N

= (W kn

N

)⇤

Periodicity in n and k :

W kn

N

= W k(n+N)

N

= W (k+N)n

N

Power:W 2

N

= WN/2


Most FFT algorithms decompose the computation of a DFTinto successively smaller DFT computations.

Decimation-in-time algorithms decompose x [n] intosuccessively smaller subsequences.Decimation-in-frequency algorithms decompose X [k] intosuccessively smaller subsequences.

We mostly discuss decimation-in-time algorithms here.

Assume length of x [n] is power of 2 ( N = 2⌫). If smallerzero-pad to closest power.


Decimation-in-Time Fast Fourier Transform

We start with the DFT

X [k] =N�1X

n=0

x [n]W kn

N

, k = 0, . . . ,N � 1

Separate the sum into even and odd terms:

X [k] =X

n even

x [n]W kn

N

+X

n odd

x [n]W kn

N

These are two DFT’s, each with half of the samples.



Let n = 2r (n even) and n = 2r + 1 (n odd):

X [k] =

(N/2)�1X

r=0

x [2r ]W 2rk

N

+

(N/2)�1X

r=0

x [2r + 1]W (2r+1)k

N

=

(N/2)�1X

r=0

x [2r ]W 2rk

N

+W k

N

(N/2)�1X

r=0

x [2r + 1]W 2rk

N

Note that:

W 2rk

N

= e�j( 2⇡N

)(2rk) = e�j

⇣2⇡N/2

⌘rk

= W rk

N/2

Remember this trick, it will turn up often.



Hence:

X [k] =

(N/2)�1X

r=0

x [2r ]W rk

N/2 +W k

N

(N/2)�1X

r=0

x [2r + 1]W rk

N/2

�

= G [k] +W k

N

H[k], k = 0, . . . ,N � 1

where we have defined:

G [k]�

=

(N/2)�1X

r=0

x [2r ]W rk

N/2 ) DFT of even idx

H[k]�

=

(N/2)�1X

r=0

x [2r + 1]W rk

N/2 ) DFT of odd idx



An 8 sample DFT can then be diagrammed as

x[0]

x[2]

x[4]

x[6]

x[1]

x[3]

x[5]

x[7]

N/2 - Point DFT

N/2 - Point DFT

G[0]

G[1]

G[2]

G[3]

H[0]

H[1]

H[2]

H[3]

X[0]

X[1]

X[2]

X[3]

X[4]

X[5]

X[6]

X[7]

WN0

WN1

WN2

WN3

WN4

WN5

WN6

WN7

Eve

n S

am

ple

sO

dd

Sa

mp

les



Both G [k] and H[k] are periodic, with period N/2. Forexample

G [k + N/2] =

(N/2)�1X

r=0

x [2r ]W r(k+N/2)N/2

=

(N/2)�1X

r=0

x [2r ]W rk

N/2Wr(N/2)N/2

=

(N/2)�1X

r=0

x [2r ]W rk

N/2

= G [k]

so

G [k + (N/2)] = G [k]

H[k + (N/2)] = H[k]



The periodicity of G [k] and H[k] allows us to further simplify.For the first N/2 points we calculate G [k] and W k

N

H[k], andthen compute the sum

X [k] = G [k] +W k

N

H[k] 8{k : 0 k <N

2}.

How does periodicity help for N

2

k < N?



X [k] = G [k] +W k

N

H[k] 8{k : 0 k <N

2}.

for N

2

k < N:

W k+(N/2)N

=?

X [k + (N/2)] =?



X [k + (N/2)] = G [k]�W k

N

H[k]

We previously calculated G [k] and W k

N

H[k].

Now we only have to compute their di↵erence to obtain the secondhalf of the spectrum. No additional multiplies are required.



The N-point DFT has been reduced two N/2-point DFTs,plus N/2 complex multiplications. The 8 sample DFT is then:

x[0]

x[2]

x[4]

x[6]

x[1]

x[3]

x[5]

x[7]

N/2 - Point DFT

N/2 - Point DFT

G[k]

H[k]

X[0]

X[1]

X[2]

X[3]

X[4]

X[5]

X[6]

X[7]

WN0

WN1

WN2

WN3

Eve

n S

am

ple

sO

dd

Sa

mp

les

WNk

-1

-1

-1

-1



Note that the inputs have been reordered so that the outputscome out in their proper sequence.We can define a butterfly operation, e.g., the computation ofX [0] and X [4] from G [0] and H[0]:

G[0] X[0] =G[0] + WN0

H[0]

WN0

-1

H[0] X[4] =G[0] - WN0

H[0]

This is an important operation in DSP.



Still O(N2) operations..... What shall we do?

x[0]

x[2]

x[4]

x[6]

x[1]

x[3]

x[5]

x[7]

N/2 - Point DFT

N/2 - Point DFT

G[k]

H[k]

X[0]

X[1]

X[2]

X[3]

X[4]

X[5]

X[6]

X[7]

WN0

WN1

WN2

WN3

Eve

n S

am

ple

sO

dd

Sa

mp

les

WNk

-1

-1

-1

-1



We can use the same approach for each of the N/2 pointDFT’s. For the N = 8 case, the N/2 DFTs look like

x[0]

x[2]

x[4]

x[6]

N/4 - Point DFT

G[1]

G[2]

G[3]

N/4 - Point DFT

G[0]

WN/20

WN/21

-1

-1

*Note that the inputs have been reordered again.



At this point for the 8 sample DFT, we can replace theN/4 = 2 sample DFT’s with a single butterfly.The coe�cient is

WN/4 = W

8/4 = W2

= e�j⇡ = �1

The diagram of this stage is then

-1

x[0]

x[4]

1

x[0] + x[4]

x[0] - x[4]



Combining all these stages, the diagram for the 8 sample DFT is:

x[0]

x[2]

x[4]

x[6]

x[1]

x[3]

x[5]

x[7]

X[0]

X[1]

X[2]

X[3]

X[4]

X[5]

X[6]

X[7]

WN0

WN1

WN2

WN3

-1

-1

-1

-1

WN/20

WN/21

-1

-1

WN/20

WN/21

-1

-1

-1

-1

-1

-1

This the decimation-in-time FFT algorithm.



In general, there are log2

N stages of decimation-in-time.

Each stage requires N/2 complex multiplications, some ofwhich are trivial.

The total number of complex multiplications is (N/2) log2

N.

The order of the input to the decimation-in-time FFTalgorithm must be permuted.

First stage: split into odd and even. Zero low-order bit firstNext stage repeats with next zero-lower bit first.Net e↵ect is reversing the bit order of indexes



This is illustrated in the following table for N = 8.

Decimal Binary Bit-Reversed Binary Bit-Reversed Decimal

0 000 000 01 001 100 42 010 010 23 011 110 64 100 001 15 101 101 56 110 011 37 111 111 7


EE123 Digital Signal Processingee123/sp18/Notes/Lecture3C.pdfBased on Course Notes by J.M Kahn Fall...

Documents

Transcript of EE123 Digital Signal Processingee123/sp18/Notes/Lecture3C.pdfBased on Course Notes by J.M Kahn Fall...