dspbook

An Introduction to Digital Signal Processing

Edmund M-K. Lai

17th November 2003

Contents

Preface xi

1 Introduction 1

1.1 Advantages of Processing Signals Digitally . . . . . . . . . . . . . . . . 2

1.2 Definition of Some Terms . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 DSP Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.1 Sample-by-sample Processing . . . . . . . . . . . . . . . . . . . 4

1.3.2 Block Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.3 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4 Application Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4.1 Speech and Audio Processing . . . . . . . . . . . . . . . . . . . 5

1.4.1.1 Speech Coding . . . . . . . . . . . . . . . . . . . . . . 5

1.4.1.2 Linear Prediction . . . . . . . . . . . . . . . . . . . . 7

1.4.1.3 Speech Synthesis . . . . . . . . . . . . . . . . . . . . 9

1.4.1.4 Speech Recognition . . . . . . . . . . . . . . . . . . . 9

1.4.2 Image and Video Processing . . . . . . . . . . . . . . . . . . . . 9

1.4.2.1 Image Enhancement . . . . . . . . . . . . . . . . . . . 9

1.4.2.2 Image Restoration . . . . . . . . . . . . . . . . . . . . 10

1.4.2.3 Image Compression and Coding . . . . . . . . . . . . 10

1.4.3 Adaptive Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 11

iv CONTENTS

1.4.3.1 Noise Cancellation . . . . . . . . . . . . . . . . . . . . 11

1.4.3.2 Echo Cancellation . . . . . . . . . . . . . . . . . . . . 11

1.4.3.3 Channel Equalization . . . . . . . . . . . . . . . . . . 12

1.4.4 Control Applications . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4.5 Sensor or Antenna Array Processing . . . . . . . . . . . . . . . 15

1.4.6 Digital Communication Transceivers . . . . . . . . . . . . . . . 16

1.5 Brief Overview of Chapters . . . . . . . . . . . . . . . . . . . . . . . . . 16

2 Discrete-time Signals and Systems 19

2.1 Discrete-time signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.1.1 Some Elementary Sequences . . . . . . . . . . . . . . . . . . . . 20

2.1.1.1 Unit Impulse Sequence . . . . . . . . . . . . . . . . . 20

2.1.1.2 Unit Step Sequence . . . . . . . . . . . . . . . . . . . 20

2.1.1.3 Sinusoidal Sequences . . . . . . . . . . . . . . . . . . 21

2.1.1.4 Complex Exponential Sequences . . . . . . . . . . . . 21

2.1.1.5 Random Sequences . . . . . . . . . . . . . . . . . . . 22

2.1.2 Types of Sequences . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.1.3 Some Basic Operations on Sequences . . . . . . . . . . . . . . . 27

2.2 Discrete-time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.2.1 Classification of Systems . . . . . . . . . . . . . . . . . . . . . . 30

2.2.2 Linear Shift-Invariant Systems . . . . . . . . . . . . . . . . . . . 34

2.2.2.1 Linear Convolution . . . . . . . . . . . . . . . . . . . 34

2.2.2.2 Properties of Linear Convolution . . . . . . . . . . . . 38

2.2.2.3 Condition for Stability . . . . . . . . . . . . . . . . . . 38

2.2.2.4 Condition for Causality . . . . . . . . . . . . . . . . . 39

2.2.3 FIR and IIR Systems . . . . . . . . . . . . . . . . . . . . . . . . 40

2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

CONTENTS v

3 Discrete-time Fourier Analysis 43

3.1 Continuous-time Fourier Analysis . . . . . . . . . . . . . . . . . . . . . 43

3.2 Discrete-time Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . 44

3.2.1 Average Power . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.2.2 Normalized Frequency . . . . . . . . . . . . . . . . . . . . . . . 47

3.2.3 Power Density Spectrum . . . . . . . . . . . . . . . . . . . . . . 48

3.3 Discrete-time Fourier Transform . . . . . . . . . . . . . . . . . . . . . . 49

3.3.1 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.3.1.1 Uniform Convergence . . . . . . . . . . . . . . . . . . 50

3.3.1.2 Mean Square Convergence . . . . . . . . . . . . . . . 50

3.3.2 Energy Density Spectrum . . . . . . . . . . . . . . . . . . . . . 52

3.3.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.3.1 Periodicity . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.3.2 Symmetry . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.3.3 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.3.3.4 Time Shifting . . . . . . . . . . . . . . . . . . . . . . 54

3.3.3.5 Time Reversal . . . . . . . . . . . . . . . . . . . . . . 55

3.3.3.6 Frequency Shifting . . . . . . . . . . . . . . . . . . . 55

3.3.3.7 Linear Convolution . . . . . . . . . . . . . . . . . . . 55

3.3.4 Inverse DTFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.3.5 Autocorrelation Sequence . . . . . . . . . . . . . . . . . . . . . 56

3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4 Discrete Fourier Transform and Its Computation 59

4.1 Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.1.1 Sampling the DTFT . . . . . . . . . . . . . . . . . . . . . . . . 59

4.1.2 Inverse DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

vi CONTENTS

4.1.3 Matrix Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.1.4 Zero-padding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.1.5 Circular Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1.6 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1.7 Circular Convolution . . . . . . . . . . . . . . . . . . . . . . . . 67

4.1.8 Linear Convolution Using DFT . . . . . . . . . . . . . . . . . . 69

4.1.9 Computational Complexity . . . . . . . . . . . . . . . . . . . . . 75

4.2 The Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 76

4.2.1 Radix-2 Decimation-in-time Algorithm . . . . . . . . . . . . . . 77

4.2.2 Radix-2 Decimation-in-frequency Algorithm . . . . . . . . . . . 81

4.2.3 Other Fast Algorithms . . . . . . . . . . . . . . . . . . . . . . . 84

4.3 Spectral Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

4.3.1 Spectral Leakage . . . . . . . . . . . . . . . . . . . . . . . . . . 84

4.3.2 Rectangular Windowing . . . . . . . . . . . . . . . . . . . . . . 86

4.3.3 Frequency Resolution . . . . . . . . . . . . . . . . . . . . . . . 89

4.3.4 Computational Frequency Resolution . . . . . . . . . . . . . . . 92

4.3.5 Non-stationary Signals . . . . . . . . . . . . . . . . . . . . . . . 93

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

5 Sampling, Reconstruction and Quantization 97

5.1 The Sampling Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

5.1.1 Spectra of Sampled Signals . . . . . . . . . . . . . . . . . . . . 99

5.1.2 Aliasing and The Uniform Sampling Theorem . . . . . . . . . . 100

5.1.3 Anti-aliasing Prefilters . . . . . . . . . . . . . . . . . . . . . . . 107

5.1.4 Practical Limits on Sampling Rates . . . . . . . . . . . . . . . . 108

5.2 Reconstruction of Analog Signals . . . . . . . . . . . . . . . . . . . . . 108

5.2.1 Ideal Reconstructor . . . . . . . . . . . . . . . . . . . . . . . . . 109

CONTENTS vii

5.2.2 Staircase Reconstructor . . . . . . . . . . . . . . . . . . . . . . . 111

5.2.3 Anti-image Postfilters . . . . . . . . . . . . . . . . . . . . . . . 111

5.3 Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

5.3.1 Sample-and-Hold . . . . . . . . . . . . . . . . . . . . . . . . . . 114

5.3.2 Uniform Quantization . . . . . . . . . . . . . . . . . . . . . . . 115

5.3.3 Quantization Noise . . . . . . . . . . . . . . . . . . . . . . . . . 116

5.3.4 Signal-to-Quantization Noise Ratio . . . . . . . . . . . . . . . . 117

5.4 Sampling Rate and Quantization Tradeoff . . . . . . . . . . . . . . . . . 118

5.4.1 Power Spectral Density . . . . . . . . . . . . . . . . . . . . . . . 118

5.4.2 Oversampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

5.4.3 Noise Shaping . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.4.4 Non-uniform Quantization . . . . . . . . . . . . . . . . . . . . . 122

5.4.4.1 Companding . . . . . . . . . . . . . . . . . . . . . . . 123

5.5 Dithering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

6 Digital Filters and Their Realizations 131

6.1 A Review of Z Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6.1.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

6.1.3 Inverse Z Transform . . . . . . . . . . . . . . . . . . . . . . . . 133

6.2 System Transfer Functions . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.2.1 Poles and Zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.2.2 Frequency Response . . . . . . . . . . . . . . . . . . . . . . . . 134

6.3 Digital Filter Realization . . . . . . . . . . . . . . . . . . . . . . . . . . 137

6.3.1 Direct Form I . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

6.3.2 Canonical Form or Direct Form II . . . . . . . . . . . . . . . . . 139

viii CONTENTS

6.3.3 Transposed Structures . . . . . . . . . . . . . . . . . . . . . . . 143

6.3.4 FIR Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

6.3.5 Cascade Realization . . . . . . . . . . . . . . . . . . . . . . . . 146

6.3.6 Parallel Realization . . . . . . . . . . . . . . . . . . . . . . . . . 149

6.3.7 Circular Buffers . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.4 Finite Wordlength Effects . . . . . . . . . . . . . . . . . . . . . . . . . . 154

6.4.1 Fixed-point Representation . . . . . . . . . . . . . . . . . . . . . 155

6.4.2 Floating-point Representation . . . . . . . . . . . . . . . . . . . 157

6.4.3 Coefficient Quantization . . . . . . . . . . . . . . . . . . . . . . 158

6.4.4 Round-off Noise . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.4.5 Limit Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.4.6 Arithmetic Overflow . . . . . . . . . . . . . . . . . . . . . . . . 167

6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

7 FIR Filter Design 171

7.1 Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

7.2 Window Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

7.2.1 Rectangular Window . . . . . . . . . . . . . . . . . . . . . . . . 177

7.2.2 Kaiser Window . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

7.3 Highpass Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7.4 Bandpass Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

7.5 Frequency Sampling Method . . . . . . . . . . . . . . . . . . . . . . . . 183

7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

8 IIR Filter Design 187

8.1 Design Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

8.2 Some Classic Analog Filters . . . . . . . . . . . . . . . . . . . . . . . . 188

8.2.1 Butterworth Filters . . . . . . . . . . . . . . . . . . . . . . . . . 188

CONTENTS ix

8.2.2 Chebyshev Filters . . . . . . . . . . . . . . . . . . . . . . . . . . 192

8.3 Bilinear Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

8.3.1 Frequency Warping . . . . . . . . . . . . . . . . . . . . . . . . . 196

8.3.2 Frequency Transformation . . . . . . . . . . . . . . . . . . . . . 199

8.4 Impulse Invariant Method . . . . . . . . . . . . . . . . . . . . . . . . . . 202

8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Bibliography 207

Index 209

x CONTENTS

Preface

Many excellent textbooks have been written on the subject of Digital Signal Processing(DSP). See the bibliography at the end of the book for examples. They typically cover awide range of topics and in much detail. Since DSP is such a broad area, there are veryfew DSP textbooks that are less than 500 pages in length. Students often find themselvesoverwhelmed by the depth and breadth covered by these books. Moreover, the amount ofmathematics used gave students the impression that DSP is a very difficult subject.

I have been teaching DSP to undergraduates and practising engineers and techniciansover a number of years. Often the students both in universities and in the industry do notrealize that it is some of the fundamental concepts in DSP that will take time to digest.Once they grasped some of these concepts, the mathematics becomes obvious and simple.The emphasis of this book is on the fundamental concepts and ideas that are importantand common to most DSP applications. I have consciously keep the length of the bookto a minimum by carefully selecting the topics to be covered. That means of course thatthere are a number of interesting topics in DSP that are not even touched upon. Readersare encouraged to consult the more comprehensive DSP texts if they are interested inextending their knowledge to these other areas. The fundamentals that they have learntfrom this book should enable them to pursue the subject further with emphases based ontheir areas of interest.

The materials in this book are based on the course I give to senior Computer Engineeringundergraduates in Nanyang Technological University, Singapore. They are suitable foruse in a single semester course for senior undergraduate or first year graduate students.Readers are assumed to have some basic knowledge of continuous-time signals and sys-tems, and Z transform. A typical mathematics or signal and systems course in the earlypart of an electrical and computer engineering curriculum will suffice.

The book is typeset entirely using LATEX and Lyx on a Linux platform. Most of thediagrams are drawn using the Xfig program. All these are open source programs that

xii Preface

are the efforts of thousands of volunteers. I wish to thank them for producing such highquality software for the common good. The remaining graphs and plots are producedusing MATLAB R©.

Many former students of The University of Western Australia, The Chinese Universityof Hong Kong, Edith Cowan University in Perth, Australia, and Nanyang TechnologicalUniversity, Singapore since the late 1980’s have been taught using part or all of the ma-terials in this book. To see them learn and develop an interest in DSP has always beena joy. They in turn inspired me by asking insightful questions and by challenging mypresentations. I wish to thank all of them.

I also wish to thank my wife, Susie, and my daughters, Rhoda and Sarah, for bring somuch joy and purpose to my life. Without them it would not have been worth the effortat all. This book is dedicated to them.

Singapore

Chapter 1

Introduction

Digital signal processing (DSP) is concerned with the use of digital processors to analyze,modify and extract information from a signal. Signals are normally physical quantitiesthat can be measured. DSP is a field which is primarily technology driven. It started fromaround the mid-1960’s when digital computers and digital circuitry became fast enoughto process large amount of data efficiently.

When the term “digital” is used, often it loosely refers to a finite set of distinct values.This is in contrast to “analog” which refers to a continuous range of values. Indigitalsignal processingwe are concerned with the processing of signals which arediscrete intime (sampled) and, in most cases,discrete in amplitude(quantized) as well. In otherwords, we are primarily dealing withdata sequences– sequences of numbers.

Such discrete (or digital) signals may arise in one of the following two distinct circum-stances:

1. The signal may be inherently discrete in time (and/or amplitude); or

2. The signal may be a sampled version of a continuous-time signal.

Examples of the first type of data sequences include monthly sales figures, daily high-est/lowest temperatures, stock market indices and students’ examination marks. Businesspeople, meteorologists, economists, and teachers process these types of data sequences todetermine cyclic patterns, trends, and averages. The processing usually involves filteringto remove as much “noise” as possible so that the pattern of interest will be enhanced orhighlighted.

2 Introduction

Examples of the second type of discrete-time signals can readily be found in many en-gineering applications. For instance, speech and audio signals are sampled and thenencoded for storage or transmission. A compact disc player reads the encoded digitalaudio signals and reconstructs the continuous-time signals for playback.

1.1 Advantages of Processing Signals Digitally

A typical question one may ask is: why process signals digitally? For the first type ofsignals discussed previously, the reason is obvious. If the signals are inherently discrete intime, the most natural way to process them is using digital methods. But for continuous-time signals, we have a choice.

Analog signals are processed by analog electronics while digital signals can be processedby computers or microprocessors. Analog methods are potentially faster since the ana-log circuits process signals as they arrive in real-time, provided the settling time is fastenough. On the other hand, digital techniques are algorithmic in nature. If the com-puter is fast and the algorithms are efficient, then digital processing can be performed in“real-time” provided the data rate is “slow enough”. However, with the speed of digitallogic increasing exponentially, the upper limit in data rate that can still be considered asreal-time processing is becoming higher and higher.

The major advantage of digital signal processing isconsistency. For the same signal,the output of a digital process will always be the same. It is not sensitive to offsets anddrifts in electronic components. The second main advantage of DSP is that very complexdigital logic circuits can be packed onto a single chip, thus reducing the component countand the size and reliability of the system.

1.2 Definition of Some Terms

DSP has its origin in electrical/electronic engineering (EE). Therefore the terminologiesused in DSP are typically that of EE. We shall now attempt to explain a few terms thatwe shall be using throughout the book.

Signals — we have already started using this term in the previous section. A signal issimply a quantity that we can measure over a period of time. This quantity usually

Definition of Some Terms 3

changes with time and that is what makes it interesting. Such quantities couldbe voltage or current. They could also be the pressure, fluid level and temperature.Other quantities of interest include financial indices such as the stock market index.You will be surprised how much of the concepts in DSP has been used to analysethe financial market.

Frequency — some signals change slowly over time and others change rapidly. Forinstance, the (AC) voltage available at our household electrical mains goes up anddown like a sine function and they complete one cycle in 50 times or 60 times asecond. This signal is said to have a frequency of 50 or 60 Hertz (Hz).

Spectrum — while some signals consist of only a single frequency, others have a com-bination of a range of frequencies. If you play a string on the violin, there is afundamental tone (frequency) corresponding to the musical note that is played.But there are other harmonics (integer multiples of the fundamental frequency)present. This musical sound signal is said to have a spectrum of frequencies. Thespectrum is a frequency (domain) representation of the time (domain) signal. Thetwo representations are equivalent.

Low-pass Filter — filters let a certain range of frequency components of a signal throughwhile rejecting the other frequency components. A low-pass filter lets the "low-frequency" components through. Low-pass filters have acutoff frequencybelowwhich the frequency components can pass through the filter. For instance, if a sig-nal has two frequency components, say, 10 Hz and 20 Hz. Applying a low-passfilter to this signal with a cutoff frequency of 15 Hz will result in an output signalwhich has only one frequency component at 10 Hz; the 20 Hz component has beenrejected by the filter.

Bandpass Filter — bandpass filters are similar to low-pass filters in that only a rangeof frequency components can pass through it intact. This range (thepassband)is usually above the DC (zero frequency) and somewhere in the mid-range. Forinstance, we can have a bandpass filter with a passband between 15 and 25 Hz.Applying this filter to the signal discussed above will result in a signal having onlya 20 Hz component.

High-pass Filter — these filters allows frequency componentsabovea certain frequency(cutoff) to pass through intact, rejecting the ones lower than the cutoff frequency.

This should be enough to get us going. New terms will arise from time to time and theywill be explained as we come across them.

4 Introduction

1.3 DSP Systems

DSP systems arediscrete-time systemswhich means that they accept digital signals asinput and output digital signals (or information extracted). Digital signals are simplysequences of numbers. The output sequence of samplesy[n] is computed from the inputsequence of samplesx[n] according to some rules which the discrete-time systemHdefines.

There are two main methods by which the output sequence is computed from the inputsequence. They are calledsample-by-sample processingandblock processingrespec-tively. We shall encounter both types of processing in later chapters. Most systems canbe implemented with either processing method. The output obtained in both cases shouldbe equivalent if the input and the system are the same.

1.3.1 Sample-by-sample Processing

With the sample-by-sample processing method, normally one output sample is obtainedwhen one input sample is presented to the system.

For instance, if the sequencey0, y1, y2, . . . , yn, . . . is obtained when the input sequencex0, x1, x2, . . . , xn, . . . is presented to the system. The sampley0 appears at the outputwhen the inputx0 is available at the input. The sampley1 appears at the output when theinputx1 is available at the input, etc.

The delay between the input and output for sample-by-sample processing is at most onesample. The processing has to be completed before the next sample appears at the input.

1.3.2 Block Processing

With block processing methods, a block of signal samples are being processing at a time.A block of samples are usually treated as a vector which is transformed to an outputvector of samples by the system transformationH.

x=

x0

x1

x2...

H−→

y0

y1

y2...

= y (1.1)

Application Areas 5

The delay between input and output in this case is dependent on the number of samplesin each block. For example, if we use 8 samples per block, then the first 8 input sampleshave to be buffered (or collected) before processing can proceed. So the block of 8output samples will appear at least 8 samples after the first samplex0 appears. The blockcomputation (according toH) has to be completed before the next block of 8 samples arecollected.

1.3.3 Remarks

Both processing methods are extensively used in real applications. We shall encounterDSP algorithms and implementation that uses one or the other. The reader might find ituseful in understanding the algorithms or techniques being discussed by being consciousof which processing method is being used.

1.4 Application Areas

Digital signal processing is being applied to a large range of applications. No attempt ismade to include all areas of application here. In fact, new applications are constantly ap-pearing. In this section, we shall try to describe a sufficiently broad range of applicationsso that the reader can get a feel of what DSP is about.

1.4.1 Speech and Audio Processing

An area where DSP has found a lot of application is in speech processing. It is also one ofthe earliest applications of DSP. Digital speech processing includes three main sub-areas:encoding, synthesis, andrecognition.

1.4.1.1 Speech Coding

There is a considerable amount of redundancy in the speech signal. The encoding processremoves as much redundancy as possible while retaining an acceptable quality of theremaining signal. Speech coding can be further divided into two areas:

6 Introduction

1. compression – a compact representation of the speech waveform without regard toits meaning.

2. parametrization – a model that characterizes the speech in some linguistically oracoustically meaningful form.

The minimum channel bandwidth required for the transmission of an acceptable qualityof speech is around 3kHz, with a dynamic range of 72dB. This is normally referred toas telephone quality. Converting into digital form, a sampling rate of 8k samples persecond with a 12-bit quantization (212 amplitude levels) is commonly used, resulting in96k bits per second of data. This data rate can be significantly reduced without affectingthe quality of the reconstructed speech as far as the listener is concerned. We shall brieflydescribe three of them:

Companding or Non-uniform Quantization The dynamic range of speech signalsis very large. This is due to the fact that voiced sounds such asvowelscontains a lotof energy and exhibits wide fluctuations in amplitude while unvoiced sounds likefrica-tivesgenerally have much lower amplitudes. Acompander(compressor-expander) com-presses the amplitude of the signal at the transmitter end and expands it at the receiverend. The process is illustrated schematically in Figure 1.1. The compressor compressesthe large amplitude samples and expands the small amplitude ones while the expanderdoes the opposite.

Theµ-law compander (withµ = 255) is a North American standard. A-law compandingwith A = 87.56 is a European (CCITT) standard. The difference in performance isminimal. A-law companding gives slightly better performance at high signal levels whileµ-law is better at low levels.

Adaptive Differential Quantization At any adequate sampling rate, consecutive sam-ples of the speech signal are highly correlated, except for those sounds that contain asignificant amount ofwideband noise. The data rate can be greatly reduced by quantiz-ing thedifferencebetween two samples instead. Since the dynamic range will be muchreduced by differencing, the number of levels required for the quantizer will also be re-duced.

The concept of differential quantization can be extended further. Suppose we have anestimateof the value of the current sample based on information from the previous sam-ples, then we can quantize the difference between the current sample and its estimate. Ifthe prediction is accurate enough, this difference will be quite small.

Application Areas 7

Compressor Expander

1

0 1

1

0 1

Transmitter Receiver

Signal

x(t)x’(t)SignalRecovered

y(t) y’(t)

y(t)

x(t) y’(t)

x’(t)

Figure 1.1: The companding process

Figure 1.2 shows the block diagram of an adaptive differential pulse code modulator(ADPCM). It takes a 64 kbits per second pulse code modulated (PCM) signal and encodesit into 32kbit per second adaptive differential pulse code modulated (ADPCM) signal.

1.4.1.2 Linear Prediction

The linear predictive coding method of speech coding is based on a (simplified) model ofspeech production shown in Figure 1.3.

The time-varying digital filter models the vocal tract and is driven by an excitation signal.For voiced speech, this excitation signal is typically a train of scaled unit impulses at pitchfrequency. For unvoiced sounds it is random noise.

The analysis system (or encoder) estimates the filter coefficients, detects whether thespeech is voiced or unvoiced and estimates the pitch frequency if necessary. This isperformed for each overlapping section of speech usually around 10 milliseconds in du-ration. This information is then encoded and transmitted. The receiver reconstructs thespeech signal using these parameters based on the speech production model. It is inter-esting to note that the reconstructed speech is similar to the original perceptually but thephysical appearance of the signal is very different. This is an illustration of the redun-dancies inherent in speech signals.

8 Introduction

Predictor +

EncoderQuantizer

∆[n]

c[n]

Step SizeAdjustment

x[n]signalerror

−

+

Figure 1.2: Block diagram of an adaptive differential pulse code modulator

RandomNoise

For voicedspeech

ImpulseTrain

For unvoicedspeech

Voiced/Unvoiced

Decision

Filter

Time−varyingGain

Synthesized

Speech

ModelVocal Tract

Figure 1.3: A model of speech production

Application Areas 9

1.4.1.3 Speech Synthesis

The synthesis or generation of speech can be done through the speech production modelmentioned above. Although the duplication of the acoustics of the vocal tract can becarried out quite accurately, the excitation model turns out to be more problematic.

For synthetic speech to sound natural, it is essential that the correct allophone be pro-duced. Despite the fact that different allophones are perceived as the same sound, if thewrong allophone is selected, the synthesized speech will not sound natural. Translationfrom phonemes to allophones is usually controlled by a set of rules. The control of timingof a word is also very important. But these rules are beyond the realm of DSP.

1.4.1.4 Speech Recognition

One of the major goal of speech recognition is to provide an alternative interface betweenhuman user and machine. Speech recognition systems can either be speaker dependent orindependent, and they can either accept isolated utterances or continuous speech. Eachsystem is capable of handling a certain vocabulary.

The basic approach to speech recognition is to extract features of the speech signals in thetraining phase. In therecognition phase, the features extracted from the incoming signalare compared to those that have been stored. Owing to the fact that our voices changewith time and the rate at which we speak also varies, speech recognition is a very toughproblem. However, there are now commercially available some relatively simple smallvocabulary, isolated utterance recognition systems. This comes about after 30 years ofresearch and the advances made in DSP hardware and software.

1.4.2 Image and Video Processing

Image processing involves the processing of signals which are two dimensional. A digitalimage consists of a two dimensional array of pixel values instead of a one dimensionalone for, say, speech signals. We shall briefly describe three areas of image processing.

1.4.2.1 Image Enhancement

Image enhancement is used when we need to focus or pick out some important featuresof an image. For example, we may want to sharpen the image to bring out details such

10 Introduction

as a car licence plate number or some areas of an X-ray film. In aerial photographs, theedges or lines may need to be enhanced in order to pick out buildings or other objects.Certain spectral components of an image may need to be enhanced in images obtainedfrom telescopes or space probes. In some cases, the contrast may need to be enhanced.

While linear filtering may be all that is required for certain types of enhancement, mostuseful enhancement operations are nonlinear in nature.

1.4.2.2 Image Restoration

Image restoration deals with techniques for reconstructing an image that may have beenblurred by sensor or camera motion, and in which additive noise may be present. Theblurring process is usually modeled as a linear filtering operation and the problem ofimage restoration then becomes one of identifying the type of blur and estimating theparameters of the model. The image is then filtered by the inverse of the filter.

1.4.2.3 Image Compression and Coding

The amount of data in a visual image is very large. A simple black-and-white still picturedigitized to a512×512 array of pixels using 8 bits per pixel involves more than 2 millionbits of information. In the case of sequences of images such as in video or televisionimages, the amount of data involved will be even greater. Image compression, like speechcompression, seeks to reduce the number of bits required to store or transmit the imagewith either no loss or an acceptable level of loss or distortion. A number of differenttechniques have been proposed, including prediction or coding in the (spatial) frequencydomain. The most successful techniques typically combine several basic methods. Verysophisticated methods have been developed for digital cameras and digital video discs(DVD).

Standards have been developed for the coding of both image and video signals for dif-ferent kinds of applications. For still images, the most common one is JPEG. For highquality motion video, there is MPEG and MPEG-2. MPEG-2 was developed with high-definition television in mind. It is now used in satellite transmission of broadcast qualityvideo signals.

Application Areas 11

1.4.3 Adaptive Filtering

A major advantage of digital processing is its ability of adapt to changing environments.Even though adaptive signal processing is a more advanced topic which we will not coverin this course, we shall describe the basic ideas involved in adaptive signal processing andsome of its applications.

A basic component in an adaptive digital signal processing system is a digital filter withadjustable filter coefficients – a time-varying digital filter. Changing the characteristicsof a filter by a change in the coefficient values is a very simple operation in DSP. Theadaptation occurs through an algorithm which takes a reference (or desired) signal andan error signal produced by the difference between the current output of the filter andthe current input signal. The algorithm adjusts the filter coefficients so that the averagederror is minimized.

1.4.3.1 Noise Cancellation

One example of noise cancellation is the suppression of the maternal ECG componentin fetal ECG. The fetal heart rate signal can be obtained from a sensor placed in theabdominal region of the mother. However, this signal is very noisy due to the mother’sheartbeat and fetal motion.

The idea behind noise cancellation in this case is to take a direct recording of the mother’sheartbeat and after filtering of this signal, subtract it off the fetal heart rate signal to geta relatively noise-freee heart rate signal. A schematic diagram of the system is shown inFigure 1.4.

There are two inputs: a primary and a reference. The primary signal is of interest buthas a noisy interference component which is correlated with the reference signal. Theadaptive filter is used to produce an estimate of this interference or noise componentwhich is then subtracted off the primary signal. The filter should be chosen to ensure thatthe error signal and the reference signal areuncorrelated.

1.4.3.2 Echo Cancellation

Echoes are signals that are identical to the original signals but are attenuated and delayedin time. They are typically generated in long distance telephone communication due toimpedance mismatch. Such a mismatch usually occurs at the junction or hybrid between

12 Introduction

Reference signalfrom chest sensorNoise

Primary signal

abdominal sensorfrom

FilterAdjustable

Algorithm

Adaptive

adjustmentcoefficient

Filter

−

Figure 1.4: An adaptive noise cancellation system

the local subscriber loop and the long distance loop. As a result of the mismatch, incidentelectromagnetics waves are reflected which sound like echoes to the telephone user.

The idea behind echo cancellation is to predict the echo signal values and thus subtractit out. The basic mechanism is illustrated in Figure 1.5. Since the speech signal isconstantly changing, the system has to be adaptive.

1.4.3.3 Channel Equalization

Consider the transmission of a signal over a communication channel (e.g. coaxial cable,optical fibre, wireless). The signal will be subject to channel noise and dispersion caused,for example, by reflection from objects such as buildings in the transmission path. Thisdistorted signals will have to be reconstructed by the receiver.

One way to restore the original signal is to pass the received signal through anequalizingfilter to undo the dispersion effects. Theequalizershould ideally be the inverse of thechannel characteristics. However, channel characteristics typically drift in time and sothe equalizer (a digital filter) coefficients will need to be adjusted continuously. If thetransmission medium is a cable, the drift will occur very slowly. But for wireless channelsin mobile communications the channel characteristics change rapidly and the equalizerfilter will have to adapt very quickly.


Near−end signal

far−end signaland echo of

To far−end

−

junction hybrid

4 wire to 2 wire

Filter

Adjustable

AlgorithmAdaptive

Near−endsignal

signalFar−end

Cleanednear−endsignal

adjustmentFilter coefficient

Figure 1.5: An adaptive echo cancellation system

In order to “learn” the channel characteristics, the adaptive equalizer operates in a train-ing mode where a pre-determined training signal is transmitted to the receiver. Normalsignal transmission has to be regularly interrupted by a brief training session so that theequalizer filter coefficients can be adjusted. Figure 1.6 shows an adaptive equalizer intraining mode.

1.4.4 Control Applications

A digital controller is a system used for controlling closed-loop feedback systems asshown in Figure 1.7. The controller implements algebraic algorithms such as filters andcompensators to regulate, correct, or change the behaviour of the controlled system.

Digital control has the advantage that complex control algorithms are implemented insoftware rather than specialized hardware. Thus the controller design and its parameterscan easily be altered. Furthermore, increased noise immunity is guaranteed and parame-

14 Introduction

−

+

Delay

Channel

Signal

Training

SignalError

Noise

(Equalizer)Filter

Adjustable

Adaptive

Algorithm

+−

Figure 1.6: An adaptive equaliser in training mode

ParameterAdjustment

signal+Controller System

AlgorithmAdaptive

errorSignal

Reference −Output

Figure 1.7: A digital closed loop control system


ter drift is eliminated. Consequently, they tend to be more reliable and at the same time,features reduced size, power, weight and cost.

Digital signal processors are very useful for implementing digital controllers since theyare typically optimized for digital filtering operations with single instruction arithmeticoperations. Furthermore, if the system being controlled changes with time, adaptivecontrol algorithms, similar to adaptive filtering discussed above, can be implemented.

1.4.5 Sensor or Antenna Array Processing

In some applications, a number of spatially distributed sensors are used for receivingsignals from some sources. The problem of coherently summing the outputs from thesesensors is known asbeamforming. Beyond the directivity provided by an individual sen-sor, a beamformer permits one to “listen” preferentially to wavefronts propagating fromone direction over another. Thus a beamformer implements a spatial filter. Applicationsof beamforming can be found inseismology, underwater acoustics, biomedical engineer-ing, radio communication systemsandastronomy.

In cellular mobile communication systems, smart antennas (an antenna array with dig-itally steerable beams) are being used to increase user capacity and expand geographiccoverage. In order to increase capacity, an array which can increase the carrier to inter-ference ratio (C/I) at both the base station and the mobile terminal is required. There arethree approaches to maximizing C/I with an antenna array.

1. The first one is to create higher gain on the antenna in the intended direction usingantenna aperture. This is done by combining the outputs of each individual antennato create aperture.

2. The second approach is the mitigation of multipath fading. In mobile communi-cation, fast fading induced by multipath propagation requires an additional linkmargin of 8dB. This margin can be recovered by removing the destructive multi-path effects.

3. The third approach is the identification and nulling of interferers. It is not difficultfor a digital beamformer to create sharpnulls, removing the effects of interference.

Direction of arrival estimation can also be performed using sensor arrays. In the simplestconfiguration, signals are received at two spatially separated sensors with one signal be-ing an attenuated, delayed and noisy version of the other. If the distance between the

16 Introduction

sensors is known, and the signal velocity is known, then the direction of arrival can beestimated. If the direction does not change, or changes very slowly with time, then it canbe determined by cross-correlating the two signals and finding the global maximum of thecross-correlation function. If the direction changes rapidly, then an adaptive algorithm isneeded.

1.4.6 Digital Communication Transceivers

One of the most exciting applications of DSP is in the design and implementation of dig-ital communication equipment. Throughout the 1970’s and 80’s radio systems migratedfrom analog to digital in almost every aspect, from system control to source and channelcoding to hardware technology. A new architecture known generally as “software radio”is emerging. This technology liberates radio-based services from dependence on hard-wired characteristics such as frequency band, channel bandwidth, and channel coding.

The software radio architecture centres on the use of wideband analog-to-digital anddigital-to-analog converters that are placed as close to the antenna as possible. Since thesignal is being digitized earlier in the system, as much radio functionality as possiblecan be defined and implemented in software. Thus the hardware is relatively simple andfunctions are software defined as illustrated in Figure 1.8.

Software definable channel modulations across the entire 25MHz cellular band have beendeveloped.

In an advanced application, a software radio does not just transmit; it characterizes theavailable transmission channels, probes the propagation path, constructs an appropriatechannel modulation, electronically steers its transmit beam in the right direction for sys-tems with antenna arrays, and selects the optimum power level. It does not just receive; itcharacterizes the energy distribution in the channel and in adjacent channels, recognizesthe mode of the incoming transmission, adaptively null interferers, estimates the dynamicproperties of multipath propagation, equalizes and decodes the channel codes. The mainadvantage of software radio is that it supports incremental service enhancements throughupgrades to its software. This whole area is not possible without the advancements inDSP technology.

1.5 Brief Overview of Chapters

An overview of the remaining chapters in this book are as follows:

Brief Overview of Chapters 17

WidebandADC & DAC

RFConversion

WidebandADC & DAC

RFConversion

Video

Mobile Unit

Real−time

ADCDAC

ProgrammableProcessor(s)

Speech

Data

Real−timeOnlineOffline

Processor(s)Programmable

ProcessorsProgrammable

Software

ServiceDevelopment

Public SwitchedNetwork

Base−station

Figure 1.8: Software radio architecture

• Digital signals and systems can either be described as sequences in time or in fre-quency. In Chapter 2, digital signals are viewed as sequences in time. Digital linearsystems are characterized in the time domain by a sequence called the impulse re-sponse. We shall discuss the properties of digital signals and systems and theirinteraction.

• Signals and systems can also be represented as functions of frequency. Just likethe continuous-time Fourier Transform connects the time-domain and frequency-domain view of continuous-time (analog) signals and systems, the discrete-timeFourier Transform (DTFT) is the tool for analyzing discrete-time signals and sys-tems in the frequency domain. Chapter 3 covers the foundamental concepts of thediscrete-time Fourier Transform.

• The DTFT is continuous in frequency. So it is not suitable for computation using acomputer. The discrete Fourier transform (DFT) is an approximation of the DTFTthat is suitable for numerical computation. The basic characteristics of the DFTand some ways by which the transform can be computed efficiently are describedin Chapter 4. The application of DFT to linear convolution and spectral analysis is

18 Introduction

also discussed.

• Chapter 5 discusses the process of converting a continuous-time signal to a discrete-time and discrete-amplitude one and vice versa. Concepts of sampling and quan-tization and their relation to aliasing are described. The process of reconstructingan analog signal from data sequences is also presented. Special emphasis is placedon the design tradeoffs between sampling rate, quantization resolution, and thecomplexity of analog filter for sampling and reconstruction.

• The processingof digital signals are most often performed by digital filters. Thedifferent ways by which these FIR and IIR digital filters can be realized by hard-ware or software will be discussed in Chapter 6. Even though these realizations arebased on a single equation, there are surprisingly many variations with differentproperties. Here the effects of using a finite wordlength to represent numbers andfilter coefficients are discussed.

• The design of the two major types of digital filters: finite impulse response (FIR)and infinite impulse response (IIR) filters are coverd in chapters 7and 8. Thereare many design techniques available. Here we only describe two methods foreach type of filters to illustrate the principles involved. Chapters 6 to 8 combinedshould gives the reader a good understanding of the digital filter approximation andimplementation issues.

Since this is an introductory course, a number of important but more advanced topics indigital signal processing are not covered. These topics include adaptive filtering, multi-rate processing, parametric signal modelling and spectral estimation, two (and higher) di-mensional digital signal processing, and other efficient fast Fourier transform algorithms.I have also restrict the materials to cover only deterministic signals. However, those in-terested in exploring these more advanced topics should find the foundation provided bythis book very useful.

Chapter 2

Discrete-time Signals and Systems

2.1 Discrete-time signals

Digital signals are discrete in both time (the independent variable) and amplitude (thedependent variable). Signals that are disrete in time but continuous in amplitude arereferred to asdiscrete-time signals.

Discrete-time signals are data sequences. A sequence of data is denotedx[n] or simplyx [n] when the meaning is clear. The elements of the sequence are calledsamples. Theindexn associated with each sample is an integer. If appropriate, the range ofn will bespecified. Quite often, we are interested in identifying the sample wheren = 0. This isdone by putting an arrow under that sample. For instance,

x[n] =. . . ,0.35,1,1.5,−0.6,− 2,. . .↑

The arrow is often omitted if it is clear from the context which sample isx[0]. Samplevalues can either be real or complex. In the rest of this book, the terms discrete-timesignals and sequences are used interchangeably.

The time interval between samples is not explicitly shown. It can be assumed to benormalized to 1 unit of time. So the corresponding normalized sampling frequency is 1Hz. If the actual sampling interval isT seconds, then the sampling frequency is given byfs = 1

T.

20 Discrete-time Signals and Systems

−3 −2 −1 0 1 2 3 40

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Index (n)

Am

plitu

de

Figure 2.1: The Unit Impulse Sequence

2.1.1 Some Elementary Sequences

There are some sequence that we shall encounter frequently. They are described here.

2.1.1.1 Unit Impulse Sequence

The unit impulse sequence is defined by

δ[n] =

1, n = 00, n 6= 0

(2.1)

This is depicted graphically in Figure 2.1. Note that while the continuous-time unit im-pulse function is a mathematical object that cannot be physically realized, the unit im-pulse sequence can easily be generated.

2.1.1.2 Unit Step Sequence

The unit step sequence is one that has an amplitude of zero for negative indices and anamplitude of one for non-negative indices. It is shown in Figure 2.2.

Discrete-time signals 21

−3 −2 −1 0 1 2 3 40

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Am

plitu

de

Index (n)

Figure 2.2: The Unit Step Sequence

2.1.1.3 Sinusoidal Sequences

A sinusoidal sequence has the form

x[n] = A cos (ω0n + φ) (2.2)

This function can also be decomposed into itsin-phasexi[q] andquadraturexq[n] com-ponents.

x[n] = A cos φ cos ω0n− A sin φ sin ω0n (2.3)

= xi[n] + xq[n] (2.4)

This is a common practice in communications signal processing.

2.1.1.4 Complex Exponential Sequences

Complex exponential sequences are essentially complex sinusoids.

x[n] = Aej(ωon+φ) (2.5)

= A cos(ω0n + φ) + jA sin(ω0n + φ) (2.6)


0 5 10 15 20 25 30−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

Index (n)

Am

plitu

de

Figure 2.3: Uniformly distributed random sequence with amplitudes between -0.5 and0.5.

2.1.1.5 Random Sequences

The sample values of a random sequence are randomly drawn from a certain probabilitydistribution. They are also called stochastic sequences. The two most common distribu-tions are the Gaussian (normal) distribution and the uniform distribution. The zero-meanGaussian distribution is often used to model noise. Figure 2.3 and figure 2.4 show exam-ples of uniformly distributed and Gaussian distributed random sequences respectively.

2.1.2 Types of Sequences

The discrete-time signals that we encounter can be classified in a number of ways. Somebasic classifications that are of interest to us are described below.

Real vs. Complex Signals

A sequence is considered complex at least one sample is complex-valued.


0 5 10 15 20 25 30−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

Index (n)

Am

plitu

de

Figure 2.4: Gaussian distributed random sequence with zero mean and unit variance.

Finite vs. Infinite Length Signals Finite length sequences are defined only for a rangeof indices, sayN1 toN2. The length of this finite length sequence is given by|N2 −N1 + 1|.

Causal Signals A sequencex[n] is a causal sequence ifx[n] = 0 for n < 0.

Symmetric Signals First consider a real-valued sequencex[n]. Even symmetryim-plies thatx[n] = x[−n] and forodd symmetryx[n] = −x[n] for all n.

Any real-valued sequence can be decomposed into odd and even parts so that

x[n] = xe[n] + xo[n]

where the even part is given by

xe[n] =1

2(x[n] + x[−n])

and the odd part is given by

xo[n] =1

2(x[n]− x[−n])


A complex-valued sequenceis conjugate symmetricif x[n] = x?[−n]. The sequence hasconjugate anti-symmetry ifx[n] = −x?[−n]. Analogous to real-valued sequences, anycomplex-valued sequence can be decomposed into its conjugate symmetric and conjugateanti-symmetric parts:

x[n] = xcs[n] + xca[n] (2.7)

xcs[n] =1

2(x[n] + x?[−n]) (2.8)

xca[n] =1

2(x[n]− x?[−n]) (2.9)

Periodic Signals A discrete-time sequence isperiodicwith a period ofN samples if

x[n] = x[n + kN ] (2.10)

for all integer values ofk. Note thatN has to be a positive integer. If a sequence is notperiodic, it isaperiodicor non-periodic.

We know that continuous-time sinusoids are periodic. For instance, the continuous-timesignal

x(t) = cos (ω0t) (2.11)

has a frequency ofω0 radians per second orf0 = ω0/2π Hz. The period of this sinusoidalsignal isT = 1/f0 seconds.

Now consider a discrete-time sequence based on a sinusoid.

x[n] = cos (ω0n) (2.12)

If this sequence is periodic with a period ofN samples, then

cos ω0 (n + N) = cos ω0n (2.13)

Soω0N must be an integer multiple of2π, or if r is any integer,

ω0N =2πr

⇒2πf0N=2πr

⇒f0 =r

N(2.14)

Since bothr andN are integers, a discrete-time sinusoidal sequence is periodic if itsfrequency is a rational number. Otherwise, it is non-periodic.


Example 2.1. Is x[n] = cos(

π8n)

periodic? If so, what is the period?

The sequence can be expressed as

x[n] = cos

(2π

(1

16

)n

)So in this case,f0 = 1/16 is a rational number and the sinusoidal sequence is periodicwith a periodN = 16.

It is interesting to note that for discrete-time sinusoidal sequences, a small change infrequency can lead to a large change in period. For example, a certain sequence hasfrequencyf1 = 31/60. So its period is 60 samples. Another sinusoidal sequence withfrequencyf2 = 30/60 has a period of only 2 samples sincef2 can be simplified to1/2.

Another point to note is that discrete-time sinusoidal sequences with frequencies sepa-rated by an integer multiple of2π areidentical.

Energy and Power Signals The energy of a finite length sequencex[n] is defined as

E =

N2∑n=N1

|x[n]|2 (2.15)

while that for an infinite sequence is

E =∞∑

n=−∞

|x[n]|2 (2.16)

Note that the energy of an infinite length sequence may not be infinite. A signal withfinite energy is usually referred to as anenergy signal.

Example 2.2.Find the energy of the infinite length sequence

x[n] =

2−n, n ≥ 00, n < 0

According to the definition, the energy is given by

E =∞∑

n=0

2−2n =∞∑

n=0

(1

4

)n


To evaluate the finite sum, first consider

SN =N−1∑n=0

an = 1 + a + a2 + · · ·+ aN−1

Multiplying this equation bya, we have

aSN = a + a2 + · · ·+ aN

and the difference between these two equations give

SN − aSN = (1− a) SN = 1− aN

Hence ifa 6= 1,

SN =1− aN

1− a(2.17)

For a = 1, it is obvious thatSN = N . For a < 1, the infinite sum is therefore

S∞ =1

1− a(2.18)

Making use of this equation, the energy of the signal is

E =1

1− 1/4=

4

3

Equations (2.17) and (2.18) are very useful and we shall be making use of them later.

The average power of a periodic sequence with a period ofN samples is defined as

Px =1

N

N−1∑n=0

|x[n]|2

and for non-periodic sequences, it is defined in terms of the following limit if it exists:

Px = limK→∞

1

2K + 1

K∑n=−K

|x[n]|2

A signal with finite average power is called apower signal.


Example 2.3.Find the average power of the unit step sequenceu[n].

The unit step sequence is non-periodic, therefore the average power is

P = limK→∞

1

2K + 1

∞∑n=0

u2[n]

= limK→∞

K + 1

2K + 1

=1

2

Therefore the unit step sequence is a power signal. Note that its energy is infinite and soit is not an energy signal.

Bounded Signals A sequence is bounded if every sample of the sequence has a mag-nitude which is less than or equal to a finite positive value. That is,

|x[n]| ≤ Bx < ∞

Summable Signals A sequence isabsolutely summableif the sum of the absolute valueof all its samples is finite.

∞∑n=−∞

|x[n]| < ∞

A sequence issquare summableif the sum of the magnitude squared of all its samples isfinite.

∞∑n=−∞

|x[n]|2 < ∞

2.1.3 Some Basic Operations on Sequences

Scaling Scaling is the multiplication of a scalar constant with each and every samplevalue of the sequence. This operation is depicted schematically in Figure 2.5.

Addition Addition of two sequences usually refers to point-by-point addition as shownin Figure 2.6.


x[n] Ax[n]A

Figure 2.5: Scalar Multiplication byA.

w[n]

y[n]=x[n]+w[n]x[n]

Figure 2.6: Point-by-point Addition

Delay A unit delay shifts a sequence by one unit of time as shown in Figure 2.7. Asequencex[n] is delayed byN samples to producey[n] if y[n] = x[n−N ].

Up/Down Sampling Down-sampling by a factor ofL (a positive integer) is an opera-tion by which only one everyL-th sample of the original sequence is kept, with the restdiscarded. The schematic for a down-sampler is shown in Figure 2.8. The down-sampledsignaly[n] is given by

y[n] = x[nM ]

Up-sampling is the opposite operation to down-sampling. It increases the number ofsamples of the original sequence by a certain factorL (a positive integer). This is done byinsertingL− 1 zeros between each pair of original samples. Figure shows the schematic

x[n] x[n−1]Z−1

Figure 2.7: A Unit Delay


x[n] y[n]L

Figure 2.8: Down-sampling by a factor ofL.

Lx[n] y[n]

Figure 2.9: Up-sample by a factor ofL.

diagram of an up-sampler and the up-sampled sequencey[n] is given by

y[n] =

x[

nL

], n = 0,±L,±2L, . . .

0, otherwise

An interpolated signal can be obtained by passing the up-sampled signal through a low-pass filter with an appropriate bandwidth. This process will be discussed in more detailin a later chapter.

Modulation Given two sequencesx[n] andw[n], and

y[n] = x[n] · w[n] (2.19)

Then we say thaty[n] is w[n] modulated byx[n]. This is analogous to carrier modulationin communication systems.

Correlation The correlation, or more precisely cross-correlation, between two finitelength data sequencesx[n] andw[n] is defined by

r =1

N

N∑n=0

x[n]w[n] (2.20)

if each sequence is of lengthN . The correlation coefficientr is often used as a measureof how similar the two sequences are. If they are very different, then the value ofr is low.

The matched filter used in digital communication receivers for optimal detection is alsoeffectively a correlator between the incoming and the template signals.


2.2 Discrete-time Systems

A discrete-time system is one that processes a discrete-time input sequence to produce adiscrete-time output sequence. There are many different kinds of such systems.

2.2.1 Classification of Systems

Discrete-time systems, like continuous-time systems, can be classified in a variety ofways.

Linearity A linear system is one which obeys the superposition principle. For a cer-tain system, let the outputs corresponding to inputsx1[n] andx2[n] arey1[n] andy2[n]respectively. Now if the input is given by

x[n] = Ax1[n] + Bx2[n] (2.21)

whereA and B are arbitrary constants, then the system is linear if its correspondingoutput is

y[n] = Ay1[n] + By2[n]

Superposition is a very nice property which makes analysis much simpler. Althoughmany real systems are not entirely linear throughout its operating region (for instance, thebipolar transistor), they can be considered approximately linear for certain input ranges.Linearization is a very useful approach to analyzing nonlinear systems. Almost all thediscrete-time systems considered in this book are linear systems.

Example 2.4.Are the down-sampler and up-sampler linear systems?

Consider the down-samplery[n] = x[nM ]

For inputx1[n], the corresponding output isy1[n] = x1[nM ]. For inputx2[n], the outputis y2[n] = x2[nM ]. Let x[n] be a linear combination of these two input with arbitraryconstantsA andB so that

x[n] = Ax1[n] + Bx2[n]

The output is given by

y[n] = Ax1[nM ] + Bx2[nM ]

= Ay1[n] + By2[n]

Discrete-time Systems 31

Therefore the down-sampler is a linear system.

Now consider the up-sampler

y[n] =

x[n/L], n = 0,±L,±2L, . . .0, otherwise

Lety1[n] andy2[n] be the outputs for inputsx1[n] andx2[n] respectively. For

x[n] = Ax1[n] + Bx2[n]

then the output is

y[n] =

Ax1[n/L] + Bx2[n/L], n = 0,±L,±2L, . . .0, otherwise

= Ay1[n] + By2[n]

Hence the up-sampler is also linear.

Shift Invariance A shift (or time) invariant system is one that does not change withtime. Let a system response to an inputx[n] bey[n]. If the input is now shifted byn0 (aninteger) samples,

x1[n] = x[n− n0] (2.22)

then the system is shift invariant if its response tox1[n] is

y1[n] = y[n− n0] (2.23)

In the remainder of this book, we shall use the terms linear time-invariant (LTI) and linearshift-invariant interchangeably.

Example 2.5.A system has input-output relationship given by

y[n] =n∑

k=−∞

x[k]

Is this system shift-invariant?


If the input is nowx1[n] = x[n− n0], then the corresponding output is

y1[n] =n∑

k=−∞

x1[k]

=n∑

k=−∞

x[k − n0]

=

n−n0∑k=−∞

x[k]

= y[n− n0]

Therefore the system is shift invariant.

Example 2.6. Is the down-sampler a shift invariant system?

LetM (a positive integer) be the down-sampling ratio. So for an inputx[n] the output isy[n] = x[nM ]. Now ifx1[n] is x[n] delayed byn0 samples, then

x1[n] = x[n− n0]

and the corresponding output is

y1[n] = x[nM − n0]

= x[(

n− n0

M

)M]

If the system is shift invariant, one would expect the output to be

y[n− n0] = x [(n− n0) M ]

Since this is not the case, the down-sampler must be shift variant.

Causality The response of a causal system at any time depends only on the input at thecurrent and past instants, not on any “future” samples. In other words, the output sampley[n0] for anyn0 only depends onx[n] for n ≤ n0.

Example 2.7.Determine if the following system is causal:

y[n] =∞∑

k=−∞

(n− k) u[n− k]x[k]


Note thatu[n − k] = 0 for n < k because the unit step sequence is zero for negativeindices. In other words, for a certainn, u[n − k] = 0 for k > n. So the output can bewritten as

y[n] =n∑

k=−∞

(n− k) x[k]

Soy[n] depends onx[k] for k ≤ n and therefore the system is causal.

Stability There are two common criteria for system stability. They are exponentialstability and bounded-input bounded-output (BIBO) stability. The first criterion is morestringent. It requires the response of the system to decay exponentially fast for a finiteduration input. The second one merely requires that the output be a bounded sequence ifthe input is a bounded sequence.

Example 2.8. Determine if the system with the following input-output relationship isBIBO stable.

y[n] =∞∑

k=−∞

(n− k) u[n− k]x[k]

Consider inputx[n] = δ[n]. Then

y[n] =∞∑

k=−∞

(n− k) u[n− k]δ[k]

= nu[n]

which is unbounded as it grows linearly withn. Therefore the system is not BIBO stable.

Example 2.9. Determine if the following system is BIBO stable. Note that this system isan “averager”, taking the average of the pastM samples.

y[n] =1

M

M−1∑k=0

x[n− k]


Let the input|x[n]| ≤ B for some finite valueB. Consider the magnitude of the output

|y[n]| =

∣∣∣∣∣ 1

M

M−1∑k=0

x[n− k]

∣∣∣∣∣≤ 1

M

M−1∑k=0

|x[n− k]|

≤ 1

M(MB)

= B

Hence the output is bounded and the system is BIBO stable.

Lossy or Lossless For a passive system that does not generate any energy internally,the output should have at most the same energy as the input. So

∞∑n=−∞

|y[n]|2 ≤∞∑

n=−∞

|x[n]|2 < ∞ (2.24)

A lossless system is one which the equality holds.

2.2.2 Linear Shift-Invariant Systems

An discrete-time LTI system, like its continuous-time counterpart, is completely charac-terized by its impulse response. In other words, the impulse response tells us everythingwe need to know about an LTI system as far as signal processing is concerned. Theimpulse response is simply the observed system output when the input is an impulsesequence. For continuous-time systems, the impulse function is purely a mathematicalentity. However, for discrete-time systems, since we are dealing with sequences of num-bers, the impulse sequence can realistically (and easily) be generated.

2.2.2.1 Linear Convolution

Let us consider a discrete-time LTI system with impulse reponseh[n] as shown in Figure2.10. What would be the outputy[n] of the system if the inputx[n] is as shown in Figure2.11?


0 1 2 3 4 5 6 7 80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Index (n)

h[n]

Figure 2.10: Impulse Response of the System Considered in Section 2.2.2.1

0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5

3

Index (n)

Inpu

t Sig

nal x

[n]

Figure 2.11: Input Signal Sequence


Since the system is linear and time invariant, we can make use of the superposition prin-ciple to compute the output. The input sequence is composed of three impulses. Mathe-matically, it can be expressed as

x[n] = δ[n] + 0.5δ[n− 1] + 2δ[n− 2]

Let

x1[n] = δ[n]

x2[n] = 0.5δ[n− 1]

x3[n] = 2δ[n− 2]

and the system response to each of these inputs are respectivelyy1[n], y2[n] andy3[n].The sample values ofy1[n] are given by

y1[0] = h[0]x1[0]=0.8

y1[1] = h[1]x1[0]=0.4

y1[2] = h[2]x1[0]=0.2

y1[3] = h[3]x1[0]=0.1

which is the same as the impulse response sincex1[n] is a unit impulse. Similarly,

y2[1] = h[0]x2[1]=0.4

y2[2] = h[1]x2[1]=0.2

y2[3] = h[2]x2[1]=0.1

y2[4] = h[3]x2[1]=0.05

and

y3[2] = h[0]x3[2]=1.6

y3[3] = h[1]x3[2]=0.8

y3[4] = h[2]x3[2]=0.4

y3[5] = h[3]x3[2]=0.2

The system outputy[n] in response to inputx[n] is therefore, through the superpositionprinciple, given by

y[n] = y1[n] + y2[n] + y3[n]

= 0.8, 0.8, 2, 1, 0.45, 0.2


Note that

y[0] = h[0]x[0]

y[1] = h[1]x[0] + h[0]x[1]

y[2] = h[2]x[0] + h[1]x[1] + h[0]x[2]

y[3] = h[3]x[0] + h[2]x[1] + h[1]x[2]

y[4] = h[3]x[1] + h[2]x[2]

y[5] = h[3]x[2]

In general, we have

y[n] = h[n]x[0] + h[n− 1]x[1] + . . . + h[1]x[n− 1] + h[0]x[n] (2.25)

or

y[n] =n∑

k=0

h[k]x[n− k] (2.26)

Alternatively,

y[n] =n∑

k=0

x[n]h[n− k] (2.27)

Equations 2.26 and 2.27 are the linear convolution equations for finite length sequences.If the length ofh[n] isM and the length ofx[n] isN , then the length ofy[n] isN +M−1.

We can further generalize it for infinite length sequences:

y[n] =∞∑

k=−∞

h[k]x[n− k] (2.28)

=∞∑

k=−∞

x[k]h[n− k] (2.29)

These equations are analogous to the linear convolution equation for continuous-timesignals. Note that the linear convolution equation comes about as a result of the superpo-sition principles and therefore applies only to LTI systems.

The convolution equation for discrete-time signals is also called theconvolution sum. Itis denoted by?. So equations 2.28 and 2.29 can be written as

y[n] = x[n] ? h[n]

The convolution sum is one of the most important fundamental equations in DSP.


2.2.2.2 Properties of Linear Convolution

The convolution sum has three important properties:

1. Commutativex[n] ? y[n] = y[n] ? x[n] (2.30)

2. Associative(x[n] ? w[n]) ? y[n] = x[n] ? (w[n] ? y[n]) (2.31)

3. Distributivex[n] ? (w[n] + y[n]) = x[n] ? w[n] + x[n] ? y[n] (2.32)

2.2.2.3 Condition for Stability

Since the impulse response completely characterize an LTI system, we should be able todraw conclusions regarding the stability of a system based on its impulse response. Weshall consider BIBO stability here.

Theorem 2.1. An discrete-time LTI system is BIBO stable if its impulse response is ab-solutely summable.

Proof:

Let the input be bounded, i.e.|x[n]| < B < ∞ for some finite valueB. The magnitudeof the output is given by

|y[n]| =

∣∣∣∣∣∞∑

k=−∞

h[k]x[n− k]

∣∣∣∣∣≤

∞∑k=−∞

|h[k]| |x[n− k|

≤ B

∞∑k=−∞

|h[k]|

So the magnitude ofy[n] is bounded if∑∞

k=−∞ |h[k]| is finite. In other words, the impulseresponse must be absolutely summable.


2.2.2.4 Condition for Causality

Theorem 2.2. A discrete-time LTI system is causal if and only if its impulse response isa causal sequence.

Proof:

Consider an LTI system with impulse responseh[k]. Two different inputsx1[n] andx2[n]are the same up to a certain point in time, that isx1[n] = x2[n] for n ≤ n0 for somen0.The outputsy1[n] andy2[n] atn = n0 are given by

y1[n0] =∞∑

k=−∞

h[k]x1[n0 − k]

=−1∑

k=−∞

h[k]x1[n0 − k] +∞∑

k=0

h[k]x1[n0 − k]

and

y2[n0] =∞∑

k=−∞

h[k]x2[n0 − k]

=−1∑

k=−∞

h[k]x2[n0 − k] +∞∑

k=0

h[k]x2[n0 − k]

Sincex1[n] = x2[n] for n ≤ n0, if the system is causal, then the outputsy1[n] andy2[n]must be the same forn ≤ n0. More specifically,y1[n0] = y2[n0]. Now,

∞∑k=0

h[k]x1[n0 − k] =∞∑

k=0

h[k]x2[n0 − k]

becausex1[n0 − k] = x2[n0 − k] for non-negative values ofk. Sincex1[n] may not beequal tox2[n] for n > n0, we must have

−1∑k=−∞

h[k]x1[n0 − k] =−1∑

k=−∞

h[k]x2[n0 − k] = 0

which means thath[k] = 0 for k < 0.


2.2.3 FIR and IIR Systems

In DSP, finite impulse response (FIR) and infinite impulse response (IIR) systems andusually referred to as FIR and IIRfilters.

An causal FIR filter has impulse response

h[n] = h0, h1, h2, · · · , hM , 0, 0, · · ·

whereh0, h1, h2, · · · , hM are called filter coefficients. The length of the impulse responseis M + 1 and it is called anM -th order filter.

Example 2.10.What is the impulse response of a causal LTI system with the followinginput-output relationship?

y[n] = 2x[n] + x[n− 1]− x[n− 3]

Impulse response is the system’s output when the input is a unit impulse signal. Therefore,

h[n] = 2δ[n] + δ[n− 1]− δ[n− 3]

= 2, 1, 0,−1

This filter is an FIR filter.

The filtering equation is essentially the linear convolution equation. Since the impulseresponse has a length ofM +1, the output can be computed as (assuming a causal filter):

y[n] =M∑

k=0

h[k]x[n− k] (2.33)

The impulse response of an IIR system, however, has infinite length. So the filteringequation for a causal filter is given by

y[n] =∞∑

k=0

h[k]x[n− k] (2.34)

This equation is not computable in practice due to the infinite limit. But there is a typeof IIR systems that admits recursive computation ofy[n] and so it can be computed. This


type of IIR systems are the only IIR filters that are used in DSP. The general input-outputrelationship of these IIR filters is

y[n] =N∑

k=1

a[k]y[n− k] +M∑

k=0

b[k]x[n− k] (2.35)

wherea[k] andb[k] are the filter coefficients. The present outputy[n] is a linearcombination of previousN outputs and the present and previousM outputs. Theorderof the filter isM − 1 or N , whichever is larger.

Example 2.11.A causal LTI system has impulse response

h[n] = ah[n− 1] + δ[n]

wherea is a constant. What is the input-output equation for this system?

Sincey[n] = h[n] for x[n] = δ[n], it is obvious from the impulse response that theinput-output equation for this system is

y[n] = ay[n− 1] + x[n]

Example 2.12.Find the impulse response of the causal system with the following input-output equation:

y[n] = 0.25y[n− 2] + x[n]

The impulse response of this system is given by

h[n] = 0.25h[n− 2] + δ[n]

and so

h[0] = 0.25h[−2] + δ[0]=1

h[1] = 0.25h[−1] + δ[1]=0

h[2] = 0.25h[0] + δ[2] =(0.25)2

h[3] = 0.25h[1] + δ[3] =0

......

Hence

h[n] =

(0.5)n, n ≥ 0, and even0, otherwise


2.3 Summary

In this chapter, we have studied the fundamentals of discrete-time signals and systemsin the time domain. Discrete-time signals are essentially a sequence of numbers. Somefundamental sequences, such as the unit impulse, unit step and sinusoidal (both real andcomplex) sequences are examined because more complex sequences can be expressed asa combination of some or all of these fundamental ones.

The most important discrete-time systems are the linear time invariant (LTI) systems. Forthese systems, the superposition principle applies which leads to the linear convolutionsum. This convolution sum is the way by which we derive the output of the system givenan input. An LTI system is completely characterized by its impulse response. All theproperties of such a system is revealed by its impulse response, including causality andBIBO stability.

Chapter 3

Discrete-time Fourier Analysis

One of the most powerful mathematical tool in signal processing is Fourier analysis.Periodic continuous-time signals can be represented as a Fourier series. For non-periodicsignals, the Fourier Transform can be used to provide us with a frequency-domain viewof the signals. The Fourier transform has some very nice properties which makes it veryuseful in theory and in practice.

Since in DSP we are dealing with discrete-time signals, there is a need to have a similarset of tools for the frequency analysis of these signals. The Discrete-time Fourier Series(DTFS) and Discrete-time Fourier Transform (DTFT) are such tools. Before we studythese techniques in detail, we shall briefly review the continuous-time Fourier series andtransforms.

3.1 Continuous-time Fourier Analysis

If x(t) is a continuous-time periodic signal, then it can be expressed as a linear combina-tion of weighted exponential functions

x(t) =∞∑

k=−∞

c[k]ej2πkf0t (3.1)

44 Discrete-time Fourier Analysis

where the fundamental period ofx(t)is T0 = 1/f0. The Fourier coefficientsc[k] can becomputed by

c[k] =1

T0

∫ T0

0

x(t)e−j2πkf0tdt (3.2)

The coefficientsc[k] indicate the strength of each frequency component present in thesignal. Sincex(t) is periodic, the only frequency components present are the harmonicsof the fundamental frequency.

If x(t) is non-periodic, it can be expressed as

x(t) =

∫ ∞

−∞X(f)ej2πftdf (3.3)

where

X(f) =

∫ ∞

−∞x(t)e−j2πftdt (3.4)

(3.4) is usually referred to as the forward transform and (3.3) as the inverse transform.X(f) is the spectrum of the signalx(t). Note that the spectrum of a non-periodic signalis continuous in frequency.

3.2 Discrete-time Fourier Series

Supposex[n] is a periodic sequence with a period ofN samples. It can be expressed as alinear combination of complex exponential functions:

x[n] =N−1∑k=0

c[k]ej2πkn/N n = 0, 1, . . . , N − 1 (3.5)

The coefficientsc[k] are called discrete-time Fourier coefficients. Note that the complexexponential functionsej2πkn/N have a fundamental period ofN .

In order to obtain an equation forc[k], we start with multiplying both sides of (3.5) bye−j2πln/N wherel is an integer. So we have

x[n]e−j2πln/N =N−1∑k=0

c[k]ej2π(k−l)n/N (3.6)

Discrete-time Fourier Series 45

Then we sum both sides with respect ton as follows:

N−1∑n=0

x[n]e−j2πln/N =N−1∑n=0

N−1∑k=0

c[k]ej2π(k−l)n/N (3.7)

SinceN−1∑n=0

an =

N, a = 11−aN

1−a, a 6= 1

(3.8)

we can conclude thatN−1∑n=0

ej2π(k−l)n/N =

N, (k − l) = 0,±N,±2N, . . .0, otherwise

(3.9)

Therefore the right hand side of (3.7) becomes

N−1∑k=0

c[k]N−1∑n=0

ej2π(k−l)n/N = Nc[l] (3.10)

Put this back into (3.7) and re-arranging, we have

c[l] =1

N

N−1∑n=0

x[n]e−j2πln/N l = 0, 1, . . . , N − 1 (3.11)

Note also that these discrete-time Fourier coefficients are also periodic with a period ofN .

Example 3.1. Determine the discrete-time Fourier coefficients of the following periodicsequence with a period of 4 samples:

x[n] = . . . , 0, 1, 1, 0, . . .

The coefficients are calculated using (3.2) withN = 4.

c[k] =1

4

3∑n=0

x[n]e−j2πkn/4

=1

4

x[0] + x[1]e−jπk/2 + x[2]e−jπk + x[3]e−j3πk/2

=

1

4

x[1]e−jπk/2 + x[2]ejπk


So

c[0] = 0.5

c[1] = 0.25 (−1− j)

c[2] = 0

c[3] = 0.25 (−1 + j)

3.2.1 Average Power

The average power of a periodic sequencex[n] is defined as

P =1

N

N−1∑n=0

|x[n]|2 (3.12)

whereN is the period ofx[n].

We want to express the average power in terms ofc[k] instead ofx[n].

P =1

N

N−1∑n=0

|x[n]|2

=1

N

N−1∑n=0

x[n]x∗[n]

=1

N

N−1∑n=0

x[n]

N−1∑k=0

c∗[k]e−j2πkn/N

Interchanging the order of summation, we have

P =N−1∑k=0

c∗[k]

1

N

N−1∑n=0

x[n]e−j2πkn/N

=N−1∑k=0

c∗[k]c[k]

=N−1∑k=0

|c[k]|2 (3.13)

Discrete-time Fourier Series 47

Now we have established that the average power can be calculated using either the signalsequence or its discrete-time Fourier coefficients.

The relationship

1

N

N−1∑n=0

|x[n]|2 =N−1∑k=0

|c[k]|2 (3.14)

is called the Parseval’s relation.

Example 3.2.Verify the Parseval’s relation for the periodic sequence in Example 3.1.

Solution:

Using the sequence sample values, we have

P =1

4

3∑n=0

|x[n]|2

=1

4(1 + 1)

=1

2

Now using the Fourier coefficients, we have

P =3∑

k=0

|c[k]|2

= (0.5)2 + |0.25 (−1− j)|2 + |0.25 (−1 + j)|2

= 0.25 + 0.125 + 0.125

= 0.5

Thus the Parseval’s relation is verified.

3.2.2 Normalized Frequency

Each Fourier coefficient is associated with a particular frequency. From (3.5), we can seethatc[k] corresponds to the complex sinusoid

ej2πkn/N = cos (2πkn/N) + jsin (2πkn/N) (3.15)

= cos (ωkn) + jsin (ωkn) (3.16)


with an angular frequency

ωk =2πk

N(3.17)

Fork = 0, 1, . . . , N − 1,0 ≤ ωk < 2π

So the Fourier series expansion only gives us frequency components within a2π range.ωk is the normalized (angular) frequency because we implicity assume that the time spac-ing T between samples of the data sequence is1 second.

If the actual value ofT is known, then we can obtain the de-normalized frequencies

Ωk =2πk

NT(3.18)

and the range given by

0 ≤ Ωk <2π

T(3.19)

3.2.3 Power Density Spectrum

Since each Fourier coefficientc[k] is associated with a particular frequency,|c[k]|2 isproportional to the power of that particular frequency component within the discrete-time signal. A plot of|c[k]|2 againstωk shows the distribution of power as a functionof frequency. The sequence|c[k]|2 , k = 0, 1, . . . , N − 1 is called the power densityspectrum of the signal.

The power density spectrum provide us with a frequency-domain view, as opposed to thetime-domain view, of the signal.

Example 3.3. Plot the power density spectrum of the periodic signal given in Example3.1.

The power density spectrum is given by

|c[k]|2 =

0.25, 0.125, 0, 0.125

The plot is shown in Figure 3.1.

Discrete-time Fourier Transform 49

−1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 40

0.05

0.1

0.15

0.2

0.25

k

Mag

nitu

de S

quar

ed F

ourie

r Coe

ffici

ents

Power Density Spectrum

Figure 3.1: Power Density Spectrum in Example 3.3.

3.3 Discrete-time Fourier Transform

Using the discrete-time Fourier series, we can analyze periodic discrete-time signals inthe frequency domain. In order to deal with the more general non-periodic signals, weneed to use the discrete-time Fourier Transform (DTFT). It is defined as

X (ω) =∞∑

n=−∞

x[n]e−jωn (3.20)

Notice that the DTFT is continuous in the variableω, while x[n] is discrete-time. Alsonote that in order to compute the DTFT, we need to know the whole data sequence (n =−∞ to∞).


3.3.1 Convergence

The DTFT is a sum of an infinite sequence. As we know, not all infinite sums converge.Therefore we would like to establish the conditions under which the DTFT exists.

The DTFT is said to exist if the finite sum

XK (ω) =K∑

n=−K

x[n]e−jωn (3.21)

converges toX (ω) in some sense asK approaches infinity.

3.3.1.1 Uniform Convergence

XK (ω) as defined in (3.21) is said to convergeuniformly if

limK→∞

|X (ω)−XK (ω)| = 0 (3.22)

Notice that in this case we require that the absolute difference betweenX (ω) andXK (ω)becomes smaller and smaller asK increases at every single frequencyω.

Consider

|X (ω)| =

∣∣∣∣∣∞∑

n=−∞

x[n]e−jωn

∣∣∣∣∣ (3.23)

≤∞∑

n=−∞

|x[n]| (3.24)

So the magnitude ofX (ω) is finite if x[n] is absolutely summable.

3.3.1.2 Mean Square Convergence

If DTFT is only applicable to absolutely summable sequences, then its use will be verylimited. Most of the data sequences in practice are not absolutely summable, but theyare square summable. By relaxing the convergence criteria, we can extend the DTFT toinclude square summable sequences.


The mean square convergence criterion is defined as

limK→∞

∫ π

−π

|X (ω)−XK (ω)|2 dω = 0 (3.25)

Notice that this criterion requires that thesum of the differences squaredbecome smallerasK increases. In other words, the energy of the error decreases when more terms areincluded. However, it does not mean that for any particular frequency, the difference isreduced. This fact is illustrated through the following example.

Example 3.4.Consider the following discrete time signal

x[n] =

sin ωcn

πn, n 6= 0

ωc

π, n = 0

The DTFT ofx[n] is given by

X (ω) =∞∑

n=−∞

x[n]e−jωn

=

1, |ω| ≤ ωc

0, ωc < |ω| ≤ π

x[n] is square summable since

∞∑n=−∞

|x[n]|2 =1

2π

∫ π

−π

|X (ω)|2 dω

=1

2π

∫ ωc

−ωc

dω

=ωc

π

However, it is not absolutely summable. So its DTFT exists only in the mean square sense.

In Figure 3.2 we plottedXK (ω) for several different values ofK to illustrate mean squareconvergence. Here we useωc = 0.5π. The DTFT ofx[n] is an ideal low-pass filter. AsKincreases, it can be observed that the overall error becomes smaller due to smaller errorsaway from the jump discontinuity atωc. However, the peak of the ripple in the passbandremains very much the same height regardless ofK. This effect is due to mean squareconvergence at a jump discontinuity.


0 0.5 1 1.5 2 2.5 3 3.50

0.2

0.4

0.6

0.8

1

1.2

1.4

Normalized frequency

Mag

nitu

de

K=51

K=21 K=11

Figure 3.2: Finite term approximations to the DTFT.

At the end of the 19-th century, Albert Abraham Michelson conducted an experiment tomeasure the speed of light using an interferometer. This experiment became known as theMichelson-Morley experiment and Michelson subsequently became the first Americanto win the Nobel prize. He also built a device that performs Fourier analysis on light.He observed that where there is a jump discontinuity in the signal, some sort ofringingoccurs. He could not explain the phenomenon and consulted Josiah Willard Gibbs. Gibbseventually came up with an explanation and published it in 1899 in Nature. So thisbecame known as theGibb’s phenomenon.

3.3.2 Energy Density Spectrum

The energy density spectrum of a non-periodic discrete-time signal is given by|X (ω)|2.It tells us the energy distribution of the signal in frequency.


The magnitude and phase spectra are given by|X (ω)| and∠X (ω) respectively.

The energy of a finite-energy sequence can be calculated using the data sequence itselfor from the energy density spectrum.

E =∞∑

n=−∞

|x[n]|2 (3.26)

=1

2π

∫ π

−π

|X (ω)|2 dω (3.27)

This is the Parseval’s relation for DTFT.

3.3.3 Properties

3.3.3.1 Periodicity

The DTFT is periodic with a period of2π. This can easily be verified as follows. Letkbe any integer,

X (ω + 2πk) =∞∑

n=−∞

x[n]e−j(ω+2πk)n

=∞∑

n=−∞

x[n]e−jωne−j2πkn

=∞∑

n=−∞

x[n]e−j(ω)n

= X (ω) (3.28)

This means that the DTFT of a discrete-time sequence is only unique within a (normal-ized) frequency range of2π. The DTFT is repeated once every2π. Figure 3.3 shows theperiodic repetition of the magnitude spectrum of discrete-time sequences.

3.3.3.2 Symmetry

If x[n] is a real-valued discrete-time signal, then its DTFT has conjugate symmetry, i.e.

X (ω) = X∗ (−ω) (3.29)


|X(w)|Replica of |X(w)| Replica of |X(w)|

ω−3π π 3π−π−2π 2π

........

Figure 3.3: Periodic nature of the Fourier spectrum of discrete-time sequences.

orX∗ (ω) = X (−ω) (3.30)

Furthermore, ifx[n] is real-valued and even, then its DTFT is also real-valued and even.If x[n] is real-valued and odd, then its spectrum is imaginary-valued and odd.

3.3.3.3 Linearity

DTFT is a linear transform since it obeys the superposition principle. For any scalarconstantsa andb, and data sequencesx[n] andy[n] where

x[n] ↔ X (ω) (3.31)

y[n] ↔ Y (ω) (3.32)

are DTFT pairs,ax[n] + by[n] ↔ aX (ω) + bY (ω) (3.33)

3.3.3.4 Time Shifting

If the data sequencex[n] is shifted by an integer index ofn0, the effect in the frequencydomain is a shift in phase by an amount−ωn0 at frequencyω.

x[n− n0] ↔ e−jωn0X (ω) (3.34)


3.3.3.5 Time Reversal

Reversing the time sequence has the effect of reversing the DTFT.

x[−n] ↔ X (−ω) (3.35)

3.3.3.6 Frequency Shifting

Analogous to time-shifting, if the DTFTX (ω) is shifted in frequency by a constantω0,then the effect in the time domain is a multiplication by a complex sinusoid.

ejω0nx[n] ↔ X (ω − ω0) (3.36)

A particular case of frequency shifting is when the time sequence is modulated by asinsoidal signal.

x[n]cos (ω0n) ↔ 1

2X (ω + ω0) + X (ω − ω0) (3.37)

This is commonly encountered in communication systems. The effect of modulation isa shift of the original DTFT up and down the frequency axis by an amount equal to themodulating frequency.

3.3.3.7 Linear Convolution

One of the most useful property of DTFT is converting a linear convolution in the timedomain into multiplication in the frequency domain.

Let

x1[n] ↔ X1 (ω) (3.38)

x2[n] ↔ X2 (ω) (3.39)

Consider the linear convolution ofx1[n] andx2[n]:

y[n] = x1[n] ? x2[n]

=∞∑

k=−∞

x1[k]x2[n− k] (3.40)


Multiplying both sides bye−jωn and sum over alln, we have∞∑

n=−∞

y[n]e−jωn =∞∑

n=−∞

(∞∑

k=−∞

x1[k]x2[n− k]

)e−jωn (3.41)

The left hand side of the above equation is the DTFT ofy[n] which we shall denote byY (ω). Therefore,

Y (ω) =∞∑

k=−∞

x1[k]∞∑

n=−∞

x2[n− k]e−jωn

=∞∑

k=−∞

x1[k]∞∑

m=−∞

x2[m]e−jω(m+k)

=∞∑

k=−∞

x1[k]e−jωk

∞∑m=−∞

x2[m]e−jωm

= X1 (ω) X2 (ω) (3.42)

3.3.4 Inverse DTFT

The time domain sequence can be recovered given its DTFT. Comparing (3.43) with(3.1), we realize that (3.43) can be viewed as a (continuous) Fourier series representationof X (ω). So the inverse DTFT should take the form of (3.2).

The inverse DTFT is given by

x[n] =1

2π

∫ π

−π

X (ω) ejωndω (3.43)

It can be shown that this is indeed the inverse transform by substitution of the aboveexpression into (3.20).

3.3.5 Autocorrelation Sequence

In many applications, the autocorrelation sequence of a discrete-time signalx[n] has tobe computed. It is defined as

rxx[l] =∞∑

n=−∞

x[n]x[n− l] l = 0,±1,±2, . . . (3.44)

Summary 57

where the parameterl is called thelag. Note thatrxx[0] is the energy of the signal.

We can express the autocorrelation with a lag ofl as a convolution sum.

rxx[l] =∞∑

n=−∞

x [n] x [− (l − n)]

= x[n] ∗ x[−n]

If x[n] is a real-valued sequence and the DTFT ofx[n] is X (ω), then the DTFT ofx[−n]is X (−ω). So

DTFT x[n] ∗ x[−n] = X (ω) X (−ω)

= |X (ω)|2

Hence we can compute the energy density spectrum of real-valued signals by taking theDTFT of the autocorrelation sequence of the signal. This is particularly useful for randomsignals.

3.4 Summary

Signals and systems can be viewed from the frequency domain perspective. We showedthat a periodic time sequence can be expressed as a sum of a finite number of sinuoids(the Fourier series) similar to that of continuous-time signals. Non-periodic discrete-time signals can also be expressed as the sum of an infinite number of sinusoids (thediscrete-time Fourier Transform). The main difference between continuous-time FourierTransforms (CTFT) and DTFT is that the DTFT is periodic. Properties of the DTFTare otherwise similar to those for CTFT. Two of the more important properties are linearconvolution and the Parseval’s theorem.

Chapter 4

Discrete Fourier Transform and ItsComputation

A discrete-time sequence can be converted to its frequency domain representation bythe Discrete-time Fourier Transform. The frequency variableω is a continuous variable.While it is not difficult to compute the DTFT at a particular frequency, it is impossibleto compute the whole DTFT numerically using a computer. One way to approximate theDTFT is to sample it at regular frequency intervals similar to the way we sample timedomain signals. We call the sampled version of the DTFT the Discrete Fourier Transform(DFT).

In this chapter, we are concerned about the DFT and its implementation and applicationsto computing the linear convolution and to spectral analysis.

4.1 Discrete Fourier Transform

4.1.1 Sampling the DTFT

Suppose we takeN evenly spaced frequency samples from the DTFTX (ω) of a discrete-time sequencex [n]. The sampled values are given by

X [k] = X (ω)|ω=2πk/N

=∞∑

n=−∞

x [n] e−j2πkn/N (4.1)

60 Discrete Fourier Transform and Its Computation

x[1+

2N]

x[0+

2N]

x[1+

N]

x[N

−1]

x[N

−1+N

]

x[0+

N]

x[0]

x[1]

x[N

−1+2

N]

N

m=1 m=3m=2

Samples with same exponential factorare added up

Figure 4.1: Illustration of Equation 4.3 Using 3 Data Blocks.

for 0 ≤ k ≤ N − 1. We can divide the summation in (4.1) into blocks ofN terms:

X [k] =∞∑

m=−∞

(m+1)N−1∑n=−mN

x [n] e−j2πkn/N (4.2)

=N−1∑n=0

(∞∑

m=−∞

x [n−mN ]

)e−j2πkn/N (4.3)

(4.3) is obtained from (4.2) by interchanging the inner and outer summations with appro-priate change of variables. The term in bracket in (4.3) is the sum of allx [n] terms withthe same exponential factor. This equation is illustrated in Figure with 3 blocks.

Supposex [n] is of finite length and the length ofx [n] is N , then

∞∑m=−∞

x [n−mN ] = x [n] (4.4)

sincex [n] = 0 for n < 0 andn ≥ N . So (4.3) becomes

X [k] =N−1∑n=0

x [n] e−j2πkn/N (4.5)

Discrete Fourier Transform 61

for k = 0, 1, . . . , N − 1. This is called theN -point Discrete Fourier Transform ofx [n].X [k] are known as the DFT coefficients.

Note that the DFT transforms a length-N data sequence into a length-N sequence of DFTcoefficients. In other words, if we need more sample points in the frequency domain, weneed more data points in the time domain.

4.1.2 Inverse DFT

Note that in (4.3), the term∑∞

m=−∞ x [n−mN ] (denoted asxp [n]) is periodic with aperiod ofN . Therefore it can be expressed as a discreter Fourier series, i.e.

xp [n] =N−1∑k=0

c [k] ej2πkn/N

wherec [k] are the Fourier series coefficients. They are given by

c [k] =1

N

N−1∑n=0

xp [n] e−j2πkn/N

=1

NX [k]

using (3.11) in Section 3.2 and (4.5).

If x [n] has a finite length ofN samples, then

x [n] = xp [n] 0 ≤ n ≤ N − 1

and we have

x [n] =1

N

N−1∑k=0

X [k] ej2πkn/N (4.6)

for 0 ≤ n ≤ N . (4.6) allows us to recover the time sequencex [n] from the DFTcoefficientsX [k]. It is therefore called the inverse Discrete Fourier Transform (IDFT).

The IDFT can be obtained through a forward DFT. Taking the complex conjugate of (4.6)we have

x? [n] =1

N

N−1∑k=0

X? [k] e−j2πkn/N (4.7)

=

(1

N

)DFT (X? [k]) (4.8)


So by computing the DFT of the complex conjugate of the DFT coefficients, we canobtain the complex conjugate ofx [n] and hencex [n] itself.

4.1.3 Matrix Form

The factore−j2π/N is central to the DFT equations. It is usually denoted as

WN = e−j2π/N (4.9)

and so the DFT equation (4.5) becomes

X [k] =N−1∑n=0

x [n] W knN (4.10)

for 0 ≤ k ≤ N − 1. The inverse DFT equation (4.6) then becomes

x [n] =1

N

N−1∑k=0

X [k] W−knN (4.11)

for 0 ≤ n ≤ N − 1.

(4.10) can be expressed in matrix form:

X = WNx (4.12)

Or X [0]X [1]X [2]

...X [N − 1]

=

1 1 1 · · · 11 W 1

N W 2N · · · WN−1

N

1 W 2N W 4

N · · · W2(N−1)N

......

..... .

...1 WN−1

N W2(N−1)N · · · W

(N−1)(N−1)N

x [0]x [1]x [2]

...x [N − 1]

(4.13)

Notice that some of the entries in theWN matrix are the same. For instance, it can easilybe shown that

Wk+N/2N = −W k

N (4.14)

Similarly for (4.11),x = W−1

N X (4.15)


where it can be shown that

W−1N =

1

NW?

N (4.16)

Example 4.1.Derive theW4 matrix and hence compute the 4-point DFT of

x [n] =

1, 2, 3, 4

Then derive theW−14 matrix and recoverx [n] from its DFT coefficients.

Solution:

W4 =

W 0

4 W 04 W 0

4 W 04

W 04 W 1

4 W 24 W 3

4

W 04 W 2

4 W 04 W 2

4

W 04 W 3

4 W 64 W 9

4

=

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

W4 =

W0

4 W04 W 0

4 W 04

W 04 W 1

4 W 24 W 3

4

W 04 W 2

4 W 04 W 2

4

W 04 W 3

4 W 64 W 9

4

=

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

SinceW k+N

N = W kN andW

k+N/2N = −W k

N ,

W 64 = W 2

4

W 94 = W 1

4

W 24 = −W 0

4

W 34 = −W 1

4

and

X =

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

1234

=

10

−2 + 2j−2

−2− 2j


NowW−1N = 1

NW?

N and so

W−14 =

1

4

1 1 1 11 j −1 −j1 −1 1 −11 −j −1 j

To recoverx [n],

x =1

4

1 1 1 11 j −1 −j1 −1 1 −11 −j −1 j

10−2 + 2j−2

−2− 2j

=

1234

4.1.4 Zero-padding

If we have a length-L data sequencex [n] and want to perform anN -point DFT whereL < N , we can appendx [n] with D = N − L zeros to make it of lengthN .

x [n] = x0, x1, · · · , xL−1 (4.17)

xD [n] = x0, x1, · · · , xL−1, 0, . . . , 0 (4.18)

Note that by zero-padding a signal, we do not alter its DTFT, i.e.XD (ω) = X (ω):

XD (ω) =N−1∑n=0

xD [n] e−jωn (4.19)

=L−1∑n=0

xD [n] e−jωn +L+D−1∑

n=L

xD [n] e−jωn (4.20)

=L−1∑n=0

x [n] e−jωn (4.21)

= X (ω) (4.22)

sincex [n] = xD [n] for n = 0, 1, . . . , L− 1 andxD [n] = 0 for n ≥ L.

However, the DFT coefficientsXD [k] andX [k] are not the same.


4.1.5 Circular Shift

Consider a lengthN data sequencex [n] which is zero outside the range0 ≤ n ≤ N − 1. This sequence is then time shifted byn0 samples so it becomesx [n− n0] for somenon-zero values ofn0. What would be theN -point DFT ofx [n− n0]?

Let’s use an example.x [n] = 1, 2, 3, 4 (4.23)

andn0 = 2. So the time-shift sequence is

xs [n] = 0, 0, 1, 2, 3, 4 (4.24)

and we want a 4-point DFT ofxs [n]. From (4.3), the DFT ofxs [n] is the same as theDFT of length-4 sequence

xc [n] = 3, 4, 1, 2 (4.25)

Note thatxc [n] is a circularly shifted version ofx [n]:

xc [n] = x [〈n− n0〉N ] (4.26)

where〈n〉N denotesn moduloN . In other words,

xc [n] =

x [n− n0] , n0 ≤ n ≤ N − 1x [N − n0 + n] , 0 ≤ n < n0

(4.27)

So a time-shift is equivalent to a circular shift as far as the DFT is concerned.

4.1.6 Properties

In this section, some of the important properties of the DFT are summaried.

Periodicity ConsiderX [k + N ] which is given by

X [k + N ] =N−1∑n=0

x [n] e−j2πkn/Ne−j2πNn/N (4.28)

=N−1∑n=0

x [n] e−j2πkn/N (4.29)

= X [k] (4.30)


This implies thatX [k] is periodic with a period ofN , even thoughx [n] is aperiodic. Ingeneral,

X [k] = X [k + pN ] p = ±1,±2, . . . (4.31)

Thus for a discrete-time signal, we cannot obtain the full spectrum (frequencies fromnegative infinity to infinity).

Linearity The DFT is a linear transformation. Hence the superpostion principle holds.If both x1 [n] andx2 [n] are of the same length and

x1 [n] ↔ X1 [k]

x2 [n] ↔ X2 [k]

thenx [n] = ax1 [n] + bx2 [n] ↔ X [k] = aX1 [k] + bX2 [k]

wherea andb are arbitrary constants.

Parseval’s Relation The Parseval’s relation holds for DFT, similar to the continuous-time Fourier Transform and the DTFT. So the energy of the signal can be calculated fromthe time sequence or from the DFT coefficients.

N−1∑n=0

|x(n)|2 =1

N

N−1∑k=0

|X(k)|2

Time Shifting We noted in Section 4.1.5 that time delaying a sequence is equivalentto circularly shift a sequence as far as the DFT is concerned. The DFT of the circularlyshifted sequence is given by

x [〈n−m〉N ] ↔ W kmN X [k] (4.32)

whereX [k] are the DFT coefficients of the original non-shifted sequence. In otherwords, a time (circular) shift in the time domain leads to a linear phase shift of an amounte−j2πkm/N in the frequency domain.

Frequency Shifting A circular shift in the DFT coefficients will have similar effecttime domain:

W−mnN x [n] ↔ X [〈k −m〉N ] (4.33)


Symmetry If x [n] is real-valued, then its DFT has the following symmetry properties:

<X [k] = <X [−k] (4.34)

=X [k] = −=X [−k] (4.35)

Furthermore,

• if x [n] is real and even, thenX [k] is also real and even; and

• if x [n] is real and odd, thenX [k] is imaginary and odd.

If x [n] is purely imaginary, then

<X [k] = −<X [−k] (4.36)

=X [k] = =X [−k] (4.37)

Also,

• if x [n] is imaginary and even, thenX [k] is also imaginary and even; and

• if x [n] is imaginary and odd, thenX [k] is real and odd.

The proof is straight-forward and is left as an exercise to the reader.

4.1.7 Circular Convolution

Let X1 [k] andX2 [k] beN -point DFTs of sequencesx1 [n] andx2 [n] respectively so that

X1 [k] =N−1∑n=0

x1 [n] e−j2πkn/N (4.38)

X2 [k] =N−1∑n=0

x2 [n] e−j2πkn/N (4.39)


Now considerY [k] = X1 [k] X2 [k] for k = 0, 1, . . . , N − 1. Inverse DFT gives

y [m] =1

N

N−1∑k=0

X1 [k] X2 [k] ej2πkm/N (4.40)

=1

N

N−1∑k=0

(N−1∑n=0

x1 [n] e−j2πkn/N

)(N−1∑l=0

x2 [l] e−j2πkl/N

)ej2πkm/N (4.41)

=1

N

N−1∑n=0

x1 [n]N−1∑l=0

x2 [l]

(N−1∑k=0

ej2πk(m−n−l)/N

)(4.42)

NowN−1∑k=0

ej2πk(m−n−l)/N =

N, a = 11−aN

1−a, a 6= 1

(4.43)

where

a = ej2π(m−n−l)/N (4.44)

=

N, l = m− n + pN = 〈m− n〉N0, otherwise

(4.45)

So

y [n] =N−1∑n=0

x1 [n] x2 [〈m− n〉N ] (4.46)

The circular convolutiony [n] between two length-N sequencesx1 [n] andx2 [n] is de-fined by (4.46). AnN -point circular convolution is denoted by

y [n] = x1 [n] ~ x2 [n] (4.47)

Therefore a circular convolution can be computed by multiplying the DFTs of the twodata sequences followed by an inverse DFT of the product.

Example 4.2. Compute the circular convolution of the following length-4 sequences us-ing DFT.

x [n] = 1, 2, 1, 2h [n] = 1, 2, 3, 4

Solution:


Circular convolution using DFT:

X [k] = DFT x [n] = 6, 0,−2, 0H [k] = DFT h [n] = 10,−2 + j2,−2,−2− j2

andY [k] = H [k] X [k] = 60, 0, 4, 0

So the circular convolution ofx [n] andh [n] gives

y [n] = IDFT Y [k] = 16, 14, 16, 14

The circular convolution operation in (4.46) can also be expressed in matrix form.y [0]y [1]

...y [N − 1]

=

x2 [0] x2 [N − 1] · · · x2 [1]x2 [1] x2 [0] · · · x2 [2]

......

......

x2 [N − 1] x2 [N − 2] · · · x2 [0]

x1 [0]x1 [1]

...x1 [N − 1]

(4.48)

Example 4.3.Check the answer obtained in Example 4.2 by using the definition (4.46).

Solution:

Using the matrix form,

y [n] =

1 4 3 22 1 4 33 2 1 44 3 2 1

1212

=

1 + 8 + 3 + 42 + 2 + 4 + 63 + 4 + 1 + 84 + 6 + 2 + 2

=

16141614

which is the same as the answer obtained in Example 4.2.

4.1.8 Linear Convolution Using DFT

Circular convolution does not have any real meaning, unlike linear convolution. There-fore we are interested in the relationship between linear and circular convolution. If wecan obtain the linear convolution through circular convolution, then we can make use ofDFT to perform linear convolution calculations.

Suppose two sequencesx [n] andh [n] have lengthsN andM respectively. The linearconvolution of these two sequences will yield a result which is a sequencey [n] of length


(N + M − 1). So if we zero-padx [n] andh [n] to length(N + M − 1) and perform a(N + M − 1)-point circular convolution, the result should be the same as that obtainedby linear convolution.

Example 4.4.Compute the linear convolution of the data sequences

x [n] = 1, 2, 0, 1h [n] = 2, 2, 1, 1

Solution:

The linear convolution result should be of length(4 + 4− 1) = 7. Therefore 3 zeros areneed for each sequence.

xD [n] = 1, 2, 0, 1, 0, 0, 0hD [n] = 2, 2, 1, 1, 0, 0, 0

So the circular convolution ofxD [n] andhD [n] is given by

y [n] =

2 0 0 0 1 1 22 2 0 0 0 1 11 2 2 0 0 0 11 1 2 2 0 0 00 1 1 2 2 0 00 0 1 1 2 2 00 0 0 1 1 2 2

1201000

=

22 + 41 + 4

1 + 2 + 22 + 2

11

=

2655411

It can be verified that this is the result of linear convolution.

Overlap-Add Method In many cases we have a relatively short sequenceh [n] (e.g. animpulse response sequence) and we need to compute the outputy [n] (e.g. the output ofa linear system) given a very long sequencex [n] (e.g. the input to the linear system).If we used the method above, the output will only be obtained when the whole input iscollected and the convolution sum computed. Ifx [n] is indeed very long, the delay willbe long as well. Ifx [n] is infinitely long, then the method cannot be applied.

An alternative way is to compute the linear convolution through circular convolution is todivide the input into finite length blocks. The idea is to obtain a block of output samplesat a time.


Given a long sequencex [n], divide it into non-overlapping sub-sequencesxr [n] of lengthL so that

x [n] =∞∑

r=0

xr [n− rL] (4.49)

Let h [n] be a shorter sequence of lengthM . Append(L− 1) zeros toh [n] and(M − 1)zeros to each sub-sequencexr [n] so that they are both of lengthN = M + L − 1.Compute the linear convolution ofxr [n] andh [n] (using DFT if desired), yielding anoutputyr [n] of lengthN .

y [n] = x [n] ? h [n] (4.50)

=

(∞∑

r=0

xr [n− rL]

)? h [n] (4.51)

=∞∑

r=0

yr [n− rL] (4.52)

whereyr [n] = xr [n] ? h [n] (4.53)

Notice in (4.52) that each block ofyr [n] is of lengthN while the shift betweenyr [n]andyr−1 [n] is L samples. Therefore there is an overlap of(N − L) or (M − 1) samplesbetween subsequent blocks ofyr [n]. According to (4.52), these overlapped sample valuesshould be added to give the correcty [n]. This process is shown in Figure 4.2 for threeblocks of data.

Example 4.5. Compute the linear convolution of the following sequences using 4-pointDFTs.

h [n] = 3, 2, 1x [n] = 1, 2, 3, 4, 4, 3, 2, 1

Solution:

Length of the shorter sequence isM = 3. Use 4-point FFTs, thereforeN = 4. So

L = N −M + 1 = 4− 3 + 1 = 2

Hencex [n] is to be divided into blocks of 2 samples.


L L L L

x [n]0

y [n]0

M−1

M−1

M−1

1x [n] x [n]

2

1y [n]

y [n]2

x [n]mx[n]

M

h[n]

L

y[n]

+

+

Figure 4.2: Illustration of the Overlap-add Method.

We have

h [n] = 3, 2, 1, 0 ⇒ H [k] = 6, 2− 2j, 2, 2 + 2jx0 [n] = 1, 2, 0, 0 ⇒ X0 [k] = 3, 1− 2j,−1, 1 + 2jx1 [n] = 3, 4, 0, 0 ⇒ X1 [k] = 7, 3− 4j,−1, 3 + 4jx2 [n] = 4, 3, 0, 0 ⇒ X2 [k] = 7, 4− 3j, 1, 4 + 3jx3 [n] = 2, 1, 0, 0 ⇒ X3 [k] = 3, 2− j, 1, 2 + j

and so

Y0 [k] = H [k] X0 [k] = 18,−2− 6j,−2,−2 + 6jY1 [k] = H [k] X1 [k] = 42,−2− 14j,−2,−2 + 14jY2 [k] = H [k] X2 [k] = 42, 2− 14j, 2, 2 + 14jY3 [k] = H [k] X3 [k] = 18, 2− 6j, 2, 2 + 6j


Inverse FFTs give us

y0 [n] = 3, 8, 5, 2y1 [n] = 9, 18, 11, 4y2 [n] = 12, 17, 10, 3y3 [n] = 6, 7, 4, 1

Align them properly and add:

3 8 5 29 18 11 4

12 17 10 3+ 6 7 4 1

3 8 14 20 23 21 16 10 4 1

which gives usy [n] = 3, 8, 14, 20, 23, 21, 16, 10, 4, 1.

Overlap-Save Method There is an alternative way to the overlap-add method. Con-sider a length-3 sequenceh [n] and a length-4 sequencex [n]. the result of the linearconvolution ofh [n] andx [n], which is of length 6, is given by

yL [0] = h [0] x [0] (4.54)

yL [1] = h [0] x [1] + h [1] x [0] (4.55)

yL [2] = h [0] x [2] + h [1] x [1] + h [2] x [0] (4.56)

yL [3] = h [0] x [3] + h [1] x [2] + h [2] x [1] (4.57)

yL [4] = h [1] x [3] + h [2] x [2] (4.58)

yL [5] = h [2] x [3] (4.59)

Compare this with the circular convolution of these sequences (after appending a zero toh [n]):

yc [0] = h [0] x [0] + h [1] x [3] + h [2] x [2] (4.60)

yc [1] = h [0] x [1] + h [1] x [0] + h [2] x [3] (4.61)

yc [2] = h [0] x [2] + h [1] x [1] + h [2] x [0] (4.62)

yc [3] = h [0] x [3] + h [1] x [2] + h [2] x [1] (4.63)


Obviously,

yc [0] 6= yL [0] (4.64)

yc [1] 6= yL [1] (4.65)

However,

yc[2] = yL [2] (4.66)

yc [3] = yL [3] (4.67)

In general, if the length of the shorter sequence isM and we use anN point circular con-volution, then the lastL = (N −M + 1) samples are the same as the linear convolutionoutput. In other words, the first(M − 1) samples of the circular convolution output areuseless as far as the linear convolution is concerned. Therefore they can be discarded andonly the remaining output samples are kept. All these saved output samples can now beconcatenated to form the correct linear convolution output. This process is illustrated inFigure 4.3 for three data blocks.

This method is called Overlap-save.

Example 4.6.Repeat example 4.5 using the overlap-save method.

Solution:

AgainL = 2. But instead of appending zeros to each block, we have

x0 [n] = 0, 0, 1, 2 ⇒ X0 [k] = 3,−1 + 2j,−1,−1− 2jx1 [n] = 1, 2, 3, 4 ⇒ X1 [k] = 10,−2 + 2j,−2,−2− 2jx2 [n] = 3, 4, 4, 3 ⇒ X2 [k] = 14,−1− j, 0,−1 + jx3 [n] = 4, 3, 2, 1 ⇒ X3 [k] = 10, 2− 2j, 2, 2 + 2jx4 [n] = 2, 1, 0, 0 ⇒ X4 [k] = 3, 2− j, 1, 2 + j

EachXi [k] , i = 0, . . . , 4 is multiplied withH [k] to give

Y0 [k] = 18, 2 + 6j,−2, 2− 6j ⇒ y0 [n] = 5, 2, 3, 8Y1 [k] = 60, 8j,−4,−8j ⇒ y1 [n] = 14, 12, 14, 20

Y2 [k] = 84,−4, 0,−4 ⇒ y2 [n] = 19, 21, 23, 21Y3 [k] = 60,−8j, 4, 8j ⇒ y3 [n] = 16, 18, 16, 10

Y4 [k] = 18, 2− 6j, 2, 2 + 6j ⇒ y4 [n] = 6, 7, 4, 1Retaining only the lastL = 2 samples of eachyi [n] gives us the linear convolutionoutputy [n] = 3, 8, 14, 20, 23, 21, 16, 10, 4, 1 which is the same as that obtained usingoverlap-add.


LL L L

x [n]0 1x [n] x [n]

2

M−1

LM−1

y [n]0

1y [n]

M−1

y [n]2

M−1

mx[n]

M

h[n]

x [n]

y[n]

Discard

Figure 4.3: Illustration of Overlap-save Method.

4.1.9 Computational Complexity

Consider the DFT equation:

X [k] =N−1∑n=0

x [n] e−j2πkn/N (4.68)

To compute eachX [k], we need to performN complex multiplications (the multiplica-tion of two complex numbers) andN − 1 complex additions (the addition of two com-plex numbers). So the completeN -point DFT requiresN2 complex multiplications andN (N − 1) complex additions in general.

Note that the multiplication of two complex numbers

(a + jb) (c + jd) = (ac− bd) + j (bc + ad)


involves 4 real number multiplications and 2 real number additions. The addition of twocomplex numbers involves 2 real number additions. Therefore anN -point DFT involves4N2 real multiplications and2N (N − 1) + 2N2 real additions.

To compute anN point circular convolution using the equation

y [n] =N−1∑n=0

x1 [n] x2 [〈m− n〉N ] (4.69)

we requireN2 multiplications andN (N − 1) additions. If the sequencesx1 [n] andx2 [n]are real-valued, then these multiplications and additions involve only real numbers.

If the circular convolution is computed usingN -point DFTs, then it involves two DFTsto transformx1 [n] andx2 [n] to X1 [k] andX2 [k], and one inverse DFT to transformY [k] = X1 [k] X2 [k] to the resulty [n]. Since the IDFT equation is essentially the sameas the DFT equation, their computation complexities are the same. So the two DFTsand one IDFT together requires3N2 complex multiplications and3N (N − 1) complexadditions. To computeY [k], we need anotherN complex multiplications. So the totalnumber of complex multiplications is3N2 + N and the number of complex additions is3N (N − 1). Note that in general, the DFT coefficients are complex-valued even thoughthe data sequencesx1 [n] andx2 [n] are real-valued.

4.2 The Fast Fourier Transform

Based on the computational complexity analysis in Section 4.1.9, the DFT does notseem very useful for performing circular convolution (and hence linear convolution us-ing overlap-add and overlap-save). In the early 1960’s, Cooley and Tukey developed anefficient algorithm to computeN -point DFTs whereN = 2r for positive integersr. Itis called thefast Fourier transform(FFT). This algorithm has a significant impact on theuse of DFT in various computations and essentially launched the field of digital signalprocessing.

The Fast Fourier Transform 77

4.2.1 Radix-2 Decimation-in-time Algorithm

The basic idea of Cooley and Tukey’s FFT algorithm, which is now known as the Radix-2decimation-in-time algorithm, is to rewrite the DFT equation (4.5) into two parts:

X [k] =

N/2−1∑n=0

x [2n] W 2nkN + W k

N

N/2−1∑n=0

x [2n + 1] W 2nkN (4.70)

for k = 0, 1, . . . , N − 1 andN = 2r for some positive integerr.

The first part of (4.70) is the DFT of the even sequence and the second part, is the DFTof the odd sequence. Notice that the factorW 2nk appears in both DFT’s and need onlybe computed once. Also note that

W 2nkN = W nk

N/2 (4.71)

Let x1 [n] = x [2n] andx2 [n] = x [2n + 1], and

x1 [n] ↔ X1 [k]

x2 [n] ↔ X2 [k]

(4.70) can be written asX [k] = X1 [k] + W k

NX2 [k] (4.72)

Or

X [k] = X1 [k] + W kNX2 [k] (4.73)

X

[k +

N

2

]= X1 [k]−W k

NX2 [k] (4.74)

for k = 0, 1, . . . , N2− 1 sinceW

k+N/2N = −W k

N . The complex factorW kN is known as the

twiddle factor.

Each of the two subsequencesx1 [n] andx2 [n] can be further broken down into even andodd sequences. This process can continue until only 2-point DFT’s are left. So each N/2-point DFT is obtained by combining two N/4-point DFT’s, each of which is obtained bycombining two N/8-point DFT’s, etc. There are a total ofr stages sinceN = 2r. Figure4.4 shows the concept for an8-point FFT.

The algorithm is called decimation-in-time because we break up the original data (ortime) sequence into two reduced number of samples (decimation). Radix-2 refers to thefact that 2-point DFTs are the basic computational blocks of the algorithm.


x[7]

x[3]

x[5]

x[1]

x[6]

x[2]

x[4]x[0]

X[7]

X[6]

X[5]

X[4]

X[3]

X[2]

X[1]

X[0]2−pointDFT

2−pointDFT

2−pointDFT

2−pointDFT

DFTs

2−point

Combine

DFTs

2−point

Combine

DFTs

4−point

Combine

Stage 1 Stage 2 Stage 3

Figure 4.4: The three stages of an 8-point Radix-2 decimation-in-time FFT.

Butterfly Operation Computation of two-point DFT’s are trivial. It involves

X2p [0] = x2p [0] + W 02 x2p [1] = x2p [0] + x2p [1] (4.75)

X2p [1] = x2p [0] + W 12 x2p [1] = x2p [0]− x2p [1] (4.76)

Combining theN2

-point DFT coefficients to form anN -point DFT involves the compu-tation of (4.73) and (4.74). The basic operation is illustrated in Figure 4.5. It is called abutterfly operation and is a basic building block of the FFT structure.

Example 4.7.Derive the Radix-2 decimation-in-time FFT structure for a 4-point DFT.

Solution:


a−W bNk

a+W bNk

−1

+

+WN

k

b

a

Figure 4.5: Decimation-in-time FFT Butterfly Operation

The 4-point DFT is first decomposed:

X [k] =3∑

n=0

x [n] W nk4 k = 0, 1, 2, 3

=(x [0] W 0

4 + x [2] W 2k4

)+(x [1] W k

4 + x [3] W 3k4

)=

(x [0] W 0

4 + x [2] W 2k4

)︸︷︷︸2−pt DFT

+W k4

(x [1] W 0

4 + x [3] W 2k4

)︸︷︷︸2−pt DFT

= X1 [k] + W k4 X2 [k] k = 0, 1, 2, 3

More explicitly, the second stage of the FFT is governed by the equations:

X [k] = X1 [k] + W k4 X2 [k] , k = 0, 1

X [k + 2] = X1 [k]−W k4 X2 [k] , k = 0, 1

The first stage consists of two 2-point DFTs governed by the equations:

X1 [0] = x [0] + W 04 x[2]

X1 [1] = x [0] + W 24 x [2] = x [0]−W 0

4 x [2]

and

X2 [0] = x [1] + W 04 x[3]

X2 [1] = x [1] + W 24 x [3] = x [1]−W 0

4 x [3]


W40

W40

W40

W41

x[0]

x[2]

x[1]

x[3]

X[0]

X[1]

X[2]

X[3]

−1

−1

−1

−1

Figure 4.6: Signal Flow Graph of the Radix-2 Decimation-in-time 4-point FFT Algo-rithm.

Figure 4.7: Signal Flow Graph of the Radix-2 Decimation-in-time 8-point FFT Algo-rithm.

Signal Flow Graph The FFT structure is usually represented in a signal flow graph.The signal flow graph for the radix-2 DIT FFT algorithm in Example 4.7 is shown inFigure 4.6. In the graph, the arrows represent the direction of the signal flow, the nodesrepresent adders and the multiplication factors are written on the segment where it is tobe performed.

Figure 4.7 shows the signal flow graph of an 8-point radix-2 DIT FFT.

Bit Reversal Note that if we want the DFT coefficients to come out in the natural order,the input sequence has to be rearranged as shown in Figures 4.6 and 4.7. This reorderingis known asbit reversal. We can see why it is called bit reversal if we represent the indexsequence in binary form. This is illustrated by Table.

A simple algorithm for generating a list of bit reversed numbers is attributed to Bruneman.It goes as follows:

1. Start with0, 1. Multiply by 2 to get0, 2.


2. Add1 to the list of numbers obtained above. In this case, it is1, 3.

3. Append the list in step 2 to that in step 1 to get0, 2, 1, 3.

4. The list obtained in step 3 now becomes the starting list in step 1. The steps arerepeated until the desired length of the list is obtained.

It should be pointed out that more efficient algorithms have been developed for bit rever-sal. But we shall not cover them here.

Computational Complexity An N -point FFT consists ofN/2 butterflies per stagewith log2N stages. Each butterfly has one complex multiplication and two complex addi-tions. Thus there are a total of(N/2)log2N complex multiplications compared withN2

for DFT, andNlog2N complex additions compared withN(N − 1) for the DFT. Thisrepresents a substantial saving whenN is large.

4.2.2 Radix-2 Decimation-in-frequency Algorithm

Another radix-2 FFT algorithm can be derived by partitioning the data sequence into twohalves as follows:

X [k] =

N/2−1∑n=0

x [n] W knN +

N−1∑n=N/2

x [n] W knN (4.77)

instead of odd and even sequences. The second term on the right of (4.77) can be ex-pressed as

N−1∑n=N/2

x [n] W knN =

N/2−1∑n=0

x

[n +

N

2

]W

k(n+N/2)N (4.78)

=

N/2−1∑n=0

x

[n +

N

2

]W kn

N WkN/2N (4.79)

= (−1)k

N/2−1∑n=0

x

[n +

N

2

]W kn

N (4.80)


So (4.77) becomes

X [k] =

N/2−1∑n=0

x [n] W knN + (−1)k

N/2−1∑n=0

x

[n +

N

2

]W kn

N

=

N/2−1∑n=0

x [n] + (−1)k x

[n +

N

2

]W kn

N (4.81)

The FFT coefficient sequence can be broken up into even and odd sequences and theyhave the form

X [2k] =

N/2−1∑n=0

x [n] + x

[n +

N

2

]W 2kn

N

=

N/2−1∑n=0

x [n] + x

[n +

N

2

]W kn

N/2

=

N/2−1∑n=0

g1 [n] W knN/2 (4.82)

X [2k + 1] =

N/2−1∑n=0

x [n]− x

[n +

N

2

]W

(2k+1)nN

=

N/2−1∑n=0

x [n]− x

[n +

N

2

]W 2kn

N W nN

=

N/2−1∑n=0

W nN

x [n]− x

[n +

N

2

]W kn

N/2

=

N/2−1∑n=0

g2 [n] W knN/2 (4.83)

for k = 0, 1, . . . , N/2− 1 where

g1 [n] = x [n] + x

[n +

N

2

](4.84)

g2 [n] =

x [n]− x

[n +

N

2

]W n

N (4.85)


The data sequences involved in the even and odd coefficient sequencesX [k] andX [2k + 1]can again be divided into the first and second halves and the same procedure repeated un-til only 2-point DFT’s are required.

This algorithm is called decimation-in-frequency (DIF) FFT because it is the DFT coef-ficients that are divided into odd and even halves. Figure shows the three stages of an8-point radix-2 decimation-in-frequency FFT. Note that the data sequence appears in itsnatural order whereas the FFT output occurs in bit reversed order.

The computational complexity is the same as the decimation-in-time algorithm.

Butterfly Operation The computation of the g1(n) and g2(n) sequences involve thebutterfly operation as shown in Figure 4-5. It is similar to the butterfly for the decimation-in-time FFT except for the position of the twiddle factor.

W kN

W kN

a+b

(a−b)

a

b−1

Figure 4.8: Radix-2 Decimation-in-Frequency Butterfly

Example 4.8.Derive the 4-point radix-2 DIF FFT algorithm structure.

Signal Flow Graph The even and odd coefficient sequences can each be divided intothe first and second halves and the same procedure repeated until only 2-point DFT’sare required. Figure 4-6 shows the three stages of an 8-point radix-2 decimation-in-frequency FFT. Note that the data sequence appears in its natural order whereas the FFToutput occurs in bit reversed order.

The computational complexity is the same as the decimation-in-time algorithm.


4.2.3 Other Fast Algorithms

There are literally tens of different variations of the basic radix-2 FFT algorithms. Somefurther computational saving can be obtained by using radix-4 or mixed-radix (split-radix) variants. Other algorithms deal with cases where the number of data points is nota power of two, typically whenN is a prime number. One of the most well known in thiscategory is theWinograd Fourier transform(WFT). It does not make use of the butterflyand the idea is completely different from the Cooley and Tukey algorithm. It is very fastin the sense that it requires fewer multiplications and is particularly useful for short lengthtransforms. But both the mathematics involved and the implementation are considerablymore complex. Thus they are outside of the scope for this introductory course.

4.3 Spectral Analysis

The two main applications of DFT are in the computation of the output of a linear systemthrough the linear convolution and the analysis of the spectral content of a signal. In thissection we shall concentrate on the latter application.

Recall that the DFT coefficients are the sampled values of the DTFT of a given discrete-time signal.

X [k] = X (ω)|ω=2πk/N (4.86)

for k = 0, 1, . . . , N − 1. For a sampling frequencyfs, each indexk corresponds to theactual frequency

Ωk =k

Nfs (4.87)

4.3.1 Spectral Leakage

Let us consider two examples.

Example 4.9.Compute the DFT of a length-32 sinusoidal sequence of frequency 10 Hz,sampled at 64 Hz and plot the magnitude spectrum.

Solution:

Since it is a single sinusoid, we expect the DFT coefficients to have non-zero values onlyat

k =ΩN

fs

Spectral Analysis 85

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time (s)

Am

plitu

de

10 Hz sinusoid, sampled at 64 Hz

(a) 10 Hz sinusoid sampled at 64 Hz

0 5 10 15 20 25 300

2

4

6

8

10

12

14

16

18

20

DFT index k

Mag

nitu

de o

f DFT

coe

ffici

ents

(b) Magnitude spectrum of the sampled 10 Hz sinusoid with N=64

Figure 4.9: Signal and Spectrum in Example 4.9.


according to (4.87). Now,N = 32, fs = 64 andΩ = 10. So this givesk = 5.

The sampled signal and its magnitude spectrum is shown in Figure 4.9. There is anothernon-zero value atk = 32 − 5 = 27 due to the symmetry of the DFT of a real-valuedsignal.

Example 4.10.Compute the DFT of a length-32 sinusoidal sequence of frequency 11 Hz,sampled at 64 Hz and plot the magnitude spectrum.

Solution:

Again, use

k =ΩN

fs

we getk = 11/2 = 5.5. The peak is expected to be in between two DFT coefficients. Themagnitude spectrum is plotted in Figure 4.10.

We may expect the magnitude spectrum in example 4.10 to consist of only two majorcomponents similar to that in example 4.9. However, this is not the case. The energycontained in a single frequency is now spread to many frequencies, with the total energyremaining the same (by Parseval’s theorem). This effect is called energy leakage.

Examining the data sequences reveals why leakage occurs. The signal in example 4.9,shown in Figure 4.9, begins and terminates at the beginning and end of a cycle respec-tively. However, the signal in example 4.10, shown in Figure 4.10, ends mid-way througha cycle. Therefore the signal that is being transformed has an abrupt jump to zero. Thisis because the assumptions we made in obtaining the DFT equation is that the signal iszero outside theN samples (see Section 4.1.1). This jump in amplitude contributes to theextra frequency components in the spectrum as shown.

In order to avoid leakage, we need to chooseN , the size of the DFT, to be contain aninteger multiple of cycles of the sinusoid. While we can do that for a single sinusoid,when we have a signal that contains many sinusoidal components, leakage becomes un-avoidable.

4.3.2 Rectangular Windowing

Another way of looking at a finite length sinusoidal signalg [n] is that it is a produceof an infinite length sinusoidx [n] of the same frequency multiplied by a finite length


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

111 Hz sinusoid, sampled at 64 Hz

Am

plitu

de

time (s)

(a) 11 Hz sinusoid sampled at 64 Hz

0 5 10 15 20 25 300

2

4

6

8

10

12

DFT index k

Mag

nitu

de o

f DFT

coe

ffici

ents

(b) Magnitude Spectrum of sampled sinusoid with N=64

Figure 4.10: Signal and Spectrum in Example 4.10.


rectangular windoww [n]. Here

w [n] =

1, N1 ≤ n ≤ N2

0, otherwise(4.88)

For convenience,N1 is usually assumed to be zero andN2 = N − 1 for a window oflengthN .

The DTFT ofw [n]is

W (ω) =N−1∑n=0

w [n] e−jωn

=1− e−jωN

1− e−jω

=e−jωN/2

(ejωN/2 − e−jωN/2

)e−jω/2 (ejω/2 − e−jω/2)

=sin (ωN/2)

sin (ω/2)· e−jω(N−1)/2 (4.89)

So the magnitude spectrum is given by

|W (ω)| =∣∣∣∣sin (ωN/2)

sin (ω/2)

∣∣∣∣ (4.90)

Sincesin θ ≈ θ for very smallθ,

|W (ω)| = ωN/2

ω/2= N (4.91)

for ω close to zero. This is shown in Figure 4.11. It consists of one main lobe and manysidelobes. The spectrum is zero when the numerator in (4.90) is zero, i.e.

sin (ωN/2) = 0 (4.92)

This occurs when

ω =2πk

N

for k = ±1,±2, . . . .


The main lobe is of heightN in agreement with (4.91). The base width of the mainlobe is4π/N . The width of the main lobe may also be defined by the 3 dB width where|W (ω)| drops by half. In this case,

∆ω =2πfs

N(4.93)

or

∆f =fs

N(4.94)

when it is denormalized by the sampling frequencyfs.

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

2

4

6

8

10

12

Normalized frequency (Hz)

Mag

nitu

de o

f DTF

T

Magnitude Spectrum of Rectangular Window of Length 10

Main lobe

Sidelobes

Figure 4.11: Magnitude Spectrum of a Rectangular Window of length 10.

4.3.3 Frequency Resolution

Suppose we wish to analyze the spectrum of a signal that is composed of the sum of twosinusoids. What is the minimum data lengthN required so that we can see the peaks ofthese sinusoids distinctly?


Two distinct peaks in a DFT spectrum will merge with one another if the main lobes cor-responding to the two sinusoids overlap substantially. This happens when the differencein frequency of the sinusoidal components is less than the main lobe width. Using the3-dB main lobe width definition, the width is given by (4.94).

If the desired frequency resolution is∆fr which may be the frequency separation ofthe sinusoidal components, then the minimum number of samples has to ensure that∆fr ≥ ∆f where∆f is the main lobe width. So the number of data samples required isgiven by

N ≥ fs

∆fr

(4.95)

In other words, the higher the resolution, the smaller the value of∆fr, and so the moredata samples we need.

Example 4.11.A signal consists of four sinusoidal frequencies: 1, 1.5, 2.5 and 2.75 kHz.It is sampled at 10 kHz. How many samples are needed for the spectrum to exhibit fourpeaks?

Solution:

If a rectangular window is used, the main lobe width is given by

∆f = 2.75− 2.5 = 0.25kHz

since this is the smallest frequency separation between the four frequency components.Let∆fr = ∆f , then

N ≥ fs

∆f=

10

0.25= 40

So we need at least 40 samples of data.

Thus the main lobe width of the DTFT of the windowing function determines the achie-veable frequency resolution. The sidelobes determine the amount of frequency leakage.Obviously we want to increase the main lobe width and the amplitudes of the sidelobes.For the rectangular window, the amplitude of the first (and thus highest) sidelobe com-pared to the main lobe is given by∣∣∣∣W (3π/N)

W (0)

∣∣∣∣ ≈ 2

3π(4.96)

or in terms of dB,

R = 20 log10

∣∣∣∣W (3π/N)

W (0)

∣∣∣∣ ≈ −13.46dB (4.97)


To obtain better performance, one way is to design a window to give us the desiredparameters.

Hamming Window The Hamming window of lengthL is defined as

w [n] =

0.54− 0.46 cos

(2πnL−1

), 0 ≤ n ≤ L− 1

0, otherwise(4.98)

A 51 point Hamming window is shown in Figure 4.12.

0 5 10 15 20 25 30 35 40 45 500

0.2

0.4

0.6

0.8

1

Index (k)

Am

plitu

de

51−point Hamming Window

Figure 4.12: A Hamming Window of Length 51.

The effective width of the main lobe of the Hamming window is given by

∆fw =2fs

L(4.99)

This translates to a main lobe width of8π/L. So for the same data length, the Hammingwindow can provide only half the frequency resolution compared to a rectangular win-dow. The advantage of the Hamming window is in the reduction of spectral leakage. Theratio of the first side lobe peak amplitude to the main lobe isR = 40dB.


There exists other windows such as the Hanning window, Blackman window and theKaiser window. They mostly possess the raised-cosine shape as the Hamming window.In general, they give better values ofR but worse frequency resolution compared to therectangular window.

4.3.4 Computational Frequency Resolution

An estimation of the spectrum of a discrete-time signalx [n] can be obtained through asegment of data. The data segment is usually obtained by windowingx [n] to obtaing [n].The estimate is the DTFT ofg [n].

In practice, DFT is used which gives us a sampled version of the DTFT. Since FFT ismuch more computationally efficient, it is generally used to compute the DFT. However,the length of the data segment being analyzed may not be of the convenient length forFFT. This problem can be overcome by appendingg [n] with the necessary number ofzeros.

Suppose the length ofg [n] is L and it is appended with zeros to produce a sequencegD [n] of lengthN whereL < N . Performing anL-point DFT ong [n] with give usLDFT coefficients. Performing anN -point FFT ongD [n] will give usN FFT coefficients.SinceN > L,does that mean we can increase the frequency resolution by appendingzeros? The answer is a definite no.

As discussed in Section 4.3.3, the frequency resolution is determined by the data length ofg [n]. This resolution will not change by appending zeros tog [n]. The apparent increasein the number of frequency points provided by the FFT ofgD [n] is actually are-samplingof theL frequency samples. No extra information can be obtained through re-sampling.

This situation is similar to that of a digital camera. The maximum resolution of the imagethat can be acquired is determined by the resolution of the image sensor on the camera.Any “increase” in resolution by interpolation is purely artificial. No extra details of theimage could be obtained through pure interpolation alone.

Example 4.12.A signal is composed of two sinusoids of frequencies 1 Hz and 2 Hz. Itis sampled at a rate of 5 Hz. How many samples are required to resolve two frequenciesusing DFT?

Solution:

If a rectangular window is used,

L ≥ fs

∆f=

5

(2− 1)= 5


samples. However, if a Hamming window is used,

L ≥ 2fs

∆fw

=2 (5)

(2− 1)= 10

samples.

4.3.5 Non-stationary Signals

So far we have only considered signals that arestationary, i.e. their parameters (suchas frequency components) do not change over time. However, many signals encounteredin real life do change over time. Examples include speech and music signals. They aregenerally known as non-stationary signals. If we want to know the spectral content ofthis kind of signals, we cannot simply analyze one segment of the signal.

Consider a chirp which has the form

x [n] = A cos(ω0n

2)

(4.100)

This signal is shown in Figure 4.13. It is obvious that the instantaneous frequency (thefrequency of the sinusoid at a particular time) changes with time. So analyzing a longsegment of the chirp signal will show the averaged frequency over that segment which isa misleading result.

One way to tackle the spectral analysis of non-stationary signals is to assume that withina reasonably short time, the signal remains stationary. So we divide the signal into suf-ficiently small segments by windowing. The spectra obtained from the spectral analysisof each segment are then displayed side by side to show the changes. To show the slowlyvarying spectrum with better time resolution, these segments may overlap one another.The resulting display is called a spectrogram. Figure 4.14 shows a spectrogram of thechirp signal using 256-point FFTs, Hamming window, and 250-point overlap. In thisfigure, the darker the colour, the higher the magnitude. It clearly shows that the instanta-neous frequency of the signal increases linearly with time.

The length of the windowed segment determines the frequency resolution while theamount of overlap between subsequent segments determines the time resolution. Tocompute a spectrogram with good frequency and time resolution will require a lot ofcomputation (FFTs). It is common to either relax the frequency resolution or the timeresolution requirements in order to lessen the computational requirements. Spectrogramswith good frequency resolution are called narrowband spectrograms while those withgood time resolution are called wideband spectrograms.


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5−1.5

−1

−0.5

0

0.5

1

1.51kHz Sampling rate from DC for 0.5s

Time (s)

Am

plitu

de

Figure 4.13: A Linear Chirp Signal.

4.4 Summary

In this chapter we discussed ways to compute the spectrum of a discrete-time signal.The Discrete Fourier Transform is introduced as a sampled version of the discrete-timeFourier transform of a discrete-time signal. It can be more efficiently computed throughfast algorithms called fast Fourier Transforms. The two main applications of DFT/FFT isin performing linear convolution (to obtain the output of discrete-time linear systems) andin the spectral analysis of signals. For the first application, the overlap-add and overlap-save methods could be employed in cases where the input signal could be infinitely long.For spectral analysis, we discussed the relationship between the frequency resolution andthe data length required. The analysis of non-stationary signals is also briefly discussed.

Summary 95

Time

Freq

uenc

y

Spectrogram of linear chirp

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

50

100

150

200

250

300

350

400

450

500

Figure 4.14: Spectrograms of the Chirp Signal.

Chapter 5

Sampling, Reconstruction andQuantization

Most of the signals encountered in engineering applications are analog. In order to pro-cess analog signals using digital techniques they must first be converted into digital sig-nals. Digital processing of analog signals proceeds in three stages:

1. The analog signal is digitized. Digitization involves two processes:sampling(dig-itization in time) andquantization(digitization in amplitude). This whole processis called analog-to-digital (A/D) conversion.

2. The digitized signal is processed by the appropriate DSP algorithms.

3. The results or outputs of the processing are converted back into analog signalsthrough interpolation. This process is called digital-to-analog (D/A) conversion.

Figure 5.1 illustrates the main functional blocks involved in the process.

5.1 The Sampling Process

The analog signal is sampled once everyT seconds, resulting in a sampled data sequence.The sampler is generally assumed to beideal in that the value of the signal at an instant(an infinitely small time) is taken. A real sampler, of course, cannot achieve that and

98 Sampling, Reconstruction and Quantization

AnalogInput

AnalogOutput

Analog−to−DigitalConversion

filterlow−pass

Sampleand

HoldQuantizer DSP to Analog

Converter

Digitallow−pass

filter

Anti−aliasingPre−filter

Anti−imagePost−filter

Analog Reconstruction

Figure 5.1: Digital Processing of Analog Signals.

Sampled

Ideal Sampler

AnalogSignal

x(t)

Discrete−timeSignal

x(nT)T

Figure 5.2: Schematic Diagram of Ideal Sampling

the “switch” in the sampler (see Figure 5.2) is actually closed for a finite, though verysmall, amount of time. This is analogous to a camera with a finite shutter speed. Even ifa camera can be built with an infinitely fast shutter, the amount of light that can reach thefilm plane will be very small indeed. However, for simplicity in analysis the samplingprocess is usually considered to be close enough to the ideal.

It should be pointed out that throughout our discussions we shall assume that the samplingperiodT is constant. In other words, the spacing between the samples isregular. Thisis calleduniform sampling. Although irregularly sampled signals can, under suitableconditions, be converted to uniformly sampled ones, the concept and mathematics arebeyond the scope of this introductory course.

The Sampling Process 99

5.1.1 Spectra of Sampled Signals

The most important parameter in the sampling process is thesampling periodT , or thesampling frequencyor sampling ratefs which is defined as

fs =1

T(5.1)

Sampling frequency is in units of “samples per second” or “Hertz”. The sampled se-quencex [n] is given by

x [n] = x (nT ) n = . . . ,−1, 0, 1, . . . (5.2)

wherex (t) is the analog signal. Alternatively, we can view the sampled signal as ananalog signal which can be expressed as:

xa (t) = x (t)∞∑

n=−∞

δ (t− nT ) (5.3)

whereδ (t) is the impulse function.

The impulse train

s (t) =∞∑

n=−∞

δ (t− nT ) (5.4)

is a period continuous-time function. So it can be expressed as a Fourier series

s (t) =∞∑

m=−∞

ckej2πmt/T (5.5)

The Fourier coefficents are

ck =1

T

∫ T

0

s (t) e−j2πkt/T dt (5.6)

=1

T

∫ T

0

δ (t) e−j2πkt/T dt (5.7)

=1

T(5.8)


So combining the above equations, we have

xa (t) = x (t)

[1

T

∞∑m=−∞

ej2πmt/T

](5.9)

From the frequency shifting property of the Fourier Transform, we know that if we havethe Fourier Transform pair

x (t) ↔ X (ω) (5.10)

then we have the pairx (t) ejω0t ↔ X (ω − ω0) (5.11)

for some constantω0. Therefore, sincexa (t) consists ofx (t) multiplied by an infinitenumber ofejω0t whereω0 = 2πm/T , the Fourier transform ofxa (t) is given by

Xa (ω) =1

T

∞∑m=−∞

X (ω −mωs) (5.12)

where

ωs =2π

T= 2πfs (5.13)

is the sampling frequency in radians per second. In other words, the spectrum of the sam-pled signal consists of frequency shifted replica of the spectrum of the original continuous-time signal.

Figure 5.3 shows the magnitude spectrum of the sampled signal when the original continuous-time signal is low-pass bandlimited. A low-pass bandlimited signal with a bandwidth ofW has a spectrum which is non-zero for the frequency range0 ≤ |ω| ≤ W .

5.1.2 Aliasing and The Uniform Sampling Theorem

Each spectrum replica in Figure 5.3 is separated from the adjacent replicas by the sam-pling frequency. If we reduce the sampling frequency, the replicas will be closer to eachother. If the sampling frequency is so low that the replicas overlap each other, as shownin Figure , then the spectrum will be corrupted. We will not be able to reconstruct theoriginal analog signal from the spectrum if it is corrupted. This situation is known asaliasing. Note that this happens whenωs < 2W .


X( )ω X( )ω−ωsX( )ω+ωsX( )ω+2ωs X( )ω−2ωs

ωs−ωs−2ω s 2ωs

MagnitudeSpectrum

0 ω

......

W−W

Figure 5.3: Spectrum of a Sampled Bandlimited Analog Signal.

Uniform Sampling Theorem If an analog signalx (t) is bandlimited toW Hz, then itcan be uniquely specified by samples taken uniformly at a rate of at least2W samplesper second, i.e. the sampling frequency and sampling period are given by

fs ≥ 2W (5.14)

T ≤ 1

2W(5.15)

This is known as the uniform sampling theorem.

A sampling rate of2W for an analog signal of bandwidthW is called theNyquist rate.A signal isunder-sampledif it is sampled below the Nyquist rate; it iscritically sampledif it is sampled at the Nyquist rate; it isover-sampledif it is sampled at higher than theNyquist rate.

The sampling theorem provides us with the lower bound on the sampling rate. If thesampling is too sparse, then important information will be missing in the sampled signal.However, if the sampling rate is too high, then a large amount of data will have to beprocessed within a given time frame. Thus the processing speed of the digital signalprocessor gives us the upper bound on the sampling rate.

It is interesting to note that even though this theorem is usually referred to as Shannon’ssampling theorem, it was originated by the British mathematicians E.T. and J.M. Whit-taker. In the Russian literature, this theorem was introduced to communications theoryby Kotel’nikov and took its name from him. C.E. Shannon used it to study what is now


X( )ω X( )ω−2ωs

X( )ω−ωsX( )ω+ωs

X( )ω+2ωs

MagnitudeSpectrum

ω

... ...

Aliasing

Figure 5.4: Aliasing of the Frequency Spectrum.

known as Information Theory in the 1940’s. Therefore in the mathematics and engi-neeirng literatures sometimes it is also calledWKS sampling theoremafter Whittaker,Kotel’nikov and Shannon.

Effect of Aliasing The effect of aliasing on an input signal in the time domain can bedemonstrated by sampling a sine wave of frequencyfb using different sampling frequen-ciesfs. Figure 5.5 shows such a sinusoidal function sampled at three different rates:fs = 4fb, fs = 2fb, andfs = 1.5fb.

In the first two cases, if we join the sample points using straight lines, it is obvious thatthe basic “up-down” nature of the sinusoid is still preserved by the resulting triangularwave as shown in Figure 5.6. If we pass this triangular wave through a low-pass filter, asmooth interpolated function will result. If the low-pass filter has the appropriate cut-offfrequency, the original sine wave can be recovered. This will be discussed in detail inSection 5.2.

For the last case, the sampling frequency is below the Nyquist rate. We would expectaliasing to occur. This is indeed the case. If we join the sampled points together, it canbe observed that the rate at which the resulting function repeats itself differs from thefrequency of the original signal. In fact, if we interpolate between the sample points, asmooth sinusoidal function with a lower frequency results, as shown in Figure 5.7.

Therefore it is no longer possible to recover the original sine wave from these sampledpoints. We say that the higher frequency sine wave now has an “alias” in the lower fre-quency sine wave inferred from the samples. In other words, these samples are no longer


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

Sampling at 4 times the frequency of the sinusoid

(a) Oversampled 2 times

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

Sampling at 2 times the frequency of the sinusoid

(b) Critically Sampled

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

Sampling at 1.5 times the frequency of the sinusoid

(c) Under-sampled

Figure 5.5: A Sinusoid Sampled at 3 Different Rates.


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

interpolated

(a) Over-sampled case

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

interpolated

(b) Critically Sampled Case

Figure 5.6: Interpolation of Sample Points without Aliasing.


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

time

ampl

itude

Sampling at 1.5 times the frequency of the sinusoid

2 Hz

1 Hz

Figure 5.7: Effect of Aliasing.

uniquely representative of the input signal and therefore any subsequent processing willbe invalid.

The baseband aliased frequency is a frequency within the range[−fs

2, fs

2

]. It is the

frequency range by which the samples will be reconstructed to form an analog signaland is known as theNyquist interval. If a signal is under-sampled, the frequency of thesignal lies outside of this frequency range. Since the spectrum of the sampled signalis replicated up and down the frequency axis, a replica appears in the baseband aliasedfrequency range. This frequency will be the one that is reconstructed. Using the aboveexample, the sinusoid has a frequency of2 Hz and it is sampled at3 Hz. So the Nyquistinterval is[−1.5, 1.5]. The2 Hz is reflected into this range as1 Hz shown in Figure. Ingeneral, the baseband aliased frequencyfa of any undersampled signal of frequencyf ata sampling ratefs is given by

fa = f − nfs (5.16)


wheren is an integer so thatfa lies within[−fs

2, fs

2

].

Example 5.1. A 7 Hz sinewave is sampled at a rate of 6 Hz. What will be the basebandaliased frequency of this sinewave?

Solution:

f = 7

fs = 6

So the apparent frequency is

f ′ = f − fs = 1 Hz

Example 5.2.Determine the Nyquist rate for the signal:

x (t) = 3 cos (πt) + cos (3πt)

Suppose this signal is sampled at half the Nyquist rate, determine the signalxa (t) thatwill be aliased withx (t).

Solution:

The signal has two frequency components:

f1 = 0.5 Hz

f2 = 1.5 Hz

So the Nyquist sampling rate, which is twice the highest frequency, is3 Hz.

Now iffs = 1.5 Hz, the Nyquist interval is[−0.75, 0.75].

f1 falls within the Nyquist interval and is not aliased. Butf2 will be aliased with thefrequency

f2a = f2 − fs = 1.5− 1.5 = 0 Hz

So the signal that will be aliased withx (t) is

xa (t) = 3 cos (πt) + cos (2πf2at) = 1 + 3 cos (πt)


5.1.3 Anti-aliasing Prefilters

The uniform sampling theorem assumes that the signal is strictly bandlimited. In the realworld, typical signals have a wide spectrum and are not bandlimited in the strict sense.For instance, we may assume that 20kHz is the highest frequency the human ears candetect. Thus we want to sample at a frequency slightly above 40kHz (say, 44.1kHz as incompact discs) as dictated by the Sampling Theorem. However, the actual audio signalsnormally have a much wider bandwidth than 20kHz. We can ensure that the signal isbandlimited at 20kHz by low-pass filtering. This low-pass filter is usually called anti-alias filter.

Anti-aliasing filters are analog filters as they process the signalbeforeit is sampled. Inmost cases they are also low-pass filters unless bandpass sampling techniques are used.(Bandpass sampling is beyond the scope of this course.) An ideal low-pass filter has aflat passband and the cut-off is very sharp. Since the cut-off frequency of this filter is halfof that of the sampling frequency, the resulting replicated spectrum of the sampled signaldo not overlap each other. Thus no aliasing occurs.

Practical low-pass filters cannot achieve the ideal characteristics. What are the implica-tions? Firstly, this would mean that we have to sample the filtered signals at a rate that ishigher than the Nyquist rate to compensate for the transition band of the filter. The band-width of a low-pass filter is usually defined as the 3-dB point (the frequency at which themagnitude response is 3dB below the peak level in the passband). But signal levels below3dB are still quite significant for most applications. For the audio signal application ex-ample in the previous section, it may be decided that signal levels below 40dB will causeinsignificant aliasing. The anti-aliasing filter used may have a bandwidth of 20kHz butthe response is 40dB down starting from 24kHz. This means that the minimum samplingfrequency has to be increased to 48kHz instead of 40kHz for the ideal filter.

Alternatively, if we fix the sampling rate, then we need an anti-alias filter with a sharpercut-off. Using the same audio example, if we want to keep the sampling rate at 44.1kHz,the anti-aliasing filter will need to have an attenuation of 40dB at about 22kHz. With abandwidth of 20kHz, the filter will need a transition from 3dB at down to 40dB within2kHz. This typically means that a higher order filter will be required. A higher orderfilter also implies that more components are needed for its implementation.


5.1.4 Practical Limits on Sampling Rates

As discussed in previous sections, the practical choice of sampling rate is determinedby two factors for a certain type of input signal. On one hand, the sampling theoremimposes alower boundon the allowed values of the sampling frequency. On the otherhand, the economics of the hardware imposes anupper bound. This economics includesthe cost of the analog-to-digital converter (ADC) and the cost of implementing the analoganti-alias filter. A higher speed ADC will allow a higher sampling frequency but maycost substantially more. However, a lower sampling frequency will put a more stringentrequirement on the cut-off of the anti-aliasing filter, necessitating a higher order filter anda more complex circuit which again may cost more.

In real-time applications, each sample is acquired (sampled), quantized and processedby a DSP. The output samples may need to be converted back to analog form. A highersampling rate will means that there are more samples to be processed within a certainamount of time. IfTproc represents the total DSP processing time, then the time intervalbetween samplesTs will need to be greater thanTproc. Otherwise the processor will notbe able to keep up. This means that if we increase the sampling rate we will need a higherspeed DSP chip.

5.2 Reconstruction of Analog Signals

A lot of applications requires an analog signal output. If the processing is done digitally,then the processed discrete-time signal will have to be converted to analog form. Sincethe information about the analog signal is contained in one of the replicas of the spectrumof the discrete-time signal, we need to isolate only one of those replicas. The easiest wayto do so is to filter the discrete-time signal with a low-pass filter that has a passband inthe Nyquist interval.

There is an alternative way the Uniform Sampling Theorem can be stated: If the highestfrequency contained in an analog signalx (t) is W and the signal is sampled uniformlyat a ratefs of 2W or higher, thenx (t) can be exactly recovered from its sample valuesusing the formula

x (t) =∞∑

n=−∞

x (nT ) hr (t− nT ) (5.17)

Reconstruction of Analog Signals 109

where

hr (t) =sin (πfst)

πfst(5.18)

T =1

fs

(5.19)

This is referred to as ideal reconstruction.

5.2.1 Ideal Reconstructor

The ideal reconstructor is simply an ideal low-pass filter with a cutoff frequencyfs/2wherefs is the sampling frequency of the discrete-time signal. Its frequency response isgiven by

Hr (f) =

T, |f | ≤ fs/20, otherwise

(5.20)

whereT = 1/fs. The impulse response is therefore the sinc function

hr (t) = sinc

(πt

T

)(5.21)

Example 5.3.An analog signal

x (t) = cos (10πt) + cos (50πt) + cos (90πt)

wheret is in milliseconds, is prefiltered by an analog filterH (f), producingy (t). Theoutput y (t) is sampled at 40kHz, givingy (nT ). This discrete-time signal is passedthrough an ideal reconstructor that producesya (t).

Determineya (t) when (i)H (f) = 1; (ii) H (f) is an ideal prefilter with cutoff frequency20kHz; and (iii)H (f) is a practical prefilter with a flat magnitude response from DC to20kHz and then attenuates monotonically at a rate of 60 dB/octave.

Solution:

(i) H (f) = 1 implies that there is no prefiltering. Now,

f1 = 5kHz

f2 = 25kHz

f3 = 45kHz


Sampling atfs = 40kHz will cause aliasing forf2 andf3 and the aliased frequenciesare

f2a = 25− 40 = −15kHz

f3a = 45− 40 = 5kHz

So the reconstructed signal is

ya (t) = cos (10πt) + cos (−30πt) + cos (10πt)

= 2 cos (10πt) + cos (30πt)

(ii) The prefilter cuts off all frequency components above 20 kHz. Therefore,

y (t) = cos (10πt)

and it is reconstructed without error with

ya (t) = cos (10πt)

(iii) Recall that 1 octave is a doubling in the frequency. So the response at 40 kHz is 60dB below that at 20 kHz. We have

log2

(25

20

)= 0.322

log2

(45

20

)= 1.170

Therefore, attenuation at 25 kHz and 45 kHz are

A2 = 60 (0.322) = 19.3dB

A3 = 60 (1.170) = 70.1dB

Amplitudes at these frequencies become

|H (f2)| = 10−A2/20 = 0.1084

|H (f3)| = 10−A3/20 = 3.126× 10−4

So

ya (t) = cos (10πt) + 0.1084 cos (−30πt) + 3.126× 10−4 cos (10πt)

= 1.0003126 cos (10πt) + 0.1084 cos (30πt)


5.2.2 Staircase Reconstructor

While the ideal reconstructor is useful for analysis purposes, it cannot be implementedin practice. In reality, we need to first convert the sequence of numbers that is the digitalsignal into some analog form. This analog signal will then be filtered to isolated only thefrequency components in the Nyquist interval.

A practical reconstructor is the staircase reconstructor. It generates a staircase approxi-mation in response to each digital sample. Its impulse response is

hr (t) =

1, 0 ≤ t ≤ T0, otherwise

(5.22)

It can be expressed ashr (t) = u (t)− u (t− T ) (5.23)

This gives us the frequency response

Hr (ω) =1

jω− e−jωT

jω

=1

jω

(1− e−jωT

)=

e−jωT/2

jω

(ejωT/2 − e−jωT/2

)=

sin (ωT/2)

ωe−jωT/2 (5.24)

and so

H (f) = Tsin (πfT )

πfTe−jπfT (5.25)

The magnitude response of the staircase reconstructor is shown in Figure 5.8 togetherwith the ideal reconstructor.

The staircase reconstruction is performed by Digital-to-Analog Converters (DACs).

5.2.3 Anti-image Postfilters

The signal at the output of the staircase reconstructoryr (t) has a spectrum that is theproduct of the spectrum of the discrete-time signal and the frequency response of the


−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.50

0.2

0.4

0.6

0.8

1

Normalized Frequency (f/fs)

Mag

nitu

de

Ideal Reconstructor StaircaseReconstructor

Sampled Signal Spectrum

Figure 5.8: Magnitude Response of the Staircase and the Ideal Reconstructors.

reconstructor. From Figure 5.8 it is obvious that the magnitude spectrumm ofyr (t) willconsists of most of the central replica together with some attenuated portions of otherreplicas. In order to obtain a good bandlimited reconstructed signal,yr (t) should be low-pass filtered so that only the central replica is retained. This low-pass filter, with a cutofffrequency offs/2, is called an anti-image postfilter.

If the sampling frequency is near critical, then we need to have an anti-image postfilterthat has a very sharp roll-off because the transition band is narrow. Alternatively, if thesampling rate is high, then the replicas are far apart and so the roll-off will be gradual.Sharp roll-off implies high order filters have to be used. Since these are analog filters,they are typically implemented using op-amps, resistors and capacitors. Higher orderfilters involve more of these components and are therefore more expensive and moredifficult to maintain. Lower order filters are therefore preferred. However, the tradeofffor using a lower order filter is to have a higher sampling rate and so we need a fasterDAC and also a faster processor that can produce more samples within a given time. Real


f (kHz)−12

x y

−8 −4 0 1284

15 dB/ octave

Figure 5.9: Magnitude Spectrum of Sampled Signal in Example 5.4.

designs therefore need to consider these issues.

Example 5.4.The spectrum of a certain analog signal is flat up to 4 kHz. Beyond 4 kHz,the spectrum attenuates at a rate of 15dB per octave. This signal is sampled at 12 kHz.What would be the amount of aliasing if no prefiltering is used?

Solution:

The spectrum of the sampled signal looks like that shown in Figure 5.9. By symmetry,x = y. Since 8 kHz is exactly one octave from 4 kHz,x = y = 15dB.

So the aliased components are 15 dB or below the signal level.

Example 5.5. For the signal given in Example 5.4, we now wish to suppress the highestaliased components within the 4 kHz band by more than 50dB. Determine the stopbandattenuation required for the prefilter.

Solution:

Referring to Figure 5.9, attenuation at 8 kHz is 15 dB. Since this is the highest aliasingcomponent within the signal’s bandwidth, we need to increase the attenuation to 50 dB.In other words, the prefilter must introduce a further attenuation of(50− 15) = 35dB.

Therefore the specfications for the prefilter is a flat passband up to 4 kHz, stopband edgeat 8 kHz with minimum stopband attenuation of 35 dB.


Example 5.6. The sampled signal in Example 5.5 is filtered by a digital low-pass filter(which can be considered ideal) with cutoff frequency of 2 kHz. The filtered digital signalis reconstructed by a staircase DAC followed by a postfilter. The overall reconstruc-tor is required to suppress spectral images by more than 80dB. What is the attenuationrequirement for the postfilter?

Solution:

Since the bandwidth of the sampled signal is reduced to 2 kHz. The edge of the passbandof the first replica starts at 10 kHz instead of 8 kHz.

Total attenuation is the sum of that introduced by the staircase reconstructor and thepostfilter (in dB),

A (f) = ADAC (f) + APOST (f)

The attenuation provided by the reconstructor is

ADAC (f) = −20 log10

|H (f)||H (0)|

= −20 log10

[sin (πf/fs)

πf/fs

]from (5.25). Atf = 10kHz, ADAC (f) = 14.4dB.

Therefore the attenuation required of the postfilter is given by

APOST (f) = 80− 14.4 = 65.6dB

starting fromf = 10kHz.

5.3 Quantization

5.3.1 Sample-and-Hold

Apart from sampling (discretization in time), the process of converting an analog signalinto digital form also involve the discretization of the sampled signal amplitude orquan-tization. In practice, because the quantization process takes a finite amount of time, thesampled signal amplitude has to be held constant during this time. The sampling processis usually performed by a sample-and-hold circuit which can be logically represented

Quantization 115

as in Figure . The quantization process is performed by theanalog-to-digital converter(ADC).

The hold capacitor hold the sampled measurement of the analog signalx(nT ) for at mostT seconds during which time a quantized valuexQ(nT ) is available at the output of theanalog-to-digital converter, represented as aB-bit binary number. The sample-and-holdand the ADC may be separate modules or may be integrated on the same chip. Typicallythe very fast ADCs require an external sample-and-hold device.

5.3.2 Uniform Quantization

The ADC assumes that the input values cover afull-scale range, sayR. Typical values ofR are between 1 to 15 volts. Since the quantized sampled valuexQ(nT ) is represented byB-bits, it can take on only one of2B possible quantization levels. If the spacing betweenthese levels is the same throughout the rangeR, then we have a uniform quantizer. Thespacing between quantization levels is calledquantization widthor quantizer resolution.

For uniform quantization, the resolution is given by

Q =R

2B(5.26)

The number of bits required to achieve a required resolution ofQ is therefore

B = log2

R

Q(5.27)

Most ADCs can take bipolar inputs which means the sampled values lie within the sym-metric range

−R

2≤ x(nT ) <

R

2(5.28)

For unipolar inputs,0 ≤ x(nT ) < R (5.29)

In practice, the input signalx(t) must be preconditioned to lie within the full-scale rangeof the quantizer. Figure 5.10 shows the quantization levels of a 3-bit quantizer for bipolarinputs.


A 1

A 2

A 3

A −4

A −3

A −2

A −1

OutputAmplitude

InputAmplitudeR/2−R/2

Figure 5.10: A Uniform 3-bit Quantizer Transfer Function.

5.3.3 Quantization Noise

Quantization error(or noise) is the difference between the actual sampled value and thequantized value. Mathematically, this is

e(nT ) = x(nT )− xQ(nT ) (5.30)

or equivalently,e [n] = x [n]− xQ [n] (5.31)

Based on these equations, the quantization noise is modelled as an additive noise thatis uncorrelated with the input value. This is not quite true unless the quantizer resolu-tion is high since obviously the noise depends on the value of the input data as shownin the equations above. However, this assumption is usually made to simplify the analy-sis. Quantization noise is also normally assumed to be uniformly distributed within thequantization width.

If x [n] lies between two quantization levels, it will either be rounded up or truncated.Rounding replacesx [n] by the value of the nearest quantization level. Truncation re-placesx [n] by the value of the levelbelowit.

Quantization 117

For rounding, the error is given by

−Q

2< e <

Q

2(5.32)

whereas for truncation, it is

0 ≤ e < Q (5.33)

It is obvious that rounding produces a less biased representation of the analog values.The average error is given by

e =

∫ Q/2

−Q/2

ep(e)de (5.34)

wherep(e) is the distribution ofe[n]. Assuming uniform distribution for quantizationnoise, i.e.p (e) = 1/Q, we have

e =1

Q

∫ Q/2

−Q/2

ede = 0 (5.35)

This means that on average half the values are rounded up and half down.

5.3.4 Signal-to-Quantization Noise Ratio

The mean-square value of the error gives us an idea of the average power of the errorsignal. It is given by

σ2e =

∫ Q/2

−Q/2

e2p (e) de (5.36)

=1

Q

∫ Q/2

−Q/2

e2de

=1

Q

[e3

3

]Q/2

−Q/2

=Q2

12(5.37)


For signal powerσ2x, the signal-to-quantization-noise ratio is therefore

SQNR = 10 log10

(σ2

x

σ2e

)(5.38)

= 10 log10

(σ2

x

Q2/12

)= 10 log10

(σ2

x

R2(12)

(2B))

(5.39)

= 20B log10 2 + 10 log10 12− 20 log10

(R

σx

)= 6.02B + 10.79− 20 log10

(R

σx

)(5.40)

by using (5.37) and (5.26). Thus if we increase the number of bits of the ADC by one,the signal to quantization noise ratio improves by 6 dB. The above equation (excludingthe bias value10.79) gives us thedynamic rangeof the quantizer.

Example 5.7. The dynamic range of the human ear is about 100 dB. If a digital audiosystem is to match this dynamic range, how many bits would be required to representeach signal sample?

Ans: A 16-bit quantizer will provide a dynamic range of 96 dB.

5.4 Sampling Rate and Quantization Tradeoff

5.4.1 Power Spectral Density

The power spectral density (PSD) of a signal (or noise) is defined as the power per unitfrequency. Thus the PSD of quantization noise is

See =σ2

e

fs

(5.41)

since the spectra of sampled signals are unique only within a range equal tofs. The noisepower within any particular range of frequencies[fa, fb] can therefore be calculated by

Pab = (fb − fa) See (5.42)

= σ2e ·

(fb − fa)

fs

(5.43)

Sampling Rate and Quantization Tradeoff 119

σe12

/f s1

σ 2e2 /f s2

−f /2s2 −f /2s1 f /2s1 f /2s2

f

Figure 5.11: Power Spectral Density of Quantization Noise at Two Different SamplingRates.

5.4.2 Oversampling

The total amount of quantization noise is dependent on the resolution of the quantizer.Given a quantizer, the total quantization noise power is the same. Since the power spectraldensity is a function of the sampling frequencyfs, sampling at a higher rate will spreadthe total quantization noise power over a wider frequency range. Figure illustrates thisfact. The total area under each distribution is the same.

Let the quantization noise PSD beSee1 when the sampling rate isfs1, giving a total noisepower ofσ2

e1. At a higher sampling ratefs2, the PSD becomesSee2 for a total noise powerσ2

e2. Since the total noise power is the same for any given quantizer,

σ2e1 = σ2

e2 (5.44)

ThusSee1 · fs1 = See2 · fs2 (5.45)

orSee1

See2

=fs2

fs1

= L (5.46)

whereL is called the oversampling ratio.

Now, suppose we have two quantizers, one isB1 bits and the other isB2 bits, withB2 < B1. We desire to keep the output of these two quantizers to be of the same quality.


In other words, the quantization noise level should be the same. One way to achieve thisis to match each quantizer with different sampling rates:fs1 andfs2(> fs1). In this case,we are keeping the PSD constant, i.e.See1 = See2. So

σ2e1

fs1

=σ2

e2

fs2

(5.47)

⇒ σ2e1 = σ2

e2 ·fs1

fs2

(5.48)

=σ2

e2

L(5.49)

Now,

σ2e1 =

Q21

12(5.50)

σ2e2 =

Q22

12(5.51)

and

Q1 = R2−B1 (5.52)

Q2 = R2−B2 (5.53)

So (5.49) becomes

L =σ2

e2

σ2e1

(5.54)

=Q2

2

Q21

= 22(B1−B2) = 22∆B (5.55)

or∆B = 0.5 log2 L (5.56)

Thus, by doubling the sampling rate (L = 2), we can reduce half a bit for the quantizerwhile maintaining the same quality of output. (5.56) is a tradeoff between sampling rateand quantization resolution.

Example 5.8.CD quality sound requires 16-bit quantization. If we decided to use a 1-bitquantizer, what should be oversampling ratio be?

Ans: L = 230.


σe22

f /2−f /2 −f /2 f /2s1 s1 s2s2

High−pass FilteredNoise Spectrum

f

/f s2

in signal bandnoise remaining

Figure 5.12: Power Spectrum of Noise Before and After Noise Shaping.

5.4.3 Noise Shaping

The tradeoff of half a bit for doubling the sampling rate is too uneconomical. We canimprove on that by using noise shaping techniques.

Note that the quantization noise is white, i.e. its power spectrum is flat. If the signalspectrum occupies just a portion offs, then we can shape the noise spectrum so thatit is lower within the signal band and higher for the rest of the frequencies by using ahigh-pass filter. The effect of the noise-shaping filter on the noise spectrum is shown inFigure . We can ensure that the signal bandwidth is small compared with the samplingfrequency by oversampling.

Consider the frequency response of ap-th order noise shaping high-pass filter

HNS (f) ≈(

2πf

fs

)2p

(5.57)

for |f | |fs/2|. The average noise power within|f | ≤ |fs1/2| is given by

σ2e =

σ2e2

fs2

∫ fs1/2

−fs1/2

(2πf

fs

)2p

df (5.58)

= σ2e2 (2π)2p

∫ fs1/2

−fs1/2

f 2p

f 2p+1s2

df (5.59)

= σ2e2

(π2p

2p + 1

)1

L2p+1(5.60)


Therefore,

∆B =

(p +

1

2

)log2 L− 1

2log2

(π2p

2p + 1

)(5.61)

By using oversampling and noise shaping, doubling the sampling frequency gives us asaving of

(p + 1

2

)bits. This concept is being used in sigma-delta ADCs.

5.4.4 Non-uniform Quantization

One of the assumptions we have made in analysing the quantization error is that the sam-pled signal amplitude is uniformly distributed over the full-scale range. This assumptionmay not hold for certain applications. For instance, speech signals are known to have awide dynamic range. Voiced speech (e.g. vowel sounds) may have amplitudes that spanthe whole full-scale range while softer unvoiced speech (e.g. consonants such as frica-tives) usually have much smaller amplitudes. Also, an average person only speaks 60%of the time while she/he is talking. The remaining 40% is silence with a negligible signalamplitude.

If uniform quantization is used, the louder voiced sounds will be adequately represented.However, the softer sounds will probably occupy only a small number of quantizationlevels with similar binary values. This means that we would not be able to distinguishbetween the softer sounds. As a result, the reconstructed analog speech from these digitalsamples will not nearly be as intelligible as the original.

To get around this problem, non-uniform quantization can be used. More quantizationlevels are assigned to the lower amplitudes while the higher amplitudes will have lessnumber of levels. This quantization scheme is shown in Figure 2-17.

Alternatively, a uniform quantizer can still be used, but the input signal is firstcompressedby a system with an input-output relationship (or transfer function) similar to that shownin Figure 5.14.

The higher amplitudes of the input signal are compressed, effectively reducing the num-ber of levels assigned to it. The lower amplitude signals are expanded (or non-uniformlyamplified), effectively making it occupy a large number of quantization levels. After pro-cessing, an inverse operation is applied to the output signal (expandingit). The systemthat expands the signal has an input-output relationship that is the inverse of the compres-sor. The expander expands the high amplitudes and compresses the low amplitudes. Thewhole process is calledcompanding(COMpressing and exPANDING).


Input

Output

Figure 5.13: Non-uniform Quantization.

5.4.4.1 Companding

Companding is widely used in public telephone systems. There are two distinct com-panding schemes. In Europe, A-law companding is used and in the United States,µ-lawcompanding is used.

µ-law compression characteristic is given by the formula:

y = ymax ·ln[1 + µ

(|x|

xmax

)]ln (1 + µ)

· sgn (x) (5.62)

where

sgn (x) =

+1, x ≥ 0−1, x < 0

(5.63)

Here,x andy represent the input and output values, andxmax andymax are the maximumpositive excursions of the input and output, respectively.µ is a positive constant. TheNorth American standard specifiesµ to be 255. Notice thatµ = 0 corresponds to a linearinput-output relationship (i.e. uniform quantization). The compression characteristic isshown in Figure 5.14.


The A-law compression characteristic is given by

y =

ymaxA( |x|

xmax)

1+ln Asgn (x) , 0 < |x|

xmax≤ 1

A

ymax1+ln[A( |x|

xmax)]

1+ln Asgn (x) , 1

A< |x|

xmax< 1

(5.64)

Here,A is a postive constant. The European standard specifiesA to be 87.6. Figure 5.15shows the characteristic graphically.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Normalized Input

Com

pres

sed

Inpu

t

mu−law compression

u=10

u=100 u=255

Figure 5.14:µ-law Compression Function

5.5 Dithering

Another assumption we have made in analysing quantization noise is that it is assumedto be uniformly distributed over the quantization width. If the noise is not uniformlydistributed, quantization distortion results.

Dithering 125

Figure 5.15: A-law Compression Function

We shall illustrate quantization distortion through an example. A low amplitude sinusoidis being sampled and quantized. The samples of the sinusoid are given by

x [n] = A cos (2πf0n) (5.65)

whereA is less than the quantization resolution. Let

fs = 40 samples per second andA = 0.75Q (5.66)

So for a 1 kHz sinusoid, the actual sampling rate is 40 kHz. Figure 5.16 shows theoriginal and the quantized signals. Note that the quantized signal only occupies threeof the available quantization levels. The frequency spectrum of this quantized signal isshown in Figure 5.17.

It has peaks atf0, and the odd harmonic frequencies3f0, 5f0, etc. Clearly the odd har-monics are artefacts of the quantization process and can be considered as the spectrum ofthe quantization noise signal which, in this case, is not white.


0 100 200 300 400 500 600 700 800 900 1000

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Index n

Uni

ts o

f Q

Undithered Quantization

Figure 5.16: Original and Quantized Signals

This problem can be overcome by adding adither v(n) to the original sampled signal sothaty(n) = x(n) + v(n).

Various types of dither can be added. Two of them which are of practical interest arerect-angularandtriangular dither. They are so called because the distribution of the randomsignal samples are rectangular and triangular in shape respectively. The distributions areshown in Figure 5.18.

The addition of dither to the original signal will increase its average quantization noisepower. Recall that the average noise power for uniform quantization isQ2/12. Theaddition of rectangular dither will double this average noise power and the addition oftriangular dither will triple it. However, if we look at the frequency spectrum of thedithered and quantized signal of the example we have been considering (Figure ), we willnotice that the noise spectrum now appears to be white and the odd harmonic artefactsare not there any more.

Summary 127

Figure 5.17: Spectrum of the Quantized Signal

It must be emphasized that in general, the sampling process will cause all the odd har-monics that lie outside of the Nyquist interval (out-of-band harmonics) to be aliased backinto the interval (in-band non-harmonic frequencies). So the overall spectrum will con-tain peaks at frequencies other than the odd harmonics.

5.6 Summary

While the uniform sampling theorem is the basis for choosing sampling rates, we haveshown that in practice, issues such as the complexity of the anti-aliasing prefilter and theanti-image postfilter will need to be considered. The sampling theorem merely provideus with a lower bound on the sampling rate. The upper bound is limited by the speedat which the processor can handle the samples and the complexity of the processingrequired.


1/Q

−Q/2 0 Q/2

p(v)

v

(a) Uniform distribution

0

p(v)

v−Q Q

1/Q

(b) Triangular Distribution

Figure 5.18: Amplitude Distributions of Rectangular and Triangular Dither

Figure 5.19: The Dithered Signal and Its Quantized Version

Summary 129

Figure 5.20: Spectrum of Quantized Signal with Dithering

One can reconstruct almost perfectly an analog signal that has been sampled fast enough.However, once the sampled signal is quantized, it is forever distorted. The amount ofdistortion can be minimized by increasing the resolution of the quantizer. Increasingquantizer resolution implies increasing the wordlength of the data to be processed. Thereis a tradeoff between the sampling rate and the quantizer resolution for a certain thequality of the output.

All these are practice design issues to be addressed for each DSP system that involvesanalog-to-digital and digital-to-analog conversions.

Chapter 6

Digital Filters and Their Realizations

In this chapter, we are concern about how the output of a digital filter is computed givenits coefficients and the input signal. Even though the digital filter equation is simple, thereare many different ways by which the output could be computed and each method has itsown characteristics. We shall also study the consequences of representing the coefficientsand the results of the computing with a finite number of bits.

But firstly, a brief review of Z transform is given. We assume that the readers are al-ready somewhat familiar with Z transform. Those who are not should consult one of therelevant references.

6.1 A Review of Z Transform

6.1.1 Definition

The Z transform of a discrete-time sequencex[n] is defined by

X (z) =∞∑

n=−∞

x[n]z−n (6.1)

if the series converges. Here,z is a complex variable.

The region of convergence (ROC) of a Z-transform is the range ofz for which the trans-form converges. Sincez is complex, the ROC is a region on the complex plane.X (z)and its ROC uniquely determines the sequencex[n].

132 Digital Filters and Their Realizations

Signalx [n] Z Transform Region of Convergence

δ [n] 1 All zu [n] 1

1−z−1 |z| > 1

anu [n] 11−az−1 |z| > |a|

nanu [n] az−1

(1−az−1)2|z| > 1

cos (ω0n) u [n] 1−z−1 cos ω0

1−2z−1 cos ω0+z−2 |z| > 1

−anu [−n− 1] 11−az−1 |z| < |a|

Table 6.1: Some Z-Transform Pairs

Example 6.1.Determine the Z-transform of the sequence

x [n] = (0.5)n u [n]

whereu [n] is the unit step sequence.

Example 6.2.Determine the Z-transform of the sequence

x [n] = − (0.5)n u [−n− 1]

Some common Z-transform pairs and their ROCs are listed in Table 6.1.

6.1.2 Properties

Linearity The Z-transform is linear and so the superposition principle holds.

a1x1 [n] + a2x2 [n] ↔ a1X1 (z) + a2X2 (z) (6.2)

wherea1 anda2 are arbitrary constants and

x1 [n] ↔ X1 (z)

x2 [n] ↔ X2 (z)

Delay Ifx [n] ↔ X (z)

then a delay ofD samplesx [n−D] ↔ z−DX (z)

A Review of Z Transform 133

Convolution Convolution in the time-domain will be transformed to multiplication inthe Z domain.

y [n] = x [n] ? h [n] ↔ Y (z) = H (z) X (z) (6.3)

Causality Consider a causal sequence

x [n] = pn1u [n] + pn

2u [n] + · · · (6.4)

with Z transform

X (z) =1

1− p1z−1+

1

1− p2z−1+ · · · (6.5)

The ROC is given by|z| > max

i|pi| (6.6)

Causal signals are characterized by ROCs that are outside the maximum pole circle. If asequencex[n] is causal, then only negative powers ofz appear inX (z). If x[n] is striclyanti-causal, thenX (z) only consists of positive powers ofz.

Stability The necessary and sufficient condition for a sequencex [n] to be stable is thatits Z-transformX (z) has a ROC which contains the unit circle. For causal signals, (6.6)holds. That means a causal and stable sequence must have all its poles inside the unitcircle.

Similarly, a causal and stable system must have a transfer function with all the polesinside the unit circle.

6.1.3 Inverse Z Transform

The inverse Z-Transform is defined by

x [n] =1

2πj

∮C

X (z) zn−1dz (6.7)

with the contour integral evaluated in the anti-clockwise direction.

Example 6.3.Find the inverse Z-transform of

X (z) =2− 2.05z−1

1− 2.05z−1 + z−2


6.2 System Transfer Functions

6.2.1 Poles and Zeros

Consider a rational Z-transform with real-valued coefficientsak andbk:

H (z) =N (z)

D (z)=

b0 + b1z−1 + · · ·+ bMz−M

a0 + a1z−1 + · · ·+ aNz−N(6.8)

which can be factorized as

H (z) =b0

a0

zN−M (z − z1) (z − z2) · · · (z − zM)

(z − p1) (z − p2) · · · (z − pN)(6.9)

H (z) is zero forz = z1, z2, . . . , zM . These are thezerosof H (z). The transfer functionis infinite atz = p1, p2, . . . , pN . These are thepolesof H (z). The poles and zeros maybe real-valued or complex-valued. In the case where they are complex, they must occurin complex conjugate pairs sinceak andbk are real-valued.

If a0 = 1 and all otherak are zeros, thenH (z) is an all-zero system. We say that it hasM zeros and anM -th order pole atz = 0. The transfer function now becomes

H (z) =M∑

k=0

bkz−k (6.10)

The impulse response sequence is therefore given byb0, b1, . . . , bM and is finite ifMis finite.

6.2.2 Frequency Response

The frequency response of an LTI system with a rational transfer functionH (z) can beobtained by

H (ω) = H (z) |z=ejω (6.11)

=

∑Mk=0 bke

−jωk∑Nk=0 ake−jωk

(6.12)

=

(b0

a0

) ∏Mk=1 (1− cke

−jω)∏Nk=1 (1− dke−jω)

(6.13)

System Transfer Functions 135

The gain is the log magnitude ofH (ω). It is usually expressed in decibels (dB):

Gain (dB) = 20 log10 |H (ω)| (6.14)

The phase response is defined as

θ (ω) = ]H (ω) (6.15)

Group Delay Consider a system with impulse response

h [n] = δ [n− nd]

which consists of only one impulse at some arbitrary indexnd. The frequency responseof this system is given by

H (ω) =∞∑

n=−∞

h [n] e−jωn (6.16)

= e−jωnd (6.17)

for |ω| ≤ π. The phase response is therefore

]H (ω) = θ (ω)

= −ωnd (6.18)

We define the phase delay as

Phase Delay = −θ (ω)

ω(6.19)

In this case it is equal tond which is independent ofω. We call systems with phase delaysindependent ofω linear phase systems.

An alternative way to measure the linearity of the phase response is to use thegroup delaydefined by

τg (ω) =dθ (ω)

dω(6.20)

If H (ω) has a linear phase response, then its group delay is a constant.


Minimum Phase Systems The stability and causality of a system are solely determinedby the poles of the transfer function; they do not concern the zeros. For certain problems,we need to ensure that the inverse system is also causal and stable. In these cases, bothzeros and poles of the transfer function must be inside the unit circle. This kind of systemsare calledminimum phase systems.

Linear Phase FIR Systems FIR transfer functions can have exact linear phase. Sincesuch systems have a constant group delayτg, they must have a frequency response of theform

H (ω) = G (ω) e−jωτg+jφ (6.21)

whereG (ω) is a real-valued function andτg andφ are constants. The impulse responseof this filter is given by the inverse DTFT:

h [n] =1

2π

∫ ∞

−∞H (ω) ejωndω

=1

2π

∫ ∞

−∞G (ω) e−jωτg+jφejωndω

=ejφ

2π

∫ ∞

−∞G (ω) ejω(n−τg)dω

= ejφg [n− τg] (6.22)

whereg [n] is the IDTFT ofG (ω). So

h [n + τg] = ejφg [n] (6.23)

SinceG (ω) is real,g [n] = g? [−n] and so from (6.23) we have

e−jφh [n + τ ] = ejφh? [−n + τg] (6.24)

orh [n + τg] = ej2φh? [−n + τg] (6.25)

If h [n] is causal and of finite length (0 ≤ n ≤ N − 1), then the group delay must be

τg =(N − 1)

2(6.26)

Digital Filter Realization 137

and so (6.25) becomesh [n] = ej2φh? [N − n− 1] (6.27)

for 0 ≤ n ≤ N . For most cases, we design filters with real-valuedh [n] and thereforeej2φ must be real, implying

φ =kπ

2

for integer values ofk. So (6.27) can be rewritten as

h [n] = (−1)k h [N − n− 1] (6.28)

This means that the filter impulse response must be either symmetric or anti-symmetric.In general, we can identify four different types of linear phase responses:

• k is even andN − 1 is even (orN is odd).

• k is even andN − 1 is odd (orN is even).

• k is odd andN − 1 is even (orN is odd).

• k is odd andN − 1 is odd (orN is even).

6.3 Digital Filter Realization

Given the transfer function of a digital filter, we have all that we needed to compute theoutput of the filter given some input. However, even though the equation looks simple,there are many ways by which the output could be computed. When we talk about re-alization, we are talking about a structure that relates the input and output of the filter.The most common way to specify a particular realization, or filter structure, is throughthe use of block diagrams. The blocks within these block diagrams can be implementedeither as a piece of digital hardware or as a program to be executed by a DSP chip.

We shall describe several filter structures. Considerations for the choice of a filter struc-ture include ease of programming on a particular DSP chip, and the regularity of theVLSI (Very Large Scale Integrated) design. Some structures are more sensitive to (quan-tization) errors in the coefficients. In some cases, such as IIR filters, the stability of thefilter may depend on an appropriate realization.


+

z −1

z −1

z −1

z −1

b

b

b

−a

−a

0

x[n] y[n]

y[n−1]

y[n−2]

x[n−1]

x[n−2]

1

2 2

1

Figure 6.1: Direct Form Realization of a Second Order IIR Filter.

6.3.1 Direct Form I

Consider a simple second order IIR filter with transfer function

H (z) =B (z)

A (z)=

b0 + b1z−1 + b2z

−2

1 + a1z−1 + a2z−2(6.29)

The input and output samples are related by

y [n] = −a1y [n− 1]− a2y [n− 2] + b0x [n] + b1x [n− 1] + b2x [n− 2] (6.30)

Direct form realization is simply a realization based on the direct implementation of thisdifference equation. This is illustrated in Figure 6.1.

In the figure, thez−1 symbol represents the delay of one sample. In actual implemen-tations, it would represent shift registers or a memory location in RAM. The three basicelements of the structure are illustrated in Figure 6.2.

This structure involves 4 unit delays, 5 multiplications and 4 additions. The structure alsorequires the storage of past values ofx andy. In this case, 4 memory units or registers,excluding those for the immediate input and output, are required. In general, for an IIRtransfer function

H (z) =

∑Mk=0 bkz

−k

1 +∑N

k=1 akz−k(6.31)


+

....

(a) Adder

K

(b) Multiplier

z−1

(c) UnitDelay

Figure 6.2: Three basic elements in the realization.

we require(M + N + 1) multiplications, (M + N) additions and(M + N) memoryunits.

Notice that the right hand side of (6.30) consists of two main operations: multiplicationsand additions. Each sample of the outputy or inputx is multiplied by a coefficient. Theresult is thenstoredor accumulatedfor addition. We call these two basic operations ofthe direct form structureMultiply-and-Accumulate(MAC). Dedicated DSP processorshave special instructions for MAC that are very efficient.

Figure 6.1 also illustrates the two parts of the filter structure. All the numerator termsshown on the left hand side of the adder block are thefeed-forwardelements. The de-nominator terms that depend on the previous output samples arefeeding back. Thisrealization is calleddirect form Ior simplydirect form.

6.3.2 Canonical Form or Direct Form II

We can view the structure in Figure 6.1 as a cascade of two sub-systemsH1 (z)andH2 (z)as shown in Figure 6.3 where

H1 (z) = B (z) (6.32)

H2 (z) =1

A (z)(6.33)

Note thatH1 (z) consists of only feed-forward terms andH2 (z) only has feedback terms.Since this is a linear system, reversing the order of the cascade will not change the overall


z −1

z −1

−a

−a

y[n]

y[n−1]

y[n−2]

2

1

+

z −1

z −1

+

H (z)1 H (z)2

b

b

b

0

x[n]

x[n−1]

x[n−2]

1

2

B(z) 1/A(z)

Figure 6.3: Direct Form I Structure as a Cascade of Two Sub-systems.

system transfer functionH (z). Figure 6.4 depicts the cascade of the sub-systems withthe order interchanged. The output of the sub-systemH2 (z) is now the input to the sub-systemH1 (z) . The two sets of delays in the middle operate on the same dataw[n].Therefore we do not need two separate sets of delays; they can be merged into one asshown in Figure 6.5. This realization is known as theDirect Form II realization or theCanonical Formrealization.

It is not difficult to see that the canonical form implements the original IIR filter equation(6.30). We have, from the adder on the left hand side of Figure 6.1,

w[n] =2∑

k=1

akw [n− k] + x [n]

= −a1w [n− 1]− a2w [n− 2] + x [n] (6.34)


H (z)1

B(z)

H (z)2

1/A(z)

+ +

z −1

z −1

z −1

z −1

b 0

1−a

−a2

b

b

1

2

x[n] y[n]w[n]

Figure 6.4: Interchanging the Order of the Sub-systems.

The output of the adder on the right is given by

y [n] =2∑

k=0

bkw [n− k]

= b0w [n] + b1w [n− 1] + b2w [n− 2] (6.35)

Substituting the expressions forw [n], w [n− 1] andw [n− 2] that can be obtained from


(6.34) into (6.35), we have

y [n] = b0 (−a1w [n− 1]− a2w [n− 2]) + b0x [n]

+b1 (−a1w [n− 2]− a2w [n− 3]) + b1x [n− 1]

+b2 (−a1w [n− 3]− a2w [n− 4]) + b2x [n− 2] (6.36)

=2∑

k=0

bkx [n− k]

−a1 (b0w [n− 1] + b1w [n− 2] + b2w [n− 3])

−a2 (b0w [n− 2] + b1w [n− 3] + b3w [n− 4]) (6.37)

=2∑

k=0

bkx [n− k]− a1y [n− 1]− a2y [n− 2] (6.38)

which is the same as (6.30).

The structure in Figure 6.5 requires 5 multiplications and 4 additions similar to directform I. However, the number of memory units required is reduced to 2 (forw [n− 1] andw [n− 2]). In fact, the word “canonical” refers to the fact that this structure requires theleast amount of storage. Referring to (6.31), the canonical form requires(M + N + 1)multiplications,(M + N) additions, andmax (M, N) memory units, excluding the cur-rent input and output samples.

Although we have been using the second order IIR filter as example, both direct formstructures can easily be generalized to filter transfer functions of higher orders.

Example 6.4.Given the digital filter transfer function

H (z) =2− 3z−1 + 4z−3

1 + 0.2z−1 − 0.3z−2 + 0.5z−4

derive the direct form I and II realizations.

Relative Merits The canonical form is commonly used because of the following prop-erties:

1. requires a minimum of storage space; and

2. good round-off noise property.


+

z −1

z −1

1−a

−a2

+b 0

b 1

b 2

x[n] y[n]w[n]

Figure 6.5: Direct Form II or the Canonical Form.

The disadvantage is that it is susceptible to internal numeric overflow. The input to thefilter can be scaled to avoid overflows. On the other hand, the direct form I structure doesnot require scaling because there is only one adder. Thus if scaling is not desirable, directform I realization is preferred.

6.3.3 Transposed Structures

The direct form I and II structures can betransposed. The transposed structures can beobtained by

1. reversing all the signal flow directions;

2. change node (connection points) into adders and adders into nodes; and

3. exchanging the input with the output.

The transposed direct form I and II are shown in Figures 6.6 and 6.7 respectively. In bothcases, the transfer function remains the same after transposition.

Note that for the tranposed structures, the inputsx [n] and outputsy [n] are no longerstored. Only some computed intermediate values are stored. This allows for some opti-mization of the computation since one sample ofx [n] is now multiplied with differentcoefficients.


+

z −1

z −1

+

z −1

+

z −1

+

−a

−a

b

b

b

y[n] x[n]

1

0

2 2

1

Figure 6.6: Transposed Direct Form I Realization of a Second Order IIR Filter.

6.3.4 FIR Filters

For an FIR filter, the denominator polynomial is simply equal to 1, i.e.A (z) = 1. Soonly the feed forward elements exist. The direct form realization is usually drawn in aslightly different way as shown in Figure 6.8.

This corresponds to the FIR equation we have been using for a second order filter

y [n] =2∑

k=0

h [k] x [n− k] (6.39)

whereh [n] is the impulse response or filter coefficients.

For linear phase FIR filters, the filter coefficients are symmetric or anti-symmetric. Sofor anN -th order filter, the number of multiplications can be reduced fromN to N/2 forN even and to(N + 1)/2 for N odd. Figure 6.9 shows the direct form realization of anodd order linear phase FIR filter that takes advantage of this saving.


b 0

b 1

+

z −1

z −1

+

b 2

+

−a1

−a2

y[n] x[n]

Figure 6.7: Transposed Direct Form II Realization of a Second Order IIR Filter.

z −1 z −1

+

x[n]

y[n]

x[n−1] x[n−2]

h[2]h[1]h[0]

Figure 6.8: Direct Form I Realization of an FIR Filter.


+

+

z −1

z −1z −1z −1z −1

z −1 z −1 z −1

+

++

+

x[n]

h[1]h[0]

x[n−1]

h[N/2−1] h[(N−1)/2]

y[n]

Figure 6.9: Direct Form Realization of Odd Order Linear Phase FIR Filter.

It is interesting to note that a transposed FIR direct structure can also be obtained usingthe method discussed for transposing IIR structures. The resulting structure for a secondorder filter is shown in Figure 6.10.

6.3.5 Cascade Realization

A general transfer functionH (z) of orderN > 2 can be factorized intoK factors so that

H (z) =K∏

i=1

Hi (z) (6.40)

=K∏

i=1

bi0 + bi1z−1 + bi2z

−2

1 + ai1z−1 + ai2z−2(6.41)

whereK = N/2 for even values ofN andK = (N + 1) /2 if N is odd. We mayview each second order section with transfer functionHi (z) as a subsystem of the whole


+z −1 z −1 +

x[n]

h[2]h[1]h[0]

y[n]

Figure 6.10: Transposed FIR Filter Structure Realization of an FIR Filter.

H(z)=H (z)H (z)...H (z)0 1 K−1

y[n]x[n] H (z)K−1H (z)1H (z)0

Figure 6.11: Cascade of Transfer Functions


systemH (z) and so the full system is made up of a cascade of these subsystems asdepicted in Figure 6.11.

Note that when we say “second order section” we really mean “up to second order sec-tion”’. Some of thebi2 andai2 can be zero. So the numerator and denominator polyno-mial orders can be less than $2K$. Also, the coefficients of each section are real-valued.Each second order section can be implemented using direct, canonical or transposedforms.

Note that the second order transfer functionsHi (z) formed are not unique. Factors inthe numerator and denominator polynomials can be paired in an arbitrary way to formHi (z). But the overall transfer functionH (z) remains the same. In practice, the pairingand ordering of the second order sections may affect the numeric accuracy of the resultingfilter. The internal multiplication in each section may generate a certain amount of round-off error. This error is then propagated to the next section. The round-off error of theoverall output is different for each combination of second order sections. Naturally wewant to achieve a minimum amount of round-off error. This is a difficult problem tosolve. In practice, some trial-and-error would be needed. Fortunately, most IIR filters donot have a high order so the number of possible combinations are not too large.

The following rules of thumb can be used for cascade realizations:

• Pair the numerator and denominator quadratic pairs with roots that are closest toone another. In this way, the poles and zeros tend to cancel one another and so thesections will not be producing output values that are very large or very small.

• Put the section with the denominator root having magnitudes that are closest to oneas the last section. Systems with poles near the unit circle are the least stable ones.

Example 6.5.Determine the cascade realization of the following filter:

H (z) =1 + 2z−1 + z−2

1− 0.75z−1 + 0.125z−2

Solution:

FactorizeH (z) into

H (z) =(1 + z−1) (1 + z−1)

(1− 0.5z−1) (1− 0.25z−1)=

(1 + z−1)

(1− 0.25z−1)· (1 + z−1)

(1− 0.5z−1)

The two sub-systems can be realized in any form.


6.3.6 Parallel Realization

An alternative to cascade form for higher order IIR filters is the parallel form. Here thetransfer functionH (z) is expanded as a sum of first and second order terms.

H (z) = C +N∑

k=1

Hk (z) (6.42)

where

C =bN

aN

Hk (z) =Ak

1− pkz−1

for anN -th order transfer function. In this case, the individual subsystem transfer func-tions are summed to form the overall transfer function. Thus the subsystems are con-nected in parallel in contrast with the cascade form. This is shown in Figure 6.12. Thewhole IIR filter now consists of a parallel bank of first order filters.

BothAk andpk can be complex-valued. Ifpk is complex, then its complex conjugate willalso be appear since the coefficients ofH (z) are real-valued. We can combine the pair ofcomplex conjugate terms to avoid having to deal with complex numbers. So the transferfunction of the combined subsystems are second order sections given by

Hk (z) =bk0 + bk1z

−1

1 + ak1z−1 + ak2z−2

The advantage of the parallel form compared with the cascade form for IIR filters is thatthe ordering of the subsystems are unimportant since they are in parallel. Scaling is easieras it can be carried out for each sub-system independently. Furthermore, round-off errorsin each block are not propagated to the next block.

Example 6.6.Realize the filter given in Example 6.5 in parallel form.

Solution:


H (z)

H (z)1

+

H (z)

x[n]y[n]

C

2

N

Figure 6.12: Parallel Form Realization

H (z) =1 + 2z−1 + z−2

1− 0.75z−1 + 0.125z−2

= 8 +−7 + 8z−1

(1− 0.5z−1) (1− 0.25z−1)

= 8 +18

1− 0.5z−1− 25

1− 0.25z−1

The realization using canonical form for each sub-system is shown in Figure.

6.3.7 Circular Buffers

Sample-by-sample processing of an IIR filter implemented in the canonical form can beexpressed in pseudo-code as follows.


+

z −1

+

+

z −1

x[n]y[n]

8

18

−25

0.5

0.25

Figure 6.13: Parallel Canonical Form Realization of the Transfer Function in Example6.6.


For each input sample x do:

w(0)=x;

for i=1,2,...,M do:

w(0)=w(0)-a(i)w(i);

y=b(L)w(L);

for i=L-1,L-2,...,0 do:

w(i+1)=w(i);

y=y+b(i)w(i);

end;

The number of multiply-accumulate (MAC) operations required is to compute each out-put sampley is M (L + 1). If each MAC can be performed in one instruction cycle withtimeTinstr, then the total time required for the MACs will be

TMAC = M (L + 1) Tinstr (6.43)

The algorithm also requiresL data shifts.

Similarly, for an(N + 1)-th order FIR filter, the sample processing can be expressed as:

For each input sample x do:

w(0)=x;

y=b(N)w(N);

for i=N-1,...,0 do:


w(i+1)=w(i);

y=y+b(i)w(i);

end;

Thus to compute each output sample,(N + 1) MAC operations andN data shifts arerequired. The processing time if only MAC operations are considered is

TMAC = (N + 1) Tinstr (6.44)

However, if each data shift also requires one instruction cycle, then the total processingtime for the IIR filter is

Tproc = M (L + 1) Tinstr + LTinstr (6.45)

and for the FIR filter it is

Tproc = (N + 1) Tinstr + NTinstr (6.46)

Obviously the time for shifting the coefficients is a significant portion of the total pro-cessing time. Since shifting data is much easier to perform than MAC, it should not havetaken so much time.

One way to avoid coefficient shifting altogether is to use acircular buffer. It is theequivalent of a circularly linked list. The input data samples are stored in the circularbuffer. A pointer points to the starting location of the buffer. In order to illustrate howthis works, let us consider a length-4 impulse response which requires a length-4 circularbuffer. If xn is the current input data sample, then to compute the output sample, we needto perform:

yn = h0xn + h1xn−1 + h2xn−2 + h3xn−3 (6.47)

wherehi, i = 0, 1, 2, 3 is the impulse response andxn−1, xn−2, xn−3 are the 3 pastsamples immediately precedingxn. The input samples are placed in the circular buffer asshown in Figure 6.14. The pointer is pointing at the current input sample. The next inputsamplexn+1 replaces the oldest samplexn−3 since it is not longer needed to computeyn+1. The pointer is updated to point at the location ofxn+1. The outputyn+1 is nowcomputed by multiplying the data that the pointer is pointing to withh0 and the rest ofthe product terms in the same anti-clockwise direction.


h 1

h 0

h 3

h 2 h 3

h 0

h 1

h 2

Next sample

instant

p

p

x n−1

x n−2

x n−3

nx x n−2

x n+1

nx

x n−1

Figure 6.14: A Length 4 Circular Buffer.

Notice that now we no longer need to shift the input data as we used to do with a lineararray. Once a new data sample is placed in the right place in the circular buffer, an updateof the pointer is all we need and the same procedures are executed.

All digital signal processors now support circular buffer operations by having dedicatedaddress generators. Once the size of the circular buffer and the addresses of the co-efficients and input data are set, the operation is performed automatically. This givea significant improvement in filtering performance, in addition to the single instructionMAC.

6.4 Finite Wordlength Effects

The number of bits that is used to represent numbers, calledwordlength, is often dictatedby the architecture of DSP processor. If specialized VLSI hardware is designed, then wehave more control over the word-length. In both cases we need to tradeoff the accuracywith the computational complexity. No matter how many bits are available for numberrepresentation, it will never be enough for all situations and some rounding and truncationwill still be required.

A number within a digital system is always represented by a finite number of bits because

Finite Wordlength Effects 155

of the finite wordlength of data registers used. Quantizing a number to a fixed number ofbits has an inherent limitation on the accuracy of the representation.

For FIR digital filters, the finite word-length affects the results in the following ways:

1. Coefficient quantization errors. The coefficient we arrived at in the approximationstage of filter design assume that we have infinite precision. In practice, however,we have the same word-length limitations on the coefficients as that on the signalsamples. The accuracy of filter coefficients will affect the frequency response ofthe implemented filter.

2. Round-off or truncation errors resulting from arithmetic operations. Arithmeticoperations such as addition and multiplication often give result that require morebits than the word-length. Thus truncation or rounding of the result is needed.Some filter structures are more sensitive to these errors than others.

3. Arithmetic overflow. This happens when some intermediate results exceed therange of numbers that can be represented by the given word-length. It can beavoided by careful design of the algorithm scale the results are appropriate stages.

For IIR filters, our analysis of finite word-length effects will need to include one morefactor: Product round-off errors. The round-off or truncation errors of the output sampleat one time instant will affect the error of the next output sample because of the recursivenature of the IIR filters. Sometimes limit cycles can occur.

We shall first review how fixed and floating point numbers are represented before goinginto the details of the above effects.

6.4.1 Fixed-point Representation

The general fixed point format is basically the same as the usual familiar decimal repre-sentation of numbers. It consists of a string of digits with a decimal point. The digits tothe left of the decimal point are the integer part and those to the right are the fractionalpart of the number.

X = (bBbB−1 · · · b1b0 · b−1b−2 · · · b−A+1b−A)r (6.48)

=B∑

i=−A

biri (6.49)


for 0 ≤ bi ≤ r − 1 wherebi are the digits andr is theradix or base.

Example 6.7.

(12.34)10 = 1× 101 + 2× 100 + 3× 10−1 + 4× 10−2

(110.01)2 = 1× 22 + 1× 21 + 0× 20 + 0× 2−1 + 1× 2−2

We shall focus on binary representations as this is the format we need to deal with inDSP. In this case, the digits are bits. Sob−A is the least significant bit (LSB) andbB isthe most significant bit (MSB). Naturally thebinary point(as opposed todecimal point)betweenb0 andb−1 does not physically exist and so the user must know beforehand howto interpret a particular binary number.

Non-negative integers can easily be represented by ann-bit pattern (B = n − 1 andA = 0). Since we need to deal with fractional numbers, the fraction format (B = 0 andA = n) is normally used. This allows us to respresent numbers in the range0 to 1− 2n.This is because multiplication of two numbers that are less than 1 will give us a resultthat is less than 1. For signed numbers, an extra sign bit is required. So each numberXis normalized to satisfy

−1 ≤ X < 1 (6.50)

Positive fractions are given by

X = 0.b−1b−2 · · · b−n+1 (6.51)

=−n∑

i=−1

bi2i (6.52)

Negative fractions can be represented in the two’s complement form. That is,

X = 1.b−1b−2 · · · b−n+2

(b−n+1 ⊕ 1

)(6.53)

where⊕ is the exclusive-OR operator.

Example 6.8.7

8= 0.111

−7

8= 1.001

Multiplication of two fixed point numbers eachn bits in length will generally give aproduct which is2n bits in length. The product therefore has to be truncated or roundedoff to n bits, producing truncation or round-off errors.


6.4.2 Floating-point Representation

Floating point representations are able to cover a much wider range of numbers for thesame number of bits compared with fixed point representations. They normally consistof a mantissaM , which is the fractional part of the number, and an exponentE, whichcan be either positive or negative. Hence a numberX is given by

X = M · 2E (6.54)

with 1/2 ≤ M < 1. The mantissa and the exponent require their individual sign bits.

Given a total number of bits available for representing a number, a number of differentfloating point formats can result. In the past, individual computer manufacturers usedtheir own format for their own products. A common standard floating point format hasbeen adopted by theInstitute of Electrical and Electronic Engineers(IEEE) which isusually referred to as theIEEE 754 standard. It defines the way zero is represented, thechoice ofM andE, the handling of overflows and other issues. For a 32-bit representa-tion, the single precision floating point number is defined as

X = (−1)S · 2E−127 (M) (6.55)

whereS is the sign bit,E occupies 8 bits, andM is 23 bits long in a format shown inFigure 6.15.

The following rules apply:

1. If E = 255 andM 6= 0, thenX is not a number (denotedNaN).

2. If E = 255 andM = 0, thenX is infinity (denotedInF).

3. If 0 < E < 255, thenX = (−1)S · 2E−126 (1.M).

4. If E = 0 andM 6= 0, thenX = (−1)S · 2126 (0.M).

5. If E = 0 andM = 0, thenX is zero.

Here(0.M) is a fraction and(1.M) is a number with one integer bit and 23 fractionalbits.


S

0 1

E

8 9 31

M

Bits

sign exponent mantissa

Figure 6.15: IEEE-754 Floating Point Format

Example 6.9. What is the decimal value of the following IEEE-754 formatted floatingpoint number?

0100000101010 · · · 00

Ans: 13.

Floating point representations can naturally represent a much larger range of numbersthan a fixed point one with the same number of bits. However, it should be noted that theresolution does not remain the same throughout this range. This means that the distancebetween two floating point numbers increases as the number is increased. On the otherhand, the resolution of fixed point numbers is constant throughout the range.

When two floating point numbers are multiplied, the mantissas are multiplied and theexponents are added. But if we want to add two floating point numbers, the exponentsof the two numbers must be equal. The one with the small exponent is adjusted byincreasing the exponent and reducing the mantissa. This adjustment could result in a lossin precision in the mantissa.

Overflow occurs in multiplication when the sum of the two exponents exceed the dynamicrange of the representation for the exponent.

6.4.3 Coefficient Quantization

Let us consider the low-pass linear phase FIR filter with a normalized cutoff frequency ofπ/4. The filter coefficients are quantized to 8 bits. The magnitude response of the filter,both before and after coefficient quantization, is shown in Figure 6.16. If the minimumstopband attenuation allowed is 48dB, then it is violated by the quantized filter. Clearlyin this case, more than 8 bits are required for the filter coefficients.


The minimum number of bits needed for the filter coefficients can be found by computingthe frequency response of the coefficient quantized filter. A trial-and-error approach canbe used. However, it will be useful to have some guideline for estimating the word-lengthrequirements of a specific filter.

The quantized coefficients and the unquantized ones are related by

hq [n] = h [n] + e [n] (6.56)

for n = 0, 1, . . . , N − 1, whereh [n] andhq [n] are the unquantized coefficients respec-tively ande [n] is the difference between them. This relationship can also be expressedin the frequency domain as

Hq (ω) = H (ω) + E (ω) (6.57)

HereH (ω) is the frequency response of the original filter andE (ω) is the error in thefrequency response due to coefficient quantization.

For IIR filters, the coefficient quantization error may have one more effect:instability.The stability of a filter depends on the location of the roots of the denominator polynomialin the transfer function. Consider a second order section of an IIR filter (since it is thebasic building block of higher order filters) with transfer function

If the direct form structure is used, assuming rounding, one of the following bounds forthe magnitude of the error spectrum can be used:

|E (ω)| = N2−B (6.58)

|E (ω)| = 2−B (N/3)1/2 (6.59)

|E (ω)| = 2−B

[1

3(N ln N)

]1/2

(6.60)

whereB is the number of bits used for representing the filter coefficients andN is thelength of the filter. (6.58) is a worst case absolute bound and is usually overly pessimistic.(6.59) and (6.60) are based on the assumption that the errore [n] is uniformly distributedwith zero mean. They generally provide better estimates for the word-length required.

Example 6.10.For a length-22 filter with magnitude response shown, at least how manybits are required if the minimum stopband attenuation is 50dB?


Figure 6.16: Magnitude Response of an FIR Filter Before and After Coefficient Quanti-zation


For IIR filters, the coefficient quantization error may have one more effect: instability.The stability of a filter depends on the location of the roots of the denominator polynomialin the transfer function. Consider a second order section of an IIR filter (since it is thebasic building block of higher order filters) with transfer function

H (z) =b0 + b1z

−1 + b2z−2

a0 + a1z−1 + a2z−2(6.61)

The roots of the denominator polynomial, orpolesof the transfer function, are located at

p1 =1

2

(−a1 +

√a2

1 − 4a2

)(6.62)

p2 =1

2

(−a1 −

√a2

1 − 4a2

)(6.63)

They may either be complex conjugate pairs or are both real. If they are complex conju-gate pairs, they can be represented as having a magnitude and an angle:

p1 = rejθ (6.64)

p2 = re−jθ (6.65)

where

r =√

a2 (6.66)

= arccos(−a1

2r

)(6.67)

For stability,r must be less than 1. This applies to both real and complex poles. So thetest for stability for the second coefficient is

0 ≤ |a2| < 1 (6.68)

The arguments to the arc-cosine function in (6.67) must have a magnitude that is less thanor equal to 1. So the test for stability for the first coefficient is

|a1| ≤ 2√

a2 (6.69)

Both (6.68) and (6.69) must be satisfied for the IIR filter to remain stable.


+

z −1

z −1

z −1

z −1

b

b

b

−a

−a

0

x[n] y[n] + f[n]

y[n−1]

y[n−2]

x[n−1]

x[n−2]

1

2 2

1

e[n]

Figure 6.17: Second Order IIR Filter Realized in Direct Form I with Round-off ErrorIncluded.

6.4.4 Round-off Noise

Round-off noise is introduced when products or sums of products are quantized. Our as-sumption will be that the rounding off is performed on the result of the (infinite precision)sum of product. Consider a second order IIR filter

y [n] =2∑

k=0

bkx [n− k] +2∑

k=1

aky [n− k] (6.70)

implemented in direct form I. Figure 6.17 shows the the addition of the round-off errormodelled bye [n] for this realization. We can view this as a two-input system. Theresponsey [n] is due to inputx [n] and the responsef [n] is due to inpute [n]. Sincee [n]is processed only by the right half of the filter

f [n] =2∑

k=1

akf [n− k] + e [n] (6.71)

If stationary random white noisee [n] with meanme and varianceσ2e is passed through a


z −1

+

z −1

+ y[n] + f[n]x[n]

0.25 0.5

e [n] e [n]21

Figure 6.18: Cascade Realization of Transfer Function in Example 6.11.

system with impulse responseh [n], then the output noisef [n] will have a mean

mf = me

∞∑n=−∞

h [n] = meH (0) (6.72)

whereH (0) is the DTFT ofh [n] atω = 0. The variance of the noise is given by

σ2f = σ2

e

∞∑n=−∞

|h [n]|2 (6.73)

or in the frequency domain

σ2f =

σ2e

2π

∫ π

−π

|H (ω)|2 dω (6.74)

Example 6.11.A filter with transfer function

H (z) =1

1− 0.75z−1 + 0.075z−2

is to be realized by a cascade of two first order sections. If the round-off noise varianceof the two adders are the same. Find the variance of the round-off noise observed at theoutput in terms of the round-off noise varianceσ2

e .

Solution:


The transfer function can be expressed as a cascade of two first order sections:

H (z) =1

1− 0.25z−1· 1

1− 0.5z−1= H1 (z) H2 (z)

Figure 6.18 shows the realization using canonical form.

Assuming that the quantization is performed after summation, there are two sources oferrors, each introduced by the two adders. The first error signale1 [n] passes through thewhole systemH (z) while the second onee2 [n] only pass throughH2 (z).

H (z) =1

1− 0.25z−1· 1

1− 0.5z−1

=2

1− 0.5z−1− 1

1− 0.25z−1

So the impulse response is given by

h [n] = 2

(1

2

)n

u [n]−(

1

4

)n

u [n]

through an inverse Z transform.

Similarly,

h2 [n] = 2

(1

2

)n

u [n]

The output noise isf [n] = e1 [n] ? h [n] + e2 [n] ? h2 [n]

The variance of this noise is

σ2f = σ2

e

∑n

|h [n]|2 + σ2e

∑n

|h2 [n]|2

whereσ2e is the variance of the noise signals and is assumed to be the same for bothe1 [n]

ande2 [n]. Now,

|h [n]|2 =

[2

(1

2

)n

−(

1

4

)n]2

= 4

(1

2

)2n

+

(1

4

)2n

− 4

(1

8

)n


Using the fact that∞∑

n=0

an =1

1− a

for a < 0, we have

∑n

|h [n]|2 =4

1− (0.5)2 +1

1− (1/4)2 −4

1− (1/8)

= 1.8286

Similarly, ∑n

|h2 [n]|2 =2

1− (1/2)2 = 2.6667

Soσ2

f = 1.8286σ2e + 2.6667σ2

e = 4.4953σ2e

6.4.5 Limit Cycles

Although we have been treating digital filters as linear systems, the fact is that a digitalfilter becomes nonlinear when we include quantization, round-off, truncation, and over-flow. A filter, designed as a linear system, that is stable may oscillate when an overflowoccurs. This type of oscillation is called alimit cycle. Limit cycles due to round-off,truncation and overflow are illustrated by two examples.

Example 6.12.A first order IIR filter has the following different equation:

y [n] = x [n] + αy [n− 1] (6.75)

For α = 0.75, the output samplesy [n] obtained using initial conditiony [0] = 6 and zeroinputx [n] = 0 for n ≥ 0 are listed in Table 6.2. It shows that the output quickly decaysto zero. Ify [n] is rounded to the nearest integer, then after some time the output remainsat 2. The output will now exhibit a DC offset.

For α = −0.75, the output oscillates briefly and decays to zero if computations aredone in infinite precision. If the output is rounded to the nearest integer, then the outputoscillates between−2 and+2. These quantities are listed in Table .


n y [n], infinite precision y [n], rounded to nearest integer

0 6 61 4.5 52 3.38 43 2.53 34 1.90 25 1.42 26 1.07 27 0.80 28 0.60 29 0.45 210 0.3375 2

Table 6.2: DC Offset Behaviour of Output Rounding

n y [n], infinite precision y [n], rounded to nearest integer

0 6 61 -4.5 -52 3.38 43 -2.53 -34 1.90 25 -1.42 -26 1.07 27 -0.80 -28 0.60 29 -0.45 -210 0.3375 2

Table 6.3: Oscillating Behaviour of Output Rounding


Example 6.13.A stable filter with transfer function

H (z) =1

z2 − z + 0.5

is implemented as shown in Figure. Note that we have added an overflow non-linearityafter the adder in the filter structure. The non-linearity is to model any possible 2’scomplement overflow and the nonlinear function is shown in Figure .

If the input is zero, we have

x1 [n + 1] = x2 [n] (6.76)

x2 [n + 1] = NL (−0.5x1 [n] + x2 [n]) (6.77)

If the initial conditions are

x1 [0] = 0.8

x2 [0] = −0.8

Then we have

x1 [1] = −0.8 (6.78)

x2 [n] = NL (−0.5x1 [0] + x2 [0])

= NL (−1.2)

= 0.8 (6.79)

In fact, forn ≥ 1,

x1 [n] = (−1)n 0.8 (6.80)

x2 [n] = (−1)n+1 0.8 (6.81)

Thus the output oscillates between−0.8 and0.8.

Note that the limit cycle will only start if there is a previous overflow. If no overflowoccurs, the system remains linear.

6.4.6 Arithmetic Overflow

Overflow occurs when the two large number of the same sign are added and the resultexceeds the word-length. If we use 2’s complement representation, as long as the final


z −1 z −1x [n]1 x [n]2

x[n]

+

+ NL y[n]

−0.51

Figure 6.19: Filter Structure with Overflow Nonlinearity.

result is within the word-length, overflow of partial results is unimportant. If the finalresult does cause overflow, it may lead to serious errors in the system. Overflow can beavoided by detecting and correcting the error when it occurs. However, this is a ratherexpensive approach. A better way is to try to avoid it by scaling the data or the filtercoefficients.

Overflow can potentially cause an IIR system to become unstable. It may also lead tolimit cycles similar to round-off errors.

Consider a filter with transfer function

H (z) =1

z3 − z + 0.5(6.82)

It can easily be verified that it is a stable system. A realization of this filter is shown inFigure 6.19. Here we have added a nonlinear functionNL (·) to the realization to modelany potential 2’s complement overflow that may occur as a result of the addition. Thisnonlinear function is shown in Figure 6.20.

Consider the case where the input is zero with the initial conditions

x1 [0] = 0.8 (6.83)

x2 [0] = −0.8 (6.84)

Then from Figure 6.19 we have, forn ≥ 0,

x1 [n + 1] = x2 [n] (6.85)

x2 [n + 1] = NL (−0.5x1 [n] + x2 [n]) (6.86)

Summary 169

1

−1

x

NL(x)

−1−3 1 3

Figure 6.20: 2’complement Overflow Modelled as a Nonlinear Function.

So

x1 [1] = −0.8 (6.87)

x2 [1] = 0.8 (6.88)

and in general, forn ≥ 1,

x1 [n] = (−1)n 0.8 (6.89)

x2 [n] = (−1)n+1 0.8 (6.90)

Thus the filter is stuck in a limit cycle.

6.5 Summary

The basic IIR filter equation (which includes the FIR equation as a special case) lookssimple. However, there are a number of ways by which it can be realized in both hardwareand software. Each realization has its chacteristics. The basic forms are the direct formsI and II and their transposed counterparts. A higher order filter is usually broken up intofirst and second order sub-systems organized in a cascade or parallel form. In digitalsignal processors, apart from optimizing the MAC operation, the use of circular buffers


also has a significant impact on filtering. We have also looked at a number of practicalimplementation issues including finite wordlength effects on the filter frequency responseand the operational issues like overflow. Whether the computations are implemented inVLSI/ASIC/FPGA or in software running on digital signal processors or general purposeprocessors, the issues are still relevant.

In the following chapters, we shall be concerned with the design of FIR and IIR filters.

Chapter 7

FIR Filter Design

Finite Impulse Response (FIR) Filters are digital filters with impulse responses that van-ish after a finite number of samples. The output of the difference equation of an FIR filterdepends only on a finite number of past and present inputs.

The filter design problem is to determine the filter coefficients that produce a filter transferfunction which meets the prescribed frequency response specifications. For FIR filters,this is equivalent to designing the finite impulse response sequence of the filter.

The main advantage of FIR filters is that it can have exact linear phase. We shall assumethat only linear phase filters are required.

7.1 Design Process

The general digital filter design process can be broken up into four main steps:

1. Approximation

2. Synthesis and realization

3. Performance analysis

4. Implementation

The process is illustrated in Figure 7.1.

172 FIR Filter Design

FilterSpecifications

Approximation

Synthesisand

RealizationChoose

alternativestructure

if necessary

Filter implementedas hardware and/or

software

Filter TransferFunction

StabilityQuantization error

AnalysisPerformance

Implementation

StructuresFilter

Figure 7.1: Digital Filter Design Process

Design Process 173

Specifications The design process normally starts with the specifications and require-ments of the filter which are intimately related to the application at hand. These spec-ifications may include frequency domain characteristics such as magnitude and phaseresponses. There may also be some time domain requirements such as maximum delay.

Most specifications define the upper and lower limits to each of these characteristicseither in absolute terms or in decibels. They are illustrated in Figure 7.2. Alternatively,a desired or ideal response may be given with the maximum amount of deviations fromthe ideal specified.

Passband ripple in dB is given by

Rp = 20 log10

1− δp

1 + δp

(7.1)

and the stopband attenuation is given by

As = 20 log10

δs

1 + δp

(7.2)

Approximation Given the filter specifications, the first step of the design process is tofind a filter transfer function that will satisfy these specifications. This process is calledapproximation. It is so called because what we are doing is in fact finding a transferfunction that approximates the ideal response that is specified.

The methods for solving the approximation problem for digital filters can be classifiedas direct or indirect. With direct methods, the problem is solved in the discrete-time(and hence discrete-frequency) domain. For indirect methods, a continuous-time trans-fer function is first obtained using well-established methods in analog filter design. Thistransfer function is then transformed into a discrete-time transfer function. Indirect meth-ods are more commonly used for IIR filters whereas FIR filter design methods are mostlydirect ones.

These solution methods can also be classified as closed-form or iterative. Closed formmethods make use of closed-form formulas and are usually completed in a definite num-ber of steps. Iterative methods make use of optimization techniques that starts with aninitial solution and the solution is refined progressively until some pre-determined per-formance criteria are satisfied. The number of iterations are unknown and depends on theinitial solution and the effectiveness of the optimization techniques employed.

The following methods will be discussed in more detail in Sections 7.2 and 7.5:


1+δ p

1−δ p

ω p ωs

δ s

|H( )|

passband rippleallowed

bandtransition

minimumstopbandattenuation

ω00 π

ω

(a) Specifications in Absolute Values

ω p ωs

|H( )| (dB)ωpassband ripple

allowed

bandtransition

minimumstopbandattenuation

ω0 π

0

s

Rp

A

(b) Specficiations in dB

Figure 7.2: Filter Specifications in Frequency Domain.

Design Process 175

1. Window method.

2. Frequency sampling method.

3. Parks-McClelland (Remez Exchange) algorithm.

Realization Once the transfer function has been determined, it has to be realized insome form. This procedure is analogous to the filter realization procedure for analogfilters where a suitable circuit topology and circuit element values are chosen to realizea certain filter transfer function. A number of realization methods has been discussedin Chapter 6. The best realization of a given transfer function depends very much onthe application. General considerations include the number of adders and multipliersrequired, and the sensitivity of the network to finite precision arithmetic effects.

Performance Analysis Even though the filter coefficients are determined to a highdegree of precision in the approximation step, digital hardware has a finite precision. Theaccuracy of the output will depend on the type of arithmetic used: fixed-point or floating-point. This is particularly so for fixed point arithmetics. The designer must ensure that theerror introduced by finite precision will not cause violations of the filter specifications.Furthermore, arithmetic overflow and underflow effects must be examined.

It cannot be over-emphasized how important this design step is, especially for IIR filters.While FIR filters are guaranteed to be stable, IIR filters can exhibit instability due toquantization errors introduced in the computational process.

Implementation Digital filters can be implemented either in software or hardware or acombination of both. Software implementations require a decision to be made on the typeof computer or microprocessor the software will eventually run on. DSP chips which aredesigned specifically for DSP type of operations are very effective.

In very demanding applications, the filter may need to be hard-wired or implemented asan application specific integrated circuit (ASIC) in order to obtain the speed required.It may also be necessary that some of the other functions such as analog-to-digital con-version and digital-to-analog conversion be integrated on the same device. However,development time will generally be longer and the cost is much higher.


7.2 Window Method

The window method is suitable for designing filters with standard low-pass, bandpassand high-pass responses. The method outline is as follows:

1. Choose an ideal filter response.

2. Calculate its (infinite) impulse response using inverse discrete-time Fourier Trans-form.

3. Multiply the impulse response with a finite-length window sequence.

4. If a causal filter is required, time shift this finite-length impulse response so that allthe non-zero values are in the non-negative part of the time axis.

Consider an ideal lowpass filter with cutoff frequencyωc. The frequency response of thisideal filter is

Hd(ω) =

e−jωα, |ω| ≤ ωc

0, ωc < |ω| < π(7.3)

Notice that we have introduced an arbitrary phase shift in the frequency response. Usinginverse DTFT, we obtain the impulse reponse of this ideal filter:

hd [n] =sin [ωc (n− α)]

π (n− α)(7.4)

This impulse response has infinite length. In order to obtain a filter with lengthN , hd [n]is windowed (or multiplied) by a window of lengthN . The impulse responseh [n] of thefilter is zero forn < 0 andn ≥ N .

Some of the window functions that can be used are essentially the same as those used forspectral analysis. The rectangular and Hamming windows have been discussed in Section4.3. Other ones include the Hanning window, Blackman window and Kaiser window.

The phase shiftα in the frequency response corresponds to a delay in the impulse re-sponse. This is useful if we want to design filters that are causal.

Window Method 177

7.2.1 Rectangular Window

The second step in the approximation process using the windowing method is to windowthe infinitely long ideal impulse reponse with a finite length window. We shall startwith the rectangular window due to its simplicity. The following example illustrates theprocess.

Example 7.1. Design a 10-th order linear phase FIR filter with cutoff frequency ofπ/4radians using a rectangular window.

Solution:

In this case,N = 10 and so

α =N

2= 5

The windowed impulse response is given by (7.4) and

hd [n] =sin ωc (n− α)

π (n− α)0 ≤ n ≤ N

=sin (n− 5) π

4

π (n− 5)

=

−√

2

10π, 0,

√2

6π,

1

2π,

√2

2π,1

4,

√2

2π,

1

2π,

√2

6π, 0,−

√2

10π

How good is the design using rectangular windows? In other words, how closely doesthe resulting FIR filter frequency responseH (ω) approximate the original ideal responseHd (ω)?

To answer this question, we performed the DTFT ofhd [n] obtained in Example 7.1 andplotted the magnitude and phase responses in Figure 7.3. Note that we indeed have exactlinear phase for this filter since the filter coefficients are symmetric. Ripples exist in thestopand and the minimum stopband attenuation is about 20dB.

Figure 7.4 shows the FIR filter magnitude responses forN = 51 andN = 101. No-tice that there are now some ripples in the passband also. One would expect that asNincreases, the approximation will become better. This is indeed the case except for theregion close to the transition between passband and stopband. In this region the Gibb’sphenomenon discussed in Section 3.3.1 manifests itself. For this reason, the approxima-tion at the band edge will always be poor for the rectangular window design regardlessof how largeN is.


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−350

−300

−250

−200

−150

−100

−50

0

Normalized Frequency (×π rad/sample)

Pha

se (d

egre

es)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−70

−60

−50

−40

−30

−20

−10

0


Mag

nitu

de (d

B)

10th Order FIR Filter Response

Figure 7.3: FIR Filter Response forN = 11.

We can now summarize the characteristics of using the rectangular window:

• The ripple size decreases with increasing filter order N . Approximation well withinthe passband and stopband becomes better as the filter order increases.

• The transition width decreases with increasing filter order. For any order N , thefilter response is always equal to 0.5 at the cutoff frequency.

• Ripple size near the passband to stopband transition remains roughly the same asN increases. The maximum ripple size is about 9%. This is known as the Gibb’sphenomenon.

Using the Hamming window will provided better passband responses, especially at theband edges. Other window functions that can be used include

Window Method 179

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1400

−1200

−1000

−800

−600

−400

−200

0


Pha

se (d

egre

es)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−100

−80

−60

−40

−20

0

20


Mag

nitu

de (d

B)


(a) 50th Order FIR Filter Responses

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−2500

−2000

−1500

−1000

−500

0


Pha

se (d

egre

es)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−100

−80

−60

−40

−20

0

20


Mag

nitu

de (d

B)


(b) 100th Order Filter Responses

Figure 7.4: FIR Filter Responses forN = 51 andN = 101.


• Hanning window

w [n] =

0.5[1− cos

(2πnN−1

)], 0 ≤ n ≤ N − 1

0, otherwise(7.5)

• Blackman window

w

[n =

0.42− 0.5 cos

(2πnN−1

)+ 0.08 cos

(4πnN−1

), 0 ≤ n ≤ N − 1

0, otherwise

](7.6)

These two windows are variants of the Hamming window; they both possess the “raisedcosine” shape. The transition band width and the stopband attenuation of the resultingfilter is largely dependent on the window that is used.

In order to design filters with different parameters, a “unified” window function wouldbe useful. The Kaiser window is one such window function.

7.2.2 Kaiser Window

The Kaiser window is the most commonly used window when it comes to designingFIR filters using the window method. It is actually a class of window function that isparametrized by a variableβ. It is defined as

x [n] =

I0

(β√

1−(1− 2n

N−1

)2)I0 (β)

0 ≤ n ≤ N − 1 (7.7)

whereI0 (·) is the modified zero-order Bessel function defined by the series expansion

I0 (x) = 1 +∞∑

k=1

[(x2

)kk!

]2

(7.8)

This window is optimal in the sense that it provides the sharpest transition band widthwith the largest main lobe.

β can be chosen to give various transition widths and stopband attenuation. For example,if β = 6.658, then the transition width is7.8π/N and the minimum stopband attenuationis about 60 dB. When we setβ = 4.538, the transition width becomes5.8π/N with a

Highpass Filters 181

minimum stopband attenuation of around 50 dB. One accepted way of determiningβ isto use one of the following formulas, depending on the stopband attenuationAs:

β =

0, As ≤ 21

0.5842 (As − 21)0.4 + 0.07886 (As − 21) , 21 < As ≤ 500.1102 (As − 8.7) As > 50

(7.9)

The filter order can be determined by

N − 1 ≥ ω0D

(ωs − ωp)(7.10)

whereωp andωs are the passband and stopband edge frequencies andω0 is the samplingfrequency. The parameterD is given by

D =

0.9222, As ≤ 21(As−7.95)

14.36, As > 21

(7.11)

7.3 Highpass Filters

High-pass filter design using Kaiser windows is very similar to low-pass filter design.The only change in the steps is simply define

∆f = fpass − fstop (7.12)

sincefpass > fstop for high-pass filters.

The ideal high-pass impulse response is obtained from the inverse DTFT of the idealhigh-pass frequency response. It is given by

hd [k] = δ [k]− sin (ωck)

πk(7.13)

The windowed filter impulse response is therefore

h [n] = w [n]

(δ [n−M ]− sin ((n−M) ωc)

(n−M) π

)= δ [n−M ]− w

[n · sin ((n−M) ωc)

(n−M) π

](7.14)


+

H (z)LP

z−M

outputhighpass

outputlowpass

Figure 7.5: Highpass Filter Implemented by a Lowpass Filter and Delay.

The second term on the right hand side of this equation is the impulse response of thelow-pass filter with the same cutoff frequency. Note that with the same value ofωc, thelow-pass and high-pass filters are complementary. That is,

hLP [n] + hHP [n] = δ [n−M ] n = 0, 1, . . . , N − 1 (7.15)

In terms of Z-transform, (7.15) becomes

HLP (z) + HHP (z) = z−M (7.16)

In other words, we can obtain a high-pass filter by designing a low-pass filter with thesame cutoff frequency and using (7.15) above. Alternatively, the high-pass filter can beimplemented using a low-pass filter and a delay as shown in Figure .

7.4 Bandpass Filters

Bandpass filters can also be designed via low-pass filters. The ideal bandpass filter fre-quency response is given by

HBP (ω) =

1, ω2p ≤ |ω| ≤ ω1p

0, otherwise(7.17)

This response can be expressed as the difference between two low-pass responses

HBP (ω) = HLP1 (ω)−HLP2 (ω) (7.18)

Frequency Sampling Method 183

whereHLP1 (ω) is a low-pass response with cutoff frequencyω1p andHLP2 (ω) is onewith cutoff frequencyω2p. These low-pass filters have impulse responses

hLP1 [n] =

sin ω1pn

πn, n 6= 0

ω1p

π, n = 0

(7.19)

hLP2 [n] =

sin ω2pn

πn, n 6= 0

ω2p

π, n = 0

(7.20)

The bandpass filter impulse response is therefore given by

hBP [n] =

sin ω2pn−sin ω1pn

πn, n 6= 0

ω2p−ω1p

π, n = 0

(7.21)

7.5 Frequency Sampling Method

Another relatively simple FIR filter design method is the frequency sampling method.While the windowing method is only suitable for designing filters with simple standardfrequency responses like low-pass, high-pass and band-pass, the frequency samplingmethod can handle arbitrary frequency responses. The basic idea is that the finite im-pulse response of an FIR filter can be obtained from the DFT of its frequency response.So if we sample the frequency response atN regularly spaced frequency points (between−π to π), then we can use these frequency samples to calculate the impulse response.

If samples of the desired frequency response is given asHd [2πk/N ], then the filter coef-ficientsh [n] can be obtained by solving the followingN simultaneous equations:

N−1∑n=0

h [n] e−j2πnk/N = Hd

[2πk

N

](7.22)

The coefficients can also be computed via the IDFT which is straight forward.

The main problem of the frequency sampling method lies in the determination ofHd [2πk/N ]since they are complex-valued in general. Let

Hd

[2πk

N

]= A [k] ejΘ[k] (7.23)

whereA [k] andΘ [k] are samples of the amplitude and phase response respectively. If wewant the filter to have linear phase, then we need to have symmetrical or anti-symmetrich [n].


For symmetricalh [n] and the order of the filterN odd, we require

A [k] = A [N − k] 1 ≤ k ≤ N − 1

2

Θ [k] = −πk (N − 1)

N0 ≤ k < N

In this case, the impulse response is given by

h [n] =2

N − 1

A [0] + 2

(N−1)/2∑k=1

(−1)k A [k] cosπk (1 + 2n)

N

(7.24)

for 0≤ n ≤ N − 1.

If N is even, then

A

[N

2

]= 0 (7.25)

A [k] = A [N − k] 1 ≤ k ≤ N

2(7.26)

Θ [k] =

−πk(N−1)

N, 0 ≤ k ≤ N

2

π − πk(N−1)N

, N2

< k < N(7.27)

The impulse response is

h [n] =2

N − 1

A [0] + 2

N2−1∑

k=1

(−1)k A [k] cosπk (1 + 2n)

N

(7.28)

for 0≤ n ≤ N − 1.

While the frequency sampling method is more flexible compared to the windowing method,care should be taken with the transition region to make sure that there are enough fre-quency samples to constrain the transition band behaviour.

7.6 Summary

The FIR digital filter design process is outlined in this chapter. Two design (or approxi-mation) methods are described: windowing method and frequency sampling method. The

Summary 185

windowing method is suitable for designing “standard” low-pass, high-pass and bandpassfilters. The frequency sampling method is more flexible in that any arbitrary frequencyresponse could be specified. However, care must be taken to ensure that the transitionband behaviour conforms to what is expected.

The interested reader is encourage to consult the vast number of books available on digitalfilter design on other design methods.

Chapter 8

IIR Filter Design

While FIR filters have impulse responses that vanish after a finite number of samples.Infinite Impulse Response (IIR) Filters are ones with the output sample depending onpast input and past output samples.

y[n] = −N∑

k=1

a[k]y[n− k] +M∑

k=0

b[k]x[n− k] (8.1)

The first term on the right-hand-side of the difference equation (8.1) is the feedback ofpast outputs to the current output. It is this recursive part of the equation that causes theimpulse response to be infintely long. For this reason, IIR filters are also calledrecursivefilters.

Since the IIR filter is a linear system, the output and input are related by the convolutionsum

y [n] =∞∑

k=0

h [k] x [n− k] (8.2)

assuming a causal filter. What makes it different from FIR filters is that the upper limitof the summation is infinity because the impulse responseh [n] has infinite duration.Obviously computation using (8.2) is impractical. The recursive relationship defined in(8.1) is much more efficient and practical. It requiresN + M + 1 multiplications andN + M additions if implemented directly.

Since the impulse response is infinitely long, we cannot adapt the techniques for FIRfilter design to IIR filters. The most common techniques involve first designing an analogfilter with equivalent specifications.

188 IIR Filter Design

Specifications

Digital Filter

Specifications

Analog Filter

Filter

Analog

Filter

Digital IIR Transformation

Conversion

Design Methods

Analog Filter

Figure 8.1: IIR Filter Design Approach

8.1 Design Approach

One of the most efficient ways of designing IIR digital filters is through a correspondinganalog filter. The process is illustrated in Figure 8.1. First, the digital filter specificationsare translated to analog filter specifications. Then an analog filter is designed accord-ing to those specifications. The transfer function of the resulting analog filter is thentransformed into an equivalent digital filter transfer function.

We have a wealth of knowledge about analog filter approximations which we can makeuse of. The only step that requires some thought is the transformation from an analogfilter to a digital one. In this book, only two methods are discussed. They are the impulseinvariance method and the bilinear transformation method.

8.2 Some Classic Analog Filters

We shall briefly review two types of classic analog filters: Butterworth and Chebyshev.The interested reader is encouraged to consult textbooks on other types of analog filters.

8.2.1 Butterworth Filters

Butterworth filters have magnitude squared frequency response given by

Some Classic Analog Filters 189

|H (ω)|2 =1

1 + (ω/ω0)2n (8.3)

wheren (an integer) is the order of the filter andω0 is some constant. Designing a But-terworth filter is equivalent to computing the values ofn andω0 for a given specification.

Consider a low-pass Butterworth filter with passband edge atωp and stopband edge atωs.Suppose the passband rippleKp and stopband attenuationKs are given in dB, then forω = ωs

|H (ωs)|2 =1

1 + (ωs/ω0)2n (8.4)

and so

Ks = −10 log10 |H (ωs)|2 = 10 log10

[1 +

(ωs

ω0

)2n]

(8.5)

which gives

Ks

10= log10

[1 +

(ωs

ω0

)2n]

(8.6)

⇒(

ωs

ω0

)2n

= 100.1Ks − 1 (8.7)

Similarly for theω = ωp,

Kp = −10 log10 |H (ωp)|2 (8.8)

= 10 log10

[1 +

(ωp

ω0

)2n]

(8.9)

and therefore

Kp

10= log10

[1 +

(ωp

ω0

)2n]

(8.10)

⇒(

ωp

ω0

)2n

= 100.1Kp − 1 (8.11)


Combining (8.7) and (8.11), we obtain(ωs

ωp

)2n

=100.1Ks − 1

10−0.1Kp − 1(8.12)

⇒ 2n log

(ωs

ωp

)= log

(100.1Ks − 1

100.1Kp − 1

)(8.13)

⇒ n =log M

log Ω(8.14)

where

Ω =ωs

ωp

(8.15)

M =

√100.1Ks − 1

100.1Kp − 1(8.16)

The order of the filter is the integer closest to but higher thann obtained from (8.14).

Example 8.1. Determine the order of a Butterworth filter that meets the following spec-ifications:

ωp = 1 rad/s

ωs = 1.5 rad/s

Ks = 30dB

Kp = 1dB

Solution:

Ω = 1.5

M =

√103 − 1

100.1 − 1= 793.534

n ≥ log10 793.534

log10 1.5= 16.466

So the minimum order of the Butterworth filter that satisfies these specifications is17.Figure 8.2 shows the magnitude response of a 17-th order Butterworth filter.

Once the order of the filter is determined, how do we obtain the transfer functionH (s)of the filter? Notice that givenn, only |H (ω)|2 is known.


0 0.5 1 1.5 2 2.5 3 3.5 4−200

−150

−100

−50

0

50

Angular Frequency (normalized)

Mag

nitu

de (d

B)

Magnitude Response of a Normalized 17−th Order Butterworth Filter

Figure 8.2: Magnitude Response of a 17-th Order Butterworth Filter

However, we know that the poles of|H (ω)|2 are located in the same place asH (s) H (−s).So

H (s) H (−s) =1

1 +(

ωω0

)2n

∣∣∣∣∣∣∣(ωω0

)2=−s2

(8.17)

=1

1 + (−1)n s2n(8.18)

ands = [− (−1)n]

1/2n (8.19)

They are evenly spread out along the unit circle. The angles of the poles will be the sameon thes-plane forH (s) H (−s) while the distance from the origin is scaled byω0. Forthe filter to be a stable system, all its poles have to be on the left hand side of thes-plane.ThereforeH (s) will only consists of those poles on the left hand side of the imaginaryaxis. Thus it can be computed.


Example 8.2.Design a Butterworth low-pass filter with the following specifications:

ωp = 1 rad/s

ωs = 2 rad/s

Ks = 20dB

Kp = 1dB

Solution:

Ω = 2

M =

√102 − 1

100.1 − 1= 19.554

n ≥ log10 19.5354

log10 2= 4.289

So we need a 5-th order filter. Atω = ωp(ωp

ω0

)10

= 100.1 − 1 (8.20)

according to (8.11) and

10 (log ωp − log ω0) = log 0.2589

⇒ log ω0 = log 1− 0.1 log 0.2589 = 0.05868

⇒ ω0 = 1.145 (8.21)

The poles of the normalized Butterworth filter are located as shown in Figure 8.3.

8.2.2 Chebyshev Filters

The Butterworth filter, while possessing some desirable properties, does not provide asufficiently good approximation near the passband edge. The roll-off from passband tostopband is also relatively gradual. Hence for filters that have a narrow transition band,a very high order Butterworth filter is required. If the application can tolerate some rip-ples in the passband, the Chebyshev approximation will overcome some of the problems


−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1 poles

Figure 8.3: Pole Locations forH(s)H(-s) of a Normalized 5th Order Butterworth Filter.


associated with Butterworth filters stated above. The magnitude squared response ofChebyshev filters has the form

|H (ω)|2 =1

1 + ε2C2N (ω)

(8.22)

whereCN (ω) is theN -th order Chebyshev polynomial andε is a parameter associatedwith the size of the ripples. The Chebyshev polynomials can be obtained recursively:

C1 (ω) = ω (8.23)

C2 (ω) = 2ω2 − 1 (8.24)

Cn+1 (ω) = 2ωCn (ω)− Cn−1 (ω) (8.25)

These polynomials have the following properties

1. 0 ≤ C2n (ω) ≤ 1 for 0 ≤ ω ≤ 1.

2. C2n (ω) ≥ 1 for ω ≥ 1.

That is, they oscillate between−1 and+1 for −1 < ω < 1 and increase monotoni-cally outside this interval. So the low-pass Chebyshev filter response has a normalizedpassband from0 to 1 rad/s. Figure 8.4 shows the magnitude response of a 7-th orderChebyshev filter.

Designing a Chebyshev filter involves determining both the orderN and the parameterε.Generally the order influences the transition width and the number of oscillations withinthe passband andε influences the size of the ripples in the passband.

Within the normalized passband,

1

1 + ε2≤ |H (ω)|2 ≤ 1 (8.26)

Thus the specified maximum allowable passband rippleKp of the filter is related toε by

Kp = 10 log10

(1 + ε2

)(8.27)

orε =

√100.1Kp − 1 (8.28)


0 0.5 1 1.5 2 2.5−80

−70

−60

−50

−40

−30

−20

−10

0

10Magnitude Response of a 7−th Order Chebyshev Filter

Angular Frequency (normalized)

Mag

nitu

de (d

B)

Figure 8.4: Magnitude Response of a 7-th Order Chebyshev Filter with 2dB PassbandRipple.

Without going into the details of the derivation, the order of the filter given a specifiedminimum stopband attenuation ofKs dB with passband edge atωp and stopband edge atωs is given by

N =cosh−1 M

cosh−1 Ω(8.29)

where

Ω =ωs

ωp

(8.30)

M =

√100.1Ks − 1

100.1Kp − 1(8.31)

similar to that for Butterworth filters.


8.3 Bilinear Transformation

Once the analog filter has been designed (i.e.H (s) has been determined), we need tomap it to the digital domain to obtainH (z). Bilinear transformation is one such methodfor converting the analog filter transfer functionH(s) into a corresponding digital oneH(z). The transformation is performed by a change of variables

s =1 + z−1

1− z−1(8.32)

It is called bilinear because both the numerator and denominator of this transformationequation are linear. This transformation is reversible in thatH (s) can be obtained fromH(z) by the substitution

z =1 + s

1− s(8.33)

Through the bilinear transformation, the imaginary axis of thes-plane is mapped to theunit circle of thez-plane. This can be shown by lettings = σ + jΩ and so through (8.33)we get

z =1 + σ + jω

1− σ − jΩ(8.34)

If σ = 0, i.e. s = jΩ (the imaginary axis of thes-plane), then

z =1 + jΩ

1− jΩ(8.35)

Obviously the magnitude of the numerator and denominator are the same and so

|z| = 1 (8.36)

which is the unit circle on thez-plane.

If we considerσ < 0 in (8.34), then(1− σ) > (1 + σ) and |z| < 1. So the bilineartransformation also maps the left hand side of thes-plane to the inside of the unit circleon thez-plane.

8.3.1 Frequency Warping

In order to understand the effects of this transformation, letΩ andω be the frequencyvariables in the analog (s) and digital (z) domains respectively. The frequency response

Bilinear Transformation 197

of the digital filter can be obtained fromH(z) by the substitution

z = ejωt (8.37)

and that of the analog filter by substituting

s = jω (8.38)

into H (s). Using the above substitutions in (8.32), we have

jΩ =1− e−jω

1 + e−jω

=e−jω/2

(ejω/2 − e−jω/2

)e−jω/2 (ejω/2 + e−jω/2)

= j ·12j

(ejω/2 − e−jω/2

)12(ejω/2 + e−jω/2)

(8.39)

which can be simplified to

Ω =sin (ω/2)

cos (ω/2)

= tan(ω

2

)(8.40)

This relationship is plotted in Figure 8.5. It shows three bands of frequencies in theanalog domain that are of equal width. After applying the bilinear transformation, thecorresponding bands of frequencies in the digital domain are not of the same width.This nonlinear relationship between the analog and digital frequency variables lead to adistortion orwarpingof the digital frequency response. This means that the passband ofan analog filter will not be the same as that of the digital filter obtained through bilineartransformation.

Given the passband specifications of the digital filter, the passband frequencies of the“correct” analog filter is obtained bypre-warping. Supposefc is a critical frequency (e.g.cutoff frequency, passband edge, stopband edge, etc.), then the corresponding pre-warpedanalog frequency is obtained by

Ωc = tan

(πfc

fs

)(8.41)

wherefs is the sample frequency of the digital filter.

Let us illustrate the design process through an example.


UnequalWidth

AnalogFrequency

Digital Frequency

Ω

ω

EqualWidth

Figure 8.5: Frequency Warping in Bilinear Transformation

Example 8.3.Design a low-pass IIR digital filter with the following specifications:

0.89125 ≤ |Hω| ≤ 1, 0 ≤ |ω| ≤ 0.2π

|H (ω)| ≤ 0.17783, 0.3π ≤ |ω| ≤ π

Characteristics of the bilinear transformation are summarized below:

1. Provided that the analog filter is stable, the resulting digital filter is guaranteed tobe stable.

2. The order of the digital filter is the same as the prototype analog filter.

3. Optimal approximations to the analog filter transform into optimal digital filters.


4. The cascade of sections designed by the bilinear transformation is the same as thatobtained by transforming the total system.

8.3.2 Frequency Transformation

So far we have only considered low-pass filter designs. Designing high-pass and band-pass filters requires a transformation to be performed on the low-pass filter. Three ap-proaches can be used for IIR digital filters.

1. In the first approach, the analog prototype filter is transformed to be appropriatehigh-pass or band-pass analog filter. Then this analog filter is turned into an IIRfilter by bilinear transformation. The low-pass to high-pass transformation is givenby

s → 1

s(8.42)

and the low-pass ot band-pass transformation is

s → s2 + ωuωl

s (ωu − ωl)(8.43)

where the low-pass filter has a cutoff frequency of 1 rad/s andωu andωl are theupper and lower passband edges of the bandpass filter respectively.

2. Alternatively, the analog low-pass filter is first converted to a digital low-pass filter.The digital high-pass or band-pass filter is then obtained by a spectral transforma-tion of the digital low-pass filter. This method is less commonly used.

3. The last approach is basically a combination of the two previous methods. Newbilinear transform equations can be obtained that directly maps an analog low-passfilter to a digital high-pass, bandpass or band-reject filters. The bilinear transformfunction for low-pass to high-pass is

s =1 + z−1

1− z−1(8.44)

and the low-pass to bandpass transform is

s =1− 2cz−1 + z−2

1− z−2(8.45)

wherec is a certain constant.


Example 8.4. Design a high-pass IIR filter operating at a rate of 10 kHz. The stopbandand passband edges are 2 kHz and 4 kHz respectively. The maximum passband attenua-tion is 0.5 dB and the minimum stopband attenuation is 10 dB. Use Butterworth filter asthe analog prototype filter and the low-pass to high-pass bilinear transformation

s =1 + z−1

1− z−1

Solution:

First establish the frequency warping function between the analog and digital frequen-cies. Substitutings = jΩ andz = ejω into the above bilinear function, we have

jΩ =1 + e−jω

1− e−jω

=1

j·

12e−jω/2

(ejω/2 + e−jω/2

)12j

e−jω/2 (ejω/2 − e−jω/2)

=1

j· cos (ω/2)

sin (ω/2)

⇒ Ω = − cot(ω

2

)(8.46)

Now,

fs = 10kHz

fpass = 4kHz ⇒ ωp =2π (4)

10= 0.8π

fstop = 2kHz ⇒ ωs =2π (2)

10= 0.4π

Using (8.46) we can obtain the corresponding analog band edge frequencies

Ωp = − cot

(0.8π

2

)= −0.3249

Ωs = − cot

(0.4π

2

)= −1.3764

Since the magnitude spectrum of the filter is symmetrical aboutf = 0, we do not needto worry about the negative signs. Hence the passband edge frequency of the analog(low-pass) filter is 0.3249 radians and the stopband edge frequency is 1.3764 radians.


The Butterworth filter response is given by

|H (Ω)|2 =1

1 + (Ω/Ω0)2n (8.47)

For Ω = Ωp,|H (Ωp)|2 = 10−0.5/10 = 10−0.05

and forΩ = Ωs,|H (Ωs)|2 = 10−10/10 = 0.1

according to the specifications for passband and stopband attenuations. Substitutingthese values into (8.47) gives (

Ωp

Ω0

)2n

= 100.05 − 1

⇒ 2n (log Ωp − log Ω0) = log(100.05 − 1

)⇒ 2n (−0.4883− log Ω0) = −0.9136 (8.48)

and (Ωs

Ω0

)2n

= 10− 1 = 9

⇒ 2n (log Ωs − log Ω0) = 0.9542

⇒ log Ω0 = 0.1387− 0.9542

2n(8.49)

Substituting (8.49) into (8.48), we get

2n

(−0.4883− 0.1387 +

0.9542

2n

)= −0.9136

⇔ (−0.1254n + 0.9542) = −0.9136

⇔ n = 1.48

So the order the Butterworth filter required is 2.

Now we can compute the value ofΩ0.

log Ω0 = 0.1387− 0.9542

2 (2)= −0.09985

⇒ Ω0 = 0.7946


The poles of the Butterworth filter are those on the left-hand-side of thes-plane, i.e. at

− cos(π

4

)± j sin

(π

4

)= −0.7071± j0.7071

and so

H (s) =1(

sΩ0

)2

+√

2(

sΩ0

)+ 1

(8.50)

To obtain the digital IIR filter transfer function, we substitute

s =1 + z−1

1− z−1

into (8.50):

H (z) =1

1.5838(

1+z−1

1−z−1

)2+ 1.778

(1+z−1

1−z−1

)+ 1

=(1− z−1)

2

1.5838 (1 + z−1)2 + 1.778 (1 + z−1) (1− z−1) + (1− z−1)2

=1− 2z−1 + z−2

1.5838 + 3.1676z−1 + 1.5838z−2 + 1.778− 1.778z−2 + 1− 2z−1 + z−2

=1− 2z−1 + z−2

4.3618 + 1.1676z−1 + 0.8058z−2

=(0.2293) (1− 2z−1 + z−2)

1 + 0.2677z−1 + 0.1847z−2

8.4 Impulse Invariant Method

Let Ha (ω) be the transfer function of the analog filter that has been designed. The im-pulse responseha (t) of this filter can be obtained through Fourier transformation. Theidea behind the impulse invariance method is that the impulse response of the digital filterh [n] is a sampled version ofha (t). Thus

h [n] = ha (nT ) (8.51)

whereT is the sampling period.

Impulse Invariant Method 203

As we know, the magnitude spectrum of a sampled signal is a periodic continuation of theoriginal spectrum of the analog signal. So the magnitude responseH (ω) of the digitalfilter is periodic with a period offs = 1/T . It can be expressed mathematically as

H (ω) =1

T

∞∑k=−∞

Ha

[j (ω − 2πk)

T

](8.52)

Even though the impulse invariant method preserves the shape of the impulse response,the frequency response may be significantly different from what we expected. This isbecause the stopband magnitude response does not drop to zero atω = π. This meansthat H(ω) will be an aliased version ofHa(ω) because of the periodic nature of theresponse. This is illustrated in Figure 8.6. Due to the aliasing problem, the impulseinvariant technique is only applicable to low-pass filter designs.

0 π 2π

Analog FilterResponse

Replica ofAnalog Filter Response

Combined Response

|Η(ω)|

ω

Figure 8.6: A Sampled Magnitude Response

The stopband attenuation of the aliased version ofHa(ω) may not be sufficient to satisfythe original filter specifications. This will render the design useless. So we have to makesure that the aliased portion ofH(ω) is small enough. The passband is also affected butthe effect is usually much smaller than that in the stopband. A sufficiently high sampling


rate can be used to overcome this problem; the higher the sampling rate, the smaller theerror introduced by aliasing.

For this reason, Butterworth and Chebyshev filters are more suitable if impulse invariantmethod is used. This is because both these filters do not have ripples in the stopband andthe response in this region is monotonically decreasing.

The analog filter transfer function can be expressed in terms of partial fractions

Ha (ω) =N∑

i=1

Ai

jω − si

(8.53)

wheresi is generally complex-valued. In fact, it is usually expressed in terms of theLaplace variables:

Ha (s) =N∑

i=1

Ai

s− si

(8.54)

Inverse Laplace transform of (8.54) gives the impulse response

ha (t) =

∑Ni=1 Ake

skt, t ≥ 00, t < 0

(8.55)

The impulse response of the digital filter is obtained by sampling. Therefore,

h [n] = ha (t) (8.56)

=N∑

i=1

AkesknT u [n] (8.57)

=N∑

i=1

Ak

(eskT

)nu [n] (8.58)

Applying the Z transform pair

anu [n] ↔ 1

1− az−1(8.59)

we obtain

H (z) =N∑

i=1

Ai

1− eskT z−1(8.60)

Impulse Invariant Method 205

Since the transfer function of the digital filter can be obtained directly fromHa (s), thecoefficients of the IIR filter can be obtained directly fromHa(s) without having to com-pute the impulse response first.

In summary, the characteristics of impulse invariance designed filters are:

1. The IIR filter is stable if the analog filter is stable.

2. The frequency response of the IIR filter is an aliased version of the analog filter. Sothe optimal properties of the analog filter are not preserved.

3. The cascade of two impulse invariance designed filters is not impulse invariant withthe cascade of the two analog filter prototypes.

4. The step response (the response of the filter to a unit step input) is not a sampledversion of the step response of the analog filter. Therefore, if the step response ofthe final filter is important, then the impulse invariant method may not be a suitablechoice.

Example 8.5. A second order Butterworth filter has the following normalized transferfunction

H (s) =1

s2 +√

2s + 1

Design an IIR filter based on this analog filter using the impulse invariant method.

Solution:

First expand the transfer function in terms of partial fractions:

H (s) =A1

s− s1

+A2

s− s2

s1 = − 1√2

(1− j)

s2 = − 1√2

(1 + j)

A1 = − j√2

A2 =j√2


Then the IIR filter transfer function is given by

H (z) =A1

1− es1T z−1+

A2

1− es2T z−1

Since this is a normalized transfer function, we can use the normalized sampling periodT = 1. So the transfer function becomes

H (z) =0.2265z−1

1− 0.7497z−1 + 0.2431z−2

8.5 Summary

Digital IIR filters designs are usually based on an analog filter with equivalent specifi-cations. We have briefly reviewed two common types of analog filters: Butterworth andChebyshev. Two methods for converting the analog filter response to a digital one hasbeen discussed. The impulse invariant method seeks to preserve the shape of the impulseresponse of the analog filter. However, it is only suitable for designing low-pass filtersbecause of aliasing introduced when sampling. The second method, bilinear transforma-tion, can be applied to a larger class of filters. However, the frequency warping effect hasto be compensated for.

Bibliography

[1] S. J. Orfanidis,Introduction to Signal Processing, ser. Prentice-Hall Signal Process-ing Series. Prentice-Hall, 1996.

[2] A. Oppenheim and R. Schafer,Discrete-Time Signal Processing, second edition ed.,ser. Prentice-Hall Signal Processing Series. Prentice-Hall, 1999.

[3] S. K. Mitra, Digital Signal Processing: A Computer-Based Approach, second edi-tion ed. McGraw-Hill, 2001.

[4] J. G. Proakis and D. G. Manolakis,Digital Signal Processing: Principles, Algo-rithms and Applications, third edition ed. Prentice-Hall, 1996.

[5] C. Marven and G. Ewers,A Simple Approach to Digital Signal Processing. Wiley-Interscience, 1996.

[6] R. G. Lyons,Understanding Digital Signal Processing. Addison-Wesley, 1997.

[7] J. Y. Stein,Digital Signal Processing: A Computer Science Perspective, ser. WileySeries in Telecommunications and Signal Processing. Wiley-Interscience, 2000.

[8] E. C. Ifeachor,Digital Signal Processing: A Practical Approach, second edition ed.Prentice-Hall, 2002.

[9] A. Bateman and I. Paterson-Stephens,The DSP Handbook: Algorithms, Applica-tions and Design Techniques. Prentice-Hall, 2002.

[10] J. H. McClelland, R. W. Schafer, and M. A. Yoder,DSP First: A Multimedia Ap-proach. Prentice-Hall, 1998.

[11] P. S. Diniz, E. A. da Silva, and S. L. Netto,Digital Signal Processing: SystemAnalysis and Design. Cambridge University Press, 2002.

208 BIBLIOGRAPHY

[12] S. Salivahanan, A. Vallavaraj, and C. Gnanapriya,Digital Signal Processing. TataMcGraw-Hill, 2000.

[13] M. H. Hayes,Schaum’s Outline of Theory and Problems of Digital Signal Process-ing. McGraw-Hill, 1999.

[14] T. K. Moon and W. C. Sterling,Mathematical Methods and Algorithms for SignalProcessing. Prentice-Hall, 2000.

Index

ADPCM, 7aliasing, 100, 102arithmetic overflow, 155, 167autocorrelation, 56

beamforming, 15bilinear transformation, 196block processing, 4bounded, 27

causal, 23circular buffer, 153circular convolution, 67companding, 6, 122

µ-law, 123A-law, 124

convolution sum, 37correlation, 29critical sampling, 101

DFT, seediscrete Fourier Transformdigital signals, 19discrete Fourier transform, 17, 59discrete-time Fourier series, 43, 44discrete-time Fourier Transform, 17discrete-time Fourier transform, 43, 49

convergence, 50mean square, 51uniform, 50

inverse, 56properties, 53

frequency shift, 55linearity, 54periodicity, 53symmetry, 53time reversal, 55time shift, 54

discrete-time signals, 19discrete-time system

frequency response, 134minimum phase, 136poles, 134zeros, 134

Dithering, 124down-sampling, 28DTFT,seediscrete-time Fourier transform

echo cancellation, 12energy density spectrum, 52energy signal, 25equalizer, 12

filteradaptive, 11analog, 188

Butterworth, 188Chebyshev, 188, 192

anti-aliasing, 107anti-image postfilter, 112bandpass, 3

design, 182design process, 171

210 INDEX

equalizing, 12high-pass, 3

design, 181low-pass, 3order, 41realization, 137

canonical form, 142cascade, 148direct form, 139FIR, 144parallel, 149transposed form, 143

recursive, 187finite impulse reponse filter

window method, 176finite impulse response, 18, 40, 171finite impulse response filter

frequency sampling, 183FIR, seefinite impulse responsefixed-point, 155floating point, 157Fourier coefficients, 44

discrete-time, 45frequency, 3

normalized, 47frequency resolution

computational, 92frequency transformation, 199frequency warping, 197

Gibb’s phenomenon, 52group delay, 135

ideal reconstruction, 109IEEE 754, 157IIR, seeinfinite impulse responseimage compression, 10image enhancement, 9

image processing, 9image restoration, 10impulse invariant method, 202infinite impulse response, 18, 40, 187infinite impulse response filter

design, 188

Kotel’nikov, 102

limit cycle, 165linear convolution, 37, 55

properties, 38linear phase, 136linear predictive coding, 7

modulation, 29multiply-and-accumulate, 139

noise cancellation, 11noise-shaping, 121Nyquist interval, 105Nyquist rate, 101

over-sampling, 101, 119ratio, 119

Parseval, 47PCM, 7phase delay, 135power density spectrum, 48power signal, 26power spectral density, 118pre-warping, 197

quantization, 97, 114non-uniform, 122

quantization error,seequantization noisequantization noise, 116quantization width, 115quantizer

INDEX 211

full-scale range, 115quantizer resolution, 115

round-off, 155round-off noise, 162

sample, 19sample-by-sample processing, 4sampling, 19, 97

frequency, 99period, 98rate, 99uniform, 98

sampling frequency, 19scaling, 27sequence

average power, 26, 46bounded, 33causal, 23complex exponential, 21energy, 25finite length, 23periodic, 24random, 22sinusoidal, 21symmetry, 23unit impulse, 20unit step, 20

Shannon, 101signal, 2

chirp, 93discrete-time, 19non-stationary, 93reconstruction, 108

signal-to-quantization-noise ratio, 118smart antennas, 15software radio, 16spectral analysis, 84

spectrogram, 93spectrum, 3

leakage, 86time-varying, 93

speech coding, 5speech processing, 5speech recognition, 9speech synthesis, 9staircase reconstructor, 111superposition, 36superposition principle, 30system

causal, 32discrete-time, 30linear, 30linear time invariant, 31linear time-invariant, 34lossless, 34shift invariant, 31stability, 33

bounded-input bounded-output, 33exponential, 33

truncation, 155

under-sampling, 101uniform sampling theorem, 101, 108unit delay, 28up-sampling, 28

Whittaker, 102window

Blackman, 180Hamming, 91, 176

main lobe, 91Hanning, 180Kaiser, 180rectangular, 88, 176, 177

212 INDEX

main lobe, 88sidelobe, 88, 90

wordlength, 154

Z transform, 131inverse, 133properties, 132region of convergence, 131

zero-padding, 64

dspbook

Documents

Transcript of dspbook