Signals and Systems - ETH Z...Due to their inertia, most physical systems produce signals with power...

Signals and Systems

Lecture 7: Introduction to Filtering

Dr. Guillaume Ducard

Fall 2018

based on materials from: Prof. Dr. Raffaello D’Andrea

Institute for Dynamic Systems and Control

ETH Zurich, Switzerland

G. Ducard 1 / 46

Outline

1 Motivation2 White Noise

Primer on random variablesWhite noise definitionGenerating white noise from probability density functions

3 FiltersMain types of filtersCausal vs. Non-Causal Filtering

Non-causal filteringNon-causal filtering with causal filters

Non-Linear FilteringMedian filter

4 Anti-AliasingAnti-aliasing filteringAliasing of noiseDesign of a simple analog anti-aliasing filter G. Ducard 2 / 46

MotivationWhite Noise

FiltersAnti-Aliasing

Introduction

Definition: filtering

is the operation of rejecting undesired signal features, while maintaining desiredones. These features are often naturally described in the frequency domain;for example, “high-frequency noise”, “50Hz electrical hum”, “white noise”, etc.

In practice

Due to their inertia, most physical systems produce signals with power atlow frequencies and, therefore, typically only respond to low frequencyinput signals.

Input chatter or rapid switching from high frequency signals usually haslittle effect on the output of the system and may even cause damage dueto deformations, heat, etc.

Furthermore, the output of most sensors is affected by white noise, whichwe will define later.

⇒ Therefore, it often makes sense to low-pass filter the output signals ofmechanical systems obtained with sensors, in order to attenuate the sensornoise while keeping the relevant, low-frequency information. G. Ducard 3 / 46



Introduction

In this lecture, we introduce:

1 basic filtering

2 and noise concepts.

In the following two lectures, we will focus on the two maincategories of filter:

Finite impulse response (FIR) filters

and infinite impulse response (IIR) filters.

Both are implemented as linear time-invariant systems with, astheir names suggest, either finite or infinite impulse responses.

G. Ducard 4 / 46



Example: Continuous-time tracking problem

Consider the following feedback loop:

101

s

G

u v

−

r

d

where 1/s is the CT transfer function of the system obtained using the Laplacetransform, the CT counterpart of the z-transform (see e.g. “Control Systems1”):

V (s)

U(s)=

1

s←→ v(t) = u(t).

The signal v represents the velocity of a mass and u the force applied to it.The reference signal is r; d is the sensor noise; 10 is the gain of a proportionalcontroller; and G is a CT low-pass filter (an LTI system) with transfer function

H(s) =20

s+ 20.

G. Ducard 5 / 46




This filter attenuates high-frequencies, as described by its magnitude response,shown below.

100 101 102 103

0

−10

−20

−30

−40

Frequency (rad s−1)

|H(jω)|

(dB)

Magnitude response of filter G

G. Ducard 6 / 46




0 1 2

0

0.5

1

Time (s)

Velocity

(ms−

1)

Reference trajectory and noise signal

d(t), Noise

r(t), Reference

G. Ducard 7 / 46




Consider the closed-loop system and its tracking performance for the followingcases:

1 No filtering (H(s) = 1):

V (s) =10

s+ 10(R(s)−D(s)),

U(s) =10s

s+ 10(R(s)−D(s)).

2 With low-pass filter (H(s) = 20s+20

):

V (s) =10

s+ 10H(s)(R(s)−H(s)D(s)),

U(s) =10s

s+ 10H(s)(R(s)−H(s)D(s)).

G. Ducard 8 / 46



0 1 2

−0.1

−0.05

0

Time (s)

Velocity

(ms−

1)Tracking error

v(t), no noise, no filter

v(t), with noise, no filter

v(t), no noise, with filter

v(t), with noise, with filter

Observation: Although the noise’s influence on the tracking performance isminor, the advantage of filtering becomes more apparent when the controleffort is considered, i.e. the force u(t): G. Ducard 9 / 46



−1

0

1

2

Time (s)

Force

(N)

Control effort without filter

u(t), no noise, no filter

u(t), with noise, no filter

Observation: Without the filter, the noise directly acts on the actuator(potentially causing damage). It is therefore desirable to filter the signal.

G. Ducard 10 / 46



0 1 2−1

0

1

2

Time (s)

Force

(N)

Control effort with filter

u(t), no noise, with filter

u(t), with noise, with filter

Observation: The control signal has been greatly improved and smoothed out(less jittery), due to low-pass filtering of the measurement data. G. Ducard 11 / 46




Outline










Random variables

Let x ∈ R be :

a scalar continuous random variable

with probability density function (PDF) p(x), which satisfies

∞∫

−∞

p(x)dx = 1 and p(x) ≥ 0 ∀x ∈ R.

The expected value of x is

E (x) :=

∞∫

−∞

x p(x)dx

The variance of x is

Var (x) := E(

(x− E (x))2)

.G. Ducard 13 / 46




Outline










White noise definition

Consider a real-valued sequence of random variables with finitelength N with the following properties:

E (x[n]) = 0, for n = 0, . . . , N − 1

E (x[n]x[l]) =

0 for n 6= l

1 for n = l.

Therefore, x[n] is a random variable with zero mean and thesequence x[n] is uncorrelated across time.

Remark: For simplicity, we define white noise to have a varianceof 1, however it can be any constant.

G. Ducard 15 / 46




We now analyze the sequence in the frequency domain using thediscrete Fourier transform (DFT):

X[k] =

N−1∑

n=0

x[n]e−jk2πN

n.

By linearity, it is clear that E (X[k]) = 0 for all k. Furthermore,

E (X∗[k]X[q]) = E

((

N−1∑

n=0

x[n]ejk2πN

n

)(

N−1∑

l=0

x[l]e−jq2πN

l

))

=

N−1∑

n=0

ej(k−q)2πN

n

=

N for k = q

0 otherwise, as established in previous lectures.

G. Ducard 16 / 46




E (X∗[k]X[q]) =

N for k = q

0 otherwise, as established in previous lectures.

Conclusions : In the frequency domain, we expect all frequencies:

1 to be equally represented,

2 and be uncorrelated.

⇒ White noise : is a signal with a flat spectrum.

Remark: There are other, slightly different, definitions: we can, for example,generalize the above to allow for a time-varying variance Var (x[n]) = σ2

n,instead of constant variance, which leads to

E(

|X[k]|2)

=N−1∑

n=0

σ2n ,

but in this case it is not necessarily true that E (X∗[k]X[q]) = 0 for k 6= q.G. Ducard 17 / 46




Generic definition of a white noise sequenceIf a sequence x[n] is such that :

1 E (x[n]) = 0

2 and E (x[n]x[l]) = δ[n − l],

the sequence is referred to as a white noise sequence.

In practice : White noise is :

a good model for many types of noise that are found inphysical processes,

is simple to work with,

dealt with by applying low-pass filtering.

Remark : The formal way of dealing with how power (for periodic signals) andenergy (for finite length signals) are distributed across frequency is the power

spectral density. We will not discuss this in the class. G. Ducard 18 / 46




Outline










Uniform distribution

White noise can be generated by many different underlying PDFs,as we illustrate with the following two examples.

Uniform Distribution Let x be a random variable with PDF

p(x) =

1b−a

a ≤ x ≤ b

0 otherwise

x

p(x)

a b

G. Ducard 20 / 46





Matlab: The command rand draws independent pseudorandom numbersfrom a uniform distribution with a = 0, b = 1.

Zero mean assumption: a = −b, for b > 0

Unit variance assumption, for a = −b and b > 0:

1

b− a

∫ b

a

x2 dx = 1

1

2b

∫ b

−b

x2 dx =x3

6b

∣

∣

∣

∣

b

−b

=b2

3= 1

=⇒ b =√3.

Therefore, the Matlab command sqrt(3)*(2*rand-1) draws independentpseudorandom numbers from a uniform distribution with expected valuezero and unit variance.

G. Ducard 21 / 46





See the following figure for a discrete-time representation of white noisegenerated by a uniform distribution.

0 10 20 30 40 50 60

−√3

-1

0

1

√3

n

Amplitude

G. Ducard 22 / 46




Normal Distribution

Normal Distribution A normal-, or Gaussian-distributed continuous randomvariable x is fully defined by its mean µ and variance σ2 > 0. The PDF of x is

p(x) =1√2πσ2

e−

(x−µ)2

2σ2 ,

where the notation x ∼ N (µ, σ2) is used to denote that x is normallydistributed with mean µ and variance σ2. For white noise, we have µ = 0 andσ2 = 1:

p(x) =1√2π

e−x2

2 .

−4 −3 −2 −1 1 2 3 4

0.1

0.2

0.3

≈ 0.4

≈ 0.24

x

p(x)

G. Ducard 23 / 46




Remarks :

1 Using Matlab: The command randn draws independentpseudorandom numbers from a normal distribution with mean0 and variance 1.

2 Both probability distributions may be used to generate whitenoise.⇒ We often only care about mean and variance, so theunderlying distribution does not matter so much.

G. Ducard 24 / 46



Main types of filtersCausal vs. Non-Causal FilteringNon-Linear Filtering

Outline










Main types of filters

Filters can be classified according to their magnitude response. The main typesof filters are listed below with their ideal magnitude response.Low-Pass : Passes low frequencies and attenuates high frequencies.

|H(Ω)|

Ω00 π

High-Pass: Passes high frequencies and attenuates low frequencies.

|H(Ω)|

Ω00 πG. Ducard 26 / 46




Main types of filters

Band-Pass: Passes frequencies in a frequency band and attenuates all otherfrequencies. It can be obtained by applying a low-pass filter and a high-passfilter in series (i.e. cascading the two systems).

|H(Ω)|

Ω0 Ω10 π

Band-Stop: Attenuates frequencies in a frequency band and passes all otherfrequencies. It can be obtained by applying a low-pass filter and a high-passfilter in parallel, and adding the resulting outputs.

|H(Ω)|

Ω0 Ω10 π G. Ducard 27 / 46




Outline










Forword

As seen in previous lectures: systems, and thus filters, canbe causal or non-causal.

Recall that a system is said to be causal if the output onlydepends on the present and past inputs.

Practical conclusions

causal filters must be used for real-time applications.

In post-processing, since the entire signal is available,non-causal filters are usually used since they provide betterperformance.

G. Ducard 29 / 46




Non-causal filtering

Non-causal filtering is usually performed in the frequency domain, whereunwanted features can be easily identified and removed.

Conceptually, it works as follows:

u[n] → U [k] DFT

U [k] → Y [k] Manipulate in the frequency domain

Y [k] → y[n] Inverse DFT

In practice :

This is, however, computationally expensive.

Causal filtering is usually computationally more efficient, as we will see infuture lectures.

G. Ducard 30 / 46




Non-causal filtering with causal filters

We will now investigate an approach to non-causal filtering that has the com-putational complexity of causal filtering.

Let y = Gu, where G is a real, causal, LTI filter with transfer function H(z).Let G be the corresponding real, anti-causal, LTI filter with transfer function

H(z−1), and let y = Gy.

Then, y = GGu, and

Y (ejΩ) = H(e−jΩ)H(ejΩ)U(ejΩ)

= |H(ejΩ)|2U(ejΩ),

because for real filters, H∗(ejΩ) = H(e−jΩ).

G. Ducard 31 / 46




Non-causal filtering with causal filters

We will now investigate an approach to non-causal filtering that has the com-putational complexity of causal filtering.

Let y = Gu, where G is a real, causal, LTI filter with transfer function H(z).Let G be the corresponding real, anti-causal, LTI filter with transfer function

H(z−1), and let y = Gy.

Then, y = GGu, and

Y (ejΩ) = H(e−jΩ)H(ejΩ)U(ejΩ)

= |H(ejΩ)|2U(ejΩ),

because for real filters, H∗(ejΩ) = H(e−jΩ).

Conclusions

This approach is computationally efficient, and maintains one of the mainadvantages of non-causal filtering: No phase is introduced by the filter.

Note that the magnitude response is squared. G. Ducard 31 / 46




Let the filter G1 have transfer function H1(z) =1−αz−α

,

and the filter G2 have transfer function H2(z) = H1(z)H1(z−1) = 1−α

z−α1−α

z−1−α

.Let α = 0.8 and u[n] = s[n− 50] for n ≥ 0. The results of applying theabove filters to signal u are shown below.

0 10 20 30 40 50 60 70 80 90 100

0

0.5

1

n

Amplitude

Input

Filtered with G1

Filtered with G2

Remarks : No delay is introduced by G2 . In Matlab, the causal filter is applied with the command filter and thenon-causal one with filtfilt. G. Ducard 32 / 46




Outline










Forword

In the next lectures, we will introduce various types of linear filters.

Sometimes, however, non-linear filters might be required forspecific operations.

One such operation is outlier rejection.

An outlier

is a measurement that is distant from other measurements and isoften caused by a measurement error.s

G. Ducard 34 / 46




Median filter

The median filter is one of the simplest non-linear digital filters. Let

M be an even positive integer,

u the input sequence to the filter,

and y its output.

The non-causal version of the filter is then defined as follows:

y[n] = median(u[n−M/2], . . . , u[n], . . . , u[n+M/2]),

where the median is the numerical value separating the higher half of a datasample from the lower half and can be seen as the “middle number” in a sortedlist of numbers.

In the figures below, we compare :

a median filter with M = 2

to a second-order moving average filter, which is a special type of FIRfilter and will be discussed in the next lecture. It is obtained bysubstituting “median” with “mean” in the above equation.

G. Ducard 35 / 46




−10

0

10Original

Amplitude

−10

0

10Moving-Average Filter

Amplitude

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5−10

0

10Median Filter

Time

Amplitude

Conlcusion : It can be seen how the median filter (non-linear) greatly outperforms the moving average filter(linear) in rejecting outliers.

G. Ducard 36 / 46



Anti-aliasing filteringAliasing of noiseDesign of a simple analog anti-aliasing filter

Outline










Recall

When sampling a continuous-time signal, it is important to ensure that

the signal has minimal frequency content (ideally none) at or above theNyquist frequency π/Ts rad s−1

Why?

because these frequencies are aliased into low frequencies and corrupt theresulting discrete-time signal,

making reconstruction of the original CT signal impossible.

⇒ This effect is known as aliasing and was introduced in Lecture 6.

G. Ducard 38 / 46




How to reduce aliasing effect?

To reduce the effect of aliasing, an anti-aliasing filter is used prior to samplingthe CT signal.

⇒ Anti-aliasing filters are designed to attenuate frequencies above theNyquist frequency, which would otherwise be aliased in the sampling operation.

The sample-and-hold scheme introduced in Lecture 1 can be extended with ananti-aliasing filter Gf :

H Gc Gf Su[n] u(t) y(t) yf (t) y[n]

Here we see the inclusion of an anti-aliasing filter Gf prior to sampling thesignal. This filter is a CT system, and operates on the CT signal.

G. Ducard 39 / 46




Outline










Example of aliasing of noise

Consider a CT signal x(t), which is a combination of a nominal,low-frequency signal x(t) and a band-limited (to 150kHz) noiseν(t):

x(t) = x(t) + ν(t).

To investigate the effects of aliasing, we choose a samplingfrequency of fs = 15kHz (implying that the Nyquist frequency isfN = 7.5kHz) and generate two signals:

1 Directly sampledx1[n] where x1[n] = x(n/fs) for all n.

2 Anti-aliased, then sampledx2[n] where x2[n] = (Gfx)(n/fs) for all n, where Gf is ananti-aliasing filter with corner frequency fN (see Appendix).

G. Ducard 41 / 46




G. Ducard 42 / 46




Aliasing of noise

The result of aliasing can clearly be seen when directly samplingx(t) to generate x1[n]: since the noise component of thesignal contains frequencies up to 150kHz, high frequency noise isaliased in the low frequency band.

By applying an anti-aliasing filter to x(t) prior to sampling,frequencies above the Nyquist frequency are attenuated and theeffect of aliasing is significantly reduced when sampling the filteredsignal to generate x2[n].

G. Ducard 43 / 46




Outline










A simple analog (and thus continuous-time) anti-aliasing filter can be designedusing a resistor and a capacitor as follows:

R

C

i(t)

u(t) y(t)

Assuming the source of signal u has zero output impedance and the sink ofsignal y has infinite input impedance, we can model the input-output behaviorof the filter as:

i(t) = Cy(t), u(t) = Ri(t) + y(t) = RCy(t) + y(t)

Y (s)

U(s)=

1

RCs+ 1=

ωc

s+ ωc

, ωc =1

RC.

This is a first-order, low-pass filter with corner frequency 1/(RC). Higher orderanti-aliasing filters can be designed using active components (OP-amps) or,alternatively, specialized low-pass filter chips. G. Ducard 45 / 46




Appendix: The corner frequency

The corner frequency or cutoff frequency ωc of a filter is often defined as thefrequency at which the power P of a signal is reduced to half of its input value(Pout = Pin/2).

For example if the signal is represented as a voltage, the cutoff frequencyrepresents the frequency for which the voltage drops to 1/

√2 of its input value.

At the corner frequency, the magnitude of the filter’s transfer function H(s) is

20dB log10 |H(s = jωc)| = 20dB log10Vout

Vin

= 20dB log10

√

Pout

Pin

= 10dB log10Pout

Pin≈ −3dB.

G. Ducard 46 / 46

Signals and Systems - ETH Z...Due to their inertia, most physical systems produce signals with power...

Documents

Transcript of Signals and Systems - ETH Z...Due to their inertia, most physical systems produce signals with power...