Post on 29-Mar-2020
CMLab
What is a Digital Filter?
• In our technical society we often measure a continuously varying (analog) quantity.eg. Blood pressure, earthquake displacement, population of a city, waves falling on a beach, and the prob. of death
• All these measurement varying with time; we regard them as functions of time: u(t) in mathematical notation.
1
CMLab
• For technical reasons, instead of the signal u(t), we usually record equally spaced samples un of the function u(t).
• The sampling theorem gives the conditions on the signal that justify this sampling process.
• Moreover, when the samples are taken they are not recorded with infinite precision but are rounded off (sometimes chopped off) to comparatively few digits.
2
CMLab
• This procedure is often called quantizing the samples.
• It is these quantized samples that are available for the processing that we do. We do the processing in order to understand.
• What the function samples Un reveal about the underlying phenomena that gave rise to the observations, and Digital filters are the main processing tool.
3
CMLab
• It is necessary to emphasize that the samples are assumed to be equally spaced; any error or noise is in the measurement Un.Notation:{Un or U(n)}: the sequence of numbers of a set of equally spaced measurements of some quantity u(t), where n is an integer and t is a continuous variable.
4
CMLab
• The simplest kinds of filters are the Nonrecursive Filters; they are defined by the linear formula
∑∞
−∞=−=
kknkn UCy
The coeffs. Ck are the coefficiients of the filters, the Un-k are the input data, and the yn
are the output.
5
CMLab
Sum of products
∑−=
−=N
Nkknkn UCy
:convolution
Non-recursive digital filter
6
CMLab
When we use not only data values to compute the output values yn but use other values of the output, we have a formula of the form
Where both the ck and the dk are constants.In practice, the number of products that can be computed must be finite, that is
∑∑∞
∞−−
∞
∞−− += knkknkn yducy
∑∑=
−=
− +=M
kknk
N
kknkn yducy
107
CMLab
8
CMLab
• Filters that use only past computed values and the current value of the data are called causal, for if time is the independent variable, they do not react to future events but only past ones (causes).
• Note, however, that more and more often all the data of an experiment is recorded on a magnetic tape or other storage medium before any data processing is done. In such cases the restriction to causal filters is plainly foolish.
9
CMLab
The coeffs. ck and dk of the filter are assumed to be constants. Such filters are called time-invariant filters and are the filters most used in practice.Time-varying filters are occasionally useful – adaptive signal processing (adaptive filters) .
In practice, all computation must be done with finite-length (finite-precision) numbers.The process of quantization effects not only the input numbers, but it may affect all the internal (to the filter) arithmetic that is done.
10
CMLab
• Consequently, these are round off errors in the finite output numbers yn. It is often convenient to think in terms of infinite precision arithmetic and perfect input data un; but in the end we must deal with reality.- Finite wordlength effects!
11
CMLab
Noise amplification in a filter:Let un be the ”true”’ measurement with added noise εn whose expected value is zero. The condition of zero mean isE{εn}=0And it implies that there is no “bias” in the measurement, only local, random error. Let this noise εn have a variance σ2.
What is the corresponding noise in the out put of a nonrecursive filter (assuming that the arithmetic we do does not increase)?
12
CMLab
• To compute this, we make the additional assumption (which is often, but not always, true) that in making the measurements un+ εn the errors εn are uncorrelated.
• This assumption in mathematical notation is
• A nonrecursive filter is defined by the formula
• For the variance calculation, we begin with 13
⎩⎨⎧
≠=
=⋅nmnm
E mn ,0,
}{2σ
εε
∑ ∑∑−= −=
−−−−=
−− =+=+=N
Nk
N
Nkknkknknkn
N
Nkknknkn ucEucyEucy }){(}{ ,)( εε
})]()({[ 2n
N
Nkknknk yEucE −+∑
−=−− ε
CMLab
But this is (using different summation indices to keep things clear)
Since E{εn}=0 and, for ,Multiplying out and applying the operator E to the nleaves only the terms
Thus,the sum of the sequence of the coefficients of a filter
measures the noise amplification of the filtering process.
]}][{[}{ 2 ∑∑∑−=
−−=
−−=
− =N
Nmmnm
N
Nkknk
N
Nkknk ccEcE εεε
∑∑∑=
==N
Nkkkkk ccEc
-
222222 }{ σσε
0}{, =≠ mnEnm εε
14
CMLab
Geometric progressions
A geometric progression is a sequence of n values
The special case of symmetrically arranged terms beginning at a=z-m ,and going on for 2m+1 terms gives
differential (Δ) w.r.t. z and set a=1, we get
)(1
)1()1(][
,,,,
12
12
Δ−−
=++++= −
−
KL
L
zzazzzanS
azazazan
n
n
)'(11
)1(1],[21
21
)21(
21
11211 Δ
−
−=
−−
=−−
=++++++=−−
+−++−+−−+−− KLL
zz
zzzzz
zzzzzzzmmS
mmmmmmmmmm
2
1)2(2
)1()1(1)1(321
zznnzznzz
nnn
−−+−
=−++++−
−L
15
CMLab
• In many data-processing problems there is no natural original, and therefore an arbitrary point is selected as the origin (typically, for a time signal, n=0 is the time we set t=0, which is arbitrary).
• From the addition formula of trigonometrysin(x+y)=sinxcosy+cosxsinycos(x+y)=cosxcosy-sinxsiny
• It is an easy exercise in trigonometry to see that, when x=x’+h
• Asinx+Bcosx becomes A’sinx’+B’cosx’• Where we have A’=Acosh-Bsinh; B’=Asinh+Bcosh
16
CMLab
Invariance under translation
Squaring each of these experssions and adding them, we obtain A’2+B’2=A2+B2
⇒Under the operation of translation the pair of functions cosx and sinx constitutes the eigenfunctions, for when placed into the operation of translation by the amount h, they again emerge.
Euler identities: cosx+jsinx = ejx; cosx-jsinx=e-jx
⇒
⇒ The complex exponentials are “eigenfunctions” of translation
)(21sin );(
21cos jxjxjxjx ee
jxeex −− −=+=
17
CMLab
• In the complex exponential notation the eigenfunction property is much more simply expressed than in the trigonometric notation,
• Where is the eigenvalue
• And is independent of the variable t.• Thus the are “eigenfunctions ” of translation.
)()()( )(
tueeehtu tjhjhtj
ωλ
ωωω
=⋅==+ +
)(ωλhje ωωλ =)(
hje ω 18
CMLab
• In general, – (Where c can be a complex number) and the invariant is
• Linear Systems:The complex exponential functions and are eigenfunctions for LTI systems.
tjcetu ω=)(
2
*
))((
))((
ccc
ecce
cecetjtj
tjtj
=⋅=
= − ωω
ωω
tje ω tje ω−
{ } tjtj eeL ωω ωλ )(= 19
CMLab • For nonrecursive filters of the form
The substitutionor
Produces
Where: transfer function
Fourier Transform: DTFT
∑−=
−=N
Nkknkn ucy
tjetu ω=)( njenu ω=)(
njN
Nk
kjk
njn eecey ωωω ωλ )(== ∑
−=
−
∑−=
−=N
Nk
kjkec ωωλ )(
20
CMLab • For recursive filter we need to assume that both the input and output are of the form
and• Where the input coeff. ,and output coeff. may
be complex numbers.
: Transfer function
njIn eAu ω= nj
on eAy ω=
IA oA
∑∑ −− +=M
knk
N
knkn yducy10
∑
∑
∑∑
−
−
−−
−=
+=⇒
Mkj
k
Nkj
k
I
o
Mknj
ko
Nknj
kInj
o
ed
ec
AA
edAecAeA
1
0
1
)(
0
)(
1 ω
ω
ωωω
21
CMLab
This is a transfer function because, when you multiply the input by this ratio you get the output:you ‘transfer’ from the input to the output!
22
CMLab
• It is worth nothing that the exponential function is also the eigenfunction that is appropriate for the calculus operations– Of differentiation
– Of integration
• The exponential is also the eigenfunction for differencing , since
tjtj ejedtd ωω ω)(=
tjtj ej
dte ωω
ω)1(=∫
( ) tjjtjtjtj eeeee ωωωωω ]1[)1( −=−=Δ +23
CMLab
• The expression occurs frequently and it is often easier to make a notational change and write
• Hence for nonrecursive filter
: (mod ( ))For recursive filter
Z-transformation
ωje
∑−=
−=N
Nk
kk zc)(ωλ
ωjez−
∑
∑−
−
−= M
kk
Nk
k
I
o
zd
zc
AA
1
0
1
24
ze j =ω
CMLab
25
Some basic Applications of Digital filters :
• Least-square fitting of polynomialsGiven M data points (tm, um), m=1,2,…,M wish to fit (approximate) these data, in some sense, by a polynomial u=u(t) of degree N where M≧N+1
Principle of least square : the sum of the squares of the residuals is the least
u(t)
t
• All the errors are in the vertical direction
• The differences between the polynomial and the original data are called the residuals
CMLab
26
Ex: consider a set of 5 equally spaced data points.tm=m for m=-2, -1, 0, 1, 2um unspecifiedfit the above data by a straight line u=A+Btin least-square sense.
In the case of smoothing, we use midpoint of the line as the smoothed value in place of the original point u0. In this case, only A is needed. From (1), we have
( ) ( )[ ]
( )
( )
[ ]1 1, 1, 1, 1,51 :window running
averagemoving : 51 general,In
equation normal 0,
0,
,min
2
2
2
2
2
∑
∑
−=−
−=
=
⎪⎪⎭
⎪⎪⎬
⎫
=∂
∂
=∂
∂
+−=
mmnn
mm
uu
BBAF
ABAF
BmAuBAF
CMLab
27
From the frequency point of view :S’pose the input function is one of the eigenfunctions in the complex form eiωt of frequency ω.Since the system is linear, the same function will emerge at the output except that it is multiplied by its eigenvalue H(ω).
CMLab
28
( ) [ ]
[ ]
( )( )2sin5
25sin
2cos2cos2151
151 22
ω
ω
ωω
ω ωωωω
=
++=
++++= −− iiii eeeeH
In general :
: transfer function
( ) ( )[ ]
( )( )( )2
212
1
sinsin
121
...12
1
ω
ωωω
ω
ω
+
−−−
+=
++++
=
N
iNNiiN
N
eeeN
H
CMLab
29
the more terms used , the more rapid are the wiggles of H(w), and the more the envelope of the wiggles is squeezed toward the frequency axis.
These smoothing formulas are the same as the running average in statistics!Note that these transfer functions are periodic and symmetric about ω=0 and ω= π (f=1/2). The periodicity about f=1/2 reflect the Aliasing effect, while the periodicity about f=0 arises from the symmetry of the filter coefficients.
CMLab
30
• Least-squares Quadratics and QuarticsInstead of a straight line, fit by a quardratic (or a cubic) u(t)=A+Bt+Ct2
( ) ( )[ ][ ][ ][ ][ ]
]31217123[351 general,In
36,9,44,69,84,89,84,69,44,9,361121,14,39,54,59,54,39,14,219
2,3,6,7,6,3,273,12,17,12,35
,,min
2112
4291
2311
211
351
22
++−− −+++−=
−−=−−=
−−=−−=
++−=⇒ ∑
nnnnnn
m
uuuuuu
mmmm
CmBmAuCBAF
By differentiating w.r.t the variables A, B and C, we obtain the normal equations:
Where {·} means “sum” of all values of the quantities enclosed in the braces
By selecting the particular set of equally spaced data points –m, -(m-1), … , 0, … (m-1), m, the sum over all odd power are all zero.
CMLab
31
• The effect of the higher-degree polynomial is a higher degree of tangency at ω=0, as compared with the straight line case. That is, the curves are more flat around DC.
• The use of more terms in the smoothing formula makes the curve come down rapidly and to be slightly lower in the subsequent wiggles.
• Theorem: The more powers of t that we fit in the time domain, the higher the tangent at ω=0 in the frequency domain.
HW: please find corresponding H(ω) and ck for N=5, 7, 9, 11 if f(t)=A+Bt+Ct2+Dt3+Et4
CMLab
32
• Modified least squares :Smoothing Window for 2N+1 points
modified Smoothing Windows for 2N+1 points: reduce the two end values to one-half their assigned values
( ) ( )( )
( ) ( )22
2
222
2
212
cossinsin
21
)sin(2)]sin(cos)sin(cos)cos([sin
cossin
sin21
ωω
ω
ωωω
ω
ω
ωωω
ωωω
⎭⎬⎫
⎩⎨⎧
=
−+=
⎭⎬⎫
⎩⎨⎧
−=+
NN
NNNN
NN
HN
CMLab
33
• The curve will come down more rapidly• The main lobe is slightly wider.★ the end constants can be chosen as a parameter
• The brackets are the same as before except with 2N in place of 2N+1. This means that the zeros of the transfer function are spaced slightly farther apart, at π/N instead of at π/(N+1). This means that the main lobe is slightly wider.
• The change of the division from 2N+1 to 2N also means that the term in the brackets would end at a slightly higher value.
To ensure that the end of the interval at f=0.5, the transfer function shall be zero.
• The extra factor means that the curve will come down some what more rapidly.
• Smoothing removal of high frequencies in the signal
CMLab
34
• Differences and DerivativesThe difference operator: can be shown to “amplify “ small errors!
the operator annihilates a polynomial Pn(x) of degree n in X; that is
[ ] ( )[ ][ ]nK
nK
nnn
uu
uuu1
1−
+
ΔΔ=Δ
−=Δ
1+Δn
( ) ( ) 01 =Δ + xPnn
CMLab
35
from frequency point of view :( )
[ ]( )[ ]
[ ] [ ] ( )[ ] tiKiKKtiK
tii
tii
tititi
eeie
eie
eeeee
ωωωω
ωωω
ωω
ωωω
22
22
1
sin2
sin2
1
=Δ
=
−=
−=Δ +
CMLab
36
Since , so the amplification at frequency w is contained in the factor
; △decreases the amplitude of any frequency; there is an amplification
12 ==ωiKei K
( )[ ]K2sin2 ω
πωω
π
π
<≤<≤
3
30
CMLab
37
high-pass behavior of the difference operator Δk
CMLab
38
Differences are also used to approximate derivatives.• Central difference formula
from the formula (with h=1)( ) ( ) titi
nnn
eituetuhuuu
ωω ω=′=
−=′ −+
so , set
...(1) 2
11
ωωω sin2iee ii =− −
CMLab
39
The ratio of the calculated to the true answer (which is iω) is :
ω=0, R=1ω≠0, |R|<1 ⇒ the formula underestimates the value of the
derivatives for all other freqs. in the Nyquest interval
For the 2nd derivate :
ωω
ωω sin
2sin2
truecalculated
==Δi
iR
( ) ( ) ( ) ( )( ) 2
2
2sin
121
⎥⎦
⎤⎢⎣
⎡=
−+−+=′′
ω
ω
R
tutututu
Which is the square of the previous case but contracted by a factor of 2 in the independent variable ω
CMLab
40
Linear processing of smoothing:
Spencer’s smoothing formula:15-point: 21-point:
[ ][ ],...57,60,57,47,33,18,6,2,5,5,3,1
3,6,5,3,21,46,67,74,67,46,21,3,5,6,3
3501
3201
−−−−−
−−−−−−
not informative
Smoothing: keep the value at zero frequency correct but, in general, decrease the amount of any higher frequency that may be existed in the function being smoothed
CMLab
41
plot the logs of the numbers |H(ω)| 20 log|ratio| = decibel units (dB)20 dB = factor of 10
As shown in the above figure, the curves are not too informative, since they are as small at the high frequencies that we cannot see how good or bad they are.It is therefore better to plot the logs of the numbers |H(ω)|
The smoothing formulas generally remove some frequencies and let others pass.
40db 10060db 1000
CMLab
42
• Missing Data and Interpolation :The reasons or situations for the occurrence of
“missing data” in a long record of data :• the measurements may never have been made;
they may have been misrecorded and thus later removed
• the formula used to compute successive values of the function may have involved an indeterminate form, such as at x=0, and the computer refused to divide by zero.
xxsin
CMLab
43
Usually, an interpolation formula based on the assumption that the data locally is a polynomial of some odd degree.
This is equivalent to the assumption that the next higher-order difference is zero.
11 -11 -2 1
1 -3 3 -11 -4 6 -4 1
Difference table △un=un+1-unn f(n) △f(n) △2f(n) △3f(n)-3 0-2 0 0-1 0 10 1 -21 0 12 0 03 0 04 0
0011000
1-3-3-10
CMLab
44
For instance, k=4
note: In general case the sum of the sequences of the binomial coeffs. of order N is . Hence for the 2k th difference formula, the noise amplification is
[ ]211261
211224
440464
++−−
++−−−
−++−=⇒=+−+−=Δ
nnnnn
nnnnnn
uuuuuuuuuuu
NNC 2
( )( ) 1222
2242 −≅−
Δ kC
CCNk
k
kk
kk
Aπ
31.1,3,2
5.0,1
1817
==
==
==
A
A
A
NkNkNk
The noise amplification factor of this formula is
CMLab
45
If set then ( ) [ ]ωωω
ω
2coscos431 −=
=H
eu nin
CMLab
46
H(0)=1 However, for high freqs, the value is not too good, particularly for very high freqs.
• Negative values on the graph of the transfer function imply a change in sign.The above figure points out the danager of interpolating a missing value when the data is noisy, which means that the data has numerous high frequencies.
Which implies that we obtain the right answer for ω=0
CMLab
47
Interpolation midpoint values : linear interpolation gives
if 4 adjacent points are used and make the formula exact for all cubics gives the formula: (proof:HW)
[ ]( ) ( )ωω ω
23
281
4
161
coscos9
9923
21
21
23
−=
−++−= −−++
H
uuuuu nnnnn
Please plot H2(ω) and H4(ω) and see what are their behaviors in the high and low frequency ranges respectively.
( ) ( ) ( )221 cos
21
21
ωω =→+= −+ Huuu nnn
CMLab
48
• A Class of Nonrecursive Smoothing FiltersDesign of filters : Symmetric Filters
H(0)=1 exact at dc(lowest freq.)H(π)=0 no highest freq. get through
( ) cbaHeu
aubucubuauyni
n
nnnnnn
++=⇒=
++++= ++−−
ωωω
ω
cos22cos2let
2112
L.P.
CMLab
49
Since H(π)=0 the factor [cos ω+1] had to occur Now one can select a filter that approximately meets the specific requirement.
To convert a low-pass filter to a high-pass filter, we need only to compute un-yn as the new filter!!
( ) [ ] ( )[ ]1coscos142022
122
81
21
41
−++=
−==
⇒⎭⎬⎫
=+−=++
⇒
aaHac
bcbacba
ωωω
CMLab
50
for Ex.1, if we require ( ) 43
81
21
21 ,221 =−=⇒−+−== caaaH π
Ex.2
if we set ( )
[ ]( ) 2
121
2
1141
21
21
21
22
cos2
,022
+=++=⇒
==⇒−+−==
+−
ωω
π
Huuuy
caaaH
nnnn
[ ]
( ) [ ][ ]2cos1cos
262
21
43
2cos
42cos
211281
−+−=++−=
−+++−=⇒ ++−−
ωωω
ωω
H
uuuuuy nnnnnn
CMLab
51
Ex. 3. S’pose that we try to do as well as possible in the neighborhood of zero freq.( )( )
( )161
21
02
32
0
3
3
80
0
10
−=⇒−−==
=
=
=
=
aadHd
ddHH
ω
ω
ωω
ωω
CMLab
52
Ex: we pick our filter form as
[ ]
( ) [ ][ ]3cos1cos
4104
41
85
2cos
82cos
2112161
−+−=++=
−+++−=⇒ ++−−
ωω
ωω
wH
uuuuuy nnnnnn
( )( ) ( )
[ ] ]1,1,1[0cos)(11cos)(1
2cos)( and
0,1 and2cos2;
21
1121
21
32
31
21
361
21
21
21
11
31
61
=++=⇒⎩⎨⎧
=+=+=+=+
⇒+=⇒==⇒
==
+=++=
+−
==
−+
nnnn
ff
nnnn
uuuy
fHba
fHfHbfaHaubuauy
π
π
πω
πω
CMLab
53
• Integration : Recursive FiltersTrapezoid Rule (using y0=0)
[ ]( )
( ) ( )
( ) [ ][ ] 2
221
0
121
1
sin2cos
11 where
form theof is
0 , Set
ω
ω
ω
ω
ω
ω
ω
ω
ieeA
eAty
yetu
uuyy
i
i
ti
tinnnn
=−+
=
==
++= ++
CMLab
54
The true answer for integration of is tie ω [ ] tii e ωω1
( ) 0 , 1 , 0At
21 =====
RfR
πωω
, which gives the quadratic shape of the ratio near ω=0
which is the natural boundary, for it is where Aliasing begins
( )( )[ ]
...1
sincos
truecalculated
72012
2
221
42 ++−=
⎥⎦
⎤⎢⎣
⎡==Δ
ωω
ω
ωω
ωω
ωi
AR
CMLab
55
Simpson’s Rule :
ω=0 , R=1and has a tangency through the 3rd derivative, and rises initially like a quartic until at ω=πThe denominator becomes zero => R ∞
( )1131
11 4 −+−+ +++= nnnnn uuuyy
( ) ( )( ) ( )
[ ][ ]
( )[ ] L++=+
==
−++
= −
−
1801cos2
34
4
sin31
ωωω
ω
ωω
ω
ω
ωω
ωω
i
ii
ii
HR
eeeeH
CMLab
56
• Midpoint integration formula (using y0=0)
Simpon’s formula amplifies the upper part of the Nyquest interval (the high freqs), whereas trapezoid rule damps them out.
This expression begins at 1 and slowly rises as increases, .
( )2
2
1
sin
21
ω
ω
=
+= ++
R
uyy nnn
CMLab
57
• Leo Tick Formula ( )1111 3584.02832.13584.0 −+−+ +++= nnnnn uuuhyy
This formula was designed to have a transfer function that is as close to unity as we make it throughout the lower half of the Nyquest interval while still involving only three consecutive terms
1.57=
Tick’s integration rule was designed to be as accurate as it could be for lower half of the Nyquestinterval,
CMLab
• Simpson’s formula amplifies the upper part of Nyquist interval (the higher frequencies) where as the trapezoid rule damps them out.
• In the presence of noise. Simpson’s formula is more dangerous to use than are the trapezoid or midpoint formulas. But when there is relatively little high freq. In the function being integrated, then the flatness of Simpson’s formula for low freqs. show why it is superior.
Good Reference for Window Selection:“On the use of windows for Harmonic Analysis with the DFT”, Proc. IEEE, vol. 66, pp.58-83, Jan. 1978.