Symmetric Binary Random Signals With Given Spectral Properties
Chapter 13b: RANDOM SIGNALS AND NOISEhomepages.wmich.edu/~bazuinb/ECE3800/B_Notes13b.pdf · 2020....
Transcript of Chapter 13b: RANDOM SIGNALS AND NOISEhomepages.wmich.edu/~bazuinb/ECE3800/B_Notes13b.pdf · 2020....
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 1 of 58 ECE 3800
Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0
Chapter 13b: RANDOM SIGNALS AND NOISE
Sections 13.1 Introduction to Random Signals 13.2 A Simple Random Process 13.3 Fourier Transforms 13.4 WSS Random Processes 13.5 WSS Signals and Linear Filters 13.6 Noise
13.6.1 Probabilistic Properties of Noise 13.6.2 Spectral Properties of Noise
13.7 Example: Amplitude Modulation 13.8 Example: Discrete Time Wiener Filter 13.9 The Sampling Theorem for WSS Random Processes
13.9.1 Discussion 13.9.2 Example: Figure 13.4 13.9.3 Proof of the Random Sampling Theorem
Summary Problems
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 2 of 58 ECE 3800
13.3 Fourier Transform
Definition:PSD
Let Rxx(t) be an autocorrelation function for a WSS random process. The power spectral density is defined as the Fourier transform of the autocorrealtion function.
diwRRwS XXXXXX exp
The inverse exists in the form of the inverse transform
dwiwtwStR XXXX exp2
1
Properties:
1. Sxx(w) is purely real as Rxx(t) is conjugate symmetric
2. If X(t) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(t) is real and even.
3. Sxx(w)>= 0 for all w.
Wiener–Khinchin Theorem For WSS random processes, the autocorrelation function is time based and has a spectral decomposition given by the power spectral density.
Also see:
http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem
Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined.
I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference).
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 3 of 58 ECE 3800
Relation of Spectral Density to the Autocorrelation Function
For “the right” random processes, the power spectral density is the Fourier Transform of the autocorrelation:
diwtXtXERwS XXXX exp
For a real ergodic process, we can use time-based processing to arrive at an equivalent result …
txtxdttxtx
T
T
TT
XX 2
1lim
T
TT
XX dttxtxT
tXtXE 2
1lim
diwdttxtx
TtXtXE
T
TT
XX exp2
1lim
dtdiwtxtxT
T
TT
XX
exp2
1lim
dtdiwttiwtxtxT
T
TT
XX
exp2
1lim
dtdtiwtxiwttxT
T
TT
XX
expexp2
1lim
dtdtiwtxiwttxT
T
TT
XX
expexp2
1lim
If there exists wXX
dtwXiwttxT
T
TT
XX
exp
2
1lim
dttwitxT
wXT
TT
XX
exp
2
1lim
2wXwXwXXX
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 4 of 58 ECE 3800
Properties of the Fourier Transform:
diwxxwX exp
https://en.wikipedia.org/wiki/Fourier_transform
For x(t) purely real
dwiwxxwX sincos
dwxidwxxwX sincos
dwxidwxwOiwEwX XX sincos
dwxwEX cos and
dwxwOX sin
Notice that:
wEdwxdwxwE XX
coscos
wOdwxdwxwO XX
sinsin
Therefore, the real part is symmetric and the imaginary part is anti-symmetric!
Note also, for real signals *wXwXconjwX
X(w) is conjugate symmetric about the zero axis.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 5 of 58 ECE 3800
Relatingthistoarealautocorrelationfunctionwhere XXXX RR
wOiwER XXXX
dwiwRR XXXX sincos
dtwtiwttRR XXXX sincos
dtwttRidtwttRR XXXXXX sincos
wOiwER XXXX
Since Rxx is symmetric, we must have that
XXXX RR and wOiwEwOiwE XXXX
For this to be true, wOiwOi XX , which can only occur if the odd portion of the
Fourier transform is zero! 0wOX .
This provides information about the power spectral density,
wERwS XXXXX
wEwS XXX
0 wS XX
The power spectral density necessarily contains no phase information!
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 6 of 58 ECE 3800
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for
Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
Example9.5‐3S&WandBonceletp.343
Find the psd of the following autocorrelation function … of the random telegraph.
0,exp forRXX
Find a good Fourier Transform Table … otherwise
dwjRwS XXXX exp
dwjwS XX expexp
0
0
expexpexpexp dwjdwjwS XX
0
0
expexp dwjdwjwS XX
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 7 of 58 ECE 3800
0
0
expexp
wj
wj
wj
wjwS XX
wj
wj
wj
wj
wj
wj
wj
wjwS XX
exp0exp
0expexp
wjwj
wjwj
wjwjwS XX
11
2222
22
ww
wS XX
For a=3
Figure 9.5-2 Plot of psd for exponential autocorrelation function.
Laplace vs. Fourier
𝑤 𝑗 ∙ 𝑠
𝑆 𝑠2 ∙ 𝑎𝑠 𝑎
2 ∙ 𝑎𝑠 𝑎 ∙ 𝑠 𝑎
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 8 of 58 ECE 3800
Example9.5‐4
Find the psd of the triangle autocorrelation function … autocorrelation of rect.
TtriRXX
or TT
RXX
,1
T
T
XX dwjT
wS
exp1
T
T
XX dwjT
dwjT
wS0
0
exp1exp1
TT
TT
XX
dwjT
dwj
dwjT
dwjwS
00
00
expexp
expexp
T
T
T
T
XX
wj
wj
wj
wj
T
wj
wj
wj
wj
T
wj
wj
wj
wjwS
0
2
0
2
0
0
expexp1
expexp1
expexp
22
22
1expexp1
expexp11
1expexp1
ww
Twj
wj
TwjT
T
w
Twj
wj
TwjT
wT
wjwj
Twj
wj
Twj
wjwS XX
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 9 of 58 ECE 3800
222
expexp121
expexp1
expexp
w
Twj
w
Twj
TwT
wj
TwjT
wj
TwjT
T
wj
Twj
wj
TwjwS XX
22
cos212sin2sin2
w
Tw
TwTw
Tw
w
TwwS XX
TwwT
wS XX cos112
2
2
2
2
2
2
2sin
2sin2
12
Tw
Tw
TTw
jwT
wS XX
Don’t you love the math ?!
Using a table is much faster and easier ….
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 10 of 58 ECE 3800
Properties of the Fourier Transform
ThediscreteFourierTransformalsoexists(ifx(n)isbounded).Dr.Bazuinmayalsocallthisthediscrete‐time,continuous‐frequencyFourierTransform(somethingfromECE4550)
n
nwjnxnxwX exp , for w
and X(w) periodic in 2 pi outside the range (it repeats in the frequency domain)
The inverse transform is defined as
dwnwjwXwXnx exp
2
11
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 11 of 58 ECE 3800
Deriving the Mean-Square Values from the Power Spectral Density
Using the Fourier transform relation between the Autocorrelation and PSD
diwRwS XXXX exp
dwiwtwStR XXXX exp2
1
The mean squared value of a random process is equal to the 0th lag of the autocorrelation
dwwSdwiwwSRXE XXXXXX 2
10exp
2
102
dffSdwfifSRXE XXXXXX 02exp02
Therefore, to find the second moment, integrate the PSD over all frequencies.
As a note, since the PSD is real and symmetric, the integral can be performed as
0
2
2
120 dwwSRXE XXXX
0
2 20 dffSRXE XXXX
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 12 of 58 ECE 3800
Converting between Autocorrelation and Power Spectral Density
Using the properties of the functions we can actually define variations of Transforms!
The power spectral density as a function is always real, positive, and an even function in w/f.
You can convert between the domains using any of the following …
The Fourier Transform in w
diwRwS XXXX exp
dwiwtwStR XXXX exp2
1
The Fourier Transform in f
dfiRfS XXXX 2exp
dfftifStR XXXX 2exp
The 2-sided Laplace Transform (the jw axis of the s-plane)
dsRsS XXXX exp
j
j
XXXX dsstsSj
tR exp2
1
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 13 of 58 ECE 3800
NotesonusingtheLaplaceTransform
ECE 3100 and ECE 3710 stuff …
(1) When converting from the s-domain to the frequency domain use:
jws or jsw
(2) As an even function, the PSD may be expected to have a polynomial form as: (Hint: no odd powers of w in the numerator or denominator!)
0
22
4242
2222
20
22
4242
2222
2
0bwbwbwbw
awawawawSwS
mm
mm
m
nn
nn
n
XX
This can be factored and expressed as:
sTsT
sdsd
scscwS XX
To compute the autocorrelation function for 0 use a partial fraction expansion such that
sd
sg
sd
sgwS XX
and solve for 0 using the LHP poles and zeros as
0,exp
2
1
tfordsstsd
sgtR
j
j
XX
for determining 0 , use the RHP expansion, replace –s with s, perform the Laplace transform and replace t with –t.
Another hint, once you have 0 , make the image and skip the math tRtR XXXX .
Final Note … for sTsT
sdsd
scscwS XX
If you “define” T(s) as all the LHP poles and zeros and T(-s) as all the RHP poles and zeros, then T(s) will represent a (1) causal, (2) minimum phase, and (3) stable filter (if there are no poles on the jw axis).
You give me a power spectral density and I can design a filter that passes “signal energy” and filters out as much of the rest as possible!
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 14 of 58 ECE 3800
Example:InverseLaplaceTransform.
22
2
22
2
2
1
2
w
A
w
A
wS
X
XXX
Substitute s for w
ss
A
s
AsS XX
2
22
2 22
Partial fraction expansion
ss
A
ss
sksk
s
k
s
ksS XX
21010 2
2
02
10
1010
222
0
AkAkk
kkskk
s
A
s
AsS XX
22
Taking the LHP Laplace Transform
𝐿𝐴𝛽 𝑠
𝐴 ∙ 𝑒𝑥𝑝 𝛽 ∙ 𝑡 , 𝑓𝑜𝑟 𝑡 0
Taking the RHP with –s and then –t.
0expexpexp 2222
tfortAtAtA
s
AL
Combining the positive and negative time “halves” we have
tARXX exp2
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 15 of 58 ECE 3800
7-6.3 A stationary random process has a spectral density of.
else
wwS XX
,0
2010,5
(a) Find the mean-square value of the process.
02
2
2
10 dwwSdwwSR XXXXXX
20
10
10
20
20
10
52
125
2
15
2
10 dwdwdwRXX
50
10202
10
2
100
20
10
wRXX
(b) Find the auto-correlation function the process.
dwtwjwStR XXXX exp2
1
10
20
20
10
expexp2
5dwtwjdwtwjtRXX
10
20
20
10
expexp
2
5
tj
twj
tj
twjtRXX
tj
tj
tj
tj
tj
tj
tj
tjtRXX
20exp10exp10exp20exp
2
5
tj
tj
tj
tj
tj
tj
tj
tjtRXX
10exp10exp20exp20exp
2
5
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 16 of 58 ECE 3800
ttttj
tj
tj
tjtRXX
10sin20sin510sin220sin2
2
5
ttt
ttt
tRXX
15cos5sin
10
2
1020cos
2
1020sin2
5
ttt
t
ttRXX
15cos5
sinc50
15cos5
5sin50
(c) Find the value of the auto-correlation function at t=0..
015cos05
sinc50
015cos05
05sin500
XXR
1150
1150
0 XXR
50
0 XXR
It must produce the same result!
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 17 of 58 ECE 3800
White Noise
Noise is inherently defined as a random process. You may be familiar with “thermal” noise, based on the energy of an atom and the mean-free path that it can travel.
As a random process, whenever “white noise” is measured, the values are uncorrelated with each other, not matter how close together the samples are taken in time.
Further, we envision “white noise” as containing all spectral content, with no explicit peaks or valleys in the power spectral density.
As a result, we define “White Noise” as tSRXX 0
𝑆 𝑓 𝑆𝑁2
This is an approximation or simplification because the area of the power spectral density is infinite!
Nominally, noise is defined within a bandwidth to describe the power. For example,
Thermal noise at the input of a receiver is defined in terms of kT, Boltzmann’s constant times absolute temperature, in terms of Watts/Hz. Thus there is kT Watts of noise power in every Hz of bandwidth. For communications at “room temperature”, this is equivalent to –174 dBm/Hz or –204 dBW/Hz.
For typical applications, we are interested in Band-Limited White Noise where
𝑆 𝑓 𝑆𝑁2
, |𝑓| 𝐵
0, 𝐵 |𝑓|
The equivalent noise power is then:
𝐸 𝑋 𝑅 0 𝑆 ∙ 𝑑𝑓 2 ∙ 𝐵 ∙ 𝑆 2 ∙ 𝐵 ∙𝑁2
𝑁 ∙ 𝐵
For communications, we use kTB where W=B and N0=kT.
How much noise power, in dBm, would I say that there is in a 1 MHz bandwidth?
dBmBdBkTdBkTBdB 11460174
See: https://en.wikipedia.org/wiki/Johnson%E2%80%93Nyquist_noise
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 18 of 58 ECE 3800
Band Limited White Noise
𝑆 𝑓 𝑆𝑁2
, |𝑓| 𝑊
0, 𝐵 |𝑓|
The equivalent noise power is then:
𝐸 𝑋 𝑅 0 𝑆 ∙ 𝑑𝑓 2 ∙ 𝑊 ∙ 𝑆 2 ∙ 𝑊 ∙𝑁2
𝑁 ∙ 𝑊
Butwhatabouttheautocorrelation?
W
W
XX dfftiStR 2exp0
ti
Wti
ti
WtiS
ti
ftiStR
W
WXX
2
2exp
2
2exp
2
2exp00
ti
WtiiStRXX
2
2sin20
For xt
xtxt
sinc
WtSWtRXX 2sinc2 0
Using the concept of correlation, for what values will the autocorrelation be zero? (At these delays in time, sampled data would be uncorrelated with previous samples!)
,2,12
2
kforW
kt
kWt
Sampling at 1/2W seems to be a good idea, but isn’t that the Nyquist rate!!
Also note, noise passed through a filter becomes band-limited, and the narrower the filter the smaller the noise power … but the wider is the sinc autocorrelation function.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 19 of 58 ECE 3800
Noise and Filtered Noise Matlab Simulation
Based on Cooper and McGillem HW Problem 6-4.6.
x=randn(N,1); % zero mean, unit power random signal [b,a] = butter(4,20/500); y=filter(b,a,x); % applying a digital filter y=y/std(y); % normalizing the output power Rxx=xcorr(x)/(N+1); Ryy=xcorr(y)/(N+1); DFTx = fftshift(fft(x))/N; DFTy = fftshift(fft(y))/N; DFTRxx = fftshift(fft(Rxx,2*N))/N; DFTRyy = fftshift(fft(Ryy,2*N))/N;
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 20 of 58 ECE 3800
Pink_Noise … If we call constant at all frequencies white noise, then noise in a limited low frequency band is sometimes called pink noise.
OK. I’m getting ahead …. we just did this.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 21 of 58 ECE 3800
Section 9.3 S&W or Boncelet Section 13.5 Continuous-Time Linear Systems with Random Inputs
Linear system requirements:
Definition 9.3-1 Let x1(t) and x2(t) be two deterministic time functions and let a1 and a2 be two scalar constants. Let the linear system be described by the operator equation
txLty
then the system is linear if “linear super-position holds”
txLatxLatxatxaL 22112211
for all admissible functions x1 and x2 and all scalars a1 and a2.
For x(t), a random process, y(t) will also be a random process.
Linear transformation of signals: convolution in the time domain
txthty
th ty
System
tx
Linear transformation of signals: multiplication in the Laplace domain
sXsHsY
sX sH sY
The convolution Integrals (applying a causal filter)
0
dhtxty
or
t
dxthty
Where for physical realize-ability, causality, and stability constraints we require
00 tforth and
dtth
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 22 of 58 ECE 3800
Example: Applying a linear filter to a random process
03exp5 tfortth
tMtX 2cos4
where M and are independent random variables, uniformly distributed [0,2].
What can the linear filter do to the input signal?
(1) If a DC term passes through a linear filter, what can happen? The magnitude can change.
(2) If a cos or sin signal passes through a linear filter, what can happen? The magnitude and phase can change!
What do we expect at the output … a signal with a DC offset plus a cos where the magnitude and phase has changed! Now determine the value of the change!!
We can perform the filter function since an explicit formula for the random process is known.
t
dxthty
t
dMtty 2cos43exp5
tt
dtdtMty 2cos3exp203exp5
t
t
diiiit
tMty
2exp2exp3exp10
3
3exp5
t
i
iit
i
iitMty
23
2exp3exp
23
2exp3exp10
3
5
23
2exp
23
2exp10
3
5
i
iti
i
itiMty
49
2exp232exp2310
3
5 itiiitiiMty
ttM
ty 2sin22cos313
20
3
5
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 23 of 58 ECE 3800
Linear filtering will change the magnitude and phase of a sinusoidal signals (DC too!).
tMtX 2cos4
Using trig identities on the previous solution the result is …
69.33,2cos413
5
3
5 tMty
Expectedvalueoperatorwithlinearsystems
For a causal linear system we would have
0
dhtxty
and taking the expected value of the output
0
dhtxEtyE
0
dhtxEtyE
0
dhttyE
For x(t) WSS
00
dhdhtyE
Notice the condition for a physically realizable system!
The coherent gain of a filter is defined as:
00
Hdtthhgain
Therefore, 0HXEhXEtYE gain
The coherent gain of the filter is the sum/integral of the filter impulse respose!
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 24 of 58 ECE 3800
The Fourier Transform of the filter can help us – magnitude and phase at frequencies!
Note that:
dttfithfH 2exp
For a causal filter
0
2exp dttfithfH
At f=0
0
0 dtthH
And 0HtyE
Filters change magnitude and phase for deterministic signals and random signals the same amount at each defined frequency.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 25 of 58 ECE 3800
Whataboutacross‐correlation?(Convertinganauto‐correlationtocross‐correlation)
For a linear system we would have
dhtxty
And performing a cross-correlation (assuming real R.V. and processing)
dhtxtxEtytxE 2121
dhtxtxEtytxE 2121
dhtxtxEtytxE 2121
dhttRtytxE XX 2121 ,
For x(t) WSS
dhRRtytxE XXXY
hRRtytxE XXXY
What about the other way … YX instead of XY
And performing a cross-correlation (assuming real R.V. and processing)
2121 txdhtxEtxtyE
dhtxtxEtxtyE 2121
dhtxtxEtxtyE 2121
dhttRtxtyE XX 2121 ,
For x(t) WSS … see the next page
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 26 of 58 ECE 3800
For x(t) WSS
dhttRRtxtyE XXYX
dhRRtxtyE XXYX
Perform a change of variable for lamba to “-kappa” (assuming h(t) is real, see text for complex)
dhRRtxtyE XXYX
Therefore
dhRRtxtyE XXYX
hRRtxtyE XXYX
Whatabouttheauto‐correlationofy(t)?
And performing an auto-correlation (assuming real R.V. and processing)
222211112121 , dhtxdhtxEttRtytyE YY
112222112121 , dhdhtxtxEttRtytyE YY
112222112121 , dhdhtxtxEttRtytyE YY
112222112121 ,, dhdhttRttRtytyE XXYY
For x(t) WSS
122112 ddhhRRtytyE XXYY
112221 dhdhRRtytyE XXYY
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 27 of 58 ECE 3800
The output autocorrelation can also be defined in terms of the cross-correlation as
111 dhRRtytyE XYYY
hRRtytyE XYYY
The cross-correlation can be used to determine the output auto-correlation!
Continue in this concept, the cross correlation is also a convolution. Therefore,
hhRRtytyE XXYY
If h(t) is complex, the term in h(-t) must be a conjugate.
Looking forward … convolution in the time domain is multiplication in the frequency domain …
TheMeanSquareValueataSystemOutput
Based on the output autocorrelation formula
1221122 0 ddhhRRtyE XXYY
211122
2 0 ddhRhRtyE XXYY
dhRhRtyE XXYY 02
Based on the input to output cross-correlation formula
dhRRtytyE XYYY
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 28 of 58 ECE 3800
Example:WhiteNoiseInputstoacausalfilter
Let tNtRXX
20
0
122
0
2112 0 ddhRhRtYE XXYY
0
122
0
210
12
20 ddh
NhRtYE YY
0
11102
20 dhh
NRtYE YY
0
12
102
20 dh
NRtYE YY
For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 29 of 58 ECE 3800
Example:RCfilter
The RC low-pass filter
CRs
CRCRs
CsR
CssH
1
1
1
11
1
Inverse Laplace Transform
tuCR
t
CRth
exp1
Coherent Gain of the RC Filter
00
Hdtthhgain
0
exp1
dtCR
t
CRhgain
CR
CR
t
CRCR
CR
t
CRhgain
1
exp1
1
exp1 0
10
expexp1
CRCR
hgain
If driven by a white noise process, what is the output power?
0
202
2 dh
NtYE
0
2
02 exp1
2
dCRCR
NtYE
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 30 of 58 ECE 3800
0
2
02 2exp
1
2
dCRCR
NtYE
CR
CR
CR
NtYE
2
2exp
1
20
2
02
CR
NCR
NtYE
4
11
2
1
2 002
ComparingNoisePowerinthefilterbandwidth(equivalentnoisebandwidth)
Power in “rectangular” band-limited noise
𝐸 𝑁𝑁2∙
12𝜋
∙ 1 ∙ 𝑑𝑤𝑁2∙ 1 ∙ 𝑑𝑓
𝐸 𝑁𝑁2∙
12𝜋
∙ 2 ∙ 𝑊𝑁2∙ 2 ∙ 𝐵
𝐸 𝑁 𝑁 ∙𝑊2𝜋
𝑁 ∙ 𝐵
where WEQ is in rad/sec and BEQ in Hz
The noise power in an RC filter 𝐸 𝑌 𝑁 ∙∙ ∙
For an equivalent band-limited noise process to have the same power (assume a brick wall filter)
𝐸 𝑁 𝑁 ∙𝑊2𝜋
𝑁 ∙1
4 ∙ 𝑅 ∙ 𝐶 𝐸 𝑌
Solving for the equivalent noise bandwidth
𝑊2𝜋
14 ∙ 𝑅 ∙ 𝐶
Therefore in rad/sec 𝑊∙ ∙
of in Hz 𝐵∙ ∙
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 31 of 58 ECE 3800
Note that the nominal -3dB band (½ power) of an RC network is
RCW dB
13 or
RCB dB
2
13
Comparing these two, the equivalent noise bandwidth is greater than the –3dB bandwidth by
𝑊 ∙𝑊 or 𝐵 ∙ 𝐵
Note: B in Hz and W in rad/sec.
The equivalent noise bandwidth must take into account all the noise that would pass through the filter, not just the -3dB cut-off frequency value. Therefore, for low pass filters, we expect the equivalent noise bandwidth to be greater than the desired filter passband bandwidth.
0 0.5 1 1.5 2 2.5 3 3.5 4-0.2
0
0.2
0.4
0.6
0.8
1
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 32 of 58 ECE 3800
The power spectral density output of linear systems
The first cross-spectral density
hRR XXXY
diwRwS XYXY exp
diwhRwS XXXY exp
Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)
wHwSwS XXXY
The second cross-spectral density
hRR XXYX
diwRwS YXYX exp
diwhRwS XXYX exp*
Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)
*wHwSwS XXYX
The output power spectral density becomes
hhRR XXYY
diwRwS YYYY exp
diwhhRwS XXYY exp
Using convolution identities of the Fourier Transform
*wHwHwSwS XXYY
2wHwSwS XXYY
This is a very significant result that provides a similar advantage for the power spectral density computation as the Fourier transform does for the convolution.
This leads to the following table.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 33 of 58 ECE 3800
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 34 of 58 ECE 3800
Example 13.1
X(n) is a discrete WSS random sequence with a defined PSD.
𝑌 𝑛 𝑋 𝑛 𝛼 ∙ 𝑋 𝑛 1
This defines a causal digital filter.
ℎ 𝑛 𝛿 𝑛 𝛼 ∙ 𝛿 𝑛 1
Determine the discrete-time Fourier transform
𝐻 𝑤 ℎ 𝑛 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑛
𝐻 𝑤 𝛿 𝑛 𝛼 ∙ 𝛿 𝑛 1 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑛
𝐻 𝑤 1 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 0 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 1
𝐻 𝑤 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤
Finding the PSD of Y
𝑆 𝑤 𝑆 𝑤 ∙ |𝐻 𝑤 |
for
|𝐻 𝑤 | 𝐻 𝑤 ∙ 𝑐𝑜𝑛𝑗 𝐻 𝑤
|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 𝑐𝑜𝑛𝑗 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤
|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 ∙ 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤
|𝐻 𝑤 | 1 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 𝛼 ∙ 𝑒𝑥𝑝 𝑗 ∙ 𝑤 𝑎
|𝐻 𝑤 | 1 2 ∙ 𝛼 ∙ 𝑐𝑜𝑠 𝑤 𝑎
𝑆 𝑤 𝑆 𝑤 ∙ 1 2 ∙ 𝛼 ∙ 𝑐𝑜𝑠 𝑤 𝑎
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 35 of 58 ECE 3800
Signal plus noise processing
System analysis with a noise input …
tx
tn
th ty tr
Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered.
We have tntxtr
Assuming WSS with x and n independent and n zero mean
tntxtntxEtrtrERRR
tntntxtntntxtxtxERRR
NNXXRR RtxtnEtntxERR
NNNXXXRR RRR 2
For 0 mean noise …
NNXXRR RRR
And then
* hhRRR NNXXYY
hhRhhRR NNXXYY
Design considerations
the filter should not “modify” the signal of interest (unity gain, no phase)
the filter should remove as much noise as possible.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 36 of 58 ECE 3800
Signal‐to‐Noise‐(Power)‐RatioSNR(alwaysdoneforpowers)
The signal-to-noise ratio is the power ratio of the signal power to the noise power.
The input SNR is defined as
0
02
2
NN
XX
Noise
Signal
R
R
tNE
tXE
P
P
The output SNR is defined as
0
02
2
hhR
hhR
thtNE
thtXE
P
P
NN
XX
Noise
Signal
For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter.
We have
dhN
dwwHwSR XXYY202
22
10
With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise
𝑅 01
2𝜋∙ 𝑆 𝑤 ∙ |𝐻 𝑤 | 𝑁 ∙ 𝐵
or 𝑅 0 𝑅 0 𝑁 ∙ 𝐵
The output SNR is defined as
EQ
XX
Noise
Signal
BN
R
P
P
0
0
The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest!
From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example)
12
10
12
12
20 dh
NdhRthtNE NN
dffHdtthBEQ
22
2
1
2
1
Under the unity gain condition
dtth1
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 37 of 58 ECE 3800
Otherwise, the equivalent noise bandwidth can be defined as
dffHfH
BEQ
2
2max
12
For a real, low pass filter this simplifies to
dffHH
BEQ
2
20
12
Using Parseval’s Theorem this can be also defined as
dffHdwwHdtth222
2
1
2
2
2
2
02
dtth
dtth
H
dtth
BEQ
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 38 of 58 ECE 3800
Example Section 13.7 Amplitude Modulation.
𝑋 𝑡 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 Where m(t) is a random message, typically WSS, zero mean and bounded by +/-1. A is a an amplitude, f is the center frequency and there is a random phase angle (uniform distribution around a circle). The R.V. are independent.
𝐸 𝑋 𝑡 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃
𝐸 𝑋 𝑡 𝐴 ∙ 𝐸 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝐸 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃
𝐸 𝑋 𝑡 𝐴 ∙ 1 0 ∙ 0 0
Autocorrelation
𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝑅 𝑡, 𝑡 𝜏
𝑅 𝑡, 𝑡 𝜏 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃
∙ 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃
𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝐴 ∙ 𝐸1 𝛽 ∙ 𝑚 𝑡 𝛽 ∙ 𝑚 𝑡 𝜏 𝛽 ∙ 𝑚 𝑡 ∙ 𝑚 𝑡 𝜏∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃
𝑅 𝑡, 𝑡 𝜏 𝐴 ∙ 1 𝛽 ∙ 𝑅 𝜏 ∙12∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏
𝑅 𝜏𝐴2∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏
𝐴2∙ 𝛽 ∙ 𝑅 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏
Forming the PSD
𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓
𝐴4∙ 𝛽 ∙ 𝑆 𝑤 ∗ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓
Which after performing the convolution becomes
𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓
𝐴4∙ 𝛽 ∙ 𝑆 𝑤 2𝜋 ∙ 𝑓 𝑆 𝑤 2𝜋 ∙ 𝑓
Your textbook simplifies the problem a bit … dealing with only the message and not the additional carrier component.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 39 of 58 ECE 3800
ExamplesofLinearSystemFrequency‐DomainAnalysis
Noise in a linear feedback system loop.
sX sY
1
1 ssA
sN
Linear superposition of X to Y and N to Y.
sNsYsXss
AsY
1
sNsXss
A
ss
AsY
11
1
sNsXss
A
ss
AsssY
11
2
sNAss
sssX
Ass
AsY
2
2
2
There are effectively two filters, one applied to X and a second apply to N.
Ass
AsH X
2 and
Ass
sssH N
2
2
sNsHsXsHsY NX
Generic definition of output Power Spectral Density:
wSwHwSwHwS NNNXXXYY 22
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 40 of 58 ECE 3800
Change in the input to output signal to noise ratio.
dwwS
dwwS
SNR
NN
XX
In
dwwHN
wSH
dwwSwH
dwwSwH
SNR
N
XXX
NNN
XXX
Out20
2
2
2
2
0
EQ
XX
X
N
XX
Out BN
wS
dwH
wHN
wS
SNR
0
2
2
0
02
Where for this special case … (HN not a low pass or band pass filter … bad example)
dwH
wHB
X
NEQ 2
2
0
If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 41 of 58 ECE 3800
Systems that Maximize Signal-to-Noise Ratio – Advanced Concept
SNR is defined as
EQNoise
Signal
BN
tsE
P
P
0
2
Define for an input signal tnts
Define for a filtered output signal tnts oo
For a linear system, we have:
0
dtntshtnts oo
The input SNR can be describe as
2
2
tnE
tsE
P
PSNR
Noise
Signalin
The output SNR can be described as
EQo
o
o
o
Noise
Signalout BN
tsE
tnE
tsE
P
PSNR
2
2
2
0
2
2
0
2
1dtthN
dtshE
SNR
o
out
Using Schwartz’s Inequality the numerator becomes
0
2
0
2
2
0
dtsEdhEdtshE
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 42 of 58 ECE 3800
Applying this inequality, an SNR inequality can be defined as
0
2
0
2
0
2
2
1dtthN
dtsEdh
SNR
o
out
Canceling the filter terms, we have
0
22 dtsEN
SNRo
out
To achieve the maximum SNR, the equality condition of Schwartz’s Inequality must hold, or
0
2
0
2
2
0
dtsdhdtsh
This condition can be met for utsKh
where K is an arbitrary gain constant.
The desired impulse response is simply the time inverse of the signal waveform at time t, a fixed moment chosen for optimality. If this is done, the maximum filter power (with K=1) can be computed as
tdsdtsdht
2
0
2
0
2
tN
SNRo
out 2
max
This filter concept is called a matched filter. – Advanced Concept
If you wanted to detect a burst waveform that has been transmitted, to maximize the received SNR in white noise, the receiving filter should be the time inverse of the signal transmitted!
Note and caution: when using such a filter, the received signal maximum SNR will occur when the signal and convolved filter perfectly overlap. This moment in time occurs when the “complete” burst has been received by the system. If measuring the time-of-flight of the burst, the moment is exactly the filter length longer than the time-of-flight. (Think about where the leading edge of the signal-of-interest is when transmitted, when first received, and when fully present in the filter).
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 43 of 58 ECE 3800
TheMatchedFilter–AdvancedConcept
Wikipedia: https://en.wikipedia.org/wiki/Matched_filter
“In signal processing, a matched filter is obtained by correlating a known signal, or template, with an unknown signal to detect the presence of the template in the unknown signal.[1][2] This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the template. The matched filter is the optimal linear filter for maximizing the signal to noise ratio (SNR) in the presence of additive stochastic noise. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the out-going signal. Pulse compression is an example of matched filtering. It is so called because impulse response is matched to input pulse signals. Two-dimensional matched filters are commonly used in image processing, e.g., to improve SNR for X-ray. Matched filtering is a demodulation technique with LTI (linear time invariant) filters to maximize SNR.[3].”
Applications:
Radar Sonar Pulse Compression Digital Communications (Correlation detectors) GPS pseudo-random sequence correlation
If you are looking for a signal, maximize the output of the filter when the signal is input! You will have a matched filter!
See ChirpCorrelationReceiver.m
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 44 of 58 ECE 3800
Cooper & McGillem 9-6 Another optimal solution: Systems that Minimize the Mean-Square Error between the desired output and actual output – Advanced Concept
The error function tYtXtErr
where
0
dtNtXhtY
Performed in the Laplace Domain
sFsFsHsFsFsFsF NXXYXE
sFsHsHsFsFsFsF NXYXE 1
Computing the error power
j
j
NNXX dssHsHsSsHsHsSj
ErrE 112
12
j
j XXXXXX
NNXX dssSsHsSsHsS
sHsHsSsS
jErrE
2
12
Defining the input PSD sSsSsFsF NNXXCC
j
j
CC
NNXX
C
XXC
C
XXC
ds
sFsF
sSsS
sF
sSsHsF
sF
sSsHsF
jErrE
2
12
We can not do much about the last term, but we can minimize the terms containing H(s). Therefore, we focus on making the following happen
0
sF
sSsHsF
C
XXC
Step 1:Note that for this filter 1s- F
1
s F
1
CC sSsS NNXX
This is called a whitening filter as it forces the signal plus noise PSD to unity (white noise).
Step 2: Letting sF
sHsHsHsHC
221
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 45 of 58 ECE 3800
Minimizing the terms containing H2(s), now we must focus on
sF
sSsH
C
XX
2 and
sF
sSsH
C
XX2
Letting H2 be defined for the appropriate Left or Right half-plane poles
Let LHPC
XX
sF
sSsH
2 and RHPC
XX
sF
sSsH
2
The composite filter is then
LHPC
XX
C sF
sS
sFsHsHsH
1
21
This solution is often called a Wiener Filter and is widely applied when the signal and noise statistics are known a-priori!
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 46 of 58 ECE 3800
13.8 Example: Discrete Time Wiener Filter – Advanced Concept
Your textbook works to derive a discrete form of the Wiener Filter.
In it, you must calculate the coefficients of a Finite-Impulse-Response (FIR) digital filter.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 47 of 58 ECE 3800
Cooper & McGillem - Eigenvalue Based Filters – Advanced Concept We can continue a derivation started in the previous class discussion about time-sample filters, matrices and eigenvalues.
kxkwky
knkskx The expected value
HH kxkwkxkwEkykyE
HXXH kwkRkwkykyE
For a WSS input
HXXH kwRkwkykyE
If the signal and noise are zero mean, this becomes
HNNSSH kwRRkwkykyE
How do we maximize the output SNR
HNN
HSS
Noise
Signal
kwRkw
kwRkw
P
P
If we assume that the noise is white, IR NNN 2
H
HSS
NH
HSS
NNoise
Signal
kwkw
kwRkw
kwIkw
kwRkw
P
P
22
11
Performing a cholesky factorization of the signal autocorrelation matrix generates the following. Here, the numerator should suggest that an eigenvalue computation could provide a degree of simplification.
H
HHSS
NNoise
Signal
kwkw
kwRRkw
P
P
2
1
Once formed, the eigenvalue equation to solve is kwRkw S
which result in solutions for the resulting eigenvalues and eigenvectors of the form
2
2
2
1
NH
H
NNoise
Signal
kwkw
kwkw
P
P
Selecting the maximum eigenvalue and it’s eigenvector for the weight that maximizes the SNR!
2
2max
NNoise
Signal
P
P
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 48 of 58 ECE 3800
13.9 The sampling Theorem for WSS Random Processes – Advanced Concept
If you take ECE 4550, you will be dealing with the sampling theorem.
It involves discrete time sampling of a continuous signal and the conditions under which the continuous signal can be regenerated from the discrete time samples. .
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 49 of 58 ECE 3800
Advanced Topic Adaptive Filter – Advanced Concept
If a desired signal reference is available, we may wish to adapt a system to minimize the difference between the desired signal or signal characteristics and a filter input signal.
The following information is based on:
S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014.
There are four classes of Adaptive Filter Applications
Identification
Inverse Modeling
Prediction
Interference Cancellation
Identification
The mathematical Model of an “unknown plant”
In state space control system this is an adaptive observer of the Plant
Examples: Seismology predicting earth strata
InverseModeling
Providing an “Inverse Model” of the plant
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 50 of 58 ECE 3800
For a transmission medium, the inverse model corrects non-ideal transmission characteristics.
An adaptive equalizer
Prediction
Based on past values, provide the best prediction possible of the present values.
Positioning/Navigation systems often need to predict where an object will be based on past observations
InterferenceCancellationExample
Cancellation of unknown interference that is present along with a desired signal of interest. Two sensors of signal + interference and just interference Reference signal (interference) is used to cancel the interference in the Primary signal
(noise + interference)
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 51 of 58 ECE 3800
Classic Examples: Fetal heart tone monitors, spatial beamforming, noise cancelling headphones.
From: S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
The “Reference signal” contains the unwanted interference. The goal of the adaptive filter is to match the reference signal with the “interference” in the “Primary signal and force the output “difference error” to be minimized in power. Since “interference” is the only thing available to work with, the “power minimum” solution would be one where the interference is completely removed!
These techniques are based on the Weiner filter solution. While the signal and interference statistics are not known a-priori (before the filter gets started), after a number of input samples they can be estimated and used to form the filter coefficients. Then, as time continues, there is a sense that the estimates should improve until the adaptive coefficients are equal to those that would be computed with a-priori information.
The advantage ... adaptive filter can work when the statistics are slowly time varying!
Note: An application using an LMS adaptive filter is not too difficult for a senior project! (noise cancelling headphones, remove 60 cycle hum, etc.)
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 52 of 58 ECE 3800
Textbook:Cancellinganinterferingwaveform–AdvancedConcept
The example in your textbook p. 407-411.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-20
-10
0
10
20or
igin
al
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-10
-5
0
5
10
15
Filt
ered
Time (sec)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-8
-6
-4
-2
0
2
4
6Adaptive Weights in Time
Tim
e (s
ec)
Weights
a1
a2
a3
a4
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 53 of 58 ECE 3800
Matlab % % Text Adaptive Filter Example % clear close all t=0:1/200:1; % Interfereing Signal - 60 Hz n=10*sin(2*pi*60*t+(pi/4)*ones(size(t))); % Signal of interest and S + I x1=1.1*(sin(2*pi*11*t)); x2=x1+n; % Reference signal to excise r=cos(2*pi*60*t); m=0.15; a=zeros(1,4); z=zeros(1,201); z(1:4)=x2(1:4); w(1,:)=a'; w(2,:)=a'; w(3,:)=a'; w(4,:)=a'; % Adaptive weight computation and application for k=4:200 a(1)=a(1)+2*m*z(k)*r(k); a(2)=a(2)+2*m*z(k)*r(k-1); a(3)=a(3)+2*m*z(k)*r(k-2); a(4)=a(4)+2*m*z(k)*r(k-3); z(k+1)=x2(k+1)-a(1)*r(k+1)-a(2)*r(k)-a(3)*r(k-1)-a(4)*r(k-2); w(k+1,:)=a'; end figure(1) subplot(2,1,1); plot(t,x2,'k') ylabel('original') subplot(2,1,2) plot(t,z,'k');grid; ylabel('Filtered'); xlabel('Time (sec)'); figure(2) plot(t,w);grid; title('Adaptive Weights in Time') ylabel('Time (sec)') xlabel('Weights') legend('a1','a2','a3','a4');
Review Skills 7 and Skills 8 from the solution web site. Exam based questions! Hopefully you have looked at them and potentially tried a few.
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 54 of 58 ECE 3800
Point totals beyond 3 for this homework will be considered extra credit …
Highlights from Skills 8 include the following
36.1 Suppose the circuit shown below has input ttx 9sin6 , where is a random
variable uniformly distributed on [0,2pi]. Assuming the R=1 M and C=1uF,
a. If the output signal is y(t), find the transfer function of the circuit. (H(s) possible given)
sX
sCR
RsY
1
sCR
sCRsH
1 and
s
ssH
1
111 2
2
s
s
s
s
s
ssHsH
b. Find the spectral density Syy(w) of the output and simplify. (Need AC. of Rxx and PSD Sxx)
99sin69sin6 ttERXX
99sin9sin36 ttERXX
2918cos9cos2
36 tERXX
9cos18 XXR
dtjwttwS XX exp9cos18
dtjwttjtjwS XX exp9exp9exp9
999 wwwS XX
Now the PSD of the output can be computer
1
9992
2
w
wwwwSYY
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 55 of 58 ECE 3800
9982
81999
19
99
2
2
wwwwwSYY
c. Sketch the spectral density Syy(w).
Syy(w) is almost identical to Sxx(w). The filter used is a high-pass filter with a relative cutoff frequency w0 of 1.
d. What would happen if the input term had a “DC” component? What would the filter output for signals at w=0 be?
36.5 Consider the following linear circuit, where x(t) is the input voltage signal and y(t) is the output voltage signal.
x(t)
C
y(t)
LR
a. Find the transfer function of this system. For R=25 ohms, L=5H and C=0.05F (Note: these are not realistic values!)
sXsLsCR
sCsY
1
1
05.052505.01
1
1
122
ssLCssCR
sH
41
4
125.01
1
25.025.11
12
ssssss
sH
161
16
41
4
41
422
ssssss
sHsH
b. Assume the input signal is 21 5cos42cos312 tttx , where 1 and 2 are independent random variables that are uniform on the interval [0,2pi]. From this compute (1) the autocorrelation of the input signal and (2) the power spectral density of the input signal.
21
21
55cos422cos312
5cos42cos312
tt
ttERXX
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 56 of 58 ECE 3800
22
122
21
111
21
55cos5cos16
22cos5cos125cos48
55cos2cos12
22cos2cos92cos36
55cos4822cos36144
tt
ttt
tt
ttt
tt
ERXX
2
2121
1212
1
2510cos2
165cos
2
16
27cos2
1223cos
2
12
58cos2
1253cos
2
12
224cos2
92cos
2
9144
t
tt
tt
t
ERXX
5cos82cos2
9144 XXR
dtjwtwS XX exp5cos82cos
2
9144
558222
9288 wwwwwwS XX
c. Compute (1) the autocorrelation of the output signal and (2) the spectral density of the output signal. Simplify all answers.
Output Power Spectrum sHsHsSsS XXYY
161
1622
ww
wSwS XXYY
161
16
558222
9288
22
ww
wwwwwwSYY
161
16558
161
1622
2
9
161
16288
22
22
22
wwww
wwww
www
wSYY
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 57 of 58 ECE 3800
16515
16558
16212
1622
2
9
16010
16288
22
22
22
ww
ww
w
wSYY
55533
64
22100
72
288
551066
168
22100
16
2
9
288
ww
ww
w
ww
ww
w
wSYY
dwjwwSR YYYY
exp2
1
dwjw
ww
wwwRYY
exp
55533
64
22100
72288
2
1
5cos533
642cos
100
72144 YYR
d. Compute the ratio of the output to input total average power of the signals.
5cos82cos2
9144
5cos533
642cos
100
72144
0
0
XX
YY
in
out
R
R
P
P
9255.0
5.156
84.144
82
9144
533
64
100
72144
0
0
XX
YY
in
out
R
R
P
P
e. Compute the ratio of the output to input dc average power of the two signals. 1
144
144
XX
YY
in
out
R
R
P
P
f. From the above, comment on the filtering effect of the original LCR circuit. Is it a low pass, band pass, or high pass filter?
Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and
Random Signals, Oxford University Press, February 2016.
B.J. Bazuin, Fall 2020 58 of 58 ECE 3800
This is a low-pass filter where dc is passed from the input to the output. For the selected input signal, with oscillation at w=2 and w=5, the filter first rolls off at w=1 and then continues at w=4.
The input to output power for w=3 is 16.025
4
9
72
100
2
2
9100
72
in
out
P
P
The input to output power for w=5 is 015.0533
8
8533
64
in
out
P
P