Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal...
Transcript of Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal...
![Page 1: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/1.jpg)
Slide 1
Gerhard Schmidt
Christian-Albrechts-Universität zu KielFaculty of Engineering Institute of Electrical and Information EngineeringDigital Signal Processing and System Theory
Adaptive Filters – Linear Prediction
![Page 2: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/2.jpg)
Slide 2Slide 2Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Today:
Contents of the Lecture
Source-filter model for speech generation
Derivation of linear prediction
Levinson-Durbin recursion
Application example
![Page 3: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/3.jpg)
Slide 3Slide 3Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Motivation
Human Speech Generation
and Appropriate Modelling
![Page 4: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/4.jpg)
Slide 4Slide 4Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Motivation
Speech Production
Principle:
An airflow, coming from the lungs, excites the vocal cordsfor voiced excitation or causesa noise-like signal (openedvocal cords).
The mouth, nasal, and pharynx cavity are behaving like controllable resonators and only a few frequencies (calledformant frequencies) are notattenuated.
Sourcepart
Muscleforce
Lungvolume
Vocal cords
Pharynxcavity
Mouthcavity
Nasalcavity
Filterpart
![Page 5: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/5.jpg)
Slide 5Slide 5Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Speech Production
Source-Filter Model
¾(n)
Vocal tract filter
Impulse generator
Noise generator
Fundamentalfrequency
Source partof the model
Filter partof the model
![Page 6: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/6.jpg)
Slide 6Slide 6Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Literature
Books
E. Hänsler / G. Schmidt: Acoustic Echo and Noise Control – Chapter 6 (Linear Prediction),Wiley, 2004
Basic text:
Further basics:
E. Hänsler: Statistische Signale: Grundlagen und Anwendungen – Chapter 6 (Linearer
Prädiktor), Springer, 2001 (in German)
M. S. Hayes: Statistical Digital Signal Processing and Modeling – Chapters 4 und 5 (SignalModeling, The Levinson Recursion), Wiley, 1996
Speech processing:
P. Vary, R. Martin: Digital Transmission of Speech Signals – Chapter 2 (Models of SpeechProduction and Hearing), Wiley 2006
J. R. Deller, J. H. l. Hansen, J. G. Proakis: Discrete-Time Processing of Speech Signals –Chapter 3 (Modeling Speech Production), IEEE Press, 2000
![Page 7: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/7.jpg)
Slide 7Slide 7Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Basics
Basics of
Linear Prediction
![Page 8: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/8.jpg)
Slide 8Slide 8Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Basic Approach
Estimation of the current signal sample on the basis of the previous samples:
With:
: estimation of : length / order of the predictor
: predictor coefficients
Linear prediction filter
![Page 9: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/9.jpg)
Slide 9Slide 9Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Optimization Criterion
Optimization:
Estimation of the filter coefficients such that a cost function is optimized.
Cost function:
Structure:
Linearprediction filter
![Page 10: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/10.jpg)
Slide 10Slide 10Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
„Whitening“ Property
Cost function:
Strong frequencycomponents will be attenuated most (due toPerceval).
This leads to a spectral„decoloring“ (whitening) of the signal.
![Page 11: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/11.jpg)
Slide 11Slide 11Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Inverse Filter Structure
FIR filter (sender)
All-pole filter (receiver)
Properties:
The inverse predictorerror filter is anall-pole filter
The cascaded structure- consisting of a predictor error filterand an inverse predictor error filter -can be used for lossless data compression and for sending and receiving signals.
![Page 12: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/12.jpg)
Slide 12Slide 12Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Computing the Filter Coefficients
Derivation during the lecture …
![Page 13: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/13.jpg)
Slide 13Slide 13Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Examples – Part 1
First example:
Input signal : white noise with variance (zero mean)
Prediction order:
Prediction of the next sample:
This leads to:
, what means the no prediction is possible or – to be precise – the best prediction is the mean of the input signal which is zero.
, respectively
![Page 14: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/14.jpg)
Slide 14Slide 14Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Examples – Part 2
Second example:
Input signal : speech, sampled at kHz
Prediction order:
Prediction of the next sample:
New adjustment of thefilter coefficients every 64 samples
Single optimizationof the filter coefficients
for the entire signal sequence
![Page 15: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/15.jpg)
Slide 15Slide 15Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Estimation of the Autocorrealtion Function – Part 1
Problem:
Ensemble averages are usually not known in most applications.
Solution:
Estimation of the ensemble averages by temporal averaging (ergodicity assumed):
Assumption:
is a representative signal of the underlying random process.
Estimation schemes:A few schemes for estimating an autocorrelation function exist. These scheme differ in the properties (such as unbiasedness or positive definiteness) that the resulting autocorrelation gets significantly.
![Page 16: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/16.jpg)
Slide 16Slide 16Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Estimation of the Autocorrealtion Function – Part 2
Example: „Autocorrelation method“:
Computed according to:
Properties:
The estimation is biased, we achieve:
But we obtain:
The resulting (estimated) autocorrelation matrix is positive definite.
The resulting (estimated) autocorrelation matrix has Toeplitz structure.
![Page 17: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/17.jpg)
Slide 17Slide 17Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 1
Problem:
The solution of the equation system
has – depending on how the autocorrelation matrix is estimated – a complexity proportional to or , respectively. In addition numerical problems can occour if the matrix is ill-conditioned.
Goal:A robust solution method that avoids direct inversion of the matrix .
SolutionExploiting the Toeplitz structure of the matrix :
Recursion over the filter order
Combining forward and backward prediction
Literature: J. Durbin: The Fitting of Time Series Models, Rev. Int. Stat. Inst., no. 28, pp. 233 - 244, 1960
N. Levinson: The Wiener RMS Error Criterion in Filter Design and Prediction, J. Math. Phys., no. 25,
pp. 261 - 268, 1947
![Page 18: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/18.jpg)
Slide 18Slide 18Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 2 (Backward Prediction)
Equation system of the forward prediction:
Changing the equation order:
![Page 19: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/19.jpg)
Slide 19Slide 19Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 3 (Backward Prediction)
After rearranging the equations:
Changing the order of the elements on the right side:
![Page 20: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/20.jpg)
Slide 20Slide 20Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 4 (Backward Prediction)
After changing the order of the elements on the right side:
Matrix-vector notation:
![Page 21: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/21.jpg)
Slide 21Slide 21Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 5 (Backward Prediction)
Matrix-vector notation:
Due to symmetry of the autocorrelation function:
Backward prediction by N samples:
![Page 22: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/22.jpg)
Slide 22Slide 22Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 6 (Derivation of the Recursion)
Derivation during the lecture …
![Page 23: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/23.jpg)
Slide 23Slide 23Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 7 (Basic Structure of Recursive Algorithms)
Estimated signal using a prediction filter of length :
Inserting the recursion :
Forward predictorof length N-1
Backward predictorof length N-1
Additionalsample
Innovation
![Page 24: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/24.jpg)
Slide 24Slide 24Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 8 (Basic Structure of Recursive Algorithms)
Backward predictor of lenght N-1
Forward predictor of length N-1
Forward predictor of length N
Structure that shows the recursion over the order:
In short form:
New estimation = old estimation + weighting * (new sample – estimated new sample)
![Page 25: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/25.jpg)
Slide 25Slide 25Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 9 (Recursive Computation of the Error Power)
Derivation during the lecture …
![Page 26: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/26.jpg)
Slide 26Slide 26Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Levinson-Durbin Recursion – Part 10 (Summary)
Initialization
Predictor:
Error power (optional):
Recursion:
Reflection coefficient:
Forward predictor:
Backward predictor:
Error power (optional):
Condition for termination:
Numerical problems:
Order: If the desired filter order is reached, stop the recursion.
If is true, use the coefficients of the previous recursion and fill the missing coefficients with zeros.
![Page 27: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/27.jpg)
Slide 27Slide 27Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Matlab Demo
![Page 28: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/28.jpg)
Slide 28Slide 28Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Matlab Demo – Input Signal and Estimated Signal
![Page 29: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/29.jpg)
Slide 29Slide 29Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Linear Prediction
Matlab Demo – Error Signals
![Page 30: Adaptive Filters Linear Prediction - dss.tf.uni-kiel.de · M. S. Hayes: Statistical Digital Signal Processing and Modeling –Chapters 4 und 5 (Signal Modeling, The Levinson Recursion),](https://reader036.fdocuments.us/reader036/viewer/2022062504/5b15fc1f7f8b9a472e8c4a0f/html5/thumbnails/30.jpg)
Slide 30Slide 30Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction
Adaptive Filters – Linear Prediction
Summary and Outlook
This week:
Source-filter model for speech generation
Derivation of linear prediction
Levinson-Durbin recursion
Application example
Next week:
Adaptation algorithms – part 1