Digital Speech ProcessingDigital Speech Processing——Lecture 13Lecture 13Lecture 13Lecture 13

Linear Predictive Linear Predictive Coding (LPC)Coding (LPC)Coding (LPC)Coding (LPC)--IntroductionIntroduction

1

LPC MethodsLPC MethodsLPC MethodsLPC Methods• LPC methods are the most widely used in

speech coding speech synthesis speechspeech coding, speech synthesis, speech recognition, speaker recognition and verification and for speech storageand for speech storage– LPC methods provide extremely accurate estimates

of speech parameters, and does it extremely efficiently

– basic idea of Linear Prediction: current speech sample can be closely approximated as a linearsample can be closely approximated as a linearcombination of past samples, i.e.,

∑p

21

( ) ( ) for some value of , 'sα α=

= −∑ k kk

s n s n k p

LPC MethodsLPC MethodsLPC MethodsLPC Methods

for periodic signals with period , it is obvious that

( ) ( )

•

≈ −p

p

N

s n s n N

but that is not what LP is doing; it is estimating ( ) from the ( ) most recent values of ( ) by linearly

di ti

<< p

s np p N s n

it l predicting its value for LP, the predictor coefficients (the 's) are determined (computed) by

α• k

minimizing the sum of squared differences (computed) by (over a finite interval)

minimizing the sum of squared differences between the actual speech samples

and the linearly predicted ones

3

y p

LPC MethodsLPC Methods LP is based on speech production and synthesis models

- speech can be modeled as the output of a linear, time-varying system, excited by either quasi-periodic

•

pulses or noise; - assume that the model parameters remain constant over speech analysis interval LP provides a for

estimating the parameters of the linear system (the comi robust, reliable and accurate method

bined vocal tract, glottal pulse, and radiation characteristic for voiced speech)

4

LPC MethodsLPC MethodsLPC MethodsLPC Methods• LP methods have been used in control and

information theory—called methods of system estimation and system identification

d t i l i h d f– used extensively in speech under group of names including

1. covariance method2. autocorrelation method3. lattice method4. inverse filter formulation5. spectral estimation formulation6. maximum likelihood method7. inner product method

5

Basic Principles of LPBasic Principles of LPBasic Principles of LPBasic Principles of LP( ) ( ) ( )= − +∑

p

ks n a s n k Gu n1=∑k

the time varying digital filter• the time-varying digital filter represents the effects of the glottal pulse shape, the vocal tract IR, and radiation at the lips

1( )( ) = =S zH z

p

• the system is excited by an impulse train for voiced speech, or a random noise sequence for unvoiced speech

1

1

( )( ) −

=

= =

−∑p

kk

k

H zGU z

a z • this ‘all-pole’ model is a natural representation for non-nasal voiced speech—but it also works reasonably

6

well for nasals and unvoiced sounds

LP Basic EquationsLP Basic EquationsLP Basic EquationsLP Basic Equationsa order linear predictor is a system of the form• thp

1 1

a order linear predictor is a system of the form

( ) ( ) ( ) ( )( )

th di ti ( ) i f th f

α α −

= =

= − ⇔ = =∑ ∑p p

kk k

k k

p

S zs n s n k P z zS z

1

the prediction error, ( ), is of the form

( ) ( ) ( ) ( ) ( )α=

•

= − = − −∑p

kk

e n

e n s n s n s n s n k

the pr•

1

1 1

ediction error is the output of a system with transfer function

( ) ( ) ( )( )

α −= = − = −∑p

kk

k

E zA z P z zS z 1

1

( )

if the speech signal obeys the production model exactly, and if ,( ) ( )and ( )

α=

• = ≤ ≤

⇒ =

k

k ka k pe n Gu n A z

1

is an inverse filter for ( ), i.e., H z

7

1 ( )( )

=H zA z

LP Estimation IssuesLP Estimation IssuesLP Estimation IssuesLP Estimation Issues

need to determine { } directly from speech suchα• need to determine { } directly from speech such that they give good estimates of the time-varying spectrum

α• k

need to estimate { } from short segments of speech

need to minimize mean-squared pr

α•

•k

ediction error over

short segments of speech resulting { } assumed to be the actual { } in the

h d ti d lα• k ka

speech production model=> intend to show that all of this can be done efficiently, reliably, and for speechaccurately

8

y, py

Solution forSolution for {αk}Solution for Solution for {αk} short-time average prediction error is defined as•

( )22ˆ ˆ ˆ ˆ

g

( ) ( ) ( )= = −∑ ∑n n n nm m

E e m s m s m

2

1ˆ ˆ( ) ( )α

⎛ ⎞⎜ ⎟= − −⎜ ⎟⎝ ⎠

∑ ∑p

kn nk

s m s m k1

ˆ ˆ select segment of speech ( ) ( ) in the vicinity ˆof sample

=⎝ ⎠• = +

m k

ns m s m nnof sample n

the key issue to resolve is the range of for summation(to be discussed later)• m

9

(to be discussed later)

Solution for Solution for {αk}kˆ

ˆ

can find values of that minimize by setting:α•

∂k nE

E0 1 2 , , ,...,

giving the set of equationsα

∂= =

∂

•

n

i

Ei p

1

2 0 1ˆ ˆ ˆˆ - ( )[ ( ) ( )] ,α=

− − − = ≤ ≤∑ ∑∑

p

kn n nm k

s m i s m s m k i p

2 0 1ˆ ˆ( ) ( ) ,

ˆ where α

− − = ≤ ≤

•

∑ n nm

s m i e m i p

ˆ are the values of that minimize (from nowαk k nE

ˆ ˆ

(ˆ on just use rather than for the optimum values)

prediction error ( ( )) is orthogonal to signal ( ( )) for α α

• −

k k n

k k

n ne m s m i

10

delays ( ) of 1 to i p

Solution for Solution for {αk}k

d fi i

ˆ ˆ ˆ

defining

( , ) ( ) ( )φ

•

= − −∑n n ni k s m i s m k

we get•

∑

m

p

1

0 1 2ˆ ˆ ( , ) ( , ), , ,...,

leading to a set of equations in unknowns that can be

α φ φ=

= =

•

∑ k n nk

i k i i p

p p leading to a set of equations in unknowns that can be

solved in an efficient manner for the { }α

•

k

p p

11

Solution forSolution for {αk}Solution for Solution for {αk} minimum mean-squared prediction error has the form•

p2

1ˆ ˆ ˆ ˆ ( ) ( ) ( )

which can be written in the form

α=

= − −∑ ∑ ∑p

kn n n nm k m

E s m s m s m k

0 0 0ˆ ˆ ˆ

which can be written in the form

( , ) ( , )φ α φ

•

= −∑p

kn n nE k1

ˆ

Process: 1. compute ( , ) forφ

=k

n i k 1 0,≤ ≤ ≤ ≤i p k pp ( , )φn

ˆ

,2. solve matrix equation for

need to specify range of to compute ( , )α

φ•k

n

p p

m i k

12ˆ need to specify ( )• ns m

Autocorrelation MethodAutocorrelation MethodAutocorrelation MethodAutocorrelation Method0 1ˆ assume ( ) exists for and is ≤ ≤ −i ns m m L( )

exactly zero everywhere else (i.e., window of length samples) ( )Assumption #1

n

L0 1ˆ ˆ ( ) ( ) ( ),

where ( ) is a finite length window of = + ≤ ≤ −

ins m s m n w m m L

w m length samplesL

^ ^

m=0 m=L-1

n n+L-1^ ^

13

Autocorrelation MethodAutocorrelation Method0 1ˆ

ˆ ˆ ˆ

if ( ) is non-zero only for then

( ) ( ) ( )α

⋅ ≤ ≤ −

= − −∑n

p

k

s m m L

e m s m s m k1

12 2

0 1

ˆ ˆ ˆ

( ) ( ) ( )

is non-zero only over the interval , giving

( ) ( )

α=

− +∞

=

≤ ≤ − +

= =

∑

∑ ∑

kn n nk

L p

e m s m s m k

m L p

E e m e m0

( ) ( )

at values of near 0 (i.e., =−∞ =

=

∑ ∑i

n n nm m

E e m e m

m m 0 1 1

1 1ˆ

, ...., ) we are predicting signal from zero-valued samples outside the window range => ( ) will be (relatively) large

t l (i ) di ti l

−

n

pe m

L L L L d l1 1 at values near (i.e., , ,..., ) we are predicting zero-valu= = + + −i m L m L L L p

ˆ

ed samples (outside window range) from non-zero samples => ( ) will be (relatively) large for these reasons, normally use windows that taper the segment to zero (e.g., Hamming window)•

ne m

m=0 m=L

14m=p-1 m=L+p-1

Autocorrelation MethodAutocorrelation MethodAutocorrelation MethodAutocorrelation Method

15

The “Autocorrelation Method”The “Autocorrelation Method”

^ ^ˆ 1n L+ −

ˆ[ ] [ ] [ ]s m s m n w m= +ˆ [ ] [ ] [ ]ns m s m n w m= +

L-1

1

01 2ˆ ˆ ˆ[ ] [ ] [ ] , , ,

− −

=

= + =∑ …L k

n n nm

R k s m s m k k p

16

0=m

The “Autocorrelation Method”The “Autocorrelation Method”

n̂ ˆ 1n L+ −n̂ ˆ 1n L+ −ˆ[ ] [ ] [ ]s m s m n w m= +ˆ[ ] [ ] [ ]ns m s m n w m= +

Large errors [ ] [ ] [ ]p

e m s m s m kα= ∑1L−

ˆ ˆ[ ] [ ] [ ]ns m s m n w m= +g

at ends of window

ˆ ˆ ˆ1

[ ] [ ] [ ]n n k nk

e m s m s m kα=

= − −∑

171L − ( 1 )L p− +

Autocorrelation MethodAutocorrelation MethodAutocorrelation MethodAutocorrelation Method

1

0

0 0 1

1 0

ˆ ˆ

ˆ ˆ ˆ

for calculation of ( , ) since ( ) outside the range , then

( , ) ( ) ( ), ,

φ

φ− +

=

• = ≤ ≤ −

= − − ≤ ≤ ≤ ≤∑n n

L p

n n nm

i k s m m L

i k s m i s m k i p k p0

ˆ ˆ ˆ

which is equivalent to the form

( , ) ( ) (φ

•

=

m

n n ni k s m s1

0

1 0( )

), ,− + −

+ − ≤ ≤ ≤ ≤∑L i k

m i k i p k p0

1 0

ˆ

ˆ ˆ

there are | | non-zero terms in the computation of ( , ) for each value of and ; can easily show that

( ) ( ) ( )

φ

φ

=

• − −

= − = − ≤ ≤ ≤ ≤

m

nL i k i ki k

i k f i k R i k i p k p1 0ˆ ˆ ( , ) ( ) ( ), , wher

φ = = ≤ ≤ ≤ ≤

•n ni k f i k R i k i p k p

1ˆ ˆ

ˆ ˆ ˆ

e ( ) is the short-time autocorrelation of ( ) evaluated at where

( ) ( ) ( )− −

− −

= +∑n n

L k

n n n

R i k s m i k

R k s m s m k

18

0=∑n n nm

Autocorrelation MethodAutocorrelation Method

1 0ˆ

ˆ ˆ

since ( ) is even, then

( , ) ( ), ,φ

•

= − ≤ ≤ ≤ ≤n

n n

R k

i k R i k i p k p

0 1

( , ) ( ), , thus the basic equation becomes

( ) ( )

φ

α φ φ

•

= ≤ ≤∑

n n

p

p p

i k i i p1

0 1

1

ˆ ˆ

ˆ ˆ

( ) ( , ),

( ) ( ),

α φ φ

α

=

− = ≤ ≤

− = ≤ ≤

∑

∑

k n nkp

k n n

i k i i p

R i k R i i p1

( ) ( ),

with the mi=

•

∑ k n nk

p

nimum mean-squared prediction error of the formp

1

0 0 0ˆ ˆ ˆ ( , ) ( , )

( ) ( )

φ α φ=

= −∑

∑

kn n nk

p

E k

R R k191

0ˆ ˆ( ) ( )α=

= −∑ kn nk

R R k

Autocorrelation MethodAutocorrelation Method

as expressed in matrix form•

1

2

0 1 1 11 0 2 2

ˆ ˆ ˆ ˆ

ˆ ˆ ˆ ˆ

as expressed in matrix form( ) ( ) . . ( ) ( )( ) ( ) . . ( ) ( )

αα

•

⎡ ⎤−⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥− ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥

n n n n

n n n n

R R R p RR R R p R

1 2 0ˆ ˆ ˆ ˆ

.. . . . . .

.. . . . . .( ) ( ) . . ( ) ( )α

⎢ ⎥⎢ ⎥ ⎢ ⎥=⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦⎣ ⎦pn n n nR p R p R R p

with so

⎣ ⎦ ⎣ ⎦⎣ ⎦ℜ

p

α = r

1

lution−ℜ 1

is a Toeplitz Matrix => symmetric with all diagonal elements equal=> there exist more efficient algorithms to solve for { } than simple matrixα

ℜ• ℜ ×

k

p pα = r

20

inversion

Covariance MethodCovariance MethodCovariance MethodCovariance Method there is a second basic approach to defining the speech •

ˆ

t e e s a seco d bas c app oac to de g t e speec segment ( ) and the limits on the sums, namely

over which the mean-squared error is computed, ns m fix the

interval q p , giving ( )Assumption #2

21 1

:

− − ⎡ ⎤pL L2

0 0 1

1

ˆ ˆ ˆ ˆ ( ) ( ) ( )α= = =

⎡ ⎤= = − −⎢ ⎥

⎢ ⎥⎣ ⎦∑ ∑ ∑

p

kn n n nm m k

L

E e m s m s m k

1

0

1 0ˆ ˆ ˆ( , ) ( ) ( ), ,φ−

=

= − − ≤ ≤ ≤ ≤∑L

n n nm

i k s m i s m k i p k p

21

Covariance MethodCovariance MethodCovariance MethodCovariance Method1

changing the summation index gives− −

•L i 1

1

1 0

1 0

ˆ ˆ ˆ ( , ) ( ) ( ), ,

( ) ( ) ( )

φ

φ

=−− −

= + − ≤ ≤ ≤ ≤

+ ≤ ≤ ≤ ≤

∑

∑

L i

n n nm i

L k

i k s m s m i k i p k p

i k s m s m k i i p k p1 0ˆ ˆ ˆ( , ) ( ) ( ), ,

key difference from Autocorrelation Method is that limi

φ=−

= + − ≤ ≤ ≤ ≤

•

∑n n nm k

i k s m s m k i i p k p

0ts of summation

include terms before => window extends samples backwards=m p01

include terms before => window extends samples backwards ˆ ˆ from ( - ) to ( - )

since we are extending window backwards, don't need to taper it using HW i th

=

+•

m ps n p s n L

i t iti t i d d a HW- since there is no transition at window edges

22m=-p m=L-1

Covariance MethodCovariance MethodCovariance MethodCovariance Method

23

Covariance MethodCovariance MethodCovariance MethodCovariance Method cannot use autocorrelation formulation => this is a true cross correlation need to solve set of equations of the form••

1

0 1 2ˆ ˆ

q

( , ) ( , ), , ,..., ,α φ φ=

= =∑p

k n nk

i k i i p

1

0 0 0

11

ˆ ˆ ˆ( , ) ( , )

(

φ α φ

φ=

= −∑p

kn n nk

E k

1 2 1 1 0) ( ) ( ) ( )αφ φ φ⎡ ⎤⎡ ⎤ ⎡ ⎤p11ˆ ( ,φn 1

2

1 2 1 1 02 1 2 2 2 2 0

ˆ ˆ ˆ

ˆ ˆ ˆ ˆ

) ( , ) . . ( , ) ( , )( , ) ( , ) . . ( , ) ( , )

.. . . . . .

αφ φ φαφ φ φ φ⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=⎢ ⎥⎢ ⎥ ⎢ ⎥

n n n

n n n n

pp

1 2 0ˆ ˆ ˆ ˆ

.. . . . . .( , ) ( , ) . . ( , ) ( , )

or

αφ φ φ φ

ψ ψ

⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎣ ⎦pn n n np p p p p

−1φα α φ

24

orψ ψφα = α = φ

Covariance MethodCovariance MethodCovariance MethodCovariance Method

1 1 1 1 1 1

ˆ ˆ

ˆ ˆ ˆ ˆ ˆ ˆ

we have ( , ) ( , ) => symmetric but not Toeplitz matrix whose diagonal elements are related as

( , ) ( , ) ( ) ( ) ( ) ( )

φ φ

φ φ

• =

+ + = + − − − − − − − − −

n ni k k i

i k i k s i s k s L i s L k1 1 1 1 1 12 2 11 2ˆ ˆ ˆ ˆ

( , ) ( , ) ( ) ( ) ( ) ( )( , ) ( , ) ( )

φ φφ φ

+ + +

= + −n n n n n n

n n n n

i k i k s i s k s L i s L ks s 2 2 2ˆ ˆ

ˆ

( ) ( ) ( ) all terms ( , ) have a fixed number of terms contributing to the φ

− − − −

•n n

n

s L s Li k

ˆ

computed values ( terms) ( , ) is a covariance matrix => specialized solution for { } called the Coφ α• kn

Li k

variance Method called the Covariance Method

25

Summary of LPSummary of LPSummary of LPSummary of LPˆ use order linear predictor to predict ( ) from previous samples

i i i d l i i d f d ti l• thp s n p

E Lˆ minimize mean-squared error, , over analysis window of duration -samples solution for optimum predictor coefficients, {α•

•nE L

}, is based on solving a matrix equation => two solutions have evolved

- => signal is windowed by a tapering window in order to

k

autocorrelation method - => signal is windowed by a tapering window in order to minimize discontinuities at beginn

autocorrelation method

ˆ

ing (predicting speech from zero-valued samples) and end (predicting zero-valued samples from speech samples) of the interval; the matrix ( , ) is shown to be an autocorrelation functφn i k ion; the resulting autocorrelation ( )φn g matrix is Toeplitz and can be readily solved using standard matrix solutions - => the signal is extended by samples outside the normalpcovariance method

0 1 0range

of - to include samples occurring prior to ; this eliminates large errors ≤ ≤ =m L p m0 in computing the signal from values prior to (they are available) and eliminates the

=m

need for a tapering window; resulting matrix of correlations is symmetric but not Toeplitz => different method of solution with somewhat different set of optimal prediction

coefficients { }

26

coefficients, { }αk

LPC SummaryLPC SummaryLPC SummaryLPC Summary1. Speech Production Model:

( ) ( ) ( )∑p

k G1

1

( ) ( ) ( )

( )( )( )

=

= − +

= =

∑

∑

kk

pk

s n a s n k Gu n

S zH zGU z

11( )

2. Linear Prediction Model:

−

=

−∑ kk

k

p

GU z a z

1

ˆ ˆ( ) ( )

( )( )( )

α

α

=

−

= −

= =

∑

∑

p

kk

pk

k

s n s n k

S zP z zS 1

( )( )

ˆ ˆ ˆ ˆ ˆ( ) ( ) ( ) ( ) (α

=

= − = − −

∑ kk

k

S z

e n s n s n s n s n1

)=∑

p

kk

2711( )( )

( )α −

=

= = −∑p

kk

k

E zA z zS z

LPC SummaryLPC SummaryLPC SummaryLPC Summary22

3. LPC Minimization:

( ) ( ) ( )= = ⎡ ⎤⎣ ⎦∑ ∑E e m s m s m

2

ˆ ˆ ˆ ˆ

ˆ ˆ

( ) ( ) ( )

( ) ( )α

= = −⎡ ⎤⎣ ⎦

⎡ ⎤= − −⎢ ⎥

⎣ ⎦

∑ ∑

∑ ∑

n n n nm m

p

kn n

E e m s m s m

s m s m k1

0 1 2ˆ , , ,...,α

=⎣ ⎦∂

= =∂

m k

n

i

Ei p

1ˆ ˆ ˆ ˆ

ˆ ˆ ˆ

( ) ( ) ( ) ( )

( , ) ( ) ( )

α

φ=

− = − −

= − −

∑ ∑ ∑

∑

p

kn n n nm k m

n n n

s m i s m s m i s m k

i k s m i s m k

ˆ

( , ) ( ) ( )

( ,

φ

α φ

∑n n nm

k n i1

0 1 2ˆ) ( , ), , ,...,φ=

= =∑p

nk

k i i p

2810 0 0ˆ ˆ ˆ( , ) ( , )φ α φ

=

= −∑p

kn n nk

E k

LPC SummaryLPC SummaryLPC SummaryLPC Summary

0 14. Autocorrelation Method:

ˆ( ) ( ) ( )= + ≤ ≤s m s m n w m m L

1

0 1

0 1

ˆ

ˆ ˆ ˆ

( ) ( ) ( ),

( ) ( ) ( ),α=

= + ≤ ≤ −

= − − ≤ ≤ − +∑n

p

kn n nk

s m s m n w m m L

e m s m s m k m L p

0 1 0 10 1

ˆ ˆ ( ) defined for ; ( ) defined for large errors for and for

≤ ≤ − ≤ ≤ − +

⇒ ≤ ≤ − ≤

i n ns m m L e m m L pm p L

12

1

( )− +

≤ + −

∑L p

m L p

E

( )

2

0

1

ˆ ˆ

( )

ˆ ˆ ˆ ˆ ˆ

( )

( , ) ( ) ( ) ( )φ

=

− − −

=

= − = + − = −

∑

∑

n nm

L i k

n n n n n

E e m

i k R i k s m s m i k R i k0

11ˆ ˆ( ) ( ),α

=

=

− = ≤ ≤∑m

p

k n nk

p

R i k R i i p

2910ˆ ˆ ˆ( ) ( )α

=

= −∑p

kn n nk

E R R k

LPC SummaryLPC SummaryLPC SummaryLPC Summary4. Autocorrelation Method:

1

resulting matrix equation:

ℜ ℜ

i

1

10 1 1 11 0 2 2

ˆ ˆ ˆ ˆ

or( ) ( ) . . ( ) ( )( ) ( ) ( ) ( )

α ααα

−ℜ = = ℜ

− ⎡ ⎤⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

n n n n

r rR R R p RR R R p R

⎡ ⎤⎢ ⎥⎢ ⎥21 0 2 2ˆ ˆ ˆ ˆ( ) ( ) . . ( ) ( )

.. . . . . .

.. . . . .

α− ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ =⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

n n n nR R R p R

.

⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

1 2 0ˆ ˆ ˆ( ) ( ) . . ( ) α⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥− −⎣ ⎦ ⎣ ⎦pn n nR p R p R ˆ ( )

⎢ ⎥⎢ ⎥⎣ ⎦nR p

30 matrix equation solved using Levinson or Durbin methodi

LPC SummaryLPC SummaryLPC SummaryLPC Summary2

5. Covariance Method: fix interval for error signal

⎡ ⎤

i21 1

2

0 0 1

1

ˆ ˆ ˆ ˆ ( ) ( ) ( )

ˆ ˆ need signal for from ( ) to ( ) samples

α− −

= = =

⎡ ⎤= = − −⎢ ⎥

⎣ ⎦− + − ⇒ +

∑ ∑ ∑i

pL L

kn n n nm m k

E e m s m s m k

s n p s n L L p

1ˆ ˆ( , ) (α φ φ

=

=∑p

k n nk

i k 0 1 2

0 0 0ˆ ˆ ˆ

, ), , ,...,

( , ) ( , )φ α φ

=

= −∑p

k

i i p

E k1

1

0 0 0( , ) ( , )

expressed as a matrix equation: or , symmetric matrix

φ α φ

φα ψ α φ ψ φ

=

−= =

∑i

kn n nk

E k

11 1 2 12 1 2 2 2

ˆ ˆ ˆ

ˆ ˆ ˆ

( , ) ( , ) . . ( , )( , ) ( , ) . . ( , ). . . . .

φ φ φφ φ φ

n n n

n n n

pp

1

2

1 02 0

ˆ

ˆ

( , )( , )

. .

α φα φ⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=⎢ ⎥⎢ ⎥ ⎢ ⎥

n

n

311ˆ

. . . . .( ,φn p 2 0ˆ ˆ ˆ

. .) ( , ) . . ( , ) ( , )αφ φ φ

⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎣ ⎦pn n np p p p

Computation of Model GainComputation of Model GainComputation of Model GainComputation of Model Gain it is reasonable to expect the model gain, , to be determined by matching

the signal energy with the energy of the linearly predicted samples• G

g gy gy y p p from the basic model equations we have

•

1

( ) ( ) ( ) model= − − ⇒∑p

kk

Gu n s n a s n k1

whereas for the prediction error we have

( ) ( ) ( ) best fit to modelα

=

•

= − − ⇒∑

k

p

ke n s n s n k1

when (i.e., perfect match to model), then ( ) ( )

α=

• =

=•

k

k kae n Gu n

since it is virtually impossible to guarantee that cannot use thisα a • since it is virtually impossible to guarantee that , cannot use this simple matching property for determining the gain; instead use energy matching criterion (energy in error signal=energy in

α =k ka

1 1

excitation)− + − +L p L p

32

1 12 2 2

0 0ˆ ( ) ( )

+ +

= =

= =∑ ∑L p L p

nm m

G u m e m E

Gain AssumptionsGain AssumptionsGain AssumptionsGain Assumptions

assumptions about excitation to solve for -- ( ) ( ) order of a δ

•− = ⇒

Gu n n Lvoiced speech

single pitch period; predictor order, , large enough to model glottal pulse shape, vocal tract IR, and

p

radiation -- ( )-zero mean, unity variance, − u nunvoiced speech stationary white noise process

33

Solution for Gain (Voiced)Solution for Gain (Voiced)( )( ) for voiced speech the excitation is ( ) with output ( ) (since it is the IR of the system)

δ• G n h n

1 1

the IR of the system),

( ) ( ) ( ); ( )( )

α δ

α= −

= − + = =

−∑

∑

p

k pk k

k

G Gh n h n k G n H zA z

z1

with autocorrelation ( ) (of the impu=

•

∑ kk

R m lse response) satisfying the relation shown below

0

0 ( ) ( ) ( ) [ ],∞

=

= + = − ≤ < ∞∑np

R m h n h m n R m m

1

2

1

0 0

( ) ( ),

( ) ( )

α

α

=

= − ≤ < ∞

+

∑

∑

p

kkp

R m R m k m

R R k G m

341

0 0( ) ( ) ,α=

= + =∑ kk

R R k G m

Solution for Gain (Voiced)Solution for Gain (Voiced)( )( )

0ˆ

Since ( ) and ( ) have the identical form, it follows that

( ) ( )

•

= ⋅ ≤ ≤nR m R m

R m c R m m p0

0

( ) ( ),where is a constant to be determined. Since the total energies in the signal ( ( )) and the impulse

≤ ≤

•

nR m c R m m pc

R

2

0

0

response ( ( )) must be equal, the constant must be 1, and we obtain the relation

( ) ( )∑p

R c

G R R k E2

1

0

0

ˆ ˆ ˆ

ˆ

( ) ( )

since ( ) ( ), , and the energy of the impulse

α=

= − =

• = ≤ ≤

∑ kn n nk

n

G R R k E

R m R m m p1f th i l fi t ffi i t f th response=en 1

1

ergy of the signal => first coefficients of the autocorrelation of the impulse response of the model are identical to the first coefficients of the autocorrelation function of the

+

+

p

p

35

speech signal. This condition called the of the autocorrelation method.

autocorrelation matchingproperty

Solution for Gain (Unvoiced)Solution for Gain (Unvoiced) for unvoiced speech the input is white noise with zero mean

and unity variance, i.e., ( ) ( ) ( )δ

•

− =⎡ ⎤⎣ ⎦E u n u n m m

if we excite the system with input ( ) and call the output ( ) then

• Gu ng n

( ) ( ) ( )α= − +∑p

kg n g n k Gu n1

( ) ( ) ( )

Since the autocorrelation function for the output is the convolution of the autocorrelation function of the impulse response with the autocorrelation function

=∑

i

kk

g g

of the white noise input thenautocorrelation function of the white noise input, then

[ [ ] [ ]] [ ] [ ] [ ]

letting ( ) denote the autocorrelation of ( ) gives

δ− = ∗ =

•p

E g n g n m R m m R m

R m g n

1

( ) ( ) ( ) [ ( ) ( )] (α=

= − = − − +⎡ ⎤⎣ ⎦ ∑p

kk

R m E g n g n m E g n k g n m E Gu n

0

) ( )

( ),α

−⎡ ⎤⎣ ⎦

= − ≠∑p

k

g n m

R m k m

36

1

0 0

( )

since ( ) ( ) for because ( ) is uncorrelated

with any signal prior to ( )

=

• − = >⎡ ⎤⎣ ⎦

∑ kk

E Gu n g n m m u n

u n

Solution for Gain (Unvoiced)Solution for Gain (Unvoiced)Solution for Gain (Unvoiced)Solution for Gain (Unvoiced)0 for we get• =

p

m

1

0 ( ) ( ) ( ) ( )α=

= + ⎡ ⎤⎣ ⎦∑p

kkp

R R k GE u n g n

2

12

( )

i ( ) ( ) ( )( ( ) t i t

α=

= +

⎡ ⎤ ⎡ ⎤⎣ ⎦⎣ ⎦

∑p

kk

R k G

E E G G2 since ( ) ( ) ( )( ( ) terms prior to

since the energy in the signal must equal the energy in the

• = + =⎡ ⎤ ⎡ ⎤⎣ ⎦⎣ ⎦•

E u n g n E u n Gu n n G

response to ( ) we getGu n the

ˆ

response to ( ) we get

( ) ( )= np

Gu n

R m R m

37

2

1

0ˆ ˆ ˆ( ) ( )α=

= − =∑ kn n nk

G R R k E

FrequencyFrequencyFrequency Frequency Domain Domain

Interpretations Interpretations of Linearof Linearof Linear of Linear

Predictive Predictive AnalysisAnalysis

38

The Resulting LPC ModelThe Resulting LPC ModelThe Resulting LPC ModelThe Resulting LPC Model{ } 1 2

The final LPC model consists of the LPC parameters, and the gain which togetherα =

i

k k p G{ } 1 2 , , ,..., , and the gain, , which together define the system function

( )

α

=

k

p

k p G

GH z

11

( )

with frequency response

α −

=

−∑i

pk

kk

z

H

11

( )( )

ωω

ωα −

=

= =−∑

jp j

j kk

k

G GeA ee

1

with the gain determined by matching the energy of the model to the short-time energy of the speech signal, i.e.,i

k

p

39

( )22

10ˆ ˆ ˆ ˆ ( ) ( ) ( )α

=

= = = −∑ ∑p

kn n n nm k

G E e m R R k

LPC SpectrumLPC Spectrum

x = s .* hamming(301);X = fft( x 1000 )( )ω =j

pGH e X = fft( x , 1000 )

[ A , G , r ] = autolpc( x , 10 )H = G ./ fft(A,1000);1

1( )

ωα −

=

−∑p

j kk

ke

40LP Analysis is seen to be a method of short-time spectrum estimation with removal of excitation fine structure (a form of wideband spectrum analysis)

LP ShortLP Short--Time Spectrum AnalysisTime Spectrum AnalysisLP ShortLP Short Time Spectrum AnalysisTime Spectrum Analysis

• Defined speech segment as:Defined speech segment as:

ˆ ˆ[ ] [ ] [ ]ns m s m n w m= +

• The discrete-time Fourier transform of this windowed segment is:windowed segment is:

( )ˆ ˆ[ ] [ ]j j mnS e s m n w m eω ω

∞−= +∑

• Short-time FT and the LP spectrum are linked via short-time autocorrelation

( )m=−∞

41

via short time autocorrelation

LP ShortLP Short--Time Spectrum AnalysisTime Spectrum Analysis

(a) Voiced speech segment obtained using aobtained using a Hamming window

(b) Corresponding short-time autocorrelationtime autocorrelation function used in LP analysis (heavy line shows values used in LP analysis)

(c) Corresponding short-time log magnitude Fourier transform and short-time log magnitude LPC spectrum (FS=16 kHz)

42

kHz)

LP ShortLP Short--Time Spectrum AnalysisTime Spectrum Analysis

(a) Unvoiced speech segment obtained usingsegment obtained using a Hamming window

(b) Corresponding short-time autocorrelationtime autocorrelation function used in LP analysis (heavy line shows values used in LP analysis)

(c) Corresponding short-time log magnitude Fourier transform and short-time log magnitude LPC spectrum (FS=16 kHz)

43

kHz)

Frequency Domain Interpretation of Frequency Domain Interpretation of MM S d P di ti ES d P di ti EMeanMean--Squared Prediction ErrorSquared Prediction Error

The LP spectrum provides a basis for examining the propertiesi

ˆof the prediction error (or equivalently the excitation of the VT) The mean-squared prediction error at sample is:

L p

n+

i1

ˆ

2ˆ

0

n

L p

nm

E e+

=

=1

[ ]

which, by Parseval's Theorem, can be expressed as:

m−

∑i

2 2 2 2ˆ ˆ ˆ

1 1| ( ) | | ( ) | | ( ) |2 2

y p

j j jn n nE e d S e A e d G

π πω ω ω

π π

ω ωπ π− −

= = =∫ ∫E

ˆ ˆ( ) [ ] ( ) where is the FT of and is the correspondingprediction

j jn nS e s m A eω ωierror frequency response

p

441( ) 1

pj j k

kk

A e eω ωα −

=

= −∑

Frequency Domain Interpretation of Frequency Domain Interpretation of MeanMean Squared Prediction ErrorSquared Prediction ErrorMeanMean--Squared Prediction ErrorSquared Prediction Error

The LP spectrum is of the form:

j Gi

22

( )( )

Thus we can express the mean-squared error as:

jj

j

GH eA e

ωω

π

=

i22

2ˆˆ 2

| ( ) |2 | ( ) |

We see that minimizing total squared prediction error

jn

n j

S eGE d GH e

π ω

ωπ

ωπ −

= =∫i g q pis equivalent to finding gain and predictor coefficientssuch that the integral of the ratio of the energy spectrum ofthe speech segment to the magnitude squared of the frequencythe speech segment to the magnitude squared of the frequencyresponse of the model l

2ˆ| ( ) |

inear system is unity. Thus can be interpreted as a frequency-domainj

nS e ωi

45

2ˆ

2ˆ

| ( ) |

| ( ) |

weighting function LP weights frequencies where is large more heavily than when is small.

jn

jn

S e

S e

ω

ω

⇒

LP Interpretation Example1LP Interpretation Example1LP Interpretation Example1LP Interpretation Example1

Much better spectral matches to STFT spectral

peaks than topeaks than to STFT spectral

valleys as predicted by

spectral interpretation of

error minimizationminimization.

46

LP Interpretation Example2LP Interpretation Example2LP Interpretation Example2LP Interpretation Example2

N t llNote small differences in

spectral shape between STFT, ,autocorrelation spectrum and

covariance spectrum whenspectrum when

using short window duration (L=51 samples).( p )

47

Effects of Model OrderEffects of Model OrderEffects of Model OrderEffects of Model Orderˆ ˆ[ ] [ ], The AC function, of the speech segment, n nR m s mi

[ ], [ ],

( ),

and the AC function, of the impulse response, corresponding to the system function, are equal for

R m h m

H z( 1)the first values. Thp +

2 2

,

li | ( ) | | ( ) |

us, as the AC functions are equal for all values and thus:

j j

p

H Sω ω

→∞

2 2ˆlim | ( ) | | ( ) |

( )

Thus if is large enough, the FR of the all-poled l i t th i l t

j jnp

j

H e S e

p

H

ω ω

ω

→∞=

i( ),model, can approximate the signal spectrum

wi

jH e ω

th arbitrarily small error.

48

Effects of Model OrderEffects of Model OrderEffects of Model OrderEffects of Model Order

49

Effects of Model OrderEffects of Model OrderEffects of Model OrderEffects of Model Order

50

Effects of Model OrderEffects of Model OrderEffects of Model OrderEffects of Model Order

51

Effects of Model OrderEffects of Model Order

plots show Fourier transform of segment •and LP spectra for various orders - as increases, more details of the spectrum are preserved

p

- need to choose a value of that represents

p the spectral effects of the glottal pulse, vocal

tract and radiation--nothing else

52

Linear Prediction SpectrogramLinear Prediction SpectrogramLinear Prediction SpectrogramLinear Prediction Spectrogram

1

Speech spectrogram previously defined as:L

i1

(2 / )

020 log | [ ] | 20 log| [ ] [ ] |

, / , 1, 2,..., / 2

for set of times, and set of frequencies, f (

Lj N km

rm

r k S

S k s rR m w m e

t rRT F kF N k N

π−

−

=

= +

= = =

∑

) Swhere is the time shift (in sR1/

amples) between adjacent STFTs, is the sampling period, is the sampling frequency,

and is the size of the discrete Fourier transform used toST F T

N=

compute each STFT estimate. Similarly we can defi

20log | [ ] | 20 log

ine the LP spectrogram as an image plot of:

rGH k = (2 / )

(2 / )

20 log | [ ] | 20 log( )

( ).

where and are the gain and prediction error polynomialat analysis time

r j N kr

j N kr r

H kA e

G A erR

π

π

=

53

y

Linear Prediction SpectrogramLinear Prediction Spectrogram

L=81, R=3, N=1000, 40 db dynamic range

54

Comparison to Other Spectrum Comparison to Other Spectrum A l i M h dA l i M h dAnalysis MethodsAnalysis Methods

Spectra of synthetic vowel /IY/vowel /IY/

(a) Narrowband spectrum using 40 msec window40 msec window

(b) Wideband spectrum using a 10 msec window10 msec window

(c) Cepstrally smoothed spectrumspectrum

(d) LPC spectrum from a 40 msec section using a

55

section using a p=12 order LPC analysis

Comparison to Other Spectrum Comparison to Other Spectrum A l i M h dA l i M h dAnalysis MethodsAnalysis Methods

Natural speech spectral estimates using cepstral smoothing (solid line) and linear prediction analysis (dashedprediction analysis (dashed line).

Note the fewer (spurious) peaks in the LP analysis spectrumin the LP analysis spectrum since LP used p=12 which restricted the spectral match to a maximum of 6 resonance

kpeaks.

Note the narrow bandwidths of the LP resonances versus the

t ll th d

56

cepstrally smoothed resonances.

Selective Linear PredictionSelective Linear Prediction it is possible to apply LP methods to selected parts of spectrum

0 4 kH f i d d di t f d•

1

2

- 0-4 kHz for voiced sounds use a predictor of order - 4-8 kHz for unvoiced sounds use a predictor of order

the key id

⇒

⇒

•

pp

0 5ea is to map the frequency region { , } linearly to { ,. }A Bf f the key id

{ } { }0 5

2 2 0

ea is to map the frequency region { , } linearly to { ,. } or, equivalently, the region , maps linearly to , via

the transformationπ π π

A B

A B

f f

f f

22

2 2=

ω πω π

π π−

′ ⋅−

AB

B A

ff

f f we must modify the calculation •

212 ˆ

for the autocorrelation to give:

( ) | ( ) |π

ω ω ω′ ′′ ′= ∫ j j mnR m S e e d

57

2π

π−∫ n

Selective Linear PredictionSelective Linear PredictionSelective Linear PredictionSelective Linear Prediction• 0-10 kHz region modeled

i 28using p=28

•no discontinuity in model spectra at 5 kHz

• 0-5 kHz region modeled using p1=14

• 5-10 kHz region modeled using p2=5

• discontinuity in model t t 5 kH

58

spectra at 5 kHz

Solutions of LPC EquationsSolutions of LPC EquationsSolutions of LPC EquationsSolutions of LPC Equations

Covariance Method (Cholesky Covariance Method (Cholesky Decomposition Method)Decomposition Method)))

59

LPC SolutionsLPC Solutions--Covariance MethodCovariance Method

for the covariance method we need to solve the matrix equation•

1

0 1 2ˆ ˆ ( , ) ( , ), , ,...,

(in matri notation)

α φ φ

φ=

= =∑p

k n nk

i k i i p

ˆ

(in matrix notation) is a positive definite, symmetric matrix with ( , ) element

φα ψφ φ

=• ni j

0ˆ

( , ), and and are column vectors with elements and ( , )α ψ α φi n

i ji

t

the solution of the matrix equation is called the Cholesky decomposition, or square root method

VDV V l t i lφ

•

t i ith 1' th i di lt =VDV ; V lower triangulaφ = r matrix with 1's on the main diagonal D=diagonal matrix

60

LPC SolutionsLPC Solutions--Covariance MethodCovariance Method can readily determine elements of V and D by solving for ( , ) elements

of the matrix equation, as follows•

j

i j

1

1

1 1ˆ ( , ) ,

giving

φ=

= ≤ ≤ −

•

∑j

ik k jknk

j

i j V d V j i

1

1

1ˆ ( , ) ,φ−

=

= − ≤ ≤∑j

ij j ik k jknk

V d i j V d V j 1

and for the diagonal elements

−

•

i

1ˆ ( , )

giving

φ=

=

•

∑i

ik k iknk

i i V d V

12

1

2ˆ

g g

( , ) ,

with

φ−

=

= − ≥

•

∑ ik

i

i knk

d i i V d i

611 11ˆ

with ( , )φ•

= nd

Cholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition Example

4consider example with and matrix elements ( )φ φ• = =p i j

11 21 31 41

4 ˆ consider example with , and matrix elements ( , )φ φ

φ φ φ φφ φ φ φ

• = =

⎡ ⎤⎢ ⎥⎢ ⎥

ijnp i j

21 22 32 42

31 32 33 43

41 42 34 44

φ φ φ φφ φ φ φφ φ φ φ

⎢ ⎥ =⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦41 42 34 44

1

21 2

1 0 0 0 0 0 01 0 0 0 0 0

φ φ φ φ⎢ ⎥⎣ ⎦⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥

dV d

21 31 41

32 42

10 1⎡ ⎤⎢ ⎥⎢ ⎥

V V VV V21 2

31 32 3

41 42 43 4

1 0 0 0 01 0 0 0

⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥

⎢ ⎥⎢ ⎥ ⎣ ⎦⎣ ⎦

V V dV V V d

32 42

430 0 10 0 0 1

⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

V

62

Cholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition Example

1 21 31 41 2 32 42 3 43 4solve matrix for , , , , , , , , ,• d V V V d V V d V d1 21 31 41 2 32 42 3 43 4

1 11

21 1 21 31 1 31 41 1 41

solve matrix for , , , ,step 1

, , , , ,

; ;φφ φ φ=

= = =

d V V V d V V d V dd

V d V d V d21 1 21 31 1 31 41 1 41

21 21 1 31 31 1 41 41 12

2 22 1

step 2

st

; ;/ ; / ; /

e 3 p

φ φ φφ φ φ

φ

= = =

= −

V d V d V d

d V d212 22 1

32 2 32 31 1 21 32 32 31 1 2

pφ

φ φ= − ⇒ = −V d V d V V V d V( )( )

1 2

step

/

/ 4φ φ= ⇒ =

d

V d V d V V V d V d( )42 2 42 41 1 21 42 42 41 1 21 2

3 43 4

step/

iterate procedure to solve for , ,

4φ φ= − ⇒ = −

•

V d V d V V V d V d

d V d

63

LPC SolutionsLPC Solutions--Covariance MethodCovariance MethodLPC SolutionsLPC Solutions Covariance MethodCovariance Method now need to solve for using a 2-step procedureα•

t VDV writing this as

VY= with

α ψ

ψ

=•

t

t 1

VY= with

DV or

V D

ψ

α

α −

=

=

Y

Y from V (which is now known) solve for column vector Y

using a •

1

simple recursion of the formi 1

1

2 = ,

with initial condition

ψ−

=

− ≥ ≥

•

∑i

i i ij jj

Y V Y p i

641 1

with initial condition ψ•

=Y

LPC SolutionsLPC Solutions--Covariance MethodCovariance MethodLPC SolutionsLPC Solutions Covariance MethodCovariance Method

now can solve for using the recursionα•

1 1

now can solve for using the recursion

/ ,

α

α α

•

= − ≤ ≤ −∑p

i i i ji jY d V i p1

1 1 / ,

with initial condition

α α= +

≤ ≤

•

∑i i i ji jj i

Y d V i p

with initial condition /

l l ti d b k d f

α =p p pY d

i 11

calculation proceeds backwards from down to • = −

=i p

i

65

Cholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition Example continuing the exam ple w e solve for Y•

1 1

21 2 2

1 0 0 01 0 0

ψψ

⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥ ⎢ ⎥

YV Y

31 32 3 3

41 42 43 4 4

1 01

ψψ

⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥

⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦

V V YV V V Y

1 4

1 1

first so lv ing for w e get ψ• −

=

Y YY

2 2 21 1

3 3 31 1 32 2

ψψ

= −

= − −

Y V YY V Y V Y

66

3 3 31 1 32 2

4

ψY 4 41 1 42 2 43 3ψ= − − −V Y V Y V Y

Cholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition ExampleCholesky Decomposition Example next solve for from equationα•

1 1 1 1 121 31 41

2 2 2 2 232 42

1 0 0 010 1 0 00 10 0 1 00 0 1

/ // /

/ /

ααα

⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ = =⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥

d Y Y dV V Vd Y Y dV V

d Y Y dV 3 3 3 3 343

4 4 4 4 4

0 0 1 00 0 10 0 0 10 0 0 1

/ // /

givi

αα⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦

•

d Y Y dVd Y Y d

ng the results

4 4 4

3 3 3 43 4

//

αα α

=

= −

Y dY d V

2 2 2 32 3 42 4

1 1 1 21 2 31 3 41 4

//

completing the solution

α α αα α α α

= − −

= − − −

•

Y d V VY d V V V

67

completing the solution•

Covariance Method Minimum ErrorCovariance Method Minimum ErrorCovariance Method Minimum ErrorCovariance Method Minimum Error

the minimum mean squared error can be written in the form•

1

0 0 0ˆ ˆ ˆ

q

( , ) ( , )φ α φ=

= −∑p

kn n nk

E k

1 1

1

0 0ˆ ( , )

since can write this as

φ α ψ

α − −

= −

• =

tn

t t

t

Y D V1

2

0 0

0 0

ˆ ˆ

ˆ

( , )

( , ) /

φ

φ

−= −

= −

tn n

k kn

E Y D Y

Y d∑p

k 1

ˆ this computation for can be used for all values of LP order from 1 to can understand how LP order reduces mean-squared error

=

•

⇒

∑nE

p

68

qp

69

Solutions of LPC EquationsSolutions of LPC EquationsSolutions of LPC EquationsSolutions of LPC Equations

Autocorrelation Method via Autocorrelation Method via LevinsonLevinson--Durbin AlgorithmDurbin Algorithmgg

70

LevinsonLevinson--Durbin Algorithm 1Durbin Algorithm 1LevinsonLevinson Durbin Algorithm 1Durbin Algorithm 1ˆ) Autocorrelation equations (at each frame :

p

ni

1[| |] [ ] 1

p

kk

R i k R i i pα=

− = ≤ ≤∑Rα = r

is a positive definite symmetric Toeplitz matrix The set of optimum predictor coefficients satisfy:ii

Rα rR

1[ ]

p p y

kk

R i α=

− [| |] 0, 1p

R i k i p− = ≤ ≤∑

( )[0] [ ]

with minimum mean-squared prediction error of:

p

pkR R k Eα− =∑

i

71

1[ ] [ ]k

k=∑

LevinsonLevinson--Durbin Algorithm 2Durbin Algorithm 2LevinsonLevinson Durbin Algorithm 2Durbin Algorithm 2 By combining the last two equations we get a larger matrix i

( )

1[0] [1] [2] ... [ ]

equation of the form:R R R R p⎡ ⎤⎢ ⎥

( )pE⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥( )

1( )2

[1] [0] [1] ... [ 1][2] [1] [0] ... [ 2]

p

p

R R R R pR R R R p

αα

⎢ ⎥ −−⎢ ⎥−⎢ ⎥−

⎢ ⎥⎢ ⎥

00

⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥. . . . .

[ ] [ 1] [ 2] ... [0]R p R p R p R⎢ ⎥⎢ ⎥− −⎣ ⎦

( )

. .0

expanded matrix is still Toeplitz and can be solved iteratively

ppα

⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− ⎣ ⎦⎣ ⎦

i p p yby incorporating new correlation value at each iteration andsolving for next higher order predictor in terms of new correlation

72

value and previous predictor

LevinsonLevinson--Durbin Algorithm 3Durbin Algorithm 3

( 1) Show how order solution can be derived from th sti i −i( 1) ( 1) ( 1) ( 1)

( ) ( ) ( )

,order solution; i.e., given the solution to we derive solution to

i i i i

i i i

R eR e

α α

α

− − − −=

=

( 1) The solution can be expressti −i( 1)

( 1)

1[0] [1] [2] ... [ 1]

sed as:i

i

R R R R i E −− ⎡ ⎤⎡ ⎤⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ ( 1)

1( 1)2

[1] [0] [1] ... [ 2] 0[2] [1] [0] ... [ 3] 0

i

i

R R R R iR R R R i

αα

−

−

⎢ ⎥⎢ ⎥⎢ ⎥ −− ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ − =−⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

( 1)1

.. . . . . .[ 1] [ 2] [ 3] ... [0] 0i

iR i R i R i R α −−

⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ −− − −⎣ ⎦ ⎣ ⎦ ⎣ ⎦

73

LevinsonLevinson--Durbin Algorithm 4Durbin Algorithm 4

( 1) ( )

[0] [1] [2] ... [ ]

Appending a 0 to vector and multiplying by the matrix gives:i iR

R R R R i

α −

⎡ ⎤⎢

i( 1)1 iE −⎡ ⎤⎡ ⎤

⎢ ⎥⎥ ⎢ ⎥[1] [0] [1] ... [ 1][2] [1] [0] ... [ 2]

R R R R iR R R R i

⎢ −⎢⎢ −⎢⎢

( 1)1( 1)2

00

i

i

αα

−

−

⎢ ⎥⎥ ⎢ ⎥− ⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎥ ⎢ ⎥−

= ⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎥ ⎢ ⎥. . . ... .

[ 1] [ 2] [ 3] ... [1][ ] [ 1] [ 2] ... [0]

R i R i R i RR i R i R i R

⎢⎢ − − −⎢

− −⎣

( 1)1

( 1)

. .0

0

ii

i

αγ

−−

−

⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎥ ⎢ ⎥−⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎦ ⎣ ⎦ ⎣ ⎦⎣

1( 1) ( 1)

1[ ] [ ] [ ] where and are introduced

ii i

jj

R i R i j R i

γ

γ α−

− −

=

⎦ ⎣ ⎦ ⎣ ⎦

= − −∑i

74

LevinsonLevinson--Durbin Algorithm 5Durbin Algorithm 5LevinsonLevinson Durbin Algorithm 5Durbin Algorithm 5

Key step is that since Toeplitz matrix has special symmetryi Key step is that since Toeplitz matrix has special symmetrywe can reverse the order of the equations (first equation last,last equation first), giving:

i

[0] [1] [2] ... [ ][1] [0] [1] ... [ 1]

q ) g gR R R R iR R R R i −

( 1)

( 1)1

00

i

i

γα

−

−

⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥− ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥[2] [1R R ( 1)

2

( 1)

] [0] ... [ 2] 0. . . ... . . .

[ 1] [ 2] [ 3] [1] 0

i

i

R R i

R i R i R i R

α

α

−

−

⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− −

= ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥( )

1( 1)

[ 1] [ 2] [ 3] ... [1] 0[ ] [ 1] [ 2] ... [0] 1

ii

R i R i R i RR i R i R i R E

α −−

⎢ ⎥⎢ ⎥ ⎢ ⎥− − − −⎢ ⎥⎢ ⎥ ⎢ ⎥

− − ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦

75

LevinsonLevinson--Durbin Algorithm 6Durbin Algorithm 6LevinsonLevinson Durbin Algorithm 6Durbin Algorithm 6( )

To get the equation into the desired form (a singlecomponent in the vector ) we combine the twoiei

( 1)

1

p )sets of matrices (with a multiplicative factor ) giving:i

i

k

⎡⎢

( 1) ( 1)

( 1)

0 i i

i

E γ− −⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥( 1)

1( 1)2( )

.

i

iiR

αα

−

−

⎢−⎢⎢−⎢⎢⎢

( 1)1( 1)2

0 00 0

. . .

i

i

i ik k

αα

−

−

⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥− ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥−

− = −⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥( 1)

1

0

iiα−−

⎢⎢−

⎣

( 1)1

( 1) ( 1)

( 1)

0 01

Choose so that

ii

i i

i

Eα

γ

γ

−−

− −

−

⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥−⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦

vector on right has only a single( ) Choose so that γi

1( 1)[ ] [ ]

vector on right has only a singlenon-zero entry, i.e.,

ii

jR i R i jα−

−− −∑76

( 1)1

( 1) ( 1)

[ ] [ ]

jij

i i i

R i R i jk

E E

αγ −

=− −= =

∑

LevinsonLevinson--Durbin Algorithm 7Durbin Algorithm 7LevinsonLevinson Durbin Algorithm 7Durbin Algorithm 7( ) ( 1) ( 1) ( 1) 2(1 )

The first element of the right hand side vector is now: i i i i

i iE E k E kγ− − −= − = −

i

( 1)

( )

,

The parameters are called PARCOR coefficients. With this choice of the vector of order

i i

ii th

k

i

γ

γ −

ii predictor

ffi i t i

( ) ( 1) ( 1)1 1 1

1 1 0coefficients is:

i i iiα α α− −−

⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥− − −⎢ ⎥ ⎢ ⎥ ⎢ ⎥

( ) ( 1) ( 1)2 2 2

( ) ( 1) ( 1)

. . .

i i ii

i

i i i

kα α α

α α α

− −−

− −

⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥− − −

= −⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥− − −1 1 1

( ) 0 1 yielding the updati

i ii

i

α α αα

− −⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦

i ng procedure

77

( ) ( 1) ( 1)

( )

, 1, 2,..., i i ij j i i j

ii i

k j p

k

α α α

α

− −−= − =

=

LevinsonLevinson--Durbin Algorithm 7Durbin Algorithm 7LevinsonLevinson Durbin Algorithm 7Durbin Algorithm 7

( ) 1 The final solution for order is:

p

pi( ) 1

with prediction error

pj j j pα α= ≤ ≤

i

( ) 2 2

1 1

[0] (1 ) [0] (1 )

If we use normalized autocorrelation coefficients:

p pp

m mm m

E E k R k= =

= − = −∏ ∏i

[ ] [ ] / [0] If we use normalized autocorrelation coefficients:

we get norma

r k R k R=i

i lized errors of the form: we get norma( )

( ) ( ) 2

1 1

1 [ ] (1 )[0]

lized errors of the form:

i ii

i ik m

k m

E r k kR

ν α= =

= = − = −∑ ∏

78

1 1

( )

[ ]0 1 1 1 where or

k m

iikν

=

< ≤ − < <i

LevinsonLevinson--Durbin AlgorithmDurbin AlgorithmLevinsonLevinson Durbin AlgorithmDurbin Algorithm

1

1 1

( ) ( )

( )

( ) ( )( )

−

− − −

⇒ =

−

i i

i ii

A z A zk z A z( )i

79

Autocorrelation ExampleAutocorrelation ExampleAutocorrelation ExampleAutocorrelation Example2 consider a simple solution of the form• =p

1

2

0 1 11 0 2

( ) ( ) ( )

( ) ( ) ( )αα⎡ ⎤⎡ ⎤ ⎡ ⎤

=⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎣ ⎦

R R RR R R2

0 0( )

( ) ( ) ( ) with solution

( )

⎣ ⎦ ⎣ ⎦⎣ ⎦•

E R

11

01 0

( )

( )

( )( ) / ( )=

=E Rk R R

11

2 21

1 0

0 1

( )

( )

( ) / ( )

( ) ( )

α =

−

R R

R RE80

1

0( ) ( ) ( )

( )=E

R

Autocorrelation ExampleAutocorrelation ExampleAutocorrelation ExampleAutocorrelation Example2

22 0 1( ) ( ) ( )−

=R R Rk2 2 2

22

0 1

2 0 1( )

( ) ( )

( ) ( ) ( )α

=−

−=

kR R

R R R2 2 2

21 2 2

0 11 0 1 2( )

( ) ( )( ) ( ) ( ) ( )

α

α

=−−

=

R RR R R R

1 2 2

2

0 1

( )

( ) ( ) with final coefficients

α−

•R R

212

2 2

( )1

( )

α α

α α

=

=

81

( ) prediction error for predictor of order =iE i

Prediction Error as a Function of pPrediction Error as a Function of p

11

0 0[ ]

[ ] [ ]α

=

= = −∑p

n nn k

kn n

E R kVR R

Model order is usually determinedby the following rule of thumb:y g

Fs/1000 poles for vocal tract2-4 poles for radiation2 poles for glottal pulse2 poles for glottal pulse

82

Autocorrelation Method PropertiesAutocorrelation Method Properties

• mean-squared prediction error always non-zeroq p y– decreases monotonically with increasing model order

• autocorrelation matching propertymodel and data match up to order p– model and data match up to order p

• spectrum matching property– favors peaks of short-time FTp

• minimum-phase property– zeros of A(z) are inside the unit circle

Levinson Durbin recursion• Levinson-Durbin recursion– efficient algorithm for finding prediction coefficients– PARCOR coefficients and MSE are by-products

83

y p