6 Levinson

the levinson algorithm October 2, 2011

Digtal Signal Processing (et 4235)

The Levinson Algorithm

The Levinson algorithm is a fast recursion to solve the Yule-Walker prediction-errorequations that model a random process, or more in general to implicitly invert aToeplitz matrix. It gives rise to a computational structure (lattice filter) with usefulproperties. It can be used to analyze an autocorrelation sequence, and to generatea process with this autocorrelation.

It is used for parametric spectrum estimation (based on all-pole or AR modeling),and e.g. speech coding in GSM: instead of transmitting speech samples, the filtercoefficients are transmitted so that the receiver can reconstruct the speech.

(These slides: real-valued signals. See book for the complex case.)

1


Recap: AR modeling of a random processSuppose we want to model a random signal

using an AR filter applied to awhite noise,

,

The input signal

is also known as the innovation.

White implies that

is not correlated to past samples (

, etc), and hence

is uncorrelated to

, etc:

In the -domain, the model is written as

2


Application: parametric spectrum estimation

The power spectral density of

is given by

Parametric spectrum estimation is: given the correlation sequence

of

,

find

. For this we need to find

and

, i.e. model identification.

3


Application: prediction filter modeling

We are given a signal and wish to predict the current sample

as a linear

combination of

past samples

:

To estimate the optimal coefficients, we can minimize the prediction error:

In a deterministic setting, we can minimize

.

In a stochastic setting, we minimize

.

This leads to the same equations as before (with

and

).

4


From prediction filter to Yule-Walker equationsModel for

:

In general, a random process will not precisely satisfy such a model. For anymodel order

, we try to find the best fitting parameters

that will

minimize the residual noise variance

. That is, we consider

and minimize the power of the prediction error

.

If the model holds, we can easily derive the Yule-Walker equations, as follows.

Multiply by

and take expectation:

For

this gives, with

,

5


Yule-Walker equations

These equations can be written in matrix form (with rows for

)

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The matrix is constant along diagonals: a Toeplitz matrix. It is also symmetric.

For

, we need to compute

. Because

for

(see before), we have

. We obtain an additional equation:

6


Yule-Walker equations

To solve the Yule-Walker equations is straightforward: the filter coefficients are

. The noise power (prediction error or innovation power)

follows from

the above equation for

.

This has a complexity of order

multiplications. The Levinson algorithm can

reduce this to order

, by exploiting the Toeplitz structure.

7


Levinson algorithmWe will derive a recursion, for

. For each

, we have different filter

coefficients

(

) and modeling/prediction error power

.

For order

, the YW equations give

(there is no filter)

For order

, we have

, or

, and

.

For order

, combine the YW equations for

and

into a single matrixof size

:

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Suppose we know the solution

for order

, how can we find the

solution for order

efficiently?

8


Levinson algorithm

Take as trial solution for order

the old solution for order

, extended by

:

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

This is not the correct solution of the YW equations, because in general

.

We need to modify the trial solution such that

; in that case the RHS has therequired form and we have a solution to the YW equations of order

.

To modify the trial solution, we apply the following two tricks.

9


The Levinson algorithmTrick 1: the Toeplitz structure gives rise to the following equation (for order

):

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Trick 2: The two equations can be combined:

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

10


Levinson algorithm

For

, we take as trial solutions:

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Both trial solutions are not correct solutions of the YW equations of order

,

because

. But we can take linear combinations of the two trial solutions in

such a way that

becomes 0.

11


The Levinson algorithmIndeed, for any matrix :

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

We will take linear combinations of the following form: (book uses

)

.

.

.

.

.

.

.

.

.

.

.

.

For

, this will bring the RHS into the required form. The LHS is then thesolution of the YW equations of order

. (

is called a reflection coefficient.)12


The Levinson algorithmThus, we need to set

. We obtain as solution

.

.

.

.

.

.

.

.

.

.

.

.

This is the update step in the Levinson recursion. The recursion is initiated by

With this, we enter the recursion. In the next step, we obtain the equations

where

. We set

, and follow the recursion.

The

th step in the recursion has complexity

. Overall, the complexity of solvingthe

th order YW equations is of order

: more efficient than a direct inversion.

13


The Levinson algorithmTo show that this recursion works and does not break down, we need to prove that

we always can compute a suitable

. This will follow from the main assumption:

(i.e., the covariance matrix strictly positive definiteall its eigenvalues arepositive) for all

.

. This follows from the YW equations:

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

In particular,

. Since

is strictly positive definite (hence alsoinvertible), the inverse

exists and is also strictly positive definite. This implies

that

. Hence

for any

.

In particular,

, so that we can compute

.

14


The Levinson algorithmWe can show a bit more:

The update equation is

so that

.

From this we also see that

: the modeling/prediction error always de-creases.

Special case: if we have a true AR process of order

, then we will find

and

. At this point the recursion stops (we can take

, so that the

AR coefficients do not change anymore).

Special case: if

, then

. The prediction error is zero,

can

be exactly predicted from its past, the process is called deterministic.

This can occur only if

is singular (thus the matrix is not strictly positive definite).

15


The Levinson algorithm

Szeg polynomials

We can define filter functions corresponding to the filter coefficients:

This allows to write the Levinson recursion compactly as

with initialization (

) as

.

16



FIR lattice filter

Thus, the resulting system has impulse response

(and also

, not used).If we use this system with input

, we obtain the prediction error sequence

,

see Slide 4.

The filter structure is known as a lattice filter; note it is an FIR filter.

17


The Levinson algorithmThe recursion can be reversed as follows:

18



IIR lattice filter

This allows to compute

from

: the filter impulse response is

. This is

an IIR filter, the filter structure is recursive, i.e., the filter coefficients of

are not

explicitly computed.

It can be shown that the filter is stable as long as all reflection coefficients are strictlysmaller than 1. This is the case if the original

is strictly positive definite.

If we replace

by any white noise sequence (with variance

), then the re-sulting output signal is a random process with the same correlation sequence

as

.

19



This is exploited in the GSM system, where

is the speech signal.

At the transmitter, the speech signal is split in short frames, each of 20 ms. Foreach frame, the correlation sequence

is estimated. Using theLevinson algorithm, the reflection coefficients and residual noise power

are

estimated, coded and transmitted to the receiver.

The receiver uses

as coefficients in the reverse filter, and generatesa white noise sequence in place of

. It computes the corresponding

.

It is not the same as the original speech signal, but has the same correlationsequence, which is good enough for the ear (at least for unvoiced speech).

20

6 Levinson

Documents

Transcript of 6 Levinson