1-Adaptive Signal Processing (1)

8/13/2019 1-Adaptive Signal Processing (1)

1/37

AGC

DSP

Professor A G Constantinides 1

Adaptive Signal Processing Problem: Equalise through a FIR filter the distorting

effect of a communication channel that may bechanging with time.

If the channel were fixedthen a possible solutioncould be based on the Wiener filterapproach

We need to know in such case the correlation matrixof the transmitted signal and the cross correlation

vector between the input and desired response. When the the filter is operating in an unknown

environment these required quantities need to be

found from the accumulated data.


2/37

AGC

DSP


Adaptive Signal Processing The problem is particularly acute when not

only the environment is changingbut also the

data involved are non-stationary In such cases we need temporally to follow

the behaviour of the signals, and adaptthecorrelation parameters as the environment ischanging.

This would essentially produce a temporallyadaptive filter.


3/37

AGC

DSP


Adaptive Signal ProcessingA possible framework is:

][nd][ nd]}[{ nx

][ne

w:Filter

Adaptive

Algorithm


4/37

AGC

DSP


Adaptive Signal Processing Applications are many

Digital Communications

Channel Equalisation Adaptive noise cancellation

Adaptive echo cancellation

System identification

Smart antenna systems

Blind system equalisation

And many, many others


5/37

AGC

DSP


Applications


6/37

AGC

DSP


Adaptive Signal Processing Echo Cancellers in Local Loops

-

+

-

+

Rx1

Rx2

Tx1 Rx2

Echo canceller Echo canceller

Adaptive AlgorithmAdaptive Algorithm

HybridHybrid

Local Loop


7/37


8/37

AGC

DSP


Adaptive Signal Processing System Identification

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm


9/37

AGC

DSP


Adaptive Signal Processing System Equalisation

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm

Delay


10/37

AGC

DSP


Adaptive Signal ProcessingAdaptive Predictors

Signal

-

+

FIR filter

Adaptive Algorithm

Delay


11/37

AGC

DSP


Adaptive Signal ProcessingAdaptive Arrays

Linear Combiner

Interference


12/37

AGC

DSP


Adaptive Signal Processing Basic principles:

1) Form an objective function (performance

criterion) 2) Find gradient of objective function with

respect to FIR filter weights

3) There are several different approaches

that can be used at this point 3) Form a differential/difference equation

from the gradient.


13/37

AGC

DSP


Adaptive Signal Processing Let the desired signal be

The input signal

The output Now form the vectors

So that

][nd][nx

][ny

Tmnxnxnxn ]1[.]1[][][ x

Tmhhh ]1[.]1[]0[ h

hx Tnny ][][


14/37

AGC

DSP


Adaptive Signal Processing The form the objective function

where

}][][{)( 2nyndEJ w

Rhhphhpw TTT

dJ 2)(

}][][{ TnnE xxR

]}[][{ ndnE xp


15/37

AGC

DSP


Adaptive Signal Processing We wish to minimise this function at the

instant n

Using Steepest Descentwe write

But

][

])[(

2

1][]1[

n

nJnn

h

hhh

Rhph

h22

)(

J


16/37

AGC

DSP


Adaptive Signal Processing So that the weights update equation

Since the objective function is quadratic thisexpression will converge in m steps

The equation is not practical If we knew and a priori we could find

the required solution (Wiener) as

])[(][]1[ nnn Rhphh

pR

pRh 1opt


17/37

AGC

DSP


Adaptive Signal Processing However these matrices are not known

Approximate expressions are obtained by

ignoring the expectations in the earliercomplete forms

This is very crude. However, because theupdate equation accumulates such quantities,progressive we expect the crude form toimprove

Tnnn ][][][ xxR ][][][ ndnn xp


18/37

AGC

DSP


The LMS Algorithm Thus we have

Where the error is

And hence can write

This is sometimes called the stochasticgradientdescent

])[][][]([][]1[ nnndnnn Thxxhh

])[][(])[][][(][ nyndnnndne T hx

][][][]1[ nennn xhh


19/37

AGC

DSP


ConvergenceThe parameter is the step size, and it

should be selected carefully

If too small it takes too long toconverge, if too large it can lead toinstability

Write the autocorrelation matrix in theeigen factorisation form

QQR T


20/37

AGC

DSP


Convergence Where is orthogonal and is

diagonal containing the eigenvalues

The error in the weights with respect totheir optimal values is given by (usingthe Wiener solution for

We obtain

Q

])[(][]1[ nnn optoptopt RhRhhhhh

p

][][]1[ nnn hhh Reee


21/37

AGC

DSP


Convergence Or equivalently

I.e.

Thus we have

Form a new variable

][)1(]1[ nn hh eQQe T

][)(

][)1(]1[

n

nn

h

hh

eQQQQ

eQQQQe

T

T

][)1(]1[ nn hh QeQe

][][ nn hQev


22/37

AGC

DSP


Convergence So that

Thus each element of this new variable isdependent on the previous value of it via ascaling constant

The equation will therefore have an

exponential form in the time domain, and thelargest coefficient in the right hand side willdominate

][)1(]1[ nn vv


23/37

AGC

DSP


Convergence We require that

Or

In practice we take a much smallervalue than this

11 max

max

20


24/37

AGC

DSP


Estimates Then it can be seen that as the

weight update equation yields

And on taking expectations of both sides of it

we have

Or

n

]}[{]}1[{ nEnE hh

])}[][][]([{]}[{]}1[{ nnndnEnEnE

T

hxxhh

])}[][][][][{(0 nnnndnE Thxxx


25/37

AGC

DSP


Limiting forms This indicates that the solution

ultimately tends to the Wiener form

I.e. the estimate is unbiased


26/37

AGC

DSP


Misadjustment The excess mean square error in the

objective function due to gradient noise

Assume uncorrelatedness set

Where is the variance of desired

response and is zero when uncorrelated. Then misadjustment is defined as

optT

dJ hp 2

min 2d

opth

minmin /))(( JJJJ LMSXS


27/37

AGC

DSP


Misadjustment It can be shown that the misadjustment

is given by

m

i i

iXS JJ

1min

1/


28/37

AGC

DSP


Normalised LMS To make the step size respond to the

signal needs

In this case

And misadjustment is proportional tothe step size.

][][][1

2][]1[

2 nen

nnn x

xhh

10


29/37

AGC

DSP


Transform based LMS

][nd][ nd]}[{ nx

][new:Filter

Adaptive

AlgorithmTransform

Inverse Transform


30/37

AGC

DSP


Least Squares Adaptive with

We have the Least Squares solution

However, this is computationally veryintensive to implement.

Alternative forms make use of recursiveestimates of the matrices involved.

n

i

Tiin1

][][][ xxR

n

indnn

1][][][ xp

][][][ 1 nnn pRh


31/37

AGC

DSP


Recursive Least Squares Firstly we note that

We now use the Inversion Lemma (or the

Sherman-Morrison formula) Let

][][]1[][ ndnnn xpp

Tnnnn ][][]1[][ xxRR


32/37

AGC

DSP


Recursive Least Squares (RLS) Let

Then

The quantity is known as the Kalmangain

][]1[][1

][]1[][ 1

1

nnn

nnn T xRx

xRk

1][][ nn RP

]1[][][]1[][ nnnnn T PxkRP

][nk


33/37

AGC

DSP


Recursive Least Squares Now use in the computation of

the filter weights

From the earlier expression for updates wehave

And hence

][][][ nnn xPk

])[][]1[]([][][][ ndnnnnnn xpPpPh

][nP

]1[]1[][][]1[]1[]1[][ nnnnnnnn T

pPxkpPpP

])1[][][]([]1[][ nnndnnn T

hxkhh


34/37

AGC

DSP


Kalman Filters Kalman filter is a sequential estimation

problem normally derived from

The Bayes approach The Innovations approach

Essentially they lead to the same equations

as RLS, but underlying assumptions aredifferent


35/37

AGC

DSP


Kalman Filters The problem is normally stated as:

Given a sequence of noisy observations to

estimate the sequence of state vectors of a linearsystem driven by noise.

Standard formulation

][][]1[ nnn wAxx

][][][][ nnnn xCy


36/37

AGC

DSP


Kalman Filters Kalman filters may be seen as RLS with the

following correspondenceSate space RLS

Sate-Update matrix

Sate-noise variance

Observation matrix

Observations

State estimate

][nA

][nx

Tn][x][nC

][ny

][nh

}][][{][ T

nnEn wwQ

I0

][nd


37/37

AGC

DSP

Cholesky Factorisation In situations where storage and to some

extend computational demand is at a

premium one can use the Choleskyfactorisation tecchnique for a positive definitematrix

Express , where is lower

triangular

There are many techniques for determiningthe factorisation

TLLR L

1-Adaptive Signal Processing (1)

Documents

Transcript of 1-Adaptive Signal Processing (1)