Professor A G

Constantinides© 1

AGCDSP

Adaptive Signal Processing Problem: Equalise through a FIR filter the

distorting effect of a communication channel that may be changing with time.

If the channel were fixed then a possible solution could be based on the Wiener filter approach

We need to know in such case the correlation matrix of the transmitted signal and the cross correlation vector between the input and desired response.

When the the filter is operating in an unknown environment these required quantities need to be found from the accumulated data.

Professor A G

Constantinides© 2

AGCDSP

Adaptive Signal Processing The problem is particularly acute when

not only the environment is changing but also the data involved are non-stationary

In such cases we need temporally to follow the behaviour of the signals, and adapt the correlation parameters as the environment is changing.

This would essentially produce a temporally adaptive filter.

Professor A G

Constantinides© 3

AGCDSP

Adaptive Signal Processing A possible framework is:

][nd][ˆ nd]}[{ nx

][new:Filter

Adaptive

Algorithm

Professor A G

Constantinides© 4

AGCDSP

Adaptive Signal Processing Applications are many

Digital Communications Channel Equalisation Adaptive noise cancellation Adaptive echo cancellation System identification Smart antenna systems Blind system equalisation And many, many others

Professor A G

Constantinides© 5

AGCDSP

Applications

Professor A G

Constantinides© 6

AGCDSP

Adaptive Signal Processing Echo Cancellers in Local Loops

-+

-+

Rx1 Rx2

Tx1 Rx2

Echo canceller Echo canceller

Adaptive Algorithm Adaptive Algorithm

Hybrid Hybrid

Local Loop

Professor A G

Constantinides© 7

AGCDSP

Adaptive Signal Processing Adaptive Noise Canceller

Noise

Signal +Noise

-+

FIR filter

Adaptive Algorithm

PRIMARY SIGNAL

REFERENCE SIGNAL

Professor A G

Constantinides© 8

AGCDSP

Adaptive Signal Processing System Identification

Unknown System

Signal

-+

FIR filter

Adaptive Algorithm

Professor A G

Constantinides© 9

AGCDSP

Adaptive Signal Processing System Equalisation

Unknown System

Signal

-+

FIR filter

Adaptive Algorithm

Delay

Professor A G

Constantinides© 10

AGCDSP

Adaptive Signal Processing Adaptive Predictors

Signal

-+

FIR filter

Adaptive Algorithm

Delay

Professor A G

Constantinides© 11

AGCDSP

Adaptive Signal Processing Adaptive Arrays

Linear Combiner

Interference

Professor A G

Constantinides© 12

AGCDSP

Adaptive Signal Processing Basic principles: 1) Form an objective function

(performance criterion) 2) Find gradient of objective function with

respect to FIR filter weights 3) There are several different approaches

that can be used at this point 3) Form a differential/difference equation

from the gradient.

Professor A G

Constantinides© 13

AGCDSP

Adaptive Signal Processing Let the desired signal be The input signal The output Now form the vectors

So that

][nd][nx

][ny

Tmnxnxnxn ]1[.]1[][][ x Tmhhh ]1[.]1[]0[ h

hx Tnny ][][

Professor A G

Constantinides© 14

AGCDSP

Adaptive Signal Processing The form the objective function

where

}][][{)( 2nyndEJ wRhhphhpw TTT

dJ 2)(

}][][{ TnnE xxR

]}[][{ ndnE xp

Professor A G

Constantinides© 15

AGCDSP

Adaptive Signal Processing We wish to minimise this function

at the instant n Using Steepest Descent we write

But ][])[(

21][]1[

nnJnn

hhhh

Rhphh 22)(

J

Professor A G

Constantinides© 16

AGCDSP

Adaptive Signal Processing So that the “weights update equation”

Since the objective function is quadratic this expression will converge in m steps

The equation is not practical If we knew and a priori we could

find the required solution (Wiener) as

])[(][]1[ nnn Rhphh

pR

pRh 1opt

Professor A G

Constantinides© 17

AGCDSP

Adaptive Signal Processing However these matrices are not known Approximate expressions are obtained by

ignoring the expectations in the earlier complete forms

This is very crude. However, because the update equation accumulates such quantities, progressive we expect the crude form to improve

Tnnn ][][][ˆ xxR ][][][ˆ ndnn xp

Professor A G

Constantinides© 18

AGCDSP

The LMS Algorithm Thus we have

Where the error is

And hence can write

This is sometimes called the stochastic gradient descent

])[][][]([][]1[ nnndnnn T hxxhh

])[][(])[][][(][ nyndnnndne T hx

][][][]1[ nennn xhh

Professor A G

Constantinides© 19

AGCDSP

ConvergenceThe parameter is the step size,

and it should be selected carefully If too small it takes too long to

converge, if too large it can lead to instability

Write the autocorrelation matrix in the eigen factorisation form

ΛQQR T

Professor A G

Constantinides© 20

AGCDSP

Convergence Where is orthogonal and is

diagonal containing the eigenvalues The error in the weights with respect

to their optimal values is given by (using the Wiener solution for

We obtain

Q Λ

])[(][]1[ nnn optoptopt RhRhhhhh

p

][][]1[ nnn hhh Reee

Professor A G

Constantinides© 21

AGCDSP

Convergence Or equivalently

I.e.

Thus we have

Form a new variable

][)1(]1[ nn hh eΛQQe T

][)(

][)1(]1[

n

nn

h

hh

eΛQQQQ

eΛQQQQeT

T

][)1(]1[ nn hh QeΛQe

][][ nn hQev

Professor A G

Constantinides© 22

AGCDSP

Convergence So that

Thus each element of this new variable is dependent on the previous value of it via a scaling constant

The equation will therefore have an exponential form in the time domain, and the largest coefficient in the right hand side will dominate

][)1(]1[ nn vΛv

Professor A G

Constantinides© 23

AGCDSP

Convergence We require that

Or

In practice we take a much smaller value than this

11 max

max

20

Professor A G

Constantinides© 24

AGCDSP

Estimates Then it can be seen that as the

weight update equation yields And on taking expectations of both

sides of it we have

Or

n

]}[{]}1[{ nEnE hh

])}[][][]([{]}[{]}1[{ nnndnEnEnE Thxxhh

])}[][][][][{(0 nnnndnE T hxxx

Professor A G

Constantinides© 25

AGCDSP

Limiting forms This indicates that the solution

ultimately tends to the Wiener form

I.e. the estimate is unbiased

Professor A G

Constantinides© 26

AGCDSP

Misadjustment The excess mean square error in the

objective function due to gradient noise Assume uncorrelatedness set

Where is the variance of desired response and is zero when uncorrelated.

Then misadjustment is defined as

optT

dJ hp 2min

2d

opth

minmin /))(( JJJJ LMSXS

Professor A G

Constantinides© 27

AGCDSP

Misadjustment It can be shown that the

misadjustment is given by

m

i i

iXS JJ

1min 1

/

Professor A G

Constantinides© 28

AGCDSP

Normalised LMS To make the step size respond to

the signal needs

In this case

And misadjustment is proportional to the step size.

][][][1

2][]1[ 2 nenn

nn xx

hh

10

Professor A G

Constantinides© 29

AGCDSP

Transform based LMS

][nd][ˆ nd]}[{ nx

][new:Filter

Adaptive

Algorithm

Transform

Inverse Transform

Professor A G

Constantinides© 30

AGCDSP

Least Squares Adaptive with

We have the Least Squares solution

However, this is computationally very intensive to implement.

Alternative forms make use of recursive estimates of the matrices involved.

n

i

Tiin1

][][][ xxR

n

indnn

1][][][ xp

][][][ 1 nnn pRh

Professor A G

Constantinides© 31

AGCDSP

Recursive Least Squares Firstly we note that

We now use the Inversion Lemma (or the Sherman-Morrison formula)

Let

][][]1[][ ndnnn xpp

Tnnnn ][][]1[][ xxRR

Professor A G

Constantinides© 32

AGCDSP

Recursive Least Squares (RLS)

Let

Then

The quantity is known as the Kalman gain

][]1[][1][]1[][ 1

1

nnnnnn T xRx

xRk

1][][ nn RP

]1[][][]1[][ nnnnn T PxkRP

][nk

Professor A G

Constantinides© 33

AGCDSP

Recursive Least Squares Now use in the computation

of the filter weights

From the earlier expression for updates we have

And hence

][][][ nnn xPk

])[][]1[]([][][][ ndnnnnnn xpPpPh ][nP

]1[]1[][][]1[]1[]1[][ nnnnnnnn T pPxkpPpP

])1[][][]([]1[][ nnndnnn Thxkhh

Professor A G

Constantinides© 34

AGCDSP

Kalman Filters Kalman filter is a sequential estimation

problem normally derived from The Bayes approach The Innovations approach

Essentially they lead to the same equations as RLS, but underlying assumptions are different

Professor A G

Constantinides© 35

AGCDSP

Kalman Filters The problem is normally stated as:

Given a sequence of noisy observations to estimate the sequence of state vectors of a linear system driven by noise.

Standard formulation][][]1[ nnn wAxx ][][][][ nnnn νxCy

Professor A G

Constantinides© 36

AGCDSP

Kalman Filters Kalman filters may be seen as RLS with

the following correspondenceSate space

RLS Sate-Update matrix Sate-noise variance Observation matrix Observations State estimate

][nA

][nx

Tn][x][nC][ny

][nh

}][][{][ TnnEn wwQ I0

][nd

Professor A G

Constantinides© 37

AGCDSP

Cholesky Factorisation In situations where storage and to some

extend computational demand is at a premium one can use the Cholesky factorisation tecchnique for a positive definite matrix

Express , where is lower triangular

There are many techniques for determining the factorisation

TLLR L