This article was downloaded by: [Birla Institute of Technology and Science]On: 23 March 2015, At: 12:18Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office:Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Systems SciencePublication details, including instructions for authors and subscriptioninformation:http://www.tandfonline.com/loi/tsys20

Nonlinear system identification using waveletnetworksG. P. Liu a , S A. Billings b & V Kadirkamanathan ba ABB ALSTOM Power Technology Centre , Cambridge Road, Leicester, LE8 6LH,U.K.b Department of Automatic Control and Systems Engineering , University ofSheffield , PO Box 600, Mappin Street, Sheffield, S1 3JD, U.K.Published online: 26 Nov 2010.

To cite this article: G. P. Liu , S A. Billings & V Kadirkamanathan (2000) Nonlinear system identificationusing wavelet networks, International Journal of Systems Science, 31:12, 1531-1541, DOI:10.1080/00207720050217304

To link to this article: http://dx.doi.org/10.1080/00207720050217304

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)contained in the publications on our platform. However, Taylor & Francis, our agents, and ourlicensors make no representations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressed in this publication arethe opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoevercaused arising directly or indirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes. Any substantialor systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, ordistribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use canbe found at http://www.tandfonline.com/page/terms-and-conditions

International Journal of Systems Science, 2000, volume 31, number 12, pages 1531 ± 1541

Nonlinear system identi® cation using wavelet networks

G. P. LIU{*, S. A. BILLINGS{ and V. KADIRKAMANATHAN{

A novel identi® cation scheme using wavelet networks is presented for nonlinear dyna-

mical systems. Based on ® xed wavelet networks, parameter adaptation laws are devel-

oped using a Lyapunov synthesis approach. This guarantees the stability of the overall

identi® cation scheme and the convergence of both the parameters and the state errors,

even in the presence of modelling errors. Using the decomposition and reconstructiontechniques of multiresolution decompositions , variable wavelet networks are introduced

to achieve a desired estimation accuracy and a suitable sized network, and to adapt to

variations of the characteristics and operating points in nonlinear systems. B-spline

wavelets are used to form the wavelet networks and the identi ® cation scheme is illu-

strated using a simulated example.

1. Introduction

Nonlinear system identi® cation consists of model struc-

ture selection and parameter estimation. The ® rst prob-

lem is concerned with selecting a class of mathematical

operator as a model. The second is concerned with an

estimation algorithm based on input± output data from

the process, a class of models to be identi® ed and a

suitable identi® cation criterion. A number of techniques

have been developed in recent years for model selection

and parameter estimation of nonlinear systems.

Forward and backward regression algorithms were ana-

lysed in Leontaritis and Billings (1987). Stepwise regres-

sion was used in Billings and Voon (1986), and a class of

orthogonal estimators was discussed in Korenberg et al.

(1988). Algorithms which save memory and allow fast

computations have also been proposed in Chen and

Wigger (1995) . Methods to determine a priori structural

identi® ability of a model have also been studied (Ljung

and Glad 1994). A survey of existing techniques of non-

linear system identi® cation prior to the 1980s is given in

Billings (1980), a survey of the structure detection of

input± output nonlinear systems is given in Haber and

Unbehauen (1990), and a recent survey of nonlinear

black-box modelling in system identi® cation can be

found in Sjoberg et al. (1995).

The approximation of general continuous functions

by nonlinear networks has been widely applied to

system modelling and identi® cation. Such approxima-

tion methods are particularly useful in the black-box

identi® cation of nonlinear systems where very little a

priori knowledge is available. For example, neural net-

works have been established as a general approximation

tool for ® tting nonlinear models from input± output data

on the basis of the universal approximation property of

such networks (see, e.g., Chen et al. 1990, Narendra and

Parthasarathy , 1990, Liu et al. 1996a). There has alsobeen considerable recent interest in identi® cation of gen-

eral nonlinear systems based on radial basis networks(Poggio and Girosi 1990), fuzzy sets and rules (Zadeh

1994), neural-fuzzy networks (Brown and Harris 1994;

Wang et al. 1995) and hinging hyperplanes (Beriman

1993).

The recently introduced wavelet decomposition

(Grossmannand and Morlet 1984, Daubechies 1988,

Mallat 1989a, Chui 1992, Meyer 1993, IEEE 1996)

also emerges as a new powerful tool for approximation.

In recent years, wavelets have become a very active sub-

ject in many scienti® c and engineering research areas.

Wavelet decompositions provide a useful basis for loca-

lized approximation of functions with any degree of

regularity at diŒerent scales and with a desired accuracy.

Recent advances have also shown the existence of ortho-

normal wavelet bases, from which follows the variability

of rates of convergence for approximation by wavelet-

based networks. Wavelets can therefore be viewed as a

International Journal of Systems Science ISSN 0020± 7721 print/ISSN 1464± 5319 online # 2000 Taylor & Francis Ltdhttp://www.tandf.co.uk/journals

Accepted 9 November 1999.

{ ABB ALSTOM Power Technology Centre, Cambridge Road,

Leicester LE8 6LH, U.K.

{ Department of Automatic Control and Systems Engineering,

University of She� eld, PO Box 600, Mappin Street, She� eld S1

3JD, U.K.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

new basis for representing functions. Wavelet-based net-

works (or simply wavelet networks) are inspired by both

the feedforward neural networks and wavelet decompo-

sitions and have been introduced for the identi® cation of

nonlinear static systems (Zhang and Benveniste 1992).But little attention has been paid to identi® cation of

nonlinear dynamical systems using wavelet networks

(Coca and Billings 1996, Liu et al. 1998, 1999a).

This paper presents a wavelet network-based identi® -

cation scheme for nonlinear dynamical systems. Two

kinds of wavelet networks are studied: ® xed and variablewavelet networks. The former is used for the case where

the estimation accuracy is assumed to be achieved by a

known resolution scale. But, in practice, this assumption

is not realistic because the nonlinear function to be iden-

ti® ed is unknown and the system operating point maychange with time. Thus, variable wavelet networks are

introduced to deal with this problem. The basic principle

of the variable wavelet network is that the number of

wavelets in the network can either be increased or

decreased over time according to a design strategy inan attempt to avoid over® tting or under® tting. In

order to model unknown nonlinearities, the variable

wavelet network starts with a lower resolution scale,

and then increases or reduces this according to the

novelty of the observation. Because the novelty of the

observation is tested, it is ideally suited for on-line iden-

ti® cation problems. The objective behind the develop-ment is to gradually approach the appropriate network

complexity that is su� cient to provide an approxima-

tion to the system nonlinearities and which is consistent

with the observations received.The parameters of the wavelet network are adjusted

by adaptation laws developed using a Lyapunov syn-

thesis approach. The identi® cation algorithm is per-

formed over the network parameters by taking

advantage of the decomposition and reconstructionalgorithms of a multiresolution decomposition when

the resolution scale changes in the variable wavelet net-

work. Combining the wavelet network and Lyapunov

synthesis techniques, the identi® cation algorithm devel-

oped for continuous dynamical nonlinear systems guar-

antees the stability of the whole identi® cation scheme

and the convergence of both the parameters and estima-tion errors. A simulated example shows the operation of

the proposed identi® cation scheme.

2. Wavelet networks

Wavelets are a class of functions which have some inter-

esting and special properties. Some basic concepts aboutorthonormal wavelet bases will be introduced initially.

Then the wavelet series representation of one-dimen-

sional and multi-dimensional functions will be consid-

ered. Finally, wavelet networks are introduced.

Throughout this paper, the following notations will

be used. Z and R denote the set of integers and real

numbers, respectively. L2… R † denotes the vector space

of measurable, square-integrable one-dimensional func-

tions f …x†. For f ; g 2 L2…R †, the inner product andnorm for the space L2… R † are written as

h f ; gi :ˆ…1

¡1f …x†g…x† dx …1†

k f k2 :ˆ h f ; f i1=2 …2†

where g…:† is the conjugate of the function g…:†. L2… R n† is

the vector space of measurable, square-integrable n-

dimensional functions f …x1; x2; . . . ; xn†. For

f ; g 2 L2… R n†, the inner product of f …x1; x2; . . . ; xn†with g…x1; x2; . . . ; xn† is written as

h f …x1; x2; . . . ; xn†; g…x1; x2; . . . ; xn†i

ˆ…1

¡1

…1

¡1¢ ¢ ¢

…1

¡1f …x1; x2; . . . ; xn†

£ g…x1; x2; . . . ; xn† dx1 dx2 . . . dxn …3†

The original objective of the theory of wavelets is to

construct orthogonal bases in L2… R †. These bases are

constituted by translations and dilations of the same

function Á. It is preferable to take Á as localized and

regular. The principle of wavelet construction is the fol-lowing: (i) the function ¿…x ¡ k† are mutually ortho-

gonal for k ranging over Z ; (ii) ¿ is a scaling function

and for any j, f¿…2jx ¡ k†g constitutes an orthogonal

basis of its linear span subspace; (iii) the wavelet is

de® ned as Á and the family Á…2jx ¡ k† constitutes anorthogonal basis of L2… R †. It can also be proved that

the family f¿…2j0x ¡ k†; Á…2jx ¡ k†; for j ¶ j0g also

forms an orthogonal basis of L2… R †.The wavelet subspaces Wj are de® ned as

Wj ˆ fÁ…2jx ¡ k†; k 2 Z g …4†

which satisfy

Wj \ Wi ˆ f1g; j 6ˆ i …5†

Any wavelet generates a direct sum decomposition of

L2… R †. For each j 2 Z , let us consider the closed sub-

spaces:

Vj ˆ ¢ ¢ ¢ © Wj¡2 © Wj¡1 …6†

of L2… R †, where © denotes the direct sum. These sub-

spaces have the following properties:

(1) . . . » V¡1 » V0 » V1 » . . .;

(2) closeL2

[

j2Z

Vj

Á !ˆ L2… R †;

(3)\

j2Z

Vj ˆ f1g;

1532 G. P. Liu et al.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

(4) Vj‡1 ˆ Vj ‡ Wj, j 2 Z ; and

(5) f …x† 2 Vj () f …2x† 2 Vj‡1; j 2 Z .

Hence, the sequence of subspaces Vj is nested, as

described by property (1). Property (2) shows that

every function f in L2… R † can be approximated as

closely as desirable by its projections, denoted by Pj f

in Vj . But, by decreasing j, the projections Pj f couldhave arbitrarily small energy, as guaranteed by property

(3). The most important intrinsic property of these

spaces is that more and more variations of Pj f are

removed as j ! ¡1. In fact, these variations are

peeled oŒ, level by level in decreasing order of the rate

of variations and stored in the complementary subspaces

Wj as in property (4). This process can be made verye� cient by an application of property (5).

If ¿ and Á are compactly supported, they give a local

description, at diŒerent scales j, of the considered func-

tion. The wavelet series representation of the one-dimen-sional function f …x† is given by

f …x† ˆX

k2Z

aj0k¿j0k…x† ‡X

j¶j0

X

k2Z

bjkÁjk…x† …7†

where

¿j0k…x† ˆ 2j0=2¿…2j0x ¡ k†;

Ájk…x† ˆ 2j=2Á…2jx ¡ k†;

and the wavelet coe� cients aj0k and bjk are

aj0k ˆ h f …x†; ¿j0k…x†i …8†

bjk ˆ h f …x†; Ájk…x†i …9†

The wavelet series representation can easily be general-

ized to any dimension n. For the n-dimensional case

x ˆ ‰x1; x2; . . . ; xnŠ, we introduce the scaling function

U …x† ˆ ¿…x1†¿…x2† . . . ¿…xn† …10†

and the 2n ¡ 1 mother wavelets W i…x†,i ˆ 1; 2; . . . ; 2n ¡ 1, are obtained by substituting some

¿…xj†s by Á…xj† in equation (10). Then the following

family is an orthonormal basis in L2… R n†:

f U j0k…x†; W…1†jk …x†; W

…2†jk …x†; . . . ; W

…2n¡1†jk …x†g …11†

for j ¶ j0, j 2 Z , k ˆ ‰k1; k2; . . . ; knŠ 2 Z n, and

U jk…x† ˆ 2jn=2 U …2jx1 ¡ k1; 2jx2 ¡ k2; . . . ; 2jxn ¡ kn†…12†

W…i†jk …x† ˆ 2jn=2 W i…2jx1 ¡ k1; 2jx2 ¡ k2; . . . ; 2jxn ¡ kn†

…13†

For f …x† 2 L2… R n†, the n-dimensional wavelet series

representation of the function f …x† is

f …x† ˆX

k2Z n

aj0k U j0k…x† ‡X

j¶j0

X

k2Z n

X2n¡1

iˆ1

b…i†jk W

…i†jk …x† …14†

where the wavelet coe� cients are

aj0k ˆ h f …x†; U j0k…x†i …15†

b…i†jk ˆ h f …x†; W

…i†jk …x†i …16†

For system identi® cation, f …x† is unknown. Then the

wavelet coe� cients aj0k and b…i†jk can not be calculated

simply by equations (15) and (16). As equation (11)

shows, constructing and storing orthonormal waveletbases involves a prohibitive cost for large dimensions

n. In addition, it is not realistic to use an in® nite

number of wavelets to represent the function f …x†. So,

we consider the following wavelet representation of the

function f …x†:

f̂ …x† ˆX

k2Aj0

aj0k U j0k…x† ‡XN

jˆj0

X

k2Bj

X2n¡1

iˆ1

b…i†jk W

…i†jk …x† …17†

where Aj0; Bj 2 Z n are the ® nite vector sets of integers

and N 2 R 1 is a ® nite integer. Because the convergence

of the series in equation (14) is in L2… R n†,

limN!1;Aj0

;Bj0;...;BN !Z n

k f …x† ¡ f̂ …x†k2 ˆ 0 …18†

Hence, given " > 0, there exists a number N* and vector

sets A*j0 ; B*j0 ; B*j0‡1; . . . ; B*N such that for N ¶ N*,

Aj0³ A*j0 and Bj0

³ B*j0 , Bj0‡1 ³ B*j0‡1; . . . ; BN ³ B*N ,

k f …x† ¡ f̂ …x†jj2 µ " …19†

This shows that the required approximation accuracy of

the function f by f̂ can be guaranteed by properly

choosing the number N and the vector setsAj0

; Bj0; . . . ; BN . Following neural networks, the expres-

sion (17) is called a wavelet network. In this network,

the parameters aj0k and b…i†jk , and the number N and the

vector sets Aj0; Bj0

; . . . ; BN will jointly be determined

from the data, based on the scaling functions ¿ andthe wavelets Á.

3. Identi® cation using ® xed wavelet networks

Consider the multi-input multi-state (MIMS) contin-

uous dynamical system described by,

_x ˆ f…x; u†; x…0† ˆ x0; …20†

where u 2 <r£1 is the input vector, x 2 <d£1 is the state

vector and f…¢† ˆ ‰ f1…¢†; f2…¢†; . . . ; fd…¢†ŠT 2 Ld2… R n† is an

unknown nonlinear function vector, where n ˆ r ‡ d. It

is also assumed that u; x are in compact sets.

Following the structure (17) of wavelet networks, at

the resolution 2N‡1, the estimation f̂…x; u† of the func-

Nonlinear system identi ® cation using wavelet networks 1533

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

tion f…x; u† is the output vector of the wavelet network

which is expressed by

f̂…x; u† ˆX

k2Aj0

Aj0k U j0k…x; u† ‡XN

jˆj0

X

k2Bj

X2n¡1

iˆ1

B…i†jk W

…i†jk …x; u†

…21†

where Aj0k; B…i†jk 2 R d are the wavelet coe� cient vectors,

the scaling function U j0k…x; u† and the wavelet functions

W…i†jk …x; u† are similarly de® ned as U …x† and W j…x†, re-

spectively, by replacing x with …x; u†.Here, it is assumed that the number N and the vector

sets Aj0; Bj are given. So, the wavelet network (21) for

the estimation of the nonlinear function f…x; u† is called

a ® xed wavelet network. Based on the estimation f̂…x; u†by the ® xed wavelet network, the nonlinear function

f…x; u† can be expressed by

f…x; u† ˆX

k2Aj0

A*j0k

U j0k…x; u†

‡XN

jˆj0

X

k2Bj

X2n¡1

iˆ1

B…i†*jk W

…i†jk …x; u† ‡ "N …22†

where the optimal wavelet coe� cient vectors A*j0k

and

B…i†*jk are

A*j0k ˆ

h f1…x; u†; U j0k…x; u†i

h f2…x; u†; U j0k…x; u†i

..

.

h fd…x; u†; U j0k…x; u†i

2

6666664

3

7777775…23†

B…i†*jk ˆ

h f1…x; u†; W…i†jk …x; u†i

h f2…x; u†; W…i†jk …x; u†i

..

.

h fd…x; u†; W…i†jk …x; u†i

2

66666664

3

77777775

…24†

"N ˆ ‰"N1; "N2

; . . . ; "NdŠT is the modelling error vector

which is assumed to be bounded by

"N ˆ maxiˆ1;2;...;d

supt2R ‡

fj"Ni…t†jg …25†

Modelling the nonlinear function vector f…x; u† using

wavelet networks gives the following identi® cation

model for the nonlinear dynamical system (20):

_̂x ˆ A…x̂ ¡ x† ‡ f̂…x; u†; x̂…0† ˆ x0 …26†

where x̂ denotes the state vector of the network modeland A 2 R d£d is a Hurwitz or stability matrix (i.e. all the

eigenvalues are in the open left-half complex plane).

De® ne the state error vector and wavelet coe� cient

error vectors as

ex ˆ x ¡ x̂ …27†~Aj0k ˆ A*

j0k¡ Aj0k …28†

~B…i†jk ˆ B

…i†*jk ¡ B

…i†jk …29†

so that the dynamical expression of the state error is

given by

_ex ˆ Aex ‡X

k2Aj0

~Aj0k U j0k…x; u†

‡XN

jˆj0

X

k2Bj

X2n¡1

iˆ1

~B…i†jk W

…i†jk …x; u† ‡ "N …30†

Consider the Lyapunov function

V ˆ eTx Pex ‡ 1

¬j0

X

k2Aj0

~ATj0k

~Aj0k

‡XN

jˆj0

2

­ j

X

k2Bj

X2n¡1

iˆ1

…~B…i†jk †T ~B

…i†jk …31†

where P ˆ fpijg 2 R d£d is chosen to be a positive de® -

nite matrix so that the matrix Q ˆ ¡PA ¡ ATP is also a

positive de® nite matrix, and ¬j0and ­ j are positive con-

stants which will appear in the parameter adaptation

laws, also referred to as the adaptation rates.

The ® rst derivative of the Lyapunov function V with

respect to time t is

_V ˆ _eTx Pex ‡ eT

x P _ex ‡ 2

¬j0

X

k2Aj0

_~AT

j0k~Aj0k

‡XN

jˆj0

2

­ j

X

k2Bj

X2n¡1

iˆ1

… _~B…i†jk †T ~B

…i†jk …32†

Substituting equation (30) into equation (32) gives

_V ˆ ¡eTx Qex ‡ 2

¬j0

X

k2Aj0

… _~AT

j0k~Aj0k ‡ ¬j0

eTx P~Aj0k U j0k…x; u††

‡XN

jˆj0

2

­ j

X

k2Bj

X2n¡1

iˆ1

…… _~B…i†jk †T ~B

…i†jk ‡ ­ je

Tx P~B

…i†jk W

…i†jk …x; u††

‡ 2eTx P"N …33†

Because A*j0k

and B…i†*jk

are constant vectors,_~Aj0k ˆ ¡ _Aj0k and _~B

…i†jk ˆ ¡ _B…i†

jk . If there is no modelling

error, i.e., "N ˆ 0, Aj0k and B…i†jk can simply be estimated

by the following adaptation laws:

_Aj0k ˆ ¬j0eT

x P U j0k…x; u† …34†

_B…i†jk ˆ ­ je

Tx PW

…i†jk …x; u† …35†

In the presence of a modelling error "N , several algor-

ithms can be applied to ensure the stability of the whole

1534 G. P. Liu et al.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

identi® cation scheme, e.g. the ® xed or switching ¼-mod-

i® cation (Ioannou and Kokotovic 1983), "-modi® cation

(Narendra and Annaswamy 1987) and the dead-zone

methods (Sastry and Bodson 1989).

De® ne the following sets:

F ¡… ; M† ˆ fz : kzk < M

or …kzk ˆ M and eTx Pz ¶ 0†g …36†

F ‡… ; M† ˆ fz : kzk ˆ M and eTx Pz < 0g …37†

where z 2 R d , is a function and M is a positive con-

stant. Here, in order to avoid parameter drift in the

presence of modelling error, the application of the pro-

jection algorithm (Goodwin and Mayne, 1987) gives the

following adaptive laws for the parameter estimates Aj0k

and B…i†jk :

_Aj0k ˆ

¬j0eT

x P U j0k

if Aj0k 2 F ¡… U j0k; Mj0k†

¬j0eT

x PU j0k ‡ ¬j0

M¡2j0k eT

x PAj0k U j0kAj0k

if Aj0k 2 F ‡… U j0k; Mj0k†

8>>>>><

>>>>>:

…38†

_B…i†jk ˆ

­ jeTx P W

…i†jk

if B…i†jk 2 F ¡… W …i†

jk ; M…i†jk †

­ jeTx P W

…i†jk ‡ ­ j…M

…i†jk †¡2eT

x PB…i†jk W

…i†jk B

…i†jk

if B…i†jk 2 F ‡… W …i†

jk ; M…i†jk †

8>>>>>>><

>>>>>>>:

…39†

where Mj0k, M…i†jk are the allowed largest values ofkAj0kk

and kB…i†jk k, respectively. It is clear that if the initial

parameter vectors are chosen such that

Aj0k…0† 2 F ¡… U j0k; Mj0k† [ F ‡… U j0k; Mj0k†and

B…i†jk …0† 2 F ¡… W …i†

jk ; M…i†jk † [ F ‡… W …i†

jk ; M…i†jk †;

then the vectors Aj0k and B…i†jk are con® ned to the sets

F ¡… U j0k; Mj0k† [ F ‡… U j0k; Mj0k†and

F ¡… W …i†jk ; M

…i†jk

† [ F ‡… W …i†jk ; M

…i†jk

†;

respectively. Using the adaptive laws (38) and (39),

equation (33) becomes

_V µ ¡eTx Qex ‡ 2

Xd

iˆ1

Xd

jˆ1

jpijjjexijj"Njj

µ ¡eTx Qex ‡ 2

Xd

iˆ1

Xd

jˆ1

jpijjjexij"N …40†

where ex ˆ ‰ex1; ex2; . . . ; exd ŠT.

For the sake of simplicity, the positive de® nite

matrix Q is assumed to be diagonal, i.e.

Q ˆ diag‰q1; q2; . . . ; qd Š. Also de® ne

H …±† ˆ ex :Xd

iˆ1

qi jexij ¡Xd

jˆ1

jpijjqi

±

Á !28<

:

µXd

iˆ1

Xd

jˆ1

pij

Á !2±2

qi

9=

; …41†

where ± is a positive variable, i.e., ± ¶ 0.

If there is no modelling error (i.e. "N ˆ 0), it is clear

from equation (40) that _V is negative semide® nite.

Hence the stability of the overall identi® cation schemeis guaranteed and ex ! 0, Aj0k ! 0, B

…i†jk ! 0. In the

presence of modelling errors, if ex 62 H …"N†, it is easy

to show from equation (40) that _V is still negative and

the state error ex will converge to the set H …"N†. But, if

ex 2 H …"N†, it is possible that _V > 0, which implies that

the weight vectors Aj0k and B…i†jk may drift to in® nity over

time. The adaptive laws (38) and (39) avoid this drift by

limiting the upper bounds of the parameters. Thus, the

state error ex always converges to the set H …"N† and the

whole identi® cation scheme will remain stable in the

case of modelling errors.

4. Identi® cation using variable wavelet networks

For nonlinear systems, the system operation can change

with time. This will result in an estimation error for the

® xed wavelet network that is beyond the required error.

In order to improve the identi® cation performance, both

the structure and the parameters of the wavelet network

model need to be modi® ed in response to variations ofthe plant characteristics. This section takes into account

the modi® cation of the wavelet network structure and

the adaptation of the parameters.

It is known that the modelling error "N can be

reduced arbitrarily by increasing the resolution of thewavelet network. But, generally when the resolution is

increased beyond a certain value the modelling error "N

will improve very little by any further increasing the

resolution. This will also result in a large size network

even for simple nonlinear problems and in practice, thisis not realistic. In most cases, the required modelling

error can be given by considering the design require-

ments and speci® cations of the system. Thus, the prob-

lem now is to ® nd a suitable sized network to achieve

the required modelling error. Following variable neural

networks (Liu et al. 1996b, 1999b), a variable waveletnetwork is introduced below.

Like a neuron in neural networks, a wavelon is an

information-processin g unit that is fundamental to the

operation of a wavelet network. Generally speaking, a

Nonlinear system identi ® cation using wavelet networks 1535

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

variable wavelet network has the property that the

number of wavelons in the network can be either

increased or decreased over time according to a design

strategy. For the problem of nonlinear modelling, the

variable wavelet network is initialized with a smallnumber of wavelons. As observations are received, the

network grows by adding new wavelons or is pruned by

removing old ones.

According to the multiresolution approximation

theory, increasing the resolution of the network will

improve the approximation. To improve the approxima-tion accuracy, the growing network technique

(Kadirkamanathan and Niranjan 1993, Liu et al.

1996a) is applied. This means that the wavelets at a

higher resolution need to be added to the network.

Here it is assumed that at the resolution 2N the approx-imation of the function f by the wavelet network is

denoted as f̂…N†. Based on the growing network tech-

nique and the structure of the function f̂ in equation

(21), the adding operation is de® ned as

f̂…x; u† ˆ f̂…N†…x; u† _©X

k2BN

X2n¡1

iˆ1

B…i†Nk W

…i†Nk…x; u† …42†

where _© denotes the adding operation. Equation (42)

means that wavelets at the resolution 2N‡1 are added

to the network. To add new wavelons to the networkthe following two conditions must be satis® ed. (i) The

modelling error must be greater than the required accu-

racy. (ii) The period between the two adding operations

must be greater than the minimum response time of the

adding operation.

The removing operation is de® ned as

f̂…x; u† ˆ f̂…N†…x; u† _ªX

k2BN¡1

X2n¡1

iˆ1

B…i†…N¡1†k W

…i†…N¡1†k…x; u†

…43†

where _ª denotes the removing operation. Equation (43)

implies that wavelets at the resolution 2N are removedfrom the network. Similarly, to remove some old wave-

lons from the network, the following two conditions

must be satis® ed. (i) The modelling error must be less

than the required accuracy. (ii) The period between the

two removing operations must be greater than the mini-mum response time of the removing operation.

In both the adding and removing operations, con-

dition (i) means that the change of the modelling error

in the network must be signi® cant. Condition (ii) says

the minimum response time of each operation must be

considered. The response time of the network re® nementoperation varies with diŒerent systems. It is usually esti-

mated by a trial and error process.

From the set H …"N† which gives a relationship

between the state error ex and the modelling errors "N ,

it can be shown that the state error depends on the

modelling error. If the upper bound "N of the modelling

error is known, then the set H …"N† to which the state

error will converge is also known. However, in most

cases the upper bound "N is unknown.In practice, systems are usually required to keep the

state errors within prescribed bounds, i.e.

jexij µ ei; for i ˆ 1; 2; . . . ; n; t 2 R ‡ …44†

where ei is the required accuracy. At the beginning, it is

very di� cult to know how many wavelons are needed to

achieve the above identi® cation requirements. In order

to ® nd a suitable sized network for this identi® cation

problem, lower and upper bounds are set for the stateerrors which are functions of time t. A variable network

such that

jexij 2 ‰²Li …t†; ²U

i …t† ‡ eiŠ; for i ˆ 1; 2; . . . ; n …45†

is then tried, where ²Li …t†; ²U

i …t† are monodecreasingfunctions of time t, respectively. For example,

²Ui …t† ˆ e¡­ U t²U

i …0† …46†

²Li …t† ˆ e¡­ Lt²L

i …0† …47†

where ­ U ; ­ L are positive constants, ²Ui …0†; ²L

i …0† are the

initial values. It is clear that ²Ui …t†; ²L

i …t† decrease with

time t. As t ! 0, ²Ui …t†; ²L

i …t† approach 0. Thus, in this

way the state errors reach the required accuracy given in

equation (44).From the relationship between the modelling error

and the state error, and given the lower and upper

bounds ²Ui …t†; ²L

i …t† ‡ ei of the state errors, the cor-

responding modelling error should be

"N…t† 2 ‰"L…t†; "U…t†Š …48†

From equation (41), the area that the set H …±† covers is a

hyperellipsoid with the centre

Xd

jˆ1

jp1j jq1

±;Xd

jˆ1

jp2jjq2

±; . . . ;Xd

jˆ1

jpdj jqn

±

Á !…49†

It can also be deduced from the set H …"N…t†† that the

upper bound "U…t† and the lower bound "L…t† are given

by

"L…t† ˆ miniˆ1;2;...;d

Xd

jˆ1

jpijjqi

Á(

‡Xd

kˆ1

Xd

jˆ1

pkj

Á !21

qiqk

0

@

1

A1=2

1

CA

¡1

²Li …t†

9>=

>;…50†

1536 G. P. Liu et al.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

"U…t† ˆ maxiˆ1;2;...;d

Xd

jˆ1

jpijjqi

Á(

‡Xd

kˆ1

Xd

jˆ1

pkj

Á !21

qiqk

0

@

1

A1=2

1

CA

¡1

…²Ui …t† ‡ ei†

9>=

>;

…51†

Thus, given the upper and lower bounds of the stateerror, the corresponding values for the modelling error

can be estimated by equations (50) and (51). But, if these

bounds are not known, the ® xed wavelet network with

an adequate number of wavelons may be used rather

than the variable wavelet network.To smooth the identi® cation performance when the

adding and removing operations are used, the decom-

position and reconstruction algorithms of a multiresolu-

tion decomposition are applied to the initial calculation

of the wavelet coe� cients. Here two important relationsare introduced. Firstly, because the family fU kg spans

V0, then f U …2x ¡ k†g spans the next ® lter scale

V1 ˆ V0 © W0. Both the scaling function and the

wavelet function can be expressed in terms of the scaling

function at the resolution 2j ˆ 21, i.e.

U …x† ˆ���2

p X

k2Z

ck U …2x ¡ k† …52†

W i…x† ˆ���2

p X

k2Z

d…i†k U …2x ¡ k† …53†

where ck and d…i†k are known as the two scale reconstruc-

tion sequences. Secondly, any scaling function U …2x† in

V1 can alternatively be written using the scaling function

U …x† in V0 and wavelet function W …x† in W0 as

U …2x ¡ l† ˆX

k2Z

al¡2k U …x ¡ k†…

‡X2n¡1

iˆ1

b…i†l¡2k

W i…x ¡ k†!

…54†

where ak and b…i†k are known as the decomposition

sequences, and l 2 Z n.In addition, in terms of multiresolution decomposi-

tions, the approximation of the function f…x; u† at the

resolution 2j can be written as

f̂…j†…x; u† ˆX

k2Aj¡1

A…j¡1†k U …j¡1†k…x; u†

‡X

k2Bj¡1

X2n¡1

iˆ1

B…i†…j¡1†k W

…i†…j¡1†k…x; u† …55†

where

X

k2Aj¡1

A…j¡1†k U …j¡1†k…x; u†

ˆX

k2Aj¡2

A…j¡2†k U …j¡2†k…x; u†

‡X

k2Bj¡2

X2n¡1

iˆ1

B…i†…j¡2†k W

…i†…j¡2†k…x; u† …56†

Hence, if the state error ex 62 H …"U…t††, the network

needs more wavelets. Add the wavelets at the resolution

2N‡1 into the network. Following the adding operation

(42) and the expression (55) of f̂…j†…x; u† at j ˆ N, the

structure of the approximated function f̂…x; u† is of theform:

f̂…x; u† ˆX

k2AN

ANk U Nk…x; u† ‡X

k2BN

X2n¡1

iˆ1

B…i†Nk W

…i†Nk…x; u†

…57†

The parameter vectors ANk and B…i†Nk

are adapted by the

laws (38) and (39). Using the sequences ck and d…i†k , the

initial values after the adding operation are then given

by the reconstruction algorithm (Mallat 1989b):

ANk…0† ˆX

l2AN

cl¡2kA…N¡1†l ‡X2n¡1

iˆ1

d…i†l¡2kB

…i†…N¡1†l

Á !…58†

B…i†Nk…0† ˆ 0 …59†

where A…N¡1†k and B…i†…N¡1†k are the estimated values

before the adding operation.

If the state error ex 2 H …"L…t††, some wavelets need tobe removed because the network may be over® tted. In

this case, remove the wavelets associated with the reso-

lution 2N . In terms of the removing operation (43) and

the expression (55) of f̂…j†…x; u† at j ˆ N, the structure of

the approximated function f̂…x; u† is of the followingform:

f̂…x; u† ˆX

k2AN¡2

A…N¡2†k U …N¡2†k…x; u†

‡X

k2BN¡2

X2n¡1

iˆ1

B…i†…N¡2†k W

…i†…N¡2†k…x; u† …60†

The adaptive laws for the parameters A…N¡2†k andB

…i†…N¡2†k are still given by equations (38) and (39). But,

using the sequences ak and b…i†k , the initial values after the

removing operation are then changed by the decomposi-

tion algorithm (Mallat, 1989b) as follows:

A…N¡2†k…0† ˆX

l2AN¡2

al¡2kANl …61†

B…i†…N¡2†k…0† ˆ

X

l2BN¡2

b…i†l¡2kANl …62†

Nonlinear system identi ® cation using wavelet networks 1537

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

where ANk and B…i†Nk

are the estimated values before the

removing operation.

Clearly, in both the above cases, the adaptive laws of

the parameters are still given in the form of equations

(38) and (39), based on the above changed parameters.It also follows that the convergence area of the state

error vector begins with H …"U…0†† ¡ H …"L…0†† and ends

with H …"†, where " ˆ "U…1†.The determination of the vector sets Aj and Bj, for

j ˆ j0; . . . ; N is also important but simple. The basic rule

for choosing these sets is to make sure 2jx ¡ k, fork 2 Aj or Bj, is not out of the valid range of the vari-

ables of the scaling function U …:† or W i…:†, respectively.

The basis function of the variable wavelet network

can be constructed by tensor products of scalar wavelet

functions, but other constructions are also possible.Because the wavelet network has multi-resolution cap-

abilities, it can easily achieve the diŒerent modelling

accuracy. However, the burden in computation and sto-

rage of wavelet basis functions rapidly increases with the

dimension n, so the use of the wavelet network is inpractice limited to the case where the dimension n µ 3.

5. Simulation

Consider a nonlinear system described by

_x ˆ …u ¡ xu ¡ x†e¡0:5…x2‡u2† …63†

where the input

u ˆ 0:5…cos…1:2t† sin…1:7t†‡ exp…¡ sin…t4†††.

Because n ˆ 2, we will need two-dimensional B-spline

wavelets for the wavelet network to identify this non-

linear dynamical system. The fourth order B-splines

were used as the scaling function. Thus, the two-dimen-

sional scaling function is given by

U …x; u† ˆ B4…x†B4…u† …64†

where B4…:† is the fourth-order B-spline, which is apiecewise cubic function (Chui 1992, Wang et al.

1995). For n ˆ 2, there are three two-dimensional

mother wavelets expressed by

W 1…x; u† ˆ B4…x†Á…u† …65†

W 2…x; u† ˆ Á…x†B4…u† …66†

W 3…x; u† ˆ Á…x†Á…u† …67†

where the one-dimensional mother wavelet Á…x† is

Á…x† ˆX10

kˆ0

X4

lˆ0

…¡1†k

8

4

l

³ ´B8…k ‡ 1 ¡ l†B4…2x ¡ k†

…68†

For this identi® cation, the wavelet network began with

16 wavelons at the resolution 20 and then with 81 wave-

lons at the resolution 21 using variable wavelet network

strategy. The state errors and the modelling errors forthe two diŒerent resolutions are shown in ® gures (1)± (4).

The ® gures denoted as (b) are larger scale versions of the

® gures denoted as (a).

As expected, at the beginning of the identi® cation

larger state errors and modelling errors exist. After a

1538 G. P. Liu et al.

Figure 1. The state error x(t) - x̂(t) at the resolution 20.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

while, these errors become smaller and smaller, and

® nally they converge to certain ranges. It is clear from

the simulation results that the whole identi® cation

scheme is stable from the beginning to the end. It has

also been shown that the state error and the modelling

error decrease with an increase in the resolution of the

wavelet networks by the use of the variable network

approach. Clearly, for nonlinear dynamical system iden-

ti® cation using wavelet networks, a proper resolution

can be obtained through the proposed variable network

approach to achieve the desired practical identi® cation

requirements.

Nonlinear system identi ® cation using wavelet networks 1539

Figure 2. The modelling error f (x; u) - f̂ (x; u) at the resolution 20.

Figure 3. The state error x(t) - x̂(t) at the resolution 21.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

6. Conclusions

A wavelet network based identi® cation scheme has been

proposed for nonlinear dynamical systems. Two kinds

of wavelet networks, ® xed and variable wavelet net-

works, were studied and parameter adaptation laws

were derived which achieve the required estimation

accuracy for a suitable sized network. The parameters

of the wavelet network were adjusted using laws devel-oped by the Lyapunov synthesis approach. By com-

bining wavelet networks with Lyapunov synthesis

techniques, adaptive parameter laws were developed

which guarantee the stability of the whole identi® cationscheme and the convergence of both the network par-

ameters and the state errors. A simulated example was

used to demonstrate the operation of the identi® cation

scheme which was realized using B-spline wavelets.

Acknowledgements

The authors gratefully acknowledge the support of the

Engineering and Physical Sciences Research Council

(EPSRC) of U.K. under the contract GR/J46661.

ReferencesBILLINGS, S. A., 1980, Identi® cation of nonlinear systemsÐ a survey,

IEE Proceedings, Part D, 127, 272± 285.

BILLINGS, S. A., and VOON, W. S. F ., 1986, A prediction-error and

stepwise-regression estimation algorithm for non-linear systems.

Internationa l Journal of Control, 44, 803± 822.

BERIMAN, L., 1993, Hinging hyperplanes for regression, classi® cation

and function approximation. IEEE Trans. Inf. Theory, 39, 999± 1013.

BROWN, M ., and HARRIS, C. J., 1994, Neurofuzzy Adaptive Modelling

and Control (New York: Prentice-Hall).

CHEN,S.,BILLINGS,S.A., and G RANT,P.M ., 1990, Nonlinear system

identi® cation using neural networks. Internationa l Journal of

Control, 51, 1191± 1214.

CHEN,S., and WIGGER,J., 1995, A fast orthogonal least squares algor-

ithm for e� cient subset model selection. IEEE Trans. on Signal

Processing, 43, 1713± 1715.

CHUI, C. K., 1992, An Introduction to Wavelets (San Diego: Academic

Press).

COCA, D., and BILLINGS, S. A., 1996, Continuous-time modelling of

Chua’s circuit from noisy measurements using wavelet decomposi-

tions. In The Fourth Internationa l Workshop on Nonlinear Dynamics

of electronic Systems, Seville, Spain, pp. 271± 276.

DAUBECHIES, I., 1988, Orthonormal bases of compactly supported

wavelets. Commun. on Pure and Appl. Mathematics, 16, 909± 996.

GOODWIN, G. C., and MAYNE, D . Q., 1987, A parameter estimation

perspective of continuous time model reference adaptive control.

Automatica, 23, 57± 70.

GROSSMANNAND, A., and MORLET, J., 1984, Decomposition of Hardy

functions into square integrable wavelets of constant shape. SIAM

J. Math. Anal., 15, 723± 736.

HABER, H., and UNBEHAUEN, H., 1990, Structure identi® cation of

nonlinear dynamic systemsÐ A survey on input/output approaches.

Automatica, 26, 651± 677.

IEEE, 1996, Scanning the special issue on wavelets. In The Proceedings

of the IEEE, 84, 4.

IOANNOU, P. A., and K OKOTOVIC, P. V., 1983, Adaptive Systems with

Reduced Models (New York, NY: Springer).

KADIRKAMANATHAN ,V., and N IRANJAN,M., 1993, A function estima-

tion approach to sequential learning with neural networks. Neural

Computation, 5, 954± 957.

KORENBERG, M ., BILLINGS, S. A., LIU, Y. P., and MCLLROY, P. J.,1988, Orthogonal parameter estimation algorithm for non-linear

stochastic systems. International Journal of Control, 48, 193± 210.

LEONTARITIS, I. J., and BILLINGS, S. A., 1987, Model selection and

validation methods for non-linear systems. International Journal of

Control, 45, 311± 341.

LIU, G. P., BILLINGS, S. A., and KADIRKAMANATHAN V., 1998,

Nonlinear system identi® cation using wavelet networks. In

Proceedings of Control ’98, Swansea, pp. 1248± 1253.

1540 G. P. Liu et al.

Figure 4. The modelling error f (x; u) - f̂ (x; u) at the resolution 21.

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5

LIU, G . P., BILLINGS, S. A., and KADIRKAMANATHAN V., 1999a,

Nonlinear system identi® cation via variable wavelet networks. In

Proceedings of the 14th IFAC World Congress, Beijing, Vol. H, pp.

109± 114.

LIU, G . P., KADIRKAMANATHAN V., and BILLINGS, S. A., 1996a,

Stable sequential identi® cation of continuous nonlinear dynamical

systems by growing RBF networks. Int. J. Control, 65, 53± 69.

LIU, G. P., K ADIRKAMANATHAN V., and BILLINGS, S. A., 1996b,

Variable neural networks for nonlinear adaptive control. preprints

of the 13th IFAC Congress, USA, Vol. F, pp. 181± 184.

LIU, G. P., K ADIRKAMANATHAN V., and BILLINGS, S. A., 1999b,

Variable Neural Networks for Adaptive Control of Nonlinear

Systems. IEEE Trans. on Systems, Man, and Cybernetics, 29, 34± 43.

LJUNG, L., and GLAD, T., 1994, Modelling of Dynamic Systems,

Information and System Sciences Series (Englewood CliŒs, NJ:

Prentice Hall).

MALLAT,S.G., 1989a, A theory for multiresolution signal decomposi-

tion: the wavelet representation. IEEE Trans. on Pattern Analysis

and Machine Intelligence, 11, 7, 674± 693.

MALLAT, S. G., 1989b, Multifrequency channel decompositions of

images and wavelet models. IEEE Trans. Accoust., Speech, and

Signal Process., 37, 2091± 2110.

MEYER, Y., 1993, Wavelets and Operators (Cambridge: Cambridge

University Press).

NARENDRA,K .S., and ANNASWAMY,A.M ., 1987, A new adaptive law

for robust adaptation with persistent excitation. IEEE Trans. Aut.

Control, 32, 134± 145.

NARENDRA, K. S., and PARTHASARATHY, K., 1990, Identi® cation and

control of dynamical systems using neural networks. IEEE Trans. on

Neural Networks, 1, 4± 27.

POGGIO, T., and G IROSI, F ., 1990, Networks for approximation and

learning. In Proceedings of the IEEE, 78, 1481± 1497.

SASTRY, S., and BODSON, M ., 1989, Adaptive Control: Stability,

Convergence and Robustness (Engelwood CliŒs, NJ: Prentice-Hall).

SJOBERG, J., ZHANG, Q., LJUNG, L., BENVENISTE, A., D ELYON, B.,GLORENNEC, P., H JALMARSSON, H., and JUDITSKY, A., 1995,

Nonlinear black-box modelling in system identi® cation: a uni® ed

overview. Automatica, 31, 1691± 1724.

WANG, H., LIU, G . P., HARRIS, C. J., and BROWN, M., 1995,

Advanced Adaptive Control (Oxford: Pergamon Press).

ZADEH, L., 1994, Fuzzy logic, neural networks, and soft computing.

Commun. ACM, 37, 77± 86.

ZHANG, Q., and BENVENISTE, A., 1992, Wavelet Networks. IEEE

Trans. on Neural Networks, 3, 889± 898.

Nonlinear system identi ® cation using wavelet networks 1541

Dow

nloa

ded

by [

Bir

la I

nstit

ute

of T

echn

olog

y an

d Sc

ienc

e] a

t 12:

18 2

3 M

arch

201

5