Automatica 43 (2007) 346–354www.elsevier.com/locate/automatica

Brief paper

Iterative identification of Hammerstein systems�

Yun Liua, Er-Wei Baib,∗aServo Tech Inc., Chicago, IL 60608, USA

bDepartment of Electrical and Computer Engineering, the University of Iowa, Iowa City, IA 52242, USA

Received 5 January 2006; received in revised form 10 May 2006; accepted 2 September 2006

Abstract

Convergences of iterative algorithms have been established for identification of Hammerstein systems in the case that the unknownnonlinearities are odd. Then, the results are further extended to nonsmooth nonlinearities.� 2006 Elsevier Ltd. All rights reserved.

Keywords: Hammerstein system; Iterative identification; Parameter estimation

1. Introduction

The nonlinear system consisting of a static nonlinear blockfollowed by a linear dynamic system is called a Hammersteinsystem. The Hammerstein system is a useful block-orientedmodel in many engineering practices and its identification hasbeen extensively studied (Cerone & Regruto, 2003; Chen, 2004;Greblicki, 2000; Narendra & Gallman, 1966; Stoica, 1981;Voros, 1997, 2003).

The iterative method was first proposed to estimate Ham-merstein system in Narendra and Gallman (1966). It is a verysimple and efficient algorithm. In general, the convergence ofiterative algorithm can be a problem (Stoica, 1981). It was re-cently shown that iterative algorithms with normalization pos-sess some global convergence properties in identification ofHammerstein system with smooth nonlinearity and finite im-pulse response (FIR) linear part (Bai & Li, 2004; Bai & Liu,2006; Wolodkin, Rangan, & Poolla, 1995). The convergence

� This work was not presented in any IFAC meeting. This paper wasrecommended for publication in revised form by Associate Editor AntonioVicino under the direction of Editor Torsten Söderström. The work wassupported in part by NSF ECS-0098181, ECS-0555394 and NIH/NIBIBEB004287.

∗ Corresponding author. Tel.: +1 319 335 5949; fax: +1 319 335 6028.E-mail addresses: liu_yun@servotechinc.com (Y. Liu),

erwei@engineering.uiowa.edu (E.-W. Bai).

0005-1098/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.automatica.2006.09.004

for an infinite impulse response (IIR) system was however notsolved. In this paper, the results are extended to Hammersteinsystems with an IIR linear part. The global convergence for anIIR system is established for the odd nonlinearities.

The second contribution of the paper is to extend iterative al-gorithm results to Hammerstein systems with nonsmooth non-linearities and FIR linear block (Bai, 2002). One way to dealwith such nonlinearities is to use nonparametric approach. Aproblem with nonparametric approach is the slow convergence(Chen, 2004; Greblicki, 2000), if it converges. To this end, iter-ative algorithms were proposed and developed to identify Ham-merstein systems with piecewise nonlinearity (Voros, 1997,2003). But the convergence property of the algorithms is un-known. This paper presents a normalized iterative identificationalgorithm for two common nonsmooth piecewise-linear non-linear structures, i.e., nonlinearities with saturation and preloadcharacteristics. It is shown that the algorithms converge in oneiteration step when the number of sample data is large.

The paper is organized as follows. In Section 2 the Hammer-stein model with IIR nonlinearity is described. Detailed formu-lations of the nonlinear and linear parts are given. The criterionof estimation is introduced and iterative algorithm for thistype of Hammerstein systems is proposed. The convergenceproperty of the algorithm is also established in this section. InSection 3, the iterative method are applied to two more typesof Hammerstein systems with nonsmooth nonlinear part. Theirconvergence properties are also provided. Section 4 provides

Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354 347

examples to show the effectiveness of the algorithms for eachtype of system. Finally, conclusions are found in Section 5.

2. Hammerstein system with IIR linear part

We study the scalar system in the block diagram Fig. 1. Theinput signal is ut , xt is the internal signal, vt and yt are the noiseand output, respectively. In this paper, the noise vt is assumedto be i.i.d. random sequences with zero mean, Ev2

t = �2v < ∞,

and E|vt |4+� < ∞ for some � > 0, where E denotes expectationof a random variable.

The Hammerstein system with an infinite impulse response(IIR) linear subsystem has the following representation.

The linear part is an unknown stable system which can bedescribed by

yt = d1yt−1 + d2yt−2 + · · · + dnyt−n

+ b1xt−1 + b2xt−2 + · · · + bmxt−m + vt . (1)

The nonlinear part is an unknown odd symmetric function thatcan be expressed as

xt = f (ut ) = a1g1(ut ) + a2g2(ut ) + · · · + algl(ut ). (2)

Here gi(u), i = 1, . . . , l are known odd functions of u.We can write the Hammerstein system into the following

matrix form:

yt =n∑

i=1

diyt−i +m∑

j=1

bj

[l∑

i=1

aigi(ut−j )

]+ vt

= dT�t (y) + bTGt(u)a + vt , (3)

where

�t (y) = (yt−1, yt−2, . . . , yt−n)T,

Gt(u) =⎛⎝ g1(ut−1) · · · gl(ut−1)

.... . .

...

g1(ut−m) · · · gl(ut−m)

⎞⎠ .

The unknown parameters needed to be estimated are: the non-linear part parameters

a = (a1, a2, . . . , al)T,

and the linear subsystem parameters

d = (d1, d2, . . . , dn)T, b = (b1, b2, . . . , bm)T.

The purpose of identification is to estimate the unknownparameters based on the observed input and output data{ut , yt }, t = 1, 2, . . . , N , for large enough N.

ut xt yt+

+

t

B(q-1)

D(q-1)f (·)

1

D(q-1)

Fig. 1. Hammerstein system.

For Hammerstein systems with IIR linear part (3), the es-timates a, d, b are sought to minimize the least squares costfunction,

{a, d, b} = argmina,d,b

JN(a, d, b)

= argmina,d,b

1

N

N∑t=1

(yt − yt )2

= argmina,d,b

1

N

N∑t=1

(dT�t (y) + bTGt(u)a − yt )2. (4)

Note that the model (3) is not well defined for identificationpurpose. Any pair �a and b/� for some nonzero constant �provides the same input–output data. To have identifiability, weadopt the normalization constraint on a for model (3). Assumethat ‖a‖ = 1, and the first nonzero entry of a is positive.

We propose the following normalized iterative algorithms toestimate a and d, b for system (3).

Given the initial estimate b0=(1, 0, . . . , 0)T, and an arbitraryn-dimension vector d0,

ak = argmina

JN(a, dk−1, bk−1);

Normalize ak to have positive first nonzero entry;

{dk, bk} = argmind,b

JN(ak, d, b);

Replace k by k + 1 and the process is repeated. (5)

2.1. Convergence analysis

To analyze the performance of the algorithm (5), we writethe cost function JN(a, d, b) in (4) as follows:

JN(a, d, b) = 1

N

N∑t=1

[(d − d)T�t (y)

+l∑

i=1

(gi(ut−1), · · · , gi(ut−m))

× (ai b − aib) + vt ]2. (6)

Let N → ∞, and define

C = limN→∞

1

N

N∑t=1

⎛⎝yt−1

...

yt−n

⎞⎠ (yt−1, . . . , yt−n),

A(i) = limN→∞

1

N

N∑t=1

⎛⎝yt−1

...

yt−n

⎞⎠ (gi(ut−1), . . . , gi(ut−m)),

where i = 1, . . . , l,

B(i, j) = limN→∞

1

N

N∑t=1

⎛⎝ gi(ut−1)

...

gi(ut−m)

⎞⎠

· (gj (ut−1), . . . , gj (ut−m)),

348 Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354

where i, j = 1, 2, . . . , l. When the noise vt is i.i.d. random se-quence with zero mean and finite variance �2

v , the cost function(6) becomes

J (a, d, b) = limN→∞ JN(a, d, b)

= (d−d)TC(d−d)+l∑

i=1

(d−d)TA(i)(ai b−aib)

+l∑

i=1

l∑j=1

(ai b−aib)TB(i, j)(aj b−aj b)+�2v . (7)

If the input signal is i.i.d. with symmetric distribution, the fol-lowing convergence result can be derived for the IIR Hammer-stein system (3).

Theorem 2.1. Consider the Hammerstein system (3) and thecost function (7). Suppose the first element of b is nonzero, i.e.,b1 �= 0, and the input nonlinearity is odd, i.e., −gi(u)=gi(−u),i=1, . . . , l. Further, assume that the input ut is i.i.d. which hasa symmetric probability density function with positive supportsat no less than l points, say at r1, . . . , rl , so that gi’s are linearindependent with respect to these l points, i.e.,

rank

⎛⎝g1(r1) . . . g1(rl)

.... . .

...

gl(r1) . . . gl(rl)

⎞⎠ = l.

Then, the normalized iterative algorithm (5) converges to thetrue parameters a, d, and b in one iteration step provided N →∞.

Proof. Note the output yt is independent with all inputsafter ut . Given ut i.i.d. with symmetric distribution andg1(u), . . . , gl(u) are odd functions of ut , we have the follow-ing:

A(i) =

⎛⎜⎜⎝

0 E(ytgi(ut−1)) · · · E(ytgi(ut−m))

0 0 · · · E(ytgi(ut−m−1))...

.... . .

...

0 0 · · · 0

⎞⎟⎟⎠ ,

where i = 1, 2, . . . , l. Also, we have

B(i, j) = �ij I, �ij = Egi(u)gj (u),

where i, j = 1, 2, . . . , l.The cost function (7) can now be reduced to

J (a, d, b) = (d−d)TC(d−d)+l∑

i=1

(d−d)TA(i)(ai b − aib)

+l∑

i=1

l∑j=1

(ai b − aib)T�ij (aj b − aj b) + �2v .

Given the initial estimate b0 = (1, 0, . . . , 0)T, d0, at the firstiteration, a solution a must satisfy

0 = �J

�ai

for i = 1, 2, . . . , l.

It gives

0 = (d0 − d)TA(i)b0 +l∑

j=1

(2aj�ij bT0 b0 − 2aj�ij bT

0 b).

Because the first column of matrix A(i) are all zeros, and b0 =(1, 0, . . . , 0)T, we get,

0 =l∑

j=1

(2aj�ij bT0 b0 − 2aj�ij bT

0 b).

This gives

bT0 b0

⎛⎝�11 · · · �1l

.... . .

...

�l1 · · · �ll

⎞⎠ , a = bT

0 b

⎛⎝�11 · · · �1l

.... . .

...

�l1 · · · �ll

⎞⎠ a.

Note gi(u)(∑l

j=1�j gj (u)) is an even function. Under the as-sumption, there does not exist �1, �2, . . . , �l ∈ R which are notall zeros, such that

l∑j=1

�j Egi(u)gj (u) = Egi(u)

⎛⎝ l∑

j=1

�j gj (u)

⎞⎠ = 0,

i = 1, 2, . . . , l. Then the matrix⎛⎝�11 · · · �1l

.... . .

...

�l1 · · · �ll

⎞⎠

is nonsingular. Therefore,

a = a(bT0 b)/(bT

0 b0) = ab1.

Under the assumption that b1 �= 0, we get

a = a

after normalization. Thus a converges to the true value a in thefirst iteration step.

Now we can use the estimate of a to construct the outputxt , t = 1, . . . , N of nonlinear part of the system (3), and thesystem output yt can be expressed as

yt = �t

(db

)+ vt ,

where

�t = (yt−1, . . . , yt−n, xt−1, . . . , xt−m).

This is a linear problem with i.i.d. zero mean noise. When N →∞, the least squares solutions d = d and b = b. This completesthe proof. �

3. Nonsmooth nonlinearities

In this section, we apply the iterative algorithm to twotypes of nonsmooth Hammerstein system with finite impulseresponse (FIR) linear subsystem.

Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354 349

-c

-a2

a2 = a1c

f (u)

u

a1u....................................................................

............

............

............

............

............

......

c

Fig. 2. Nonlinear part with saturation.

ut

a1u - a2

a1u+a2

-c

.........

.........

.........

.........

.........

.....

...................................................

f (u)

c

Fig. 3. Nonlinear part with preload.

The linear part has the following representation:

yt = d1xt−1 + d2xt−2 + · · · + dnxt−n. (8)

The saturation and preload nonlinearities that are encounteredoften in practical applications are considered in this paper asshown in Figs. 2 and 3 .

Let us express the saturation nonlinear part in Fig. 2 as

xt = f (ut ) ={

a2, ut > c,

a1u, −c�ut �c,

−a2, ut < − c.

(9)

Define a switching function h(u) as follows:

h(u) ={

1, u > 0,

0, u�0.

The signal xt now can be expressed in an additive form:

xt = a1g1(u, c) + a2g2(u, c), (10)

where

g1(u, c) = uh(|c| − |u|), g2(u, c) = h(u − c) − h(−c − u).

Note that both g1(u, c) and g2(u, c) are odd functions of u andhere xt is continuous piecewise-linear function of ut .

Similarly the preload nonlinear part in Fig. 3 can be ex-pressed by

xt = a1g1(u, c) + a2g2(u, c), (11)

where

g1(u, c) = u[h(u − c) + h(−c − u)],g2(u, c) = h(u − c) − h(−c − u).

and both g1(·) and g2(·) are odd functions. xt is a discontinuousfunction of ut here.

Now we can write both types of Hammerstein systems intothe following compact form:

yt =n∑

j=1

dj

[2∑

i=1

aigi(ut−j , c)

]+ vt

= dTG(ut , c)a + vt , (12)

where

G(ut , c) =⎛⎝g1(ut−1, c) g2(ut−1, c)

......

g1(ut−n, c) g2(ut−n, c)

⎞⎠ .

The unknown parameters needed to be estimated are

a = (a1, a2)T, d = (d1, d2, . . . , dn)

T,

and the discontinuity point c in the Hammerstein system withpreload nonlinear part.

To identify the Hammerstein systems with nonsmooth non-linearities (12), the estimates a, d, c are sought to minimize theleast squares cost function

{a, d, c} = argmina,d,c

JN(a, d, c)

= argmina,d,c

1

N

N∑t=1

(yt − yt )2

= argmina,d,c

1

N

N∑t=1

(yt − dTG(ut , c)a)2. (13)

Without loss of generality, we adopt the normalization con-straint on d for model (12). Assume that ‖d‖ = 1, and the firstnonzero entry of d is positive.

The following normalized iterative algorithm is proposedto estimate a and d in Hammerstein system with saturationnonlinearity in Fig. 2.

Given the initial estimate a0 �= 0 and an estimate ofan upper bound of the threshold parameter c1 > c,

dk = argmind

JN(ak−1, d, c1);Normalize dk to have positive first nonzero entry;ak = argmin

aJN(a, dk, c1);

{a2}k = the second element of ak.

Given an estimate of an lower bound of the thresholdparameter c2 < c,

ak = argmina

JN(ak−1, dk, c2);{a1}k = the first element of ak,

Replace k by k + 1 and the process is repeated.

(14)

350 Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354

Similarly, we propose the following normalized iterative algo-rithms to estimate a and d in the Hammerstein system withpreload nonlinearity in Fig. 3.

Given the initial estimate a0 �= 0 and an estimate of anupper bound of the threshold parameter c > c, definedk = argmin

dJN(ak−1, d, c);

Normalize dk to have positive first nonzero entry;ak = argmin

aJN(ak−1, dk, c);

Replace k by k + 1 and the process is repeated.

(15)

In fact, the algorithm can also be started with d0: Given theinitial estimate d0 �= 0, define

ak = argmina

JN(a, dk−1, c);dk = argmin

dJN(ak, d, c);

Normalize dk to have positive first nonzero entry;update ak accordingly with the normalization factor;Replace k by k + 1 and the process is repeated.

(16)

To analyze the convergence properties of the above algorithms(14)–(16), we write the cost function JN(a, d, c) in (13) asfollows:

JN(a, d, c) = 1

N

N∑t=1

[n∑

i=1

di(a1, a2)

(g1(ut−i , c)

g2(ut−i , c)

)

−di (a1, a2)

(g1(ut−i , c)

g2(ut−i , c)

)+ vt

]2

.

We are interested in the behavior of the algorithm when N islarge enough. Let N → ∞, and define

A(i, j) = E[(

g1(ut−i , c)

g2(ut−i , c)

)

·(g1(ut−j , c), g2(ut−j , c))

],

A(i, j) = E[(

g1(ut−i , c)

g2(ut−i , c)

)

·(g1(ut−j , c), g2(ut−j , c))

],

A1(i, j) = E[(

g1(ut−i , c)

g2(ut−i , c)

)

·(g1(ut−j , c), g2(ut−j , c))

],

A2(i, j) = AT1 (j, i) = E

[(g1(ut−i , c)

g2(ut−i , c)

)

·(g1(ut−j , c), g2(ut−j , c))

], (17)

where i, j = 1, 2, . . . , n. When the noise vt is i.i.d. random se-quence with zero mean and finite variance �2

v , the cost function

will become

J (a, d, c) = limN→∞ JN(a, d, c)

=n∑

i=1

n∑j=1

[diaTA(i, j)adj + di aTA(i, j)adj

−diaTA1(i, j)aTdj−di aTA2(i, j)adj ]+�2v . (18)

Similarly, define

B(i, j) = E

⎡⎢⎢⎣

⎛⎜⎜⎝

gi(ut−1, c)

gi(ut−2, c)...

gi(ut−n, c)

⎞⎟⎟⎠

·(gj (ut−1, c), gj (ut−2, c), . . . , gj (ut−n, c))

⎤⎥⎥⎦ ,

B(i, j) = E

⎡⎢⎢⎣

⎛⎜⎜⎝

gi(ut−1, c)

gi(ut−2, c)...

gi(ut−n, c)

⎞⎟⎟⎠

·(gj (ut−1, c), gj (ut−2, c), · · · gj (ut−n, c))

⎤⎥⎥⎦ ,

B1(i, j) = E

⎡⎢⎢⎣

⎛⎜⎜⎝

gi(ut−1, c)

gi(ut−2, c)...

gi(ut−n, c)

⎞⎟⎟⎠

·(gj (ut−1, c), gj (ut−2, c), . . . , gj (ut−n, c))

⎤⎥⎥⎦ ,

B2(i, j) = BT1 (j, i), (19)

where i, j = 1, 2. The cost function in (18) can be written inthe following equivalent form:

J (a, d, c) = limN→∞ JN(a, d, c)

=2∑

i=1

2∑j=1

[aidTB(i, j)daj + ai dTB(i, j)daj

−aidTB1(i, j)daj−ai dTB2(i, j)daj ]+�2v . (20)

The convergence property of algorithm (14) for Hammersteinsystem with saturation nonlinear part is shown in the followingtheorem.

Theorem 3.1. Consider the Hammerstein system with satura-tion nonlinearity (12). Assume an upper bound c = c1 > c, andlower bound c = c2 < c for the threshold value c are available.In addition, assume the input ut is i.i.d. with symmetric dis-tribution that have positive support at points both larger and

Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354 351

smaller than c1, c2 such that Eg2i (u, cj ) > 0, i, j = 1, 2, and

the initial estimate dT0 d �= 0. Then the normalized iterative al-

gorithm (14) converges to the true parameters a and d in onestep provided that N → ∞.

Proof. Under the given assumptions, we have

A(i, j) = 0, A(i, j) = 0, A1(i, j) = 0, A2(i, j) = 0

when i �= j , and

A(i, i) = A(j, j), A(i, i) = A(j, j),

A1(i, i) = A1(j, j), A2(i, i) = A2(j, j),

where i, j = 1, 2, . . . , n. The cost function (18) can now bereduced to

J (a, d, c) =n∑

i=1

[d2i aTA(i, i)a + d2

i aTA(i, i)a

− di diaTA1(i, i)a − di di aTA2(i, i)a] + �2v .

Given the estimate a, at each iteration, the solution dk satisfy

0 = �J

�di

⇒ 2di aTA(i, i)a − di(aTA1(i, i)a + aTA2(i, i)a) = 0

⇒ di = di

aTA1(i, i)a + aTA2(i, i)a

2aTA(i, i)a⇒ d = d · � ⇒ d = d after normalization,

where i = 1, 2, . . . , n.When the estimate c is smaller than the true parameter c, the

matrices in the cost function (20) become

B(i, j) ={

�iiI, i = j,

0, i �= j,

where �ii = Egi(u, c)gi(u, c), I is the identity matrix

B(i, j) ={

�iiI, i = j,

0, i �= j,

where �ii = Egi(u, c)gi(u, c), and

B1(j, i) = BT1 (j, i) = B2(i, j) =

⎧⎪⎨⎪⎩

�11I, i = j = 1,

0, i = 1, j = 2,

�21I, i = 2, j = 1,

�22I, i = j = 2,

where �21 = Eg2(u, c)g1(u, c), �22 = Eg2(u, c)g2(u, c), i, j =1, 2.

Given the estimate d, the solution a satisfies

0 = �J

�ai

⇒2∑

j=1

[2aj �ij‖d‖2 − 2aj �ij dTd] = 0

⇒ ‖d‖2(

�11 00 �22

)a = dTd

(�11 0�21 �22

)a

⇒ a1 = a1 · dTd/‖d‖2.

When the estimate c is larger than the true parameter c, thematrices B(i, j) and B(i, j) are the same as in the case c < c,and now

B1(j, i) = B2(i, j) =

⎧⎪⎨⎪⎩

�11I, i = j = 1,

�12I, i = 1, j = 2,

0, i = 2, j = 1,

�22I, i = j = 2,

where

�12 = Eg1(u, c)g2(u, c), �11 = Eg1(u, c)g1(u, c).

Given the estimate d, the solution a satisfies

0 = �J

�ai

⇒2∑

j=1

[2aj �ij‖d‖2 − 2aj �ij dTd] = 0

⇒ ‖d‖2(

�11 00 �22

)a = dTd

(�11 �12

0 �22

)a

⇒ a2 = a2 · dTd/‖d‖2.

So we get a = a after normalization. Thus if we know an up-per bound and lower bound of the parameter c, we can get aconvergent estimate of a in one iterative step.

Because the saturation nonlinear part xt = f (ut ) in Fig. 2 iscontinuous, we can easily get an estimate of c = a2/a1. c → c

as N → ∞ is a consequence of the convergence of the estimate(a1, a2).

The similar convergence results can be established for thepre-load nonlinearity. However, the proof is tedious. For sim-plicity, we only present the result for a uniform distributioninput.

Theorem 3.2. Consider the Hammerstein system withpreload nonlinearity (12), the cost function (18) or (20) withA(i, i) > 0, i = 1, . . . , n. Assume the initial estimate dT

0 d �= 0and the input ut is i.i.d. uniformly distributed on [−U, U ],where U �c is a known upper bound of c. Then, the normal-ized iterative algorithm (15) or (16) converges to the trueparameters a and d in one step provided that N → ∞.

Proof. The convergence of the linear subsystem parametersd is exactly the same as in the proof of Theorem 3.1. Wewill prove the convergence of the parameters of the nonlinearsubsystem (a1, a2) under different estimates of c.

Given ut i.i.d. with symmetric distribution and g1(u, c),

g2(u, c) are odd functions of ut , we have

B(i, j) = �ij I ,

where �ij = Egi(u, c)gj (u, c), and

B(i, j) = �ij I ,

352 Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354

where

�ij = Egi(u, c)gj (u, c)

and

B1(j, i) = B2(i, j) = �ij I ,

where

�ij = Egi(u, c)gj (u, c) = Egj (u, c)gi(u, c) = �ij

when c�c.

Here i, j = 1, 2. The cost function (20) can be reduced to

J (a, d, c) =2∑

i=1

2∑j=1

[‖d‖2�ij ai aj + ‖d‖2�ij aiaj

− dTd�ji aiaj − dTd�ij ai aj ] + �2v .

Given the estimate d, at each iteration, the solution ak satisfies

0 = �J

�ai

⇒2∑

j=1

[2aj �ij‖d‖2 − 2aj �ij dTd] = 0

⇒ ‖d‖2(

�11 �12

�21 �22

)a = dTd

(�11 �12

�21 �22

)a

⇒ a = a after normalization.

because the matrix

(�11

�21

�12

�22

)is nonsingular under given as-

sumptions on ut .Thus a and d converge to true values in one iteration step as

N → ∞ under the Theorem 3.2 assumptions.Next we need to obtain an estimate for the discontinuity point

c in the Hammerstein system with preload. When the inputut be i.i.d. uniformly distributed in [−U, U ], the probabilitydensity function of ut is q(u)=1/2U . Consider the correlationbetween yt and ut−1. We have

E(ytut−1) = a1d1E(ug1(u)) + a2d1E(ug2(u))

= a1d1U3 − c3

3U+ a2d1

U2 − c2

2U

⇒ − a1d1

3Uc3 − a2d1

2Uc2 + a1d1U

2

3

+ a2d1U

2− E(ytut−1) = 0. (21)

We can solve last Eq. (21) above to get an estimate c. Thecorrelation can be estimated by the sample mean,

ytut−1 = 1

N − 1

N∑t=2

ytut−1.

The parameters a1, a2, d1 in (21) are replaced by the estimatesfrom the iterative algorithm. It can be proved that c → c asN → ∞. Thus all the parameters in the systems with preloadnonlinear part are estimated. �

4. Examples

The first example shows the convergence of the iterativealgorithm (5). The true parameters in the Hammerstein systemwith IIR nonlinear part are a=(0.9545, 0.2983)T, b=(3, −2)T,d=(0.3, 0.2, 0.1)T. The simulation conditions are as follows. ut

is uniformly distributed in [−1, 1], N=2500, b0=(1, 0)T, d0=(0.1, 0.4, 0.4)T. The nonlinear functions are g1(u)=u, g2(u)=u3, the noise vt is white Gaussian with zero mean and standarddeviation 0.1. Using the normalized iterative algorithm, afterone step we get

a = (0.9537, 0.3008)T, b = (2.9793, −1.9850)T,

d = (0.2997, 0.2000, 0.1000)T.

The estimates are very close to the true values. To show howthe estimates behave with different values of N, we calculatedthe squared error eTe, e = (a, d, b) − (a, d, b) for N = 100to 10000. For each N, the squared error is the average of 100Monte Carlo simulations. The error is plotted in Fig. 4. Thisfigure shows that the squared error goes to zero as N → ∞ inprobability.

The convergence of algorithm (5) depends on the odd sym-metry of the nonlinear block. When the nonlinear block isnot perfectly odd, we give the following example to show themethod is not very sensitive on the odd assumption. Let the non-linear functions be g1(u)=u3, g2(u)=u2. The true parametersare a = (0.9998, 0.0200)T, b = (3, −2)T, d = (0.4, 0.2, 0.1)T.

The nonlinear block has a small even component and is not per-fectly odd here. The simulation conditions are as follows. ut isuniformly distributed in [−1, 1], N = 5000, b0 = (1, 0)T, d0 =(0.05, 0.05, 0.05)T. The noise vt is white Gaussian with zeromean and standard deviation 0.1. After one iterative step, weget the estimates

a = (0.9998, 0.0188)T, b = (2.9999, −1.9993)T,

d = (0.3998, 0.2000, 0.1000)T.

The estimates are still close to the true values in this exampleof nonodd case.

0 2000 4000 6000 8000 100000

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

N

Square

d E

rror

Fig. 4. Estimation error for Hammerstein models with IIR linear block.

Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354 353

0 2000 4000 6000 8000 100000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

N

Square

d E

rror

Fig. 5. Estimation error for Hammerstein models with saturation nonlinearblock.

We then provide a numerical example to show the efficiencyof the iterative algorithm (14) in Hammerstein system withsaturation nonlinear part.

The true parameters are

a = (1, 3.5)T, d = (0.9058, 0.3397, −0.2265, −0.1132)T,

and

c = 3.5.

In simulation, the input ut is uniformly distributed in[−10, 10], N = 4000, a0 = (0.5, 4.9)T. Noise vt is whiteGaussian with zero mean and standard deviation 0.1.

Using the normalized iterative algorithm, after one step weget

d = (0.9052, 0.3399, −0.2274, −0.1161)T.

Using an upper bound c1 = 9.8 and lower bound c2 = 1 of trueparameter, we get

a = (0.9878, 3.3960)T.

It then can be used to estimate the parameter c,

c = 3.4380.

The mean squared errors for different value of N are plotted inFig. 5.

Thus the algorithm (14) gives very close estimates for thetrue parameters.

The last example is a Hammerstein system with preloadpiecewise-linear nonlinear part. We will use it to verify the al-gorithms (15) and (16).

The true parameters are

a = (6, 2)T,

d = (0.7947, 0.2649, −0.5298, −0.1325)T,

and

c = 2.

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 104

0

1

2

3

4

5

6

7

N

Square

d E

rror

Fig. 6. Estimation error for Hammerstein models with preload nonlinear block.

The simulation conditions are as follows. ut is uniformly dis-tributed in [−10, 10], N=5000, a0=(4, −3)T, c=4 is an upperbound of the true value c = 2. Noise vt is white Gaussian withzero mean and standard deviation 0.1. Using the normalizediterative algorithm, after one step we get

a = (5.9495, 2.0648)T,

d = (0.7973, 0.2685, −0.5249, −0.1292)T.

To estimate c more accurately, we use the input ut uniformlydistributed in [−4, 4], and choose the real solution of Eq. (21).The estimate is

c = 2.0140.

Fig. 6 demonstrates the average squared errors with differentvalue of N. This shows that the algorithm is effective.

One point that need to be emphasized is that the estimatesin the examples are obtained by minimizing JN(a, d, b), notJ (a, d, b). But given the assumptions on the noise, the fol-lowing consistency result is available (Ljung, 1999; Ninness &Gibson, 2002):

limN→∞ N = ow.p.1, o = argmin

∈Rm

limN→∞ EJN().

Here N = {a, d, b}, o = {a, d, b}.

5. Conclusion

Iterative algorithm is a simple and efficient approach in pa-rameter estimation. The convergence of the iterative method onIIR Hammerstein system is not known in current literature. Anormalized iterative algorithm with some given initial condi-tions is proposed for the system in this paper. The convergenceproperty is analyzed and illustrated with an example.

Hammerstein systems with nonsmooth piecewise-linear non-linear part are often encountered in real processes operatingdifferently in different input intervals. In this paper, normalized

354 Y. Liu, E.-W. Bai / Automatica 43 (2007) 346–354

iterative algorithms are applied to identification of Hammer-stein systems with two kinds of odd piecewise-linear nonlin-earities, the saturation and the preload nonlinear structures. Byusing a random input with symmetric distribution, the iterativealgorithm is shown to be convergent in one step. The resultsare supported by examples of both systems.

The convergence results of the iterative algorithms in suchsystems are new. The algorithms converge fast and is easy tocompute. Although some prior knowledge of the nonlinear partstructure and lower and upper bound of the critical points isneeded, the method is still useful and provides a simple way toget convergent estimates. The numerical examples show thatthe suggested approach could be a viable method to try in suchsystems.

Future work could apply the iterative methods to more gen-eral types of nonlinear systems and study the convergence prop-erties. The technical challenge is to extend the result to non-odd systems, which may need some major revision of the al-gorithms.

References

Bai, E. W. (2002). Identification of linear system with hard input nonlinearitiesof known structure. Automatica, 38, 853–860.

Bai, E. W., & Li, D. (2004). Convergence of the iterative Hammersteinsystem identification algorithm. IEEE Transactions on Automatic Control,49, 1929–1940.

Bai, E. W., & Liu, Y. (2006). Least squares solutions of bilinear equations.System and Control Letters, 55, 466–472.

Cerone, V., & Regruto, D. (2003). Parameter bounds for discrete-timeHammerstein models with bounded output errors. IEEE Transactions onAutomatic Control, 48, 1855–1860.

Chen, H. F. (2004). Pathwise convergence of recursive identificationalgorithms for Hammerstein systems. IEEE Transactions on AutomaticControl, 49, 1641–1649.

Greblicki, W. (2000). Continuous time Hammerstein system identification.IEEE Transactions on Automatic Control, 45, 1232–1236.

Ljung, L. (1999). System Identification: Theory for the user. New Jersey:Prentice-Hall.

Narendra, K. S., & Gallman, P. G. (1966). Continuous time Hammersteinsystem identification. IEEE Transactions on Automatic Control, 11,546–550.

Ninness, B., & Gibson, S. (2002). Quantifying the accuracy of Hammersteinmodel estimation. Automatica, 38, 2037–2051.

Stoica, P. (1981). On the convergence of an iterative algorithm used forHammerstein systems. IEEE Transactions on Automatic Control, 26,967–969.

Voros, J. (1997). Parameter identification of discontinuous Hammersteinsystems. Automatica, 33, 1141–1146.

Voros, J. (2003). Recursive identification of Hammerstein systems withdiscontinuous nonlinearities containing dead-zones. IEEE Transactions onAutomatic Control, 48, 2203–2206.

Wolodkin, G., Rangan, S., & Poolla, K. (1995). New results for Hammersteinsystem identification. In Proceedings of CDC (pp. 697–702).

Yun Liu received her Ph.D. degree ofElectrical and Computer Engineeringfrom the University of Iowa in 2006,and M.E. and B.E. degree from Shang-hai Jiao Tong University (2001, 1998).Her research interests are dynamicalsystem identification, signal processingand control methods for vehicle appli-cations. She is now a control engineerwith Servo Tech Inc.

Er-Wei Bai was educated in FudanUniversity, Shanghai Jiaotong Univer-sity and the University of California atBerkeley. Dr. Bai is Professor of Elec-trical and Computer Engineering at theUniversity of Iowa where he teachesand conducts research in identification,controls, signal processing and theirapplications. He is an IEEE Fellow anda precipitant of the President Award onteaching excellence.