Multi-innovation least squares identification methods based on the auxiliary model for MISO systems

Applied Mathematics and Computation 187 (2007) 658–668

www.elsevier.com/locate/amc

Multi-innovation least squares identification methods basedon the auxiliary model for MISO systems q

Feng Ding a,*, Huibo Chen a, Ming Li b

a Control Science and Engineering Research Center, Southern Yangtze University, Wuxi, Jiangsu 214122, PR Chinab Key Laboratory of Nondestructive Test (Ministry of Education), Nanchang Institute of Aeronautical Technology,

Nanchang 330034, PR China

Abstract

For multi-input, single-output output-error systems, a difficulty in identification is that the information vector in theidentification model obtained contains unknown inner/intermediate variables; thus the standard least squares methodscannot be applied directly. In this paper, we present a multi-innovation least squares identification algorithm based onthe auxiliary model; its basic idea is to replace the unknown inner variables with their estimates computed by an auxiliarymodel. Convergence analysis indicates that the parameter estimation error converges to zero under persistent excitation.The algorithm proposed has significant computational advantage over existing identification algorithms. A simulationexample is included.� 2006 Elsevier Inc. All rights reserved.

Keywords: Recursive identification; Estimation; Least squares; Multi-innovation identification; Hierarchical identification; Auxiliarymodel; Multivariable systems; Convergence properties

1. Problem formulation

Consider a multi-input, single-output (MISO) system described by the output-error state-space model in [1]

0096-3

doi:10

q Th* Co

E-m

xðt þ 1Þ ¼ AxðtÞ þ BuðtÞ;yðtÞ ¼ CxðtÞ þDuðtÞ þ vðtÞ;

�ð1Þ

where xðtÞ 2 Rn is the state vector, uðtÞ ¼ ½u1ðtÞ; u2ðtÞ; . . . ; urðtÞ�T 2 Rr the system input vector, yðtÞ 2 R1

the system output, vðtÞ 2 R1 the observation white noise with zero mean, and A 2 Rn�n, B ¼ ½b1;b2; . . . ; br� 2 Rn�r, C 2 R1�n and D ¼ ½d1; d2; . . . ; dr� 2 R1�r the system matrices.

003/$ - see front matter � 2006 Elsevier Inc. All rights reserved.

.1016/j.amc.2006.08.090

is work was supported by the National Natural Science Foundation of China (Nos. 60474039, 60475002).rresponding author.ail address: [email protected] (F. Ding).

mailto:[email protected]

F. Ding et al. / Applied Mathematics and Computation 187 (2007) 658–668 659

Let z�1 be a unit delay operator: z�1y(t) = y(t � 1). From (1), it is easy to get� �
yðtÞ ¼ ½CðzI � AÞ�1
B þD�uðtÞ þ vðtÞ ¼ Cadj½I � Az�1�Bz�n det½zI � A� þD uðtÞ þ vðtÞ

¼:1

AðzÞXr

i¼1

BiðzÞuiðtÞ þ vðtÞ ð2Þ

with A(z) being the characteristic polynomial in z�1 of degree n and Bi(z) a polynomial in z�1, and both rep-resented as

AðzÞ ¼ z�n det½zI � A� ¼: 1þ a1z�1 þ a2z�2 þ � � � þ anz�n; ai 2 R1;

BiðzÞ ¼ Cadj½I � Az�1�bi þ diz�n det½zI � A�¼: bi0 þ bi1z�1 þ bi2z�2 þ � � � þ binz�n; bij 2 R1:

Although the bias-elimination (or bias-correction) least squares method in [1–3] may be extended to iden-tify MISO systems in (2), it requires the assumption that both the system inputs and the process noise are sta-tionary and ergodic, and also requires accurate knowledge on the autocorrelation and cross-correlationfunctions of input, output and noise [4]. These conditions are very difficult to satisfy in practice.

This paper uses the auxiliary model technique in [5–8] to study the identification problem of MISO systemsin (2). The basic idea is to replace the unknown inner (intermediate) variables in the information vector by theoutputs of the auxiliary model; we present a recursive algorithm to estimate the unknown parameters (ai,bij) in(2) from the given input–output measurement data {ui(t),y(t): t = 1,2, . . .}, and further, study convergenceperformance of the new algorithm presented. The approach here differs from the ones in [5–8] which usedthe auxiliary model technique to identify the parameters and estimate the missing outputs of dual-rate sam-pled-data systems [9–11].

On convergence or consistency of identification algorithms, Ljung’s and Solo’s results assumed that theinput and output signals have finite non-zero power, and that the noise is independent and identically distri-buted random sequence with finite 4th-order moments [12], or the process noise and input are stationary andergodic [13]. Also, Lai and Wei [14], Lai and Ying [15], and Ren and Kumar [16] assumed that high-ordermoments of the noise {v(t)} exist, i.e., E½vcðtÞjFt�1� <1, a.s. for some c > 2 (Ft is a sigma algebra generatedby the observations up to and including time t). Our analysis does not assume that the process noise is sta-tionary or ergodic, or higher-order moments exist, and thus, the problem considered is possibly non-stationaryand non-ergodic, and the noise variance is possible infinite. Under such weaker assumptions, we explore con-sistency of the estimation algorithm proposed.

Briefly, the paper is organized as follows: Section 2 derives our basic identification algorithm for multi-input, single-output systems. Section 3 analyzes the performance of the proposed algorithm. Section 4 presentsan illustrative example for the results in this paper. Finally, concluding remarks are given in Section 5.

2. The basic algorithm

Define an inner variable

xðtÞ :¼ 1

AðzÞXr

i¼1

BiðzÞuiðtÞ; ð3Þ

and the parameter vector h and information vector u0(t) as

h ¼ ½a1; a2; . . . ; an; b10; b11; . . . ; b1n; . . . ; br0; br1; . . . ; brn�T 2 Rn0 ;

u0ðtÞ ¼ ½wT0 ðtÞ;w

Tu ðtÞ�

T; n0 :¼ nþ rðnþ 1Þ;

w0ðtÞ ¼ ½�xðt � 1Þ;�xðt � 2Þ; . . . ;�xðt � nÞ�T 2 Rn;

wuðtÞ ¼ ½u1ðtÞ; . . . ; u1ðt � nÞ; u2ðtÞ; . . . ; u2ðt � nÞ; . . . ; urðtÞ; . . . ; urðt � nÞ�T 2 Rrðnþ1Þ:

Hence, from (2) and (3), we obtain the following identification model:

xðtÞ ¼ uT0 ðtÞh; yðtÞ ¼ uT

0 ðtÞhþ vðtÞ; ð4Þ

660 F. Ding et al. / Applied Mathematics and Computation 187 (2007) 658–668

where the superscript T denotes the matrix transpose. Let p be the data length (p� n0), and define the stackedvectors or matrix

YðtÞ ¼

yðtÞyðt � 1Þ

..

.

yðt � p þ 1Þ

266664377775; U0ðtÞ ¼

uT0 ðtÞ

uT0 ðt � 1Þ

..

.

uT0 ðt � p þ 1Þ

266664377775; VðtÞ ¼

vðtÞvðt � 1Þ

..

.

vðt � p þ 1Þ

266664377775: ð5Þ

From (4) and (5), we have

YðtÞ ¼ U0ðtÞhþ VðtÞ: ð6Þ

From a cost function [25]:

JðhÞ ¼ kYðtÞ � U0ðtÞhk2;

where the norm of the matrix X is defined as kXk2 :¼ tr[XXT].Because U0(t) contains unknown inner variables x(t � i), i = 1,2, . . . ,n, the standard least squares method

cannot be applied directly to obtain the least squares estimate ½UT0 ðtÞU0ðtÞ��1UT

0 ðtÞYðtÞ of the parameter vectorh by minimizing the cost function J(h) even if V(t) is a ‘‘white’’ noise vector with zero mean. Our solution isbased on the auxiliary model identification idea: these unknown x(t � i) in u0(t) are replaced by the outputsxa(t � i) of an auxiliary (or reference) model [5,6], then the identification problem of h can be solved using xa(t)instead of x(t). Further, u0(t) is replaced by u(t), and U0(t) by U(t); then it is easy from (6) to get the multi-innovation recursive least squares identification algorithm based on the auxiliary model as follows:

hðtÞ ¼ hðt � 1Þ þ PðtÞUTðtÞ½YðtÞ � UðtÞhðt � 1Þ�; ð7Þ

P�1ðtÞ ¼ P�1ðt � 1Þ þ UTðtÞUðtÞ; Pð0Þ ¼ p0I ; ð8Þ

UðtÞ ¼

uTðtÞ

uTðt � 1Þ

..

.

uTðt � p þ 1Þ

26666664

37777775; uðtÞ ¼wðtÞ

wuðtÞ

" #; ð9Þ

wðtÞ ¼ ½�xaðt � 1Þ;�xaðt � 2Þ; . . . ;�xaðt � nÞ�T;

wuðtÞ ¼ ½u1ðtÞ; . . . ; u1ðt � nÞ; u2ðtÞ; . . . ; u2ðt � nÞ; . . . ; urðtÞ; . . . ; urðt � nÞ�T; ð10Þ

xaðtÞ ¼ uTðtÞhðtÞ ðAuxiliary modelÞ; ð11Þ

where hðtÞ denote the estimate of h at time t.As p = 1, EðtÞ :¼ YðtÞ � UðtÞhðt � 1Þ ¼ yðtÞ � uTðtÞhðt � 1Þ 2 R1 is called the (single) innovation; as p P 2,

EðtÞ :¼ YðtÞ � UTðtÞhðt � 1Þ 2 Rp may be referred to as the innovation vector, namely, multi-innovation. Thisis the reason we call the algorithm in (7)–(11) the auxiliary model based multi-innovation recursive leastsquares identification algorithm (the AM-MILS algorithm for short).

To initialize this AM-MILS algorithm, we take p0 to be a large positive number, e.g., p0 = 106, and hð0Þ tobe some small real vector, e.g., h0 ¼ 1n0

=p0 with 1n0being an n0-dimensional column vector whose elements are

1. Here we refer to xa(t) as the estimate of the inner variable x(t).From here, we can see that when the number of parameters (dimension of h) is very large, the AM-MILS

algorithm will lead to increased computational complexity, which may be reduced by using the hierarchicalidentification scheme [17,18,10,19–21].

The convergence properties of the algorithm in (7)–(11) is very important and will be studied next.


3. The main convergence results

Let us introduce some notation first. The symbol I stands for an identity matrix of appropriate sizes;jXj = det[X] represents the determinant of the square matrix X; kmax[X] and kmin[X] represent the maximumand minimum eigenvalues of X, respectively; f(t) = O(g(t)) represents f(t)/g(t)! 0 as t!1; for g(t) P 0,we write f(t) = O(g(t)) or f(t)�g(t) if there exists positive constants d1 and t0 such that jf(t)j 6 d1g(t) for t P t0.

It is assumed that fvðtÞ;Ftg is a martingale sequence defined on a probability space fX;F; Pg, where fFtgis the r algebra sequence generated by {v(t)}, and the noise sequence {v(t)} satisfies [4,22]

ðA1Þ E½vðtÞjFt�1� ¼ 0; a:s:;

ðA2Þ E½v2ðtÞjFt�1� 6 r2r�1ðtÞ; a:s:; 0 6 �1 < 1; 0 6 r2 <1;

where

rðtÞ :¼ tr½P�1ðtÞ�:
Conditions (A1) and (A2) are quite relaxed – we do not assume that the noise variance is bounded, or the noiseis stationary and ergodic, or its high-order moments exist.
From the definition of r(t), it is easy to get

jP�1ðtÞj 6 rn0ðtÞ; rðtÞ 6 n0kmax½P�1ðtÞ�; ln jP�1ðtÞj ¼ Oðln rðtÞÞ: ð12Þ
In order to analyze the convergence properties of the parameter and output estimation errors, some math-
ematical preliminaries are required.

Lemma 1. For each i (i = 0,1, . . . , p � 1), the following inequality holds:
X1t¼1
uTðt � iÞPðtÞuðt � iÞreðtÞ <1; a:s: for any e > 0:

Proof. As in [6,23], from the definition of P(t) in (8), we have

P�1ðt � 1Þ ¼ P�1ðtÞ � UTðtÞUðtÞ 6 P�1ðtÞ � uðt � iÞuTðt � iÞ ¼ P�1ðtÞ½I � PðtÞuðt � iÞuTðt � iÞ�:

Taking determinants on both sides and using the formula det[I + DE] = det[I + ED] yield

jP�1ðt � 1Þj 6 jP�1ðtÞjjI � PðtÞuðt � iÞuTðt � iÞj ¼ jP�1ðtÞj½1� uTðt � iÞPðtÞuðt � iÞ�:

Hence,

uTðt � iÞPðtÞuðt � iÞ 6 jP�1ðtÞj � jP�1ðt � 1Þj

jP�1ðtÞj: ð13Þ

Dividing (13) by re(t) and summing for t give

X1t¼1

uTðt � iÞPðtÞuðt � iÞreðtÞ 6

X1t¼1

jP�1ðtÞj � jP�1ðt � 1ÞjjP�1ðtÞj1þe=n0

¼X1t¼1

Z jP�1ðtÞj

jP�1ðt�1Þj

dx

jP�1ðtÞj1þe=n06

Z jP�1ð1Þj

jP�1ð0Þj

dxx1þe=n0

¼ �n0

e1

xe=n0

��jP�1ð1Þj

jP�1ð0Þj¼ n0

e1

jP�1ð0Þje=n0� 1

jP�1ð1Þje=n0

!<1; a:s: �

Define the parameter estimation error vector ~hðtÞ and a non-negative definite function W(t) as

~hðtÞ ¼ hðtÞ � h; ð14ÞW ðtÞ ¼ ~hTðtÞP�1ðtÞ~hðtÞ; ð15Þ


and

eY ðtÞ :¼ 1

2UðtÞ~hðtÞ þ ½YðtÞ � UðtÞhðtÞ � VðtÞ�; ð16ÞeU ðtÞ :¼ �UðtÞ~hðtÞ; ð17Þ

SðtÞ :¼ 2Xt

i¼1

eU TðiÞ eY ðiÞ:
Lemma 2. For the system in (6) and the AMLS algorithm in (7)–(11), assume that (A1) and (A2) hold and
HðzÞ :¼ 1AðzÞ � 1

2 is strictly positive real. Then the following inequality holds:

E½W ðtÞ þ SðtÞjFt�1� 6 W ðt � 1Þ þ Sðt � 1Þ þ 2Xp�1

i¼0

uTðt � iÞPðtÞuðt � iÞr�1ðtÞr2; a:s:

Here, the positive realness condition guarantees that S(t) P 0.

Proof. Define the innovation vector E(t) and residue vector g(t) as follows:

EðtÞ :¼ YðtÞ � UðtÞhðt � 1Þ; ð18ÞgðtÞ :¼ YðtÞ � UðtÞhðtÞ: ð19Þ

It follows that

gðtÞ ¼ ½I � UðtÞPðtÞUTðtÞ�EðtÞ ¼ ½I þ UðtÞPðt � 1ÞUTðtÞ��1EðtÞ: ð20Þ

Substituting (7) into (14) and using (18)–(20), it is not difficult to get

~hðtÞ ¼ ~hðt � 1Þ þ PðtÞUTðtÞEðtÞ ¼ ~hðt � 1Þ þ Pðt � 1ÞUTðtÞgðtÞ: ð21Þ
Using (8) and (18)–(21), we have from (15)
W ðtÞ ¼ W ðt � 1Þ þ ~hTðtÞUTðtÞUðtÞ~hðtÞ þ 2~hTðtÞUTðtÞgðtÞ � ETðtÞ½I � UðtÞPðtÞUTðtÞ�UðtÞPðtÞUTðtÞEðtÞ6 W ðt � 1Þ þ ~hTðtÞUTðtÞUðtÞ~hðtÞ þ 2~hTðtÞUTðtÞgðtÞ

¼ W ðt � 1Þ þ 2~hTðtÞUTðtÞ 1

2UðtÞ~hðtÞ þ gðtÞ � VðtÞ

� �þ 2~hTðtÞUTðtÞVðtÞ:

Using (16), (17), (20) and (21), we have

W ðtÞ 6 W ðt � 1Þ � 2 eU TðtÞ eY ðtÞ þ 2½~hðt � 1Þ þ PðtÞUTðtÞEðtÞ�TUTðtÞVðtÞ

¼ W ðt � 1Þ � 2 eU TðtÞ eY ðtÞ þ 2~hTðt � 1ÞUTðtÞVðtÞ þ 2½EðtÞ � VðtÞ�TUðtÞPðtÞUTðtÞVðtÞþ 2VTðtÞUðtÞPðtÞUTðtÞVðtÞ: ð22Þ

Since UðtÞ~hðt � 1Þ, E(t) � V(t), U(t)P(t)UT(t) are Ft�1-measurable, taking the conditional expectation of bothsides of (22) with respect to Ft�1 and using (A1) and (A2) give

E½W ðtÞjFt�1� 6 W ðt � 1Þ � 2E½ eU TðtÞ eY ðtÞjFt�1� þ 2trfUðtÞPðtÞUTðtÞgr�1ðtÞr2

¼ W ðt � 1Þ � 2E½ eU TðtÞ eY ðtÞjFt�1� þ 2Xp�1

i¼0

uTðt � iÞPðtÞuðt � iÞr�1ðtÞr2; a:s: ð23Þ

Since

AðzÞ½gðtÞ � VðtÞ� ¼ �UðtÞ~hðtÞ ¼ eU ðtÞ; ð24Þ
using (17), (24) and (19), we get from (16)
eY ðtÞ ¼ 1

2UðtÞ~hðtÞ þ ½gðtÞ � VðtÞ� ¼ 1

AðzÞ �1þ q

2

� � eU ðtÞ þ q2eU ðtÞ ¼: eY 1ðtÞ þ

q2eU ðtÞ;


where

eY 1ðtÞ :¼ H 1ðzÞ eU ðtÞ; H 1ðzÞ :¼ 1

AðzÞ �1þ q

2:

Since H(z) is a strictly positive real function, there exists a small constant q > 0 such that H1(z) is also strictlypositive real. Referring to Appendix C in [22], we can draw that the following inequalities hold

S1ðtÞ :¼ 2Xt

i¼1

eU TðiÞ eY 1ðiÞP 0; a:s:;

SðtÞ ¼ S1ðtÞ þ qXt

i¼1

k eU ðiÞk2 P 0; a:s:

ð25Þ

Adding both sides of (23) by S(t) gives the conclusion of Lemma 2. h

The stochastic process theory and martingale theory are the main tools of studying the convergence of iden-tification algorithms [17,18,24–26]. Next, we prove the main results of this paper by formulating a martingaleprocess and by using the stochastic process theory and the martingale convergence theorem (Lemma D.5.3 in[22]).

Theorem 1. For the system in (6), assume that (A1)–(A3) hold, and A(z) is stable, i.e., all zeros of A(z) are inside

the unit circle, and that there exist positive constants c1, c2 and t0 such that the following persistent excitation

condition (bounded condition number) holds

ðC1Þ c1I 61

t

Xt

i¼1

U0ðiÞUT0 ðiÞ 6 c2I ; a:s: for t P t0:

Then for any � with 0 6 �1 < � < 1, the parameter estimation error by the AM-MILS algorithm in (7)–(11)satisfies

khðtÞ � hk2 ¼ O1

t1��

� �! 0; a:s:; as t!1:

Proof. Let

ZðtÞ :¼ W ðtÞ þ SðtÞr�ðtÞ ; 0 6 �1 < � < 1:

Since r(t) is non-decreasing, according to Lemma 2, we have

E½ZðtÞjFt�1� 6W ðt � 1Þ þ Sðt � 1Þ

r�ðtÞ þ 2Xp�1

i¼0

uTðt � iÞPðtÞuðt � iÞr�ðtÞ r�1ðtÞr2

6 Zðt � 1Þ þ 2Xp�1

i¼0

uTðt � iÞPðtÞuðt � iÞ½rðtÞ��1 r2; a:s: ð26Þ

Using Lemma 1, it is clear that the sum for t from 1 to1 of the last term on the right-hand side of (26) is finite.Now applying the martingale convergence theorem (Lemma D.5.3 in [22]) to (26), we conclude that Z(t) con-verges a.s. to a finite random variable, say, Z0; i.e.,

ZðtÞ ¼ W ðtÞ þ SðtÞr�ðtÞ ! Z0 <1; a:s:

or

W ðtÞ ¼ Oðr�ðtÞÞ; a:s:; SðtÞ ¼ Oðr�ðtÞÞ; a:s: ð27Þ


Since H(z) is a strictly positive real function, from (25), it follows that
Xt
i¼1

k eU ðiÞk2 ¼ Oðr�ðtÞÞ; a:s:

From the definition of W(t), we have

k~hðtÞk26

~hTðtÞP�1ðtÞ~hðtÞkmin½P�1ðtÞ�

¼ W ðtÞkmin½P�1ðtÞ�

:

Using (27), it follows that

k~hðtÞk2 ¼ Or�ðtÞ

kmin½P�1ðtÞ�

� �; a:s: ð28Þ

Since A(z) is stable, according to Lemma B.3.3 in [22] and (24), there exist positive constants k1 and k2 suchthat
Xt
i¼1

kgðiÞ � VðiÞk26 k1

Xt

i¼1

k eU ðiÞk2 þ k2 ¼ Oðr�ðtÞÞ: ð29Þ

Define

P�10 ðtÞ :¼

Xt

i¼1

UT0 ðiÞU0ðiÞ; r0ðtÞ :¼ tr½P�1

0 ðtÞ�:

Now we show that r(t) = O(r0(t)) and kmin½P�1ðtÞ� ¼ Oðkmin½P�10 ðtÞ� þ r�ðtÞÞ. Define the vector error eUðtÞ as

follows [refer to the definitions of U(t) and U0(t)]:

eUðtÞ :¼ UðtÞ � U0ðtÞ ¼ ½gðt � 1Þ � Vðt � 1Þ; . . . ; gðt � nÞ � Vðt � nÞ; 0; . . . ; 0�T:

Hence, we have
Xt
i¼1

keUðiÞk2 ¼Xt

i¼1

Xn

j¼1

kgði� jÞ � Vði� jÞk2 ¼ Oðr�ðtÞÞ;

rðtÞ 6 2r0ðtÞ þ 2Xt

i¼1

keUðiÞk2 ¼ 2r0ðtÞ þOðr�ðtÞÞ ¼ Oðr0ðtÞÞ; a:s:

ð30Þ

For any vector x 2 Rn with kxk = 1, we have
Xt
i¼1

kxTUðiÞk2 ¼Xt

i¼1

kxTU0ðiÞ � xT eUðiÞk26 2

Xt

i¼1

kxTU0ðiÞk2 þ 2Xt

i¼1

keUðiÞk2

¼ 2Xt

i¼1

kxTU0ðiÞk2 þOðr�ðtÞÞ:

Thus,

kmin½P�1ðtÞ� 6 2kmin½P�10 ðtÞ� þOðr�ðtÞÞ;

or

kmin½P�1ðtÞ� ¼ Oðkmin½P�10 ðtÞ� þ r�ðtÞÞ:

Combining (28) with (30) and above equation gives

khðtÞ � hk2 ¼ Or�0ðtÞ

kmin½P�10 ðtÞ� þ r�ðtÞ

!¼ O

r�0ðtÞkmin½P�1

0 ðtÞ�

!; a:s:; 0 < � < 1: ð31Þ


From (C1), we easily get

TableThe pa

t

100200300500

1000150020002500300035004000

True v

r0ðtÞ 6 n0c2t þ n0p0; a:s:; kmin½P�10 ðtÞ�P c1t; a:s:

Thus

khðtÞ � hk2 ¼ Oðn0c2t þ n0p0Þ

�

c1t

� �¼ O

1

t1��

� �! 0; a:s:; as t!1: ð32Þ

This proves Theorem 1. h

From Theorem 1, we have the following remarks:

• The parameter estimation error khðtÞ � hk2 converges to zero at the rate of O 1t1��

� (0 6 �1 < � < 1); a smaller

�1 (a lower noise level) will lead to a faster rate of convergence.• From (32), increasing c1 in condition (C1) and decreasing c2 result in faster convergence rate. In other

words, the stationarity of the input–output data can improve the parameter estimation accuracy.• We do not make such assumptions that high-order moments of the noise {v(t)} exist; thus the results of

Theorem 1 still hold for non-stationary systems.

4. Example

An example is given to demonstrate the effectiveness of the proposed algorithm. Consider the following2-input, 1-output system:

yðtÞ ¼ 1

AðzÞ ½B1ðzÞu1ðtÞ þ B2ðzÞu2ðtÞ� þ vðtÞ;

AðzÞ ¼ 1þ a1z�1 þ a2z�2 ¼ 1� 0:84z�1 þ 0:16z�2;

B1ðzÞ ¼ b11z�1 þ b12z�2 ¼ 0:12z�1 þ 0:324z�2;

B2ðzÞ ¼ b21z�1 þ b22z�2 ¼ 0:25z�1 þ 0:186z�2;

h ¼ ½a1; a2; b11; b12; b21; b12�T:

{u1(t),u2(t)} is taken as an uncorrelated persistent excitation signal vector sequence with zero mean and unitvariance r2

u1¼ r2

u2¼ 1:002, and {v(t)} as a white noise sequence with zero mean and constant variance

r2 = 0.102, r2 = 0.202 and r2 = 0.402, respectively. Apply the AM-MILS algorithm with p = 1 to estimatethe parameters of this system, the parameter estimates and their errors with different noise variances are shownin Tables 1–3, and the parameter estimation errors d ¼ khðtÞ � hk=khk versus t are shown in Fig. 1, where dns

represents the noise-to-signal ratio of the system.

1rameter estimates and their errors (r2 = 0.102,dns = 11.67%)

a1 a2 b11 b12 b21 b22 d (%)

�0.67328 0.01255 0.12774 0.34846 0.23466 0.27893 24.96523�0.71244 0.05750 0.13705 0.33692 0.24161 0.25029 18.21351�0.74065 0.07152 0.12764 0.32674 0.24292 0.23321 14.54358�0.76629 0.08950 0.12554 0.32184 0.24277 0.21467 10.92626�0.80211 0.11832 0.11705 0.31881 0.24881 0.20337 6.08732�0.81477 0.13262 0.11662 0.31659 0.24845 0.19885 4.13452�0.82596 0.14300 0.11715 0.31708 0.24853 0.19461 2.55505�0.83198 0.14991 0.11591 0.31812 0.24851 0.19230 1.65349�0.83680 0.15417 0.11638 0.31819 0.24802 0.19120 1.13526�0.83857 0.15551 0.11764 0.31989 0.24835 0.19054 0.84737�0.83965 0.15717 0.11831 0.32147 0.24982 0.19018 0.60691

alues �0.84000 0.16000 0.12000 0.32400 0.25000 0.18600

Table 2The parameter estimates and their errors (r2 = 0.202, dns = 23.35%)

t a1 a2 b11 b12 b21 b22 d (%)

100 �0.61729 �0.03305 0.12029 0.36732 0.21054 0.32082 33.83325200 �0.65620 0.01868 0.14427 0.35089 0.22959 0.28616 26.29321300 �0.69315 0.03343 0.12899 0.33264 0.23386 0.26013 21.42395500 �0.71640 0.04396 0.12784 0.32456 0.23575 0.23302 18.15027

1000 �0.76780 0.08334 0.11285 0.31812 0.24811 0.21783 11.343081500 �0.78776 0.10560 0.11246 0.31315 0.24722 0.21067 8.268412000 �0.80831 0.12413 0.11370 0.31360 0.24726 0.20299 5.372672500 �0.81970 0.13695 0.11132 0.31528 0.24720 0.19880 3.655953000 �0.82935 0.14520 0.11236 0.31500 0.24622 0.19675 2.519713500 �0.83292 0.14773 0.11494 0.31814 0.24684 0.19553 1.953274000 �0.83524 0.15112 0.11634 0.32110 0.24977 0.19484 1.45881

True values �0.84000 0.16000 0.12000 0.32400 0.25000 0.18600

Table 3The parameter estimates and their errors (r2 = 0.402, dns = 46.69%)

t a1 a2 b11 b12 b21 b22 d (%)

100 �0.52772 �0.10496 0.10412 0.39829 0.15739 0.39126 48.63959200 �0.56314 �0.04294 0.15793 0.37512 0.20355 0.34742 39.79790300 �0.61248 �0.02980 0.13155 0.34246 0.21477 0.30623 33.12154500 �0.62294 �0.04047 0.13277 0.32980 0.22146 0.26415 31.56565

1000 �0.69372 0.00912 0.10467 0.31756 0.24662 0.24638 22.528441500 �0.72432 0.04336 0.10423 0.30748 0.24484 0.23558 17.790052000 �0.76202 0.07642 0.10680 0.30790 0.24476 0.22188 12.503462500 �0.78384 0.10073 0.10213 0.31086 0.24467 0.21427 9.179633000 �0.80405 0.11756 0.10434 0.30973 0.24268 0.21019 6.642453500 �0.81197 0.12287 0.10957 0.31566 0.24388 0.20776 5.486994000 �0.81757 0.13061 0.11244 0.32130 0.24974 0.20626 4.40860

True values �0.84000 0.16000 0.12000 0.32400 0.25000 0.18600

0 500 1000 1500 2000 2500 3000 3500 40000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

t

δ

σ2 = 0.102

σ2 = 0.202

σ2 = 0.402

Fig. 1. The parameter estimation errors d versus t.


From Tables 1–3 and Fig. 1, we can see that the parameter estimation errors by the AM-MILS algorithmbecome smaller and smaller and go to zero with t increasing, and a high noise level results in a low consistencerate of the parameter estimates to the true parameters. This confirms the proposed theorem.


5. Conclusions

According to the auxiliary model identification idea, an auxiliary model based least squares identificationalgorithm is developed for multi-input, single-output systems. The analysis indicates that the algorithm pro-posed can obtain the consistent parameter estimation under persistent excitation, and require less computa-tional efforts than the existing algorithms.

Acknowledgement

The authors are grateful to Professor Tongwen Chen in the Department of Electrical and Computer Engi-neering at the University of Alberta, Edmonton, Alberta, Canada, for his helpful suggestions and carefulrevision.

References

[1] F. Ding, T. Chen, L. Qiu, Bias compensation based recursive least squares identification algorithm for MISO systems, IEEETransactions on Circuits and Systems – II: Express Briefs 53 (5) (2006) 349–353.

[2] W.X. Zheng, On a least-squares-based algorithm for identification of stochastic linear systems, IEEE Transactions on SignalProcessing 46 (6) (1998) 1631–1638.

[3] W.X. Zheng, Least-squares identification of a class of multivariable systems with correlated disturbances, Journal of the FranklinInstitute 336 (8) (1999) 1309–1324.

[4] F. Ding, Y. Shi, T. Chen, Performance analysis of estimation algorithms of non-stationary ARMA processes, IEEE Transactions onSignal Processing 54 (3) (2006) 1041–1053.

[5] F. Ding, T. Chen, Parameter estimation of dual-rate stochastic systems by using an output error method, IEEE Transactions onAutomatic Control 50 (9) (2005) 1436–1441.

[6] F. Ding, T. Chen, Combined parameter and output estimation of dual-rate systems using an auxiliary model, Automatica 40 (10)(2004) 1739–1748.

[7] F. Ding, T. Chen, Identification of dual-rate systems based on finite impulse response models, International Journal of AdaptiveControl and Signal Processing 18 (7) (2004) 589–598.

[8] F. Ding, T. Chen, Modeling and identification for multirate systems, Acta Automatica Sinica 31 (1) (2005) 105–122.[9] F. Ding, T. Chen, Parameter estimation for dual-rate systems with finite measurement data, Dynamics of Continuous, Discrete and

Impulsive Systems, Series B: Applications & Algorithms 11 (1) (2004) 101–121.[10] F. Ding, T. Chen, Hierarchical identification of lifted state-space models for general dual-rate systems, IEEE Transactions on Circuits

and Systems – I: Regular Papers 52 (6) (2005) 1179–1187.[11] F. Ding, T. Chen, Least squares based self-tuning control of dual-rate systems, International Journal of Adaptive Control and Signal

Processing 18 (8) (2004) 697–714.[12] L. Ljung, Consistency of the least-squares identification method, IEEE Transactions on Automatic Control 21 (5) (1976)

779–781.[13] V. Solo, The convergence of AML, IEEE Transactions on Automatic Control 24 (6) (1979) 958–962.[14] T.L. Lai, C.Z. Wei, Extended least squares and their applications to adaptive control and prediction in linear systems, IEEE

Transactions on Automatic Control 31 (10) (1986) 898–906.[15] T.L. Lai, Z.L. Ying, Recursive identification and adaptive prediction in linear stochastic systems, SIAM Journal on Control and

Optimization 29 (5) (1991) 1061–1090.[16] W. Ren, P.K. Kumar, Stochastic adaptive prediction and model reference control, IEEE Transactions on Automatic Control 39 (10)

(1994) 2047–2060.[17] F. Ding, T. Chen, Hierarchical gradient-based identification of multivariable discrete-time systems, Automatica 41 (2) (2005)

315–325.[18] F. Ding, T. Chen, Hierarchical least squares identification methods for multivariable systems, IEEE Transactions on Automatic

Control 50 (3) (2005) 397–402.[19] F. Ding, T. Chen, Iterative least squares solutions of coupled Sylvester matrix equations, Systems & Control Letters 54 (2) (2005)

95–107.[20] F. Ding, T. Chen, Gradient based iterative algorithms for solving a class of matrix equations, IEEE Transactions on Automatic

Control 50 (8) (2005) 1216–1221.[21] F. Ding, T. Chen, On iterative solutions of general coupled matrix equations, SIAM Journal on Control and Optimization 44 (6)

(2006) 2269–2284.[22] G.C. Goodwin, K.S. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ, 1984.[23] F. Ding, P. Liu, Y. Shi, Convergence analysis of estimation algorithms of dual-rate stochastic systems, Applied Mathematics and

Computation 176 (1) (2006) 245–261.


[24] F. Ding, T. Chen, Performance bounds of forgetting factor least squares algorithm for time-varying systems with finite measurementdata, IEEE Transactions on Circuits and Systems – I: Regular Papers 52 (3) (2005) 555–566.

[25] F. Ding, T. Chen, Identification of Hammerstein nonlinear ARMAX systems, Automatica 41 (9) (2005) 1479–1489.[26] F. Ding, Y. Shi, T. Chen, Gradient-based identification methods for Hammerstein nonlinear ARMAX models, Nonlinear Dynamics

45 (1–2) (2006) 31–43.

Multi-innovation least squares identification methods based on the auxiliary model for MISO systems

Documents

Transcript of Multi-innovation least squares identification methods based on the auxiliary model for MISO systems