IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 11 ... · The authors are with the School of...

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 11, NOVEMBER 2009 1707

A New Neuroadaptive Control Architecture forNonlinear Uncertain Dynamical Systems:

Beyond �- and �-ModificationsKostyantyn Y. Volyanskyy, Student Member, IEEE, Wassim M. Haddad, Fellow, IEEE, and

Anthony J. Calise, Senior Member, IEEE

Abstract—This paper develops a new neuroadaptive control ar-chitecture for nonlinear uncertain dynamical systems. The pro-posed framework involves a novel controller architecture involvingadditional terms in the update laws that are constructed using amoving time window of the integrated system uncertainty. Theseterms can be used to identify the ideal system weights of the neuralnetwork as well as effectively suppress and cancel system uncer-tainty without the need for persistency of excitation. A nonlinearparametrization of the system uncertainty is considered and stateand output feedback neuroadaptive controllers are developed. Toillustrate the efficacy of the proposed approach we apply our re-sults to a spacecraft model with unknown moment of inertia andcompare our results with standard neuroadaptive control methods.

Index Terms—Adaptive control, composite adaptation, fastadaptation, neural networks, nonlinear in the parameters neuralnetworks, -modification, uncertainty suppression.

I. INTRODUCTION

N EURAL networks have been extensively used for adap-tive system identification as well as adaptive and neuroad-

aptive control of highly uncertain systems [1]–[17]. The goal ofadaptive and neuroadaptive control is to achieve system perfor-mance without excessive reliance on system models. The fun-damental difference between adaptive control and neuroadap-tive control can be traced back to the modeling and treatmentof the system uncertainties as well as the structure of the basisfunctions used in constructing the regressor vector. In partic-ular, adaptive control is based on constant, linearly parameter-ized system uncertainty models of a known structure but un-known parameters [18]–[20]. This uncertainty characterizationallows for the system nonlinearities to be parameterized by afinite linear combination of basis functions within a class offunction approximators such as rational functions, spline func-tions, radial basis functions, sigmoidal functions, and wavelets.However, this linear parametrization with a given basis functioncannot, in general, exactly capture the system uncertainty.

Manuscript received November 03, 2008; revised May 28, 2009; acceptedJuly 30, 2009. First published September 29, 2009; current version publishedNovember 04, 2009. This work was supported in part by the U.S. Air ForceOffice of Scientific Research under Grant FA9550-06-1-0240 and the NationalScience Foundation under Grant ECS-0601311.

The authors are with the School of Aerospace Engineering, Georgia Instituteof Technology, Atlanta, GA 30332-0150 USA (e-mail: [email protected]; [email protected]; [email protected]).

Digital Object Identifier 10.1109/TNN.2009.2030748

To approximate a larger class of nonlinear system uncer-tainty, the uncertainty can be expressed in terms of a neuralnetwork involving a parameterized nonlinearity. Hence, incontrast to adaptive control, neuroadaptive control is based onthe universal function approximation property, wherein anycontinuous nonlinear system uncertainty can be approximatedarbitrarily closely on a compact set using a neural network withappropriate size, structure, and weights [12], [16], all of whichare not necessarily known a priori. Hence, while neuroadaptivecontrol has advantages over standard adaptive control in theability to capture a much larger class of uncertainties, furthercomplexities arise when the basis functions are not known. Inparticular, the choice and the structure of the basis functions aswell as the size of the neural network and the approximationerror over a compact domain become important issues to ad-dress in neuroadaptive control. This difference in the modelingand treatment of the system uncertainties results in the abilityof adaptive controllers to guarantee asymptotic closed-loopsystem stability versus ultimate boundness as is the case withneuroadaptive controllers [21].

To improve robustness and the speed of adaptation of adap-tive and neuroadaptive controllers several controller architec-tures have been proposed in the literature. These include the -and -modification architectures used to keep the system pa-rameter estimates from growing without bound in the face ofsystem uncertainty [12], [16]. In this paper, a new neuroad-aptive control architecture for nonlinear uncertain dynamicalsystems is developed. Specifically, the proposed framework in-volves a new and novel controller architecture involving addi-tional terms, or -modification terms, in the update laws thatare constructed using a moving time window of the integratedsystem uncertainty. The -modification terms can be used toidentify the ideal neural network system weights which can beused in the adaptive law. In addition, these terms effectively sup-press system uncertainty.

Even though the proposed approach is reminiscent to thecomposite adaptive control framework discussed in [22], the

-modification framework involves neither filtered versions ofthe control input and system state in the update laws nor a leastsquares exponential forgetting factor. Rather, the update lawsinvolve auxiliary terms predicated on an estimate of the un-known neural network weights which in turn are characterizedby an auxiliary equation involving the integrated error dynamicsover a moving time interval. For a scalar linearly parameterizeduncertainty structure, these ideas were first explored in [23]

1045-9227/$26.00 © 2009 IEEE

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on November 1, 2009 at 21:33 from IEEE Xplore. Restrictions apply.

1708 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 11, NOVEMBER 2009

and [24]. In this paper, we extend the results in [23] to vectoruncertainty structures with nonlinear parameterizations. Inaddition, state and output feedback controllers are developed.Finally, to illustrate the efficacy of the proposed approach, weapply our results to a spacecraft model involving an unknownmoment of inertia matrix and compare our results with standardneuroadaptive control methods.

II. ADAPTIVE CONTROL WITH A -MODIFICATION

ARCHITECTURE

In this section, we present the notion of the -modificationarchitecture in adaptive control. Specifically, consider the adap-tive control problem with error dynamics given by

(1)

where , , is the system error signal,is the system uncertainty, , , is the systemstate, is the adaptive signal whose purpose is to suppressor cancel the effect of the system uncertainty, is aknown Hurwitz matrix, and . For sim-plicity of exposition, in this section, we consider the case wherethe system uncertainty , , is a scalar function. Fur-thermore, in the first part of this section, we assume that thesystem uncertainty can be perfectly parameterized in terms of aconstant unknown vector and a known vector of con-tinuous basis functions

such that , , are bounded for all .In particular

(2)

The parametrization given by (2) suggests an adaptive controlsignal , , of the form

(3)

where , , is a vector of the adaptive weights.Hence, the dynamics in (1) can be rewritten as

(4)

The update law for , , can be derived using standardLyaupunov analysis by considering the Lyapunov function can-didate

(5)

where , , and satisfies

where . Note that andfor all .

Now, differentiating (5) along the trajectories of (4) yields

The standard choice of the update law is given by

(6)

so that , , whichguarantees that the error signal , , and weight error

, , are Lyapunov stable, and hence, are bounded forall . Since and are bounded for all , itfollows that , , is bounded, and hence,is bounded for all . Now, it follows from Barbalat’s lemma[25] that converges to zero asymptotically.

The above analysis outlines the salient features of the clas-sical adaptive control architecture. To improve the robustnessproperties of the adaptive controller (3) and (6), a -modifica-tion term of the form , where and is anapproximation of the ideal neural network system weights, canbe included to the update law (6) to keep the adaptive weight(i.e., parameter estimate) from growing without bound in theface of the system uncertainty. However, in this case, when the

error is small, is dominated by whichcauses to be driven to . If is not a good approximationof the actual system parameters , then the system error can in-crease. To circumvent this problem, an -modification term ofthe form , where, typically, , canbe included to the update law (6) in place of the -modificationterm. In both cases, however, the modification terms are predi-cated on involving a best guess for some .

Next, we present a new and novel modification term thatgoes beyond the aforementioned modifications by using contin-uous learning of the unknown weights to improve system uncer-tainty suppression or achieve uncertainty cancellation withoutthe need for persistency of excitation. Specifically, consider theerror dynamics given by (4) and integrate (4) over the movingtime interval , , where and

is a design parameter. Premultiplying (4) by and re-arranging terms, we obtain

(7)

where

(8)

(9)

Hence, although the vector is unknown, satisfies the linearequation (7). Geometrically, (7) characterizes an affine hyper-plane in . For example, in the case where , the affinehyperplane (7) is described by a line with being


VOLYANSKYY et al.: A NEW NEUROADAPTIVE CONTROL ARCHITECTURE FOR NONLINEAR UNCERTAIN DYNAMICAL SYSTEMS 1709

Fig. 1. Visualization of �-modification term.

a normal vector to as shown in Fig. 1. Note that the distancefrom point to point shown in Fig. 1, which is the shortestdistance from the weight estimate to affine hyperplanedefined by (7), is given by .

Next, define the square of the distance from the weight esti-mate , , to the affine hyperplane by

(10)

and note that the gradient of ,, with respect to , , is given by

(11)

Now, consider the modified update law for the adaptive weights, , given by

(12)

where and

In contrast to (6), the update law given by (12) containsthe additional term , , based on the gradient of

with respect to , .We call , , a -modification term. Note that forevery the vector is directed opposite to the gradient

and parallel to, which is a vector normal to the affine hyperplane

defined by (7). Hence, , , introduces a component inthe update law (12) that drives the trajectory , , insuch a way so that the error given by (10) is minimized.

Note that , , is zero only if , , satisfies

(13)

that is, the weight estimates , , lie on the affine hy-perplane defined by (7). If the weight estimates , ,do not satisfy (13), then , , is a nonzero vector that isorthogonal to the affine hyperplane (7) and points in the direc-tion of the hyperplane. Thus, the -modification term drives theweight estimate trajectory , , to the affine hyperplanecharacterized by (7), wherein the ideal weights lie. As shownbelow, the -modification technique can ensure convergence ofthe weight estimates , , to the ideal weights underpersistency of excitation. However, it is important to note herethat identifying the unknown weights is neither a goal of thispaper, nor is this necessary for the -modification frameworkto achieve uncertainty suppression or uncertainty cancellation.

Next, we establish stability guarantees of the adaptive law (3)with (12).

Theorem 2.1: Consider the uncertain dynamical systemgiven by (4). The adaptive feedback control law (3) withupdate law given by (12) guarantees that the solution

of the closed-loop system givenby (4) and (12) is Lyapunov stable and as forall and .

Proof: Consider the Lyapunov function candidate given by(5) and note that using (7) the Lyapunov derivativealong the trajectories of the closed-loop system (4) is given by

(14)

which proves Lyapunov stability of the closed-loop system(4) and (12). This guarantees that the error signal , ,and the weight error , , are Lyapunov stable, andhence, are bounded for all . The result now follows fromBarbalat’s lemma [25] using the fact that , and hence,

, are bounded for all .Remark 2.1: The nonnegative term

in the derivative of the Lyapunov function (14)appears due to the -modification term in the update law (12)and is a measure of the (scaled) distance between the updateweights , , and the affine hyperplane given by (7).

Remark 2.2: The -modification architecture is reminiscentto the composite adaptation technique [22], [26] and the com-bined direct and indirect adaptation technique [27]. As in the

-modification framework, composite adaptation involves a



Fig. 2. Weights identification using �-modification architecture.

linear equation of the unknown weights and uses a predictionerror-based estimation error method to construct additionalterms in the update law. However, the key difference betweenthe two methods is in how the linear equations involving theunknown weights are constructed. Specifically, in the -mod-ification technique, we use a moving time window of theintegrated system uncertainty, whereas composite adaptationuses filtered versions of the control input and system state inthe update law. In addition, composite adaptation involves aleast squares approach with exponential forgetting.

Remark 2.3: The standard adaptive control algorithm of (6)adjusts the parameters in the direction defined by , .In the absence of noise, if , , is constant for someperiod of time, then the parameters would be adjusted alongthe vector , , until the error is zero. In thepresence of noise, however, the parameter would wiggle aroundin the vicinity of the hyperplane orthogonal to , ,on which the error is small. This still allows the parameterestimate , , to grow without bound.

If time intervals , , can be appro-priately identified such that the corresponding vectors

, , given by (9) are linearly independent and

(15)

where , , are given by (8), then canbe identified exactly by solving the linear equation

(16)

where

...... (17)

In the case where , Fig. 2 shows the ideal weightis identified as the intersection of the two affine hyperplanes

and characterized by the linearly independent normal (toand ) vectors given by and ,

respectively.If the ideal weights can be identified, then no further adapta-

tion is necessary. In this case, we can drive the trajectory ,, to the point satisfying (16) and setting

for all , where , so that the uncer-tainty in (1) is completely canceled by the adaptive signal

for all . This, of course, corresponds to an idealsituation. Although for simple problems it may be possible toidentify the ideal weights using the technique discussed above,for most problems, it is difficult to find vectors ,

, such that the matrix given by (17) is nonsin-gular and well conditioned. Hence, if a batch solution for timeintervals cannot be appropriately identified such that (16) holds,then a moving time window of the integrated system uncertaintycan be used to construct the affine hyperplane (7) containingand drive the update weight trajectory , , to this hy-perplane.

As elucidated above, the -modification technique is basedon a gradient minimization of the cost function defined by (10).However, there are other cost function measures based on theintegral of the system uncertainty that can be used. For example,define the accumulated (or batch) least squares error

(18)

where the notation denotes a function defined by, , at time corresponding to the piece

of the function between and or, equivalently, the elementin the space of continuous functions defined on the interval

and taking values in , where or . The gradientof this cost function with respect to , , is given by

(19)where

(20)

For the statement of the next result define ,, and , , and consider the update

law

(21)

where and . Furthermore, let anddenote the minimum and maximum eigenvalues of a

Hermitian matrix, respectively.Theorem 2.2: Consider the linear uncertain dynamical

system given by (4). The adaptive feedback control law (3)with update law given by (21) guarantees that the solution

of the closed-loop system given by(4) and (21) is Lyapunov stable and as forall and . Moreover, if , , ispersistently excited, that is, there exists such that

(22)



where is the identity matrix and , thenand exponentially as with degree not lessthan

(23)

Proof: To show Lyapunov stability of the closed-loopsystem (4) and (21) consider the Lyapunov function candidategiven by (5). Note that since , , ,

, can be rewritten as , . Hence, thetime derivative of the Lyapunov function candidate (5) alongthe trajectories of the closed-loop system is given by

(24)

Since , , is nonnegative definite, it follows from (24)that for all , which proves Lyapunovstability of the closed-loop system (4) and (21). Hence, ,

, and , , are bounded for all . Furthermore,

since , , and , , are bounded, it follows thatis bounded for all . Now, it follows from

Barbalat’s lemma [25] that as .Next, if , , is persistently excited, then there exists

such that is positive definite for all , that is, thereexists such that , . Hence

(25)

where is given by (23). This proves that and con-verge to zero exponentially as , which completes theproof.

Remark 2.4: It follows from Theorem 2.2 that if ,, is persistently excited, then approaches exponen-

tially, where the point is the optimal solution that would re-sult from a batch solution when time intervals are appropri-ately identified and (16) holds. In the absence of persistency ofexcitation, the update weights converge to the affine hyperplane(7) containing .

Next, we show the efficacy of the -modification technique inaddressing uncertainty cancellation or suppression. Specifically,suppose that the weight estimates satisfy (13) for some

and the vector is parallel to , thatis, there exists such that . Inthis case, the uncertainty is perfectly canceled by theadaptive signal . To see this, note that it follows from (7)that

(26)

which shows uncertainty cancellation.

To show uncertainty suppression, note that since ,, are bounded continuous functions for all , it

follows from the mean value theorem [25] that, for everyand interval , , there exists

such that

(27)Hence, for all and each

(28)

where , or, in vector form

(29)

where .If , , satisfies (13), then

(30)

where denotes the Euclidean vector norm on . Now,if is chosen such that is sufficiently small,then it follows from (30) that can be madesufficiently small regardless of the magnitude of , .Hence, the -modification technique, which ensures that ,

, satisfies (13), guarantees system uncertainty suppression.Finally, note that since

the choice of can be made to depend on the time rate ofchange of , . Hence, if we assume that is atime-varying design parameter, then we can derive an optimalchoice for as a function of the rate of change of ,

. This extension will be considered in a future paper.The -modification technique described above involves the

integration of the system uncertainty. To see this, note that (7)can be rewritten as

(31)

where the integration is performed over a moving time windowof fixed length , . When the system uncer-tainty can be perfectly parameterized as in (2), integration overthe time interval , , can be used instead of integra-tion over a moving time window of fixed length. Since per-fect system uncertainty parametrization eliminates approxima-tion errors, integration over the time interval , ,does not introduce any distortion of the information of unknownweights given by (7). However, in most practical problems,system uncertainty cannot be perfectly parameterized. In this



Fig. 3. Visualization of �-modification with modeling errors.

case, neural networks can be used to approximate uncertainnonlinear continuous functions over a compact domain with abounded error [12].

In particular, let be given by

(32)

where , , is the modeling error such that, , for all , , where is a

compact set. In this case, integration of the system uncertaintyover the time interval gives

(33)

where the term can become very large over time.Hence, (33) cannot be used effectively in the update law (12)with the appropriate modifications. Alternatively, if the systemuncertainty is integrated over a moving time window ,

, then the unknown weights satisfy

(34)

where the term is bounded by . Bychoosing appropriately, one can guarantee that issufficiently small. Note that (34) defines a collection of parallelaffine hyperplanes in , or a boundary layer, where the idealweights lie. Fig. 3 shows such a collection of affine hyper-planes for the case where . Note that in Fig. 3 the widthof the boundary layer, that is, the distance between points and

, is . In the subsequent sections, we consider the caseof nonperfect parametrizations of the system uncertainty andshow how the -modification technique can be used to developstatic and dynamic neuroadaptive controllers using (34).

For illustrative purposes, in this section, we considered asimplified version of an adaptive control problem wherein thesystem uncertainty is a scalar function and the adaptive weightis a vector. Our main goal in this section was to illustrate themain idea of the -modification technique by focusing onthe salient features of the technical details. In the subsequentsections, we develop the -modification technique for generalnonlinear dynamical systems with vector uncertainty structures,nonlinear uncertainty parameterization, and state and outputfeedback neuroadaptive controllers.

III. NEUROADAPTIVE FULL-STATE FEEDBACK CONTROL FOR

NONLINEAR UNCERTAIN DYNAMICAL SYSTEMS

WITH A -MODIFICATION ARCHITECTURE

In this section, we consider the problem of characterizingneuroadaptive full-state feedback control laws for nonlinear un-certain dynamical systems to achieve reference model trajectorytracking. Specifically, consider the controlled nonlinear uncer-tain dynamical system given by

(35)

where , , is the state vector, , , isthe control input,is a vector of -delayed values of the control input withand given, and are knownmatrices, is an unknown positive-definite matrix,

is a known input matrix function such thatfor all , is Lips-

chitz continuous and bounded in a neighborhood of the origin inbut otherwise unknown, and is unknown.

Furthermore, we assume that , , is available for feed-back and the control input in (35) is restricted to the classof admissible controls consisting of measurable functions suchthat , .

In order to achieve trajectory tracking, we construct a refer-ence system given by

(36)

where , , is the reference state vector,, , is a bounded piecewise continuous reference input,

is Hurwitz, and . The goal here is todevelop an adaptive control signal , , that guaranteesthat , , where denotes theEuclidean vector norm on and is sufficiently small.

Consider the control law given by

(37)

where , , and , , are defined below. Usingthe parameterization , where is a knownpositive-definite matrix that can be chosen andis an unknown symmetric matrix such that is positivedefinite, the dynamics in (35) can be rewritten as

(38)

The following matching conditions are needed for the mainresults of this section.

Assumption 3.1: There exist andsuch that and .



Now, let , , in (37) be given by

(39)

In this case, the system dynamics (38) can be rewritten as

Defining the tracking error , , theerror dynamics is given by

(40)

where . We assume that the function canbe approximated over a compact set by a nonlinear inthe parameters neural network up to a desired accuracy. In thiscase, there exists such that, , where , and

(41)where and are optimalunknown (constant) weights that minimize theapproximation error over ,

, , , ,denotes the th row of , ,

, and is the modeling error.Since is continuous on , we can choose

from a linear space of continuous functions thatforms an algebra and separates points in . In this case, itfollows from the Stone–Weierstrass theorem [28, p. 212] thatis a dense subset of the set of continuous functions on .Now, as is the case in the standard neuroadaptive control liter-ature [12], we can construct a signal involving the estimates ofthe optimal weights and basis functions as our adaptive controlsignal. It is important to note here that we assume that we knowboth the structure and the size of the approximator. This is astandard assumption in the neural network adaptive control lit-erature. In online neural network training, the size and the struc-ture of the optimal approximator are not known and are oftenchosen by the rule that the larger the size of the neural networkand the richer the distribution class of the basis functions overa compact domain, the tighter the resulting approximation errorbound . This goes back to the Stone–Weierstrass theoremwhich only provides an existence result without any construc-tive guidelines.

Next, define

(42)

and let , , in (37) be given by

(43)

where , , , ,, , and , , are update weights.

Using (41) and (42), it follows from (43) that the error dynamics(40) can be rewritten as

(44)

where . Define ,, 2, 3, , and , . As it is often done

in the neural network literature, for and, using a Taylor series expansion about

for (see [10] for details) it follows that

(45)

where is the Jacobian ofgiven by

.... . .

...(46)

where , , andas . Since the update laws for

, , and , , will be predicated on the pro-jection operator, it follows that , , and , ,are bounded. Hence, for all and ,there exists such that , where



Using (45), the error dynamics (44) are given by

(47)

Defining

(48)

(49)

and using (48) and (49), the error dynamics (47) can be rewrittenas

(50)

Next, we develop a neuroadaptive control architecture whichinvolves additional terms in the update laws that are predicatedon auxiliary terms involving an estimate of the unknown weights

, , , and . In particular, by integrating the error dy-namics (47) over the moving time interval , where

and , we obtain

(51)

where we have (52)–(55) shown at the bottom of the page. Notethat , , and are computable, whereas

is an unknown integrated modeling error such that, where is such

that

where is the matrix norm induced by thevector norms and .

For the statement of next result, define the projection operatorgiven by

ifif and

otherwise

where , ,, is the norm bound

imposed on , and , and denotes the Fréchetderivative. Note that for a given matrix and

, it follows that

where denotes the th column of the matrix .Next, we choose such that and

for all . Define

(56)

(57)

(52)

(53)

(54)

(55)



and note that it follows from (51) that

(58)

Now, using (58), it follows that, for every and, we have (59) shown at the bottom of the page, where

is the norm bound imposed on , . Next, define the-modification term by

(60)

where for , , ,, , and .

Consider the feedback control law (37) with andgiven by (39) and (43), and update laws given by (61)–(64)shown at the bottom of the page, where , ,

, and are positive-definite matrices,is the positive-definite solution of the Lyapunov

equation

(65)

, andare given by (48) and (49), respectively, ,

, , and are given by (60),, and

is a bounded nonnega-tive function taking values between 0 and 1 such thatif , for , or , or

, then , where ,, and are the norm bounds imposed on ,, , and , , respectively.

Theorem 3.1: Consider the nonlinear uncertain dynamicalsystem given by (35) with , , given by (37) andand given by (39) and (43), respectively, and reference

model given by (36) with the tracking error dynamics given by(50). Assume Assumption 3.1 holds. Then, there exists a com-pact positively invariant set

such that , where, , , and

, and the solution , ,of the closed-loop system given by (50) and (61)–(64) is ulti-mately bounded for all

with ultimate bound , , where

(66)

(67)

(68)

, , and , are norm bounds imposed onand , respectively, and is the positive-definite

solution of the Lyapunov equation (65).Proof: To show ultimate boundedness of the closed-loop

system (50), (37), (39), (43), (61)–(64), and (60) consider theLyapunov-like function

(69)

where satisfies (65). Note that (69)satisfies with

and, where

and denotes the column stacking operator. Furthermore,note that is a class function. Now, letting ,

, denote the solution to (50) and using (61)–(64), it follows thatthe time derivative of along the closed-

(59)

(61)

(62)

(63)

(64)



loop system trajectories is given by (70) shown at the bottom ofthe page.

Next, using (56), (57), and (59), the time derivative ofalong the closed-loop system trajectories

satisfies

(71)

Now, for , whereand are given by (67) and (68), it follows that

for all ,that is, for all

and , where

Finally, define

where is the maximum value such that , and define

where

(72)

To show ultimate boundedness of the closed-loop system(50) and (61)–(64) assume1 that . Now, since

for alland , it follows that is positively in-

variant. Hence, if ,then it follows from [25, Corollary 4.4] that the so-lution , , to(50) and (61)–(64) is ultimately bounded with respectto with ultimate bound given by

, which yields (66). This completesthe proof.

Remark 3.1: Note that since , , and , ,are bounded, it follows that , , is bounded, and hence,

, , given by (39) is bounded. Furthermore, sinceis bounded for all , it is always possible to choose

and so that exists and is bounded forall . This follows from the fact that for any two square ma-trices and , if and only if there existssuch that and . Hence, it follows

1This assumption is standard in the neural network literature and ensures thatin the error space �� there exists at least one Lyapunov level set �� .In the case where the neural network approximation holds in with delayedvalues, this assumption is automatically satisfied.

(70)



that for and , ,exists for all if is sufficiently small. Hence, theadaptive signal , , given by (43) is bounded. Since

, , and , are bounded, andfor all , it follows that control input , , givenby (37) is bounded for all .

Remark 3.2: It is straightforward to show that the -mod-ification framework can be incorporated within a radial basisfunction neural-network-based adaptive controller and com-bined with the robust adaptive control laws discussed in [20],such as - or -modifications.

Remark 3.3: Note that the -modification terms in the up-date laws (61)–(64) drive the trajectories of the neural networkweights to a collection of affine hyperplanes characterized by(51) involving the unknown neural network weights.

IV. OUTPUT FEEDBACK CONTROL FOR NONLINEAR

UNCERTAIN DYNAMICAL SYSTEMS WITH A

-MODIFICATION ARCHITECTURE

In this section, we consider the problem of characterizingneuroadaptive dynamic output feedback control laws for non-linear uncertain dynamical systems to achieve reference modeltrajectory tracking. Specifically, consider the controlled non-linear uncertain dynamical system given by

(73)

(74)

where , , is the state vector, ,, is the control input, , , is the system

output, is avector of -delayed values of the control input with and

given,is a vector of -delayed values of the system output withand given, , , andare known matrices with Hurwitz, is an un-known positive-definite matrix, is Lips-chitz continuous and bounded in a neighborhood of the origin in

but otherwise unknown, is an unknownmatrix, and is a known bounded Lips-chitz continuous function. Furthermore, we assume that the con-trol input in (73) is restricted to the class of admissible con-trols consisting of measurable functions such that ,

.In order to achieve trajectory tracking, we construct a refer-

ence system given by

(75)

(76)

where , , is the reference state vector,, , is a bounded piecewise continuous reference input,

, , is the reference output,is Hurwitz, and . The goal of the controller de-sign is to develop an adaptive control signal , , predi-cated on the system measurement , , such that

, for all , where and issufficiently small.

The following matching conditions are needed for the mainresults of this section.

Assumption 4.1: There exist andsuch that and .

Consider the control law given by

(77)

where is positive-definite matrix, and , ,and , , are defined below. Using the parameteriza-tion , where is an unknown symmetricmatrix, the dynamics in (73) can be rewritten as

(78)

Now, let , , in (77) be given by

(79)

In this case, using Assumption 4.1, the system dynamics (78)can be rewritten as

(80)

Defining the tracking error , , theerror dynamics is given by

(81)

where .As in Section III, we approximate the unknown function

, , by a nonlinear in the parametersneural network. In particular, we assume that the function

can be approximated over a compact set bya nonlinear in the parameters neural network up to a desiredaccuracy. In this case, (41) holds. In order to develop an outputfeedback neural network, we use the approach developed in[29] for reconstructing the system states via the system delayedinputs and outputs. Specifically, we use a memory unit as aparticular form of a tapped delay line that takes a scalar timeseries input and provides an -dimensional vectoroutput consisting of the present values of the system outputsand system inputs, and their delayed valuesgiven by

(82)

where denotes the relative degree of with respect to theoutput , , is the (vector)relative degree of , and .



Analogous to (43), consider the adaptive signal , ,given by

(83)

where , , , ,, , and , , are update weights,

and is continuous and bounded on, where is a compact set. Furthermore, define

and .Using (41), it follows from (83) that the error dynamics (81)

can be rewritten as

(84)

where . Define ,, , , , and

, . As in Section III, for and, using a Taylor series expansion about

for (see [10] for details) it follows that

(85)

where is the Jacobian ofgiven by

.... . .

...(86)

where , . Sincethe update laws for , , and , , will bepredicated on the projection operator, it follows that ,

, and , , are bounded. Hence, for all and

, there exists such that, where

Using (85), the error dynamics (84) are given by

(87)

Defining

(88)

(89)

and using (88) and (89), the error dynamics (87) can be rewrittenas

(90)

where as long as, where .

In order to develop an output feedback neural network, con-sider the estimator given by

(91)

(92)

where , , is such that isHurwitz, and define . It follows from (74)and (76) that

(93)

Premultiplying (90) by and integrating the resulting equationover the moving time interval , whereand , and using (93), we obtain

(94)



where we have (95)–(99) shown at the bottom of the page, and. Note that , , and are

computable, whereas is an unknown term such that

where is such that for all and, and is such that

As in Section III, we choose such thatand for all . Define

(100)

(101)

and note that it follows from (94) that

(102)

Now, using (102), it follows that, for every and, we have (103) shown at the bottom of the page, where

and is the norm bound imposed on , . Next,define the -modification term by

(104)

where for , , ,, , and .

Consider the feedback control law (77) with andgiven by (79) and (83), and update laws given by (105)–(108)shown at the bottom of the next page, where ,

, , and are positive-definite ma-trices, is the positive-definite solution to (65), and

is the positive-definite solution to the Lyapunovequation

(109)

(95)

(96)

(97)

(98)

(99)

(103)



where , , andare given by (88) and (89),

respectively, , , , and aregiven by (104), ,and is a bounded nonnega-tive function taking values between 0 and 1 such thatif , for , or , or

, then , where ,, or , and are the norm bounds imposed on

, , or , , and , , respectively.Note that projection operator guarantees boundness of ,

, , , , and , . In addition,we choose the parameters and such that the inverse

in (83) exists for all . In particular,there exists such that

(110)

Next, we introduce several bounds needed to formu-late the main result of this section. Since and

are bounded for all , there existand such that , , and

, . Hence, there exist andsuch that , , and

, . Furthermore,there exist and such that ,

, and , . It follows from (77), (79), and(83) that there exist and such that

(111)

(112)

(113)

Finally, define

(114)

(115)

(116)

(117)

Theorem 4.1: Consider the nonlinear uncertain dynam-ical system given by (73) and (74) with , ,given by (77). Assume Assumption 4.1 holds, , and

. Then, there exists a compact positively invariantsetsuch that and the solu-tion , ,of the closed-loop system given by (73), (74), (91),(92), and (105)–(108), is ultimately bounded for all

with ul-timate bound , , where

(118)

(119)

Proof: Ultimate boundness can be established by consid-ering the Lyapunov-like function

The remainder of the proof is similar to the proof of Theorem3.1 and, hence, is omitted.

V. ILLUSTRATIVE NUMERICAL EXAMPLE

In this section, we present a numerical example to demon-strate the utility and efficacy of the proposed -modification ar-chitecture for neuroadaptive stabilization. Specifically, considerthe nonlinear dynamical system representing a controlled rigidspacecraft given by

(120)

where represents the angular velocities ofthe spacecraft with respect to the body-fixed frame,is an unknown positive-definite inertia matrix of the spacecraft,

is a control vector with control inputs pro-viding body-fixed torques about three mutually perpendicularaxes defining the body-fixed frame of the spacecraft, anddenotes the skew-symmetric matrix

(105)

(106)

(107)

(108)



Fig. 4. Angular velocities and control signals versus time.

Note that (120) can be rewritten in state-space form (35) with, , , ,

, and .Next, we use Theorem 3.1 to design a neuroadaptive con-

troller given by (37) with and given by (39) and(43), respectively. Here, we used six nodesin the outer layer and three nodes in thehidden layer of the neural network. For our simulation, we used

, , ,,

, , ,, , , , ,

, , , ,, , and .

With the above data, Assumption 3.1 holds with

Now, with

(121)

and initial conditions , Fig. 4 showsthe controlled angular velocities and the control signals versustime with and without the -modification architecture. It is clearfrom these simulations that the -modification neuroadaptivecontroller achieves superior performance over a standard neu-roadaptive controller. Finally, we compare the -modificationcontroller with the - and -modification schemes. For the -and -modification schemes, we modified the standard neuralnetwork update laws to include the terms and

, respectively, where is the state error. As ex-pected, the performance of the - and -modification controllersdepend on the design parameter . Assuming that the actualweights are unknown, here we let and ,and construct as a random matrix with entries cor-responding to a Gaussian distribution with zero mean and vari-ance 0.01. In this case,

With , Fig. 5 shows the angular velocities and the con-trol signals versus time of the two approaches. Though the -and -modification controllers can give better performance thanthe standard neural network controller, the -modification con-troller achieves superior performance as compared to all threedesigns.

VI. CONCLUSION

This paper presents a new and novel adaptive and neuroad-aptive controller architecture for nonlinear uncertain dynamical



Fig. 5. Angular velocities and control signals versus time with �- and �-modification controllers.

systems. This architecture goes beyond the - and -modifica-tion controller architectures by providing fast adaptation to ef-fectively suppress system uncertainty. In addition, this new con-troller architecture avoids large parameter (high gain) stabiliza-tion which can excite unmodeled dynamics and drive the systemto instability. Future work will focus on discrete-time and sam-pled-data extensions of the proposed architecture.

REFERENCES

[1] K. S. Narendra and K. Parthasarathy, “Identification and control of dy-namical systems using neural networks,” IEEE Trans. Neural Netw.,vol. 1, no. 1, pp. 4–27, Mar. 1990.

[2] J. Park and I. Sandberg, “Universal approximation using radial basisfunction networks,” Neural Comput., vol. 3, pp. 246–257, 1991.

[3] F. C. Chen and H. K. Khalil, “Adaptive control of nonlinear systemsusing neural networks,” Int. J. Control, vol. 55, no. 6, pp. 1299–1317,1992.

[4] M. Polycarpou and P. Ioannou, “Modeling, identification, and stableadaptive control of continuous-time nonlinear dynamical systems usingneural networks,” in Proc. Amer. Control Conf., Jun. 1992, pp. 36–40.

[5] K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop, “Neuralnetworks for control: A survey,” Automatica, vol. 28, pp. 1083–1112,1992.

[6] R. Sanner and J. Slotine, “Gaussian networks for direct adaptive con-trol,” IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 837–864, Nov. 1992.

[7] A. U. Levin and K. S. Narendra, “Control of nonlinear dynamical sys-tems using neural networks: Controllability and stabilization,” IEEETrans. Neural Netw., vol. 4, no. 2, pp. 192–206, Mar. 1993.

[8] M. Polycarpou, “Stable adaptive neural control scheme for nonlinearsystems,” IEEE Trans. Autom. Control, vol. 41, no. 3, pp. 447–451,Mar. 1996.

[9] S. Fabri and V. Kadirkamanathan, “Dynamic structure neural networksfor stable adaptive control of nonlinear systems,” IEEE Trans. NeuralNetw., vol. 7, no. 5, pp. 1151–1166, Sep. 1996.

[10] F. L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural-net robotcontroller with guaranteed tracking performance,” IEEE Trans. NeuralNetw., vol. 7, no. 2, pp. 388–399, Mar. 1996.

[11] B. S. Kim and A. J. Calise, “Nonlinear flight control using neural net-works,” AIAA J. Guid. Control Dyn., vol. 20, no. 1, pp. 26–33, 1997.

[12] F. L. Lewis, S. Jagannathan, and A. Yesildirak, Neural Network Controlof Robot Manipulators and Nonlinear Systems. London, U.K.: Taylor& Francis, 1999.

[13] S. Seshagiri and H. Khalil, “Output feedback control of nonlinear sys-tems using RBF neural networks,” IEEE Trans. Neural Netw., vol. 11,no. 1, pp. 69–79, Jan. 2000.

[14] J. Y. Choi and J. A. Farrell, “Adaptive observer backstepping controlusing neural networks,” IEEE Trans. Neural Netw., vol. 12, no. 5, pp.1101–1112, Sep. 2001.

[15] A. J. Calise, N. Hovakimyan, and M. Idan, “Adaptive output feedbackcontrol of nonlinear systems using neural networks,” Automatica, vol.37, no. 8, pp. 1201–1211, 2001.

[16] J. Spooner, M. Maggiore, R. Ordonez, and K. Passino, Stable AdaptiveControl and Estimation for Nonlinear Systems: Neural and Fuzzy Ap-proximator Techniques. New York: Wiley, 2002.

[17] S. S. Ge and C. Wang, “Adaptive neural control of uncertain MIMOnonlinear systems,” IEEE Trans. Neural Netw., vol. 15, no. 3, pp.674–692, May 2004.

[18] K. J.Åström and B. Wittenmark, Adaptive Control. Reading, MA:Addison-Wesley, 1989.

[19] K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems.Englewood Cliffs, NJ: Prentice-Hall, 1989.

[20] P. A. Ioannou and J. Sun, Robust Adaptive Control. Upper SaddleRiver, NJ: Prentice-Hall, 1996.

[21] T. Hayakawa, W. M. Haddad, and N. Hovakimyan, “Neural networkadaptive control for nonlinear uncertain dynamical systems withasymptotic stability guarantees,” IEEE Trans. Neural Netw., vol. 19,no. 1, pp. 80–89, Jan. 2008.

[22] J.-J. E. Slotine and W. Li, Applied Nonlinear Control. EnglewoodCliffs, NJ: Prentice-Hall, 1991.

[23] K. Y. Volyanskyy, A. J. Calise, and B.-J. Yang, “A novel �-modifi-cation term for adaptive control,” in Proc. Amer. Control Conf., Min-neapolis, MN, Jun. 2006, pp. 4072–4076.

[24] K. Y. Volyanskyy, A. J. Calise, B.-J. Yang, and E. Lavretsky, “An errorminimization method in adaptive control,” in Proc. AIAA Guid. Nav-igat. Control Conf., Keystone, CO, Aug. 2006, pp. 1–9.

[25] W. M. Haddad and V. Chellaboina, Nonlinear Dynamical Systemsand Control: A Lyapunov-Based Approach. Princeton, NJ: PrincetonUniv. Press, 2008.

[26] J.-J. E. Slotine and W. Li, “Composite adaptive control of robot ma-nipulators,” Automatica, vol. 25, no. 4, pp. 508–519, 1989.



[27] M. M. Duarte and K. S. Narendra, “Combined direct and indirect ap-proach to adaptive control,” IEEE Trans. Autom. Control, vol. 34, no.10, pp. 1071–1075, Oct. 1989.

[28] H. L. Royden, Real Analysis. New York: Macmillan, 1988.[29] E. Lavretsky, N. Hovakimyan, and A. J. Calise, “Upper bounds for ap-

proximation of continuous-time dynamics using delayed outputs andfeedforward neural networks,” IEEE Trans. Autom. Control, vol. 48,no. 9, pp. 1606–1610, Sep. 2003.

Kostyantyn Y. Volyanskyy (S’07) received theB.S. and M.S. degrees in applied mathematics fromKiev National Taras Shevchenko University, Kiev,Ukraine, in 1998 and 1999, respectively. He is cur-rently working towards the Ph.D. degree in aerospaceengineering at the School of Aerospace Engineering,Georgia Institute of Technology, Atlanta.

His research interests include nonlinear adaptivecontrol and estimation, neural networks and intelli-gent control, nonlinear analysis and control for bio-logical and physiological systems, and active control

for clinical pharmacology.

Wassim M. Haddad (S’87–M’87–SM’01–F’09)received the B.S., M.S., and Ph.D. degrees inmechanical engineering from Florida Institute ofTechnology, Melbourne, in 1983, 1984, and 1987,respectively, with specialization in dynamical sys-tems and control.

From 1987 to 1994, he served as a consultant forthe Structural Controls Group of the GovernmentAerospace Systems Division, Harris Corporation,Melbourne, FL. In 1988, he joined the faculty of theMechanical and Aerospace Engineering Department,

Florida Institute of Technology, where he founded and developed the Systemsand Control Option within the graduate program. Since 1994, he has been amember of the faculty in the School of Aerospace Engineering, Georgia Insti-tute of Technology, Atlanta, where he currently holds the rank of Professor. Hisresearch contributions in linear and nonlinear dynamical systems and controlare documented in over 490 archival journal and conference publications. Heis a coauthor of the books Hierarchical Nonlinear Switching Control Design

with Applications to Propulsion Systems (New York: Springer-Verlag, 2000),Thermodynamics: A Dynamical Systems Approach (Princeton, NJ: PrincetonUniv. Press, 2005), Impulsive and Hybrid Dynamical Systems: Stability, Dissi-pativity, and Control (Princeton, NJ: Princeton Univ. Press, 2006), NonlinearDynamical Systems and Control: A Lyapunov-Based Approach (Princeton, NJ:Princeton Univ. Press, 2008), and Nonnegative and Compartmental DynamicalSystems (Princeton, NJ: Princeton Univ. Press, 2009). His recent research isconcentrated on nonlinear robust and adaptive control, nonlinear dynamicalsystem theory, large scale systems, hierarchical nonlinear switching control,analysis and control of nonlinear impulsive and hybrid systems, adaptive andneuroadaptive control, system thermodynamics, thermodynamic modeling ofmechanical and aerospace systems, network systems, nonlinear analysis andcontrol for biological and physiological systems, and active control for clinicalpharmacology.

Dr. Haddad is a National Science Foundation (NSF) Presidential FacultyFellow and a member of the Academy of Nonlinear Sciences.

Anthony J. Calise (S’63–M’74–SM’02) receivedthe B.S. degree in electrical engineering from Vil-lanova University, Villanova, PA, in 1964, and theM.S. and Ph.D. degrees in electrical engineeringfrom University of Pennsylvania, in 1966 and 1968,respectively.

He is a Professor at the School of AerospaceEngineering, Georgia Institute of Technology(Georgia Tech), Atlanta. Prior to joining the facultyat Georgia Tech, he was a Professor of MechanicalEngineering at Drexel University, Philadelphia, PA,

for eight years. He also worked for ten years in industry for the RaytheonMissile Systems Division and Dynamics Research Corporation, where hewas involved with analysis and design of inertial navigation systems, optimalmissile guidance, and aircraft flight path optimization. He is the author ofover 250 technical reports and papers. In the area of adaptive control, he hasdeveloped a novel combination for employing neural network-based control incombination with feedback linearization.

Dr. Calise is a Fellow of the American Institute of Aeronautics and Astro-nautics (AIAA). He is a former Associate Editor for the Journal of Guidance,Control, and Dynamics. He was the recipient of the USAF Systems CommandTechnical Achievement Award and the AIAA Mechanics and Control of FlightAward.


IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 11 ... · The authors are with the School of...

Documents

Transcript of IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 11 ... · The authors are with the School of...