OPTIMAL FEEDBACK CONTROL CONTROL OF ...amadi/thesis.pdfdfkgjsklg sdfg dfsgdf dfgsdfg sfvgsdfg...

OPTIMAL FEEDBACK CONTROL

CONTROL OF CONSTRAINED

DISCRETE-TIME SYSTEMS USING

GENERAL COST FUNCTIONS

by

S. S. Keerthi

A dissertation submitted in partial fulfillmentof the requirements for the degree of

Doctor of Philosophy(Computer Information and Control Engineering)

in The University of Michigan1986

Doctoral Committee:

Professor E.G. Gilbert, ChairpersonProfessor J. BeanAssistant Professor P. KabambaProfessor N.H. Mc ClamrochProfessor K.G. Murty

c© S. S. Keerthi For A Long Long Name 2017All Rights Reserved

To all those who helped me

ii

ACKNOWLEDGEMENTS

I wish to thank all those who helped me in completing this thesis.

iii

PREFACE

This thesis deals with the existence of optimal solutions for general, discrete-time

optimal control problems.

iv

TABLE OF CONTENTS

DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . iii

PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

LIST OF FLOWCHARTS . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

LIST OF MAPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

LIST OF APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

CHAPTER

I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

II. EXISTENCE THEOREMS . . . . . . . . . . . . . . . . . . . . . 4

2.1 The Existence Theorem . . . . . . . . . . . . . . . . . . . . . 42.2 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 An Extension in Hilbert Spaces . . . . . . . . . . . . . . . . . 10

III. STABILITY OF OPTIMAL FEEDBACK LAWS . . . . . . . 12

3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . 133.2 Preliminaries on Stability . . . . . . . . . . . . . . . . . . . . 163.3 Properties C and O . . . . . . . . . . . . . . . . . . . . . . . 183.4 Stability of the Infinite-Horizon Feedback System . . . . . . . 223.5 Stability of the Moving-Horizon System . . . . . . . . . . . . 263.6 Convergence of the Moving-Horizon Cost . . . . . . . . . . . 273.7 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.8 Stability of Delayed Feedback Laws . . . . . . . . . . . . . . . 34

v

3.9 Regulation Using Nonsummation Indices . . . . . . . . . . . . 40

IV. ILLUSTRATIVE EXAMPLES . . . . . . . . . . . . . . . . . . . 44

4.1 Comparison of Performance Indices . . . . . . . . . . . . . . . 444.2 Need for Constraints . . . . . . . . . . . . . . . . . . . . . . . 474.3 The Overhead Crane Example . . . . . . . . . . . . . . . . . 48

APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

vi

LIST OF FIGURES

Figure

4.1 4.1.1. Optimal Poles for the First Order Example. . . . . . . . . . . 46

4.2 4.2.1. Responses for the Saturation and MH Feedback Laws. . . . . 49

4.3 4.3.1. Solutions of the UC-MH-l1 Problems. . . . . . . . . . . . . . 51

4.4 4.3.2. Solutions of the UC-MH-l2 Problems. . . . . . . . . . . . . . 52

4.5 4.3.3. Control Histories for UC-IH-l1 and UC-IH-l2 Solutions. . . . . 54

4.6 4.3.4. Solutions of the CC-IH-l1 and CC-IH-l2 problems. . . . . . . . 55

4.7 4.3.5. Control Histories for CC-IH-l1 and CC-IH-l2 solutions. . . . . 56

B.1 A Diagramatic Sketch of the Overhead Crane. . . . . . . . . . . . . 62

vii

LIST OF TABLES

Table

4.1 4.3.1. Properties of Moving-Horizon Solutions. . . . . . . . . . . . . 53

4.2 4.3.2. Largest Magnitudes of Variables. . . . . . . . . . . . . . . . . 53

A.1 A Test Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

viii

LIST OF FLOWCHARTS

FLOWCHARTS

1.1 A flowchart of Thisnthat. . . . . . . . . . . . . . . . . . . . . . . . . 2

ix

LIST OF MAPS

Map

1.1 A brilliant map of Australia. . . . . . . . . . . . . . . . . . . . . . . 2

x

LIST OF APPENDICES

Appendix

A. Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

B. Model of an Overhead Crane . . . . . . . . . . . . . . . . . . . . . . . 61

xi

CHAPTER I

INTRODUCTION

Actually this introduction was done by Doug Maus.

sfdgjklj dskfgjkls sfkglgjskldfgj dfszfg dkfgljsl dfkgjsklg sdfg dfsgdf dfgsdfg sfvgs-

dfg dfsgsdgf dfsgsdf fgsfdgs dsfgdf sfdgjklj dskfgjkls sfkglgjskldfgj dfszfg dkfgljsl

dfkgjsklg sdfg dfsgdf dfgsdfg sfvgsdfg dfsgsdgf dfsgsdf fgsfdgs dsfgdf














1

2

This is supposed to be a map.

Map 1.1: A brilliant map of Australia.

This is supposed to be an “anything” — what I call a flowchart.

Flowchart 1.1: A flowchart of Thisnthat.

3
























CHAPTER II

EXISTENCE THEOREMS

In this chapter we give conditions which guarantee the existence of a solution

for a general class of deterministic, infinite-horizon, discrete-time optimal control

problems. Except for special problems such as the linear quadratic control problem

[Pay.1], most existence theorems available in the literature [Ber.1], [Hin.1], [Las.1],

[Sch.1] are for stochastic problems. Some of these theorems need restrictive assump-

tions and others state hypotheses which are not easily verified. Our existence theo-

rem treats the deterministic problem directly and involves rather weak, easily verified

conditions. Some of the concepts used in our approach parallel those used by Dolezal

[Dol.1], who has given a general existence theorem for finite horizon problems.

Compactness conditions appear in the statement of the existence theorem (The-

orem 2.1.1). In a special Hilbert space setting, they may be replaced by weak com-

pactness, extending the results in [Lee.1]. This is discussed in section 2.3.

2.1 The Existence Theorem

The following notations will be used. (Z, d) denotes the metric space Z with

metric d. N and N denote, respectively, the set of positive and nonnegative integers.

4

5

Consider the discrete-time system described by

xk+1 = fk(xk, uk), x0 ∈ T0,

(xk, uk) ∈ Wk ⊂ (X, dX)× (U, dU), k ∈ N .(2.1)

A sequence pair (xk, uk)k∈N is said to be admissible if it satisfies (2.1.1). Problems

A and B consist of minimizing, respectively, the costs

JA =∑k∈N

hk(xk, uk) (2.2)

or

JB = supk∈N

hk(xk, uk) (2.3)

over the class of all admissible sequence pairs. We now state the existence theorem.

Theorem 2.1.1. Assume T0 ⊂ X is compact and for each k ∈ N the following

conditions hold: a) Wk is a closed subset of X × U ; b) fk : Wk → X is continuous;

c) hk : Wk → R is lower semicontinuous and nonnegative; d) given any compact set

P ⊂ X, the set Sk(P ) = (x, u) ∈ Wk : x ∈ P is compact in X × U . Also assume:

e) there exists an admissible sequence pair with a finite cost JA(JB). Then problem

A (B) has a solution.

By setting hK(x, u) = H(x),WK = TK ×U , and hk ≡ 0,Wk = X×U, k > K, the

infinite-horizon problems become finite-horizon problems. Thus, Theorem 2.1.1 also

applies to finite-horizon problems. In fact, our result is then very similar to the one

of Dolezal [Dol.1]. Our theorem is stronger in that it allows consideration of problem

B and has a more general formulation for the constraint sets Wk; Dolezal’s theorem is

stronger in that it applies when X and U are abstract Hausdorff topological spaces.

The proof of Theorem 2.1.1 is given in section 2.2. Although the spaces X and U are

general metric spaces, the case of greatest practical interest is X = Rn and U = Rm.

However, the proof in the general formulation is no more difficult and shows that

6

the topological concepts needed to state and prove the existence theorem are quite

elementary.

The hypotheses in Theorem 2.1.1 are not necessarily in a form which allows easy

verification in applications. Some useful conditions which may replace conditions c)

and d) are indicated in the following theorem and remark.

Theorem 2.1.2. Theorem 2.1.1 also holds if, for each k ∈ N , condition d) is

replaced by any one of the following conditions: d1) the expression (x, u) ∈ Wk can

be written as u ∈ Gk(x) where Gk(·) is a set valued function that maps elements of X

in to compact subsets of U , and is upper semicontinuous by set inclusion; d2) there

exists a compact set Ωk ⊂ U such that u ∈ U : (x, u) ∈ Wk ⊂ Ωk; d3) (U, dU) is

linear and finite dimensional and there is a function φk : U → R which satisfies the

following conditions: 1) φk(u) → ∞ whenever dU(u, 0) → ∞; 2) hk(x, u) ≥ φk(u)

whenever (x, u) ∈ Wk.

Remark 2.1.1. Adding a finite number α, to JA or JB will not change the

existence theorem. Thus, condition c) may be replaced by the following (weaker)

condition: c′) hk : Wk → R is lower semicontinuous and for problem A(B) there is

a sequence of real numbers αkk∈N such that α =∑k∈N αk(α = infk∈N αk) is finite

and hk(x, u) ≥ αk whenever (x, u) ∈ Wk.

An important application of c′) is to discounted cost functions [Ber.1], i.e., hk(x, u) =

δkhk(x, u) where 0 ≤ δ < 1 and gk is uniformly bounded on Wk for all k ∈ N .

2.2 Proofs

The essence of Dolezal’s proof is to show that the hypotheses lead to the min-

imization of a lower semicontinuous function on a compact subset of a Hausdorff

topological space. Existence then follows from a classical theorem. However, in

7

the infinite-horizon case Dolezal’s reduction does not work and a more elaborate

argument based on the following lemmas is needed.

Lemma 2.2.1. Let zk(i)k∈N,i∈N be a doubly infinite array whose elements

are in a metric space (Z, d). Also, let Zkk∈N be a sequence of compact sets

Zk ⊂ Z such that for each k ∈ N , zk(i) ∈ Zk for all i ∈ N . Then there exists an

increasing subsequence itt∈N ⊂ N , and a sequence zkk∈N such that for each

k ∈ N , limt→∞

zk(it) = zk ∈ Zk.

Proof : First consider the sequence z0(i)i∈N . Since this sequence lies in the

compact set Z0, there is an increasing subsequence i0,tt∈N ⊂ N , and z0 ∈ Z0 such

that limt→∞

z0(i0,t) = z0. Now, the subsequence z1(i0,t)t∈N lies in the compact set Z1.

Hence, there is a further increasing subsequence i1,tt∈N ⊂ i0,tt∈N , and z1 ∈ Z1 so

that limt→∞

z1(i1,t) = z1. This process is repeated inductively, and we have a sequence

of subsequences i0,tt∈N ⊃ i1,tt∈N ⊃ · · ·, forming a doubly infinite array whose

elements are in N . Out of this array, choose only the diagonal elements to form a

new subsequence itt∈N . It is then easy to verify the lemma.

Lemma 2.2.2. Suppose assumptions a), b), and d) of Theorem 2.1.1 hold.

Then it is possible to construct a sequence of sets Vkk∈N such that, for all k ∈ N :

1) Vk ⊂ Wk; 2) Vk is compact; 3) for any admissible sequence pair, (xk, uk) ∈ Vk.

Proof : Similar to [Dol.1], we use induction. Begin with T0 ⊂ X and define

V0 = (x, u) ∈ W0 : x ∈ T0. (2.4)

By the compactness of T0 and assumption d), V0 is a compact subset of W0. Next,

consider compact sets Tk ⊂ X and Vk ⊂ Wk. Define

Tk+1 = x : x = fk(y, u), (y, u) ∈ Vk ⊂ X, (2.5)

Vk+1 = (x, u) ∈ Wk+1 : x ∈ Tk+1 ⊂ Wk+1. (2.6)

8

By assumptions b) and d), it is easily confirmed that Tk+1 and Vk+1 are compact.

Thus, Vkk∈N satisfies 1) and 2). Property 3) is obvious from (2.2.2) and (2.2.3).

We now prove Theorem 2.1.1. Let IA denote the infimum of the cost function

JA among the class of all admissible sequence pairs. By assumptions c) and e), IA

exists and 0 ≤ IA < ∞. Thus, there exists a sequence of admissible sequence pairs

(xk(i), uk(i))k∈Ni∈N such that

IA ≤∑k∈N

hk(xk(i), uk(i)) ≤ IA + 1/i, i ∈ N. (2.2.4)

For each i ∈ N , define yk(i)k∈N by

y0(i) = 0, yk(i) =k−1∑j=0

hj(xj(i), uj(i)), k ∈ N. (2.2.5)

From (2.2.4),(2.2.5), and assumption c),

0 ≤ yk(i) ≤ IA + 1/i, k ∈ N , i ∈ N. (2.2.6)

Let Vkk∈N be the sequence of compact subsets of X × U constructed in Lemma

2.2.2. Then, for each k ∈ N , (yk(i), xk(i), uk(i))i∈N ⊂ [0, IA + 1] × Vk, a compact

subset of R × X × U . By Lemma 2.2.1, there is a subsequence itt∈N ⊂ N and a

sequence (yk, xk, uk)k∈N such that

limt→∞

(yk(it), xk(it), uk(it)) = (yk, xk, uk), (2.2.7)

(yk, xk, uk) ∈ [0, IA + 1]× Vk ⊂ [0, IA + 1]×Wk, k ∈ N . (2.2.8)

Also, by the continuity of fk and the compactness of T0,

xk+1 = limt→∞

xk+1(it) = limt→∞

fk(xk(it), uk(it))

= fk(xk, uk), k ∈ N , (2.2.9)

x0 = ( limt→∞

x0(it)) ∈ T0. (2.2.10)

9

Therefore, (xk, uk)k∈N is an admissible sequence pair. We will now show that it is

a solution for problem A.

First (2.2.6) and (2.2.7) imply that

yk ≤ IA, k ∈ N . (2.2.11)

Now let ykk∈N be generated by

y0 = 0, yk =k−1∑j=0

hj(xj, uj), k ∈ N. (2.2.12)

By assymption c), ykk∈N is a nonnegative, nondecreasing sequence of real numbers.

Also by (2.2.5), (2.2.7), (2.2.11), (2.2.12) and the lower semicontinuity of hj,

0 ≤ yk ≤k−1∑j=0

limt→∞

hj(xj(it), uj(it))

≤ limt→∞

yk(it) = limt→∞

yk(it) = yk ≤ IA. (2.2.13)

Hence, limk→∞

yk exists and lies in the interval [0, IA]. Note that the limit is also JA,

the cost given by the admissible sequence pair (xk, uk)k∈N . Hence JA ≥ IA and

the proof is complete for problem A. The detailed proof for problem B is similar and

will not be given. It is only necessary to give different definitions for yk and yk(i).

For example,

y0 = 0, yk+1 = max(yk, hk(xk, uk)), k ∈ N . (2.2.14)

We now prove Theorem 2.1.2. A proof that condition d1) implies d) is given in

[Dol.1, Prop. 6, p.305 ]. We now prove that d2) implies d). Let P ⊂ X be a compact

set and Sk(P ) = (x, u) ∈ Wk : x ∈ P ⊂ P × Ωk, a compact set. Then Sk(P )

is compact since it is closed. Thus, d) is satisfied. Finally, we prove the result of

Theorem 2.1.1 for d3). Because of condition e) of Theorem 2.1.1, it is sufficient to

consider only those admissible sequence pairs with cost JA(JB) ≤ JA(JB). Since

10

φk(u) → ∞ whenever dU(u, 0) → ∞, there exists µk ≥ 0 such that dU(u, 0) ≤ µk

whenever φk(u) ≤ JA(JB). Now let Ωk = u : dU(u, 0) ≤ µk and use condition d2)

to complete the proof.

2.3 An Extension in Hilbert Spaces

For convex linear optimal control problems in Hilbert spaces, we can replace the

compactness and closedness assumptions by their weak counterparts.

Theorem 2.3.1. Suppose X,U are Hilbert spaces (with inner products 〈., .〉X ,

〈., .〉U), T0 ⊂ X is weakly compact and for each k ∈ N the following hold: a) Wk is

weakly closed; b) fk(x, u) = Lk(x, u) + qk where Lk is a linear bounded operator and

qk ∈ X; c) hk is a continuous, convex, nonnegative functional; d) given any weakly

compact set P ⊂ X,Sk(P ) = (x, u) ∈ Wk : x ∈ P is weakly compact. Also assume:

e) there is an admissible sequence pair with a finite cost. Then problem A (B) has a

solution. The existence result still holds if, for each k ∈ N , condition d) is replaced

by either of the following conditions: d2′) there is a weakly compact set Ωk ⊂ U so

that u : (x, u) ∈ Wk ⊂ Ωk; d3′) there is a function φk : U → R which satisfies: 1)

φk(u)→∞ whenever 〈u, u〉U →∞; 2)hk(x, u) ≥ φk(u) whenever (x, u) ∈ Wk.

Proof : The proof is along the same lines as the proofs of Theorems 2.1.1

and 2.1.2, but with the folllowing changes. All the compactness, closedness prop-

erties, and limits should be replaced by their weak forms. In proving Lemma 2.2.2

one should use the fact that the range of a linear bounded operator with a weakly

compact domain is weakly compact. The arguments leading to (2.2.9) should be

modified as follows. Let z = xk+1−fk(xk, uk). By using weak convergence and Riesz

representation theorem we can show that

limt→∞〈xk+1(it)− fk(xk(it), uk(it)), z〉X = 〈z, z〉X . (2.3.1)

11

Since xk+1(it) = fk(xk(it), uk(it)), t ∈ N, 〈z, z〉X = 0. Thus z = 0, yielding (2.2.9).

Condition (2.2.13) should be derived using the following result [Bal.1, Corollary 1.8.3,

p.30]. If F (·) is a continuous convex functional on a Hilbert space and xn converges

weakly to x, then limF (xn) ≥ F (x). In deriving d′3) one should use the fact that a

bounded and closed set is weakly compact.

Theorem 2.3.1 is of some value in the optimal regulation and tracking of dis-

tributed parameter systems. The existence theorem for the discrete-time linear

quadratic problem in Hilbert spaces proved by Lee et al.[Lee.1] is a special case

of Theorem 2.3.1.

CHAPTER III

STABILITY OF OPTIMAL FEEDBACK LAWS

In this chapter, we give stability results for a class of feedback systems arising from

the regulation of time-varying discrete-time systems using optimal infinite-horizon

and moving-horizon feedback laws. The class is characterized by a performance

index that is a summation over time, joint constraints on the state and the control,

a general nonlinear cost function, and nonlinear equations of motion possessing two

special properties. Weak conditions on the cost function and the constraints are

sufficient to guarantee uniform asymptotic stability of both the optimal infinite-

horizon and moving-horizon feedback systems. The infinite-horizon cost associated

with the moving-horizon feedback law approaches the optimal infinite-horizon cost

as the moving-horizon is extended. A procedure for handling time delays due to

feedback law computation is described. We also consider a performance index that is

a supremum over time and prove existence and stability results for the corresponding

infinite-horizon and moving-horizon feedback laws. The following abbreviations will

be used: IH= infinite-horizon, MH= moving-horizon.

12

13

3.1 Problem Formulation

Consider the following discrete-time system with constraints:

xk+1 = fk(xk, uk) , yk = gk(xk, uk) , k ≥ i (3.1.1)

(xk, uk) ∈ Zk ⊂ Rn+m , k ≥ i , xi = a, (3.1.2)

where, for k ≥ 0 : fk : Rn+m → Rn and gk : Rn+m → Rl . For k ≥ 0 let hk :

Rl+m → R be a nonnegative function. Our problem is to determine a feedback law,

uk = ηk(xk), k ≥ 0, which, for each i ≥ 0 and feasible initial state a ∈ Rn, generates

through (3.1.1) a sequence ukk≥i that minimizes the “ cost to go ”,

Ji =∞∑k=i

hk(yk, uk) , (3.1.3)

subject to (3.1.1) and (3.1.2). The functions fk, gk and hk satisfy

fk(0, 0) = 0 , gk(0, 0) = 0 , hk(0, 0) = 0 , k ≥ 0. (3.1.4)

Thus, we have a regulator problem where the targets for xk, yk and uk is the origin.

The assumption (3.1.4) is not terribly restrictive because many interesting problems

can be made to satisfy it with a simple change of variables.

The prior literature is mostly concerned with stochastic regulator problems. See,

for instance [Ber.1]. Deterministic problems are quite different in nature and previous

results appear to be limited to the linear quadratic regulator problem (LQRP), where

the following conditions hold: (a)

fk(x, u) = Akx+Bku , gk(x, u) = Ckx+Dku , k ≥ 0, (3.1.5)

and the matrices Ak, Bk, Ck and Dk are uniformly bounded on k ≥ 0; (b) the se-

quence pairs Ak, Bkk≥0 and Ck, Akk≥0 are, respectively, uniformly completely

14

controllable and uniformly completely observable; (c) hk(y, u) = y′Qky + u′Rku,

k ≥ 0, where prime indicates transpose and Qk and Rk are symmetric positive def-

inite matrices satisfying, for some λ2 ≥ λ1 > 0, λ1y′y ≤ y′Qky ≤ λ2y

′y, λ1u′u ≤

u′Rku ≤ λ2u′u, k ≥ 0, (y, u) ∈ Rl+m; (d) Zk = Rn+m, k ≥ 0. For the LQRP, it is

known (for Dk ≡ 0) that the optimal feedback law is linear and that the resulting

feedback system is exponentially stable [Kwa.1]. The usual proof of this result uti-

lizes a discrete-time Riccati equation and relies much on the quadratic nature of the

cost function and linearity of the feedback law.

Here we use different methods and attack the wider class of problems described

by (3.1.1)-(3.1.3). Even if (3.1.5) holds, there are at least two practical motivations

for considering (3.1.1)-(3.1.3): the need to impose rigorous constraints on the state

and/or control, the possibility that nonquadratic cost functions may lead to a more

desirable quality of regulation. Unless (3.1.1)-(3.1.3) is a time-invariant LQRP it

is not possible to characterize analytically or compute the optimal feedback law

ηk(·). This leads naturally to a moving-horizon (MH) approximation of ηk, ηk, and its

computation by on-line solution of a large-scale mathematical programming problem.

As we shall see, the MH feedback law has a sound theoretical basis; the advance of

computer technology makes it possible to entertain the notion of the extensive on-line

computations.

To obtain our results the system (3.1.1) must satisfy two special properties,

which for the linear system (3.1.5) are implied, respectively, by uniform complete

controllability and uniform complete observability. Under weak conditions on hk

and Zk we show that an optimal feedback law exists and that the optimal feedback

system is uniformly asymptotically stable; also, we give conditions which guaran-

tee exponential stability of the optimal system. These results and our methods of

15

proof are extensions of those used in [Kee.2], where we considered the special case:

fk(x, u) = Ax+Bu, gk(x, u) = Cx+Du and Zk = Z, k ≥ 0.

The MH feedback law for (3.1.1)-(3.1.3) is obtained as follows. Let Mk denote the

moving-horizon at time index k. When the system is at index i ≥ 0, define xi = a.

Solve the optimal control problem of minimizing

Ji =i+Mi−1∑k=i

hk(yk, uk) , (3.1.6)

subject to (3.1.1),(3.1.2) and the alternative constraints

i ≤ k ≤ i+Mi − 1 , xi+Mi= 0. (3.1.7)

Let uk(i, a)i+Mi−1k=i be an optimal control sequence for this problem. Then the MH

feedback law is defined by the function ηi where ηi(a) = ui(i, a).

Under weak conditions on the Mk, hk and Zk we show that the feedback system

resulting from the MH feedback law enjoys the same stability properties as the op-

timal IH feedback system. Moreover, by choosing the Mk sufficiently large the IH

cost, Ji in (3.1.3) associated with uk = ηk(xk), can be made arbitrarily close to the

optimal cost of the IH control problem. The only prior results of a similar nature,

obtained by Kwon and Pearson [Kwo.2] for the time-varying LQRP, follow easily

from our results.

The organization of the remainder of the chapter is as follows. Section 3.2 intro-

duces slight generalizations of some standard ideas in stability theory. Section 3.3

defines and discusses the two special properties which are required of system (3.1.1).

Stability results for the optimal IH and MH feedback laws are then stated in Sections

3.4 and 3.5 respectively. The relationship between the costs given by these feedback

laws is described in Section 3.6. Because of the similarities in the optimal IH and

MH problems, it is more efficient and illuminating to bring the proofs together in

16

one place; this we do in Section 3.7. Section 3.8 presents a procedure by which it is

possible to take in to account delays in the computation of the feedback law. Finally,

in Section 3.9, we consider performance indices that are supremums over time.

3.2 Preliminaries on Stability

We begin with some definitions. A function W : R+ → R+ is said to belong to

class C0 if: (a) it is continuous; (b) W (s) = 0⇔ s = 0. W is in class C+ if W ∈ C0

and is nondecreasing. W is in class C∞ if W ∈ C+ and W (s)→∞ when s→∞.

Since we treat systems with a constrained state space, it is necessary to modify

slightly the usual stability definitions [Kal.1],[Wil.1] which apply to unconstrained

systems. Consider the system

xk+1 = Fk(xk) , xk ∈ Xk ⊂ Rn , k ≥ 0, (3.2.1)

where Fk : Xk → Xk+1. Define Y = (i, a) : i ≥ 0, a ∈ Xi. For (i, a) ∈ Y let

x?k(i, a), k ≥ i denote the solution of (3.2.1) given xi = a. A state xe ∈ Rn is said

to be an equilibrium state for (3.2.1) if xe ∈ Xk and Fk(xe) = xe for all k ≥ 0.

Assume hereafter that x = 0 is an equilibrium state for (3.2.1).

With these minor changes in set-up, the usual definitions for “local” stability

apply. The equilibrium state x = 0 is uniformly stable (US) if, given ε > 0 there

exists δ(ε) > 0 such that ‖x?k(i, a)‖ ≤ ε, (i, a) ∈ Y, k ≥ i, a ∈ N(δ). It is uniformly

asymptotically stable (UAS) if: (a) it is US; (b) there exist r > 0, and, for any

σ > 0 a positive integer T (σ) such that

‖x?k(i, a)‖ ≤ σ , (i, a) ∈ Y , k ≥ i+ T , a ∈ N(r). (3.2.2)

The following definitions concern global stability. The equilibrium state x = 0 is

uniformly asymptotically stable in the large (UASL) if: (a) it is US; (b) given

17

r > 0 there exists B(r) > 0 such that ‖x?k(i, a)‖ ≤ B, (i, a) ∈ Y, k ≥ i, a ∈ N(r);

(c) for any σ > 0 and r > 0 there exists a positive integer T (σ, r) such that (3.2.2)

holds. The origin is exponentially stable (ES) if there exist constants φ > 0 and

0 ≤ ζ < 1 such that

‖x?k(i, a)‖ ≤ φ(ζ)k−i‖a‖ , (i, a) ∈ Y , k ≥ i. (3.2.3)

If (3.2.3) holds, then the system in (3.2.1) is said to have degree of exponential

stability, ζ.

We now state a theorem, which is a generalization of a stability theorem first

stated by Kalman and Bertram [Kal.1],[Wil.1]. Stability properties of the optimal

infinite-horizon and moving-horizon feedback systems will be established by using

this theorem with V as the value function associated with the respective optimal

control problem.

Theorem 3.2.1. Suppose there exist V : Y → R,α ∈ C+, β ∈ C+, γ ∈ C0, λ > 0

and a positive integer L which satisfy the following conditions: (a) V (i, a) ≤ β(‖a‖)

for all (i, a) ∈ Y, a ∈ N(λ); and (b) V (i, a) ≥ α(‖a‖), V (i, a)− V (i+L, x?i+L(i, a)) ≥

γ(‖a‖), V (i, a) − V (i + 1, x?i+1(i, a)) ≥ 0, for all (i, a) ∈ Y . Then for (3.2.1), the

following results hold: (i) x = 0 is UAS; (ii) if N(λ) is replaced by Rn and α ∈ C∞,

x = 0 is UASL; (iii) if N(λ) is replaced by Rn and there exist positive numbers

α1, β1, γ1 and q such that α(s) = α1sq, β(s) = β1s

q, γ(s) = γ1sq, the origin is ES and

the degree of exponential stability is atleast ((β1 − γ1)/β1)1/qL.

The stability theorem in [Kal.1] treats the special case Xk = Rn, k ≥ 0, N(λ) =

Rn and L = 1. We omit the proof of parts (i) and (ii) of Theorem 3.2.1 since it is

elementary and follows closely the ideas in [Kal.1] and [Wil.1].

18

Proof of part (iii) : For (i, a) ∈ Y and j ≥ i,

V (j + L, x?j+L(i, a)) ≤ V (j, x?j(i, a))− γ1‖x?j(i, a)‖q

≤ V (j, x?j(i, a))− (γ1/β1)V (j, x?j(i, a))

= ψV (j, x?j(i, a)) , (3.2.4)

where, ψ = (β1 − γ1)/β1. Clearly, 0 ≤ ψ < 1. For s ∈ R, let [s] denote the largest

integer smaller than or equal to s. Using (3.2.4) we then get, for all (i, a) ∈ Y and

k ≥ i,

α1‖x?k(i, a)‖q ≤ V (k, x?k(i, a)) ≤ (ψ)[(k−i)/L]V (a)

= ψ−1ψ[((k−i)/L)+1]V (a)

≤ ψ−1ψ(k−i)/LV (a)

≤ ψ−1β1ψ(k−i)/L‖a‖q . (3.2.5)

It is then easy to check that (3.2.3) holds with

φ = (β1/α1ψ)1/q , 0 ≤ ζ = (ψ)1/qL < 1 . (3.2.6)

3.3 Properties C and O

In this section we define special properties, C and O which concern the motions

of system (3.1.1) and are crucial for our main results. For the linear case (3.1.5)

we show that they are implied, respectively, by uniform complete controllability

and uniform complete observability, and can therefore be tested easily. For general

nonlinear systems it is not clear how the two properties are verified. The properties

involve only the fk and gk and do not depend on the constraint sets Zk.

Definition 3.3.1 : System (3.1.1) is said to have property C if there exist a

positive integer Nc and a C∞ function Wc such that: given any i ≥ 0, a ∈ Rn there

19

exists a sequence pair (xk, uk)i+Nc−1k=i which satisfies (3.1.1), xi = a, xi+Nc = 0 and

i+Nc−1∑k=i

‖(xk, uk)‖ ≤ Wc(‖a‖) . (3.3.1)

Clearly, C is a uniform (with respect to i) controllability property for (3.1.1).

The inequality (3.3.1) requires the existence of a controlling motion in which both

the state and the control are suitably bounded.

Remark 3.3.1. It should be obvious that the appearance of xk in (3.3.1) can

be eliminated if the fk satisfy some additional conditions. Suppose, e.g., there exists

a C∞ function F such that ‖fk(x, u)‖ ≤ F (‖(x, u)‖), k ≥ 0, (x, u) ∈ Rn+m, and that

in Definition 3.3.1, (3.3.1) is replaced by

i+Nc−1∑k=i

‖uk‖ ≤ Wc(‖a‖). (3.3.2)

Then (3.1.1) satisfies property C.

Definition 3.3.2 : System (3.1.1) is said to have property O if there exist a

positive integer No and a C∞ function Wo such that: given any i ≥ 0 and sequence

triple (xk, yk, uk)k≥i that satisfies (3.1.1), we have

i+No−1∑k=i

‖(yk, uk)‖ ≥ Wo(‖xi‖) . (3.3.3)

For uk = 0, k ≥ i, it is obvious that O is a uniform observability property: by

observing the outputs yk for i ≤ k ≤ i+No−1, it is possible to determine a bound on

the size of the initial state xi. The uk appear in (3.3.3) because of the nonlinearity

of the fk and gk. Specifically, uk = 0, k ≥ i does not characterize all the possible

interactions between xk and yk as it does in the case of linear systems. Thus when

uk 6= 0, k ≥ i, is allowed, it must be possible to account for the fact that the uk might

be chosen in such a way that the yk are small independently of xi. A trivial situation

20

where property O is satisfied is: ‖gk(x, u)‖ ≥ Go(‖x‖), k ≥ 0, (x, u) ∈ Rn+m, and

Go ∈ C∞ (for example, l = n, gk(x, u) = x).

Now consider properties C and O for the linear case (3.1.5). Given i ≥ j ≥ 0,

let Φ(i, j) be the state-transition matrix defined by Φ(i, j) = In, for i = j and

Φ(i, j) = Ai−1Ai−2 · · ·Aj, for i > j. The following definitions are standard [Kwa.1]:

Definition 3.3.3 : The sequence pair Ak, Bkk≥0 is uniformly completely

controllable if there exist µc > 0 and a positive integer Nc such that

x′(i+Nc−1∑j=i

Φ(i+ Nc, j + 1)BjB′jΦ′(i+ Nc, j + 1))x ≥ µc‖x‖2 , i ≥ 0 , x ∈ Rn .

(3.3.4)

Definition 3.3.4 : The sequence pair Ck, Akk≥0 is uniformly completely

observable if there exist µo > 0 and a positive integer No such that

x′(i+No−1∑j=i

Φ′(j, i)C ′jCjΦ(j, i))x ≥ µo‖x‖2 , i ≥ 0 , x ∈ Rn . (3.3.5)

Remark 3.3.2. Consider the time invariant case of (3.1.5), where Ak = A,

Bk = B, Ck = C and Dk = D, k ≥ 0. Assume that (A,B) is a controllable pair,

i.e., rank [B AB · · ·An−1B] = n. Let n = minq : rank[B · · ·Aq−1B] = n. Clearly,

n ≤ n. It is also easy to see that n is the smallest value of Nc for which (3.3.4) holds.

A parallel remark holds for (3.3.5) when (C,A) is an observable pair.

Theorem 3.3.1. Consider the system (3.1.1), (3.1.5) and suppose that for

k ≥ 0, the matrices Ak, Bk, Ck and Dk are uniformly bounded. Then: (i) uniform

complete controllability implies property C and Nc = Nc,Wc(s) = pcs, where pc > 0;

(ii) uniform complete observability implies property O and No = No,Wo(s) = pos,

where po > 0 .

Proof : To prove (i), let i ≥ 0, a ∈ Rn be given. We have

xi+Nc= Φ(i+ Nc, i)a+ S(ui, ui+1, · · · , ui+Nc−1), (3.3.6)

21

where S = [Φ(i + Nc, i + 1)Bi · · ·Bi+Nc−1] ∈ Rn×mNc and (ui, ui+1, · · · , ui+Nc−1) ∈

RmNc is a column vector of the controls. Define

(ui, · · · , ui+Nc−1) = −S ′(SS ′)−1Φ(i+ Nc, i)a. (3.3.7)

Then xi+Nc= 0. By the uniform boundedness of Ak,

‖Φ(i+ j, i)‖ ≤ φj1 , i ≥ 0 , j ≥ 0, (3.3.8)

where φ1 > 0. Also by (3.3.4), ‖(SS ′)−1‖ ≤ 1/µc. This, together with (3.3.7), (3.3.8)

and Holder’s inequality yields

(i+Nc−1∑k=i

‖uk‖ )2 ≤ Nc

i+Nc−1∑k=i

‖uk‖2 ≤ µ‖a‖2, (3.3.9)

where µ = Nc(φ2Nc1 /µc). The uniform boundedness of Ak andBk implies the existence

of γc > 0 such that ‖fk(x, u)‖ ≤ γc‖(x, u)‖, k ≥ 0, (x, u) ∈ Rn+m. Then C follows

from (3.3.9) with Nc = Nc and Remark 3.3.1. The form of Wc(s) is evident from

(3.3.9) and the linearity of fk.

To prove (ii), let i ≥ 0 and (xk, yk, uk)k≥i be a sequence triple that satisfies

(3.1.1) and (3.1.5). Let a = xi. For k ≥ i,

yk = Ck(Φ(k, i)a+ Φ(k, i+ 1)Biui + · · ·+Bk−1uk−1) +Dkuk. (3.3.10)

Hencei+No−1∑k=i

‖yk‖2 =i+No−1∑k=i

‖CkΦ(k, i)a+ Lk,iUi‖2, (3.3.11)

where Ui = (ui, ui+1, · · · , ui+No−1) ∈ RmNo and the matrices Lk,i are appropriately

defined using (3.3.10). Expansion of (3.3.11) gives

i+No−1∑k=i

‖yk‖2 ≥ a′(i+No−1∑k=i

Φ′(k, i)C ′KCkΦ(k, i) )a + 2a′TiUi, (3.3.12)

22

where Ti =i+No−1∑k=i

Φ′(k, i)C ′kLk,i. Because of (3.3.8) and the uniform boundedness of

Bk, Ck and Dk, there exists φ2 > 0 independent of i such that ‖Ti‖ ≤ φ2. Also by

(3.3.5) and (3.3.12),

i+No−1∑k=i

‖yk‖2 ≥ µo‖a‖2 − 2φ2‖a‖‖Ui‖. (3.3.13)

For 0 ≤ δ ≤ 1, we then have by (3.3.13),

(i+No−1∑k=i

‖(yk, uk)‖)2 ≥i+No−1∑k=i

‖(yk, uk)‖2 ≥ δi+No−1∑k=i

‖yk‖2 + ‖Ui‖2

≥ δµo‖a‖2 − 2δφ2‖a‖‖Ui‖+ ‖Ui‖2

≥ (δµo − δ2φ22)‖a‖2.

(3.3.14)

Choose δ = min(1, µo/2φ22) and let po =

√δµo/2 to get (3.3) with No = No and

Wo(s) = pos.

3.4 Stability of the Infinite-Horizon Feedback System

In this section, we formulate precisely the IH optimal control problem and state

theorems concerning the existence and stability of the corresponding optimal feed-

back system.

For i ≥ 0, a ∈ Rn, let P (i, a) denote the problem of minimizing the cost Ji in

(3.1.3) subject to (3.1.1) and (3.1.2). A sequence (xk, yk, uk)k≥i is admissible to

P (i, a) if it satisfies (3.1.1) and (3.1.2).

To state our results, a variety of assumptions will be needed.

Assumption A.3.1. For each k ≥ 0, Zk is closed; (0, 0) ∈ interiorZ, Z =⋂k≥0

Zk.

Assumption A.3.2. For each k ≥ 0, fk : Rn+m → Rn, gk : Rn+m → Rl are

continuous and hk : Rl+m → R is lower semicontinuous.

23

Assumption A.3.3. There exists a C∞ function G such that

‖gk(x, u)‖ ≤ G(‖(x, u)‖) , k ≥ 0 , (x, u) ∈ Rn+m. (3.4.1)

Assumption A.3.4. There exists a C∞ function H1 such that

hk(y, u) ≥ H1(‖(y, u)‖) , k ≥ 0 , (y, u) ∈ Rl+m. (3.4.2)

Assumption A.3.5. There exists a C∞ function H2 such that

hk(y, u) ≤ H2(‖(y, u)‖) , k ≥ 0 , (y, u) ∈ Rl+m. (3.4.3)

Assumption A.3.6. The following conditions hold: (a) fk and gk are given by

(3.1.5) and satisfy the assumptions immediately below (3.1.5); (b) there exist positive

numbers p1, p2 and q such that

p1(‖(y, u)‖)q ≤ hk(y, u) ≤ p2(‖(y, u)‖)q , k ≥ 0 , (y, u) ∈ Rl+m ; (3.4.4)

and (c) for k ≥ 0, hk : Rl+m → R is lower semicontinuous.

Remark 3.4.1. A.3.6 implies A.3.2-A.3.5 and refers to problems with linear

equations of motion and special bounds on the cost function hk.

Remark 3.4.2. A.3.4 is implied by another condition which may be easier to

verify. Specifically, assume that there exists a function h : Rl+m → R such that

hk(y, u) ≥ h(y, u) , k ≥ 0 , (y, u) ∈ Rl+m, (3.4.5)

and the following conditions hold: (a) h is continuous and nonnegative; (b) h(y, u) =

0 ⇔ (y, u) = (0, 0); (c) h(y, u) → ∞ whenever (y, u) → ∞. To see that A.3.4 is

implied by (3.4.5), define H1(s) = minh(y, u) : ‖(y, u)‖ ≥ s, s ≥ 0. A parallel,

alternative condition holds for A.3.5.

24

For i ≥ 0 let

Xi = a : P (i, a) has an admissible sequence for which Ji is finite ,

(3.4.6)

and X =⋂i≥0

Xi. The following theorem concerns properties of these sets and the

existence of an optimal control. The existence is a direct consequence of the results

in Chapter II.

Theorem 3.4.1. Suppose (3.1.1) satisfies property C and assumptions A.3.1,

A.3.2 and A.3.4 hold. Then: (i) the Xk, k ≥ 0, are non empty; (ii) 0 ∈ interiorX;

(iii) Z = Rn+m implies X = Rn; (iv) i ≥ 0, a ∈ Xi imply P (i, a) has a solution.

Assume the hypotheses of the theorem are satisfied so that for i ≥ 0, a ∈ Xi,

there is an optimal control sequence uk(i, a)k≥i, for P (i, a). Let ηk : Xk → Rm be

defined by ηk(x) = uk(k, x). Then by the principle of optimality, ηk is an optimal

feedback law and an optimal feedback system is given by

xk+1 = fk(xk, ηk(xk)) = Fk(xk) , xk ∈ Xk , k ≥ 0. (3.4.7)

Clearly Fk : Xk → Xk+1 is defined and by (3.1.4) and A.3.4 it follows that

ηk(0) = 0 and Fk(0) = 0. We now consider the stability properties of the feedback

system (3.4.7). Let x?k(i, a), k ≥ i denote the solution of (3.4.7) given xi = a ∈ Xi.

Theorem 3.4.2. Suppose (3.1.1) satisfies properties C and O and assumptions

A.3.1-A.3.5 hold. Then: (i) for all i ≥ 0, and a ∈ Xi, limk→∞

x?k(i, a) = 0; (ii) x = 0 is

the only equilibrium state of (3.4.7) and it is UAS.

Since Theorems 3.4.1 and 3.4.2 require A.3.4, hk must include a suitable cost on

the control (for e.g., the term u′Rku in the LQRP). Under alternative hypotheses,

A.3.4 can be relaxed.

Assumption A.3.4′. The following conditions hold: (a) hk(y, u) ≥ H3(‖y‖),

25

k ≥ 0, (y, u) ∈ Rl+m, where H3 ∈ C∞; (b) for k ≥ 0, u : (x, u) ∈ Zk ⊂ Ωk and Ωk

is compact; and (c) ‖gk(x, u)‖ ≥ Go(‖x‖), k ≥ 0, (x, u) ∈ Rn+m and Go ∈ C∞ (note

that (c) implies property O).

Theorem 3.4.3. The conclusions of parts (i), (ii) and (iv) of Theorem 3.4.1,

and Theorem 3.4.2 hold if A.3.4 is replaced by A.3.4′.

The results of Theorems 3.4.2 can be viewed as a weak form of global stability. By

removing the state-control constraints true global stability properties are obtained.

Theorem 3.4.4. Suppose (3.1.1) satisfies properties C and O and Z = Rn+m.

Then for (3.4.7): (i) A.3.2-A.3.5 imply that x = 0 is UASL; (ii) A.3.6 implies that

x = 0 is ES.

Remark 3.4.3. The results in this section are extensions of those in [Kee.2],

where we considered the special case:

fk(x, u) = Ax+Bu, gk(x, u) = Cx+Du, Zk = Z, k ≥ 0. (3.4.8)

Theorems 3.4.1-3.4.4 require the controllability of (A,B) and the observability of

(C,A) to ensure (through Remark 3.3.2 and Theorem 3.3.1) that (3.1.1) satisfies

properties C and O. The results in [Kee.2] are stronger in that they allow control-

lability to be replaced by stabilizability; also, when there are only output-control

constraints (i.e., Z = (x, u) : y = Cx + Du, (y, u) ∈ W), observability can be

weakened to detectability.

It is easy to see that A.3.6 (with q = 2, p1 = λ1 and p2 = 2λ2) is satisfied for the

LQRP of Section 3.1. Thus, Theorem 3.4.1 implies the existence of an optimal feed-

back law and Theorem 3.4.4 shows that the optimal feedback system is exponentially

stable.

26

3.5 Stability of the Moving-Horizon System

We begin by completing the formulation of the MH feedback law described in

Section 3.1. For i ≥ 0, a ∈ Rn, let P (i, a;Mi) denote the problem of minimizing the

cost Ji in (3.1.6) subject to (3.1.1), (3.1.2) and (3.1.7). The finite horizons Mk, k ≥ 0,

can be chosen freely provided they satisfy the following assumption.

Assumption A.3.7. Mk+1 ≥ Mk − 1,Mk ≥ Nc, k ≥ 0, where Nc is the index

in the definition of property C.

A sequence triple (xk, yk, uk)i+Mi−1k=i is admissible for P (i, a;Mi) if it satisfies

(3.1.1), (3.1.2) and (3.1.7). For i ≥ 0, let

Xi = a : P (i, a;Mi) has an admissible sequence , (3.5.1)

and X =⋂i≥0

Xi. Because P (i, a;Mi) is a finite-horizon optimal control problem,

A.3.2 implies that all admissible sequences for P (i, a;Mi) have finite costs. It is

obvious that Xi ⊂ Xi, i ≥ 0 and X ⊂ X. The following theorem concerns other

properties of X and the existence of a solution for P (i, a;Mi).

Theorem 3.5.1. Suppose Mi ≥ Nc and the assumptions of Theorem 3.4.1 hold.

Then the conclusions of Theorem 3.4.1 hold when Xi, X and P (i, a) are replaced,

respectively by Xi, X and P (i, a;Mi).

Assume A.3.1, A.3.2, A.3.4 and A.3.7 are satisfied so that for i ≥ 0, a ∈ Xi, there

is an optimal control sequence, uk(i, a)i+Mi−1k=i , for P (i, a;Mi). Then ηk : Xk → Rm,

defined by ηk(x) = uk(k, x) is a MH feedback law and the corresponding feedback

system is

xk+1 = fk(xk, ηk(xk)) = Fk(xk) , xk ∈ Xk , k ≥ 0, (3.5.2)

where Fk : Xk → Xk+1, ηk(0) = 0 and Fk(0) = 0. We now consider the stability

properties of the feedback system (3.5.2). Let x?k(i, a), k ≥ i denote the solution of

27

(3.5.2) given xi = a ∈ Xi.

Theorem 3.5.2. Suppose A.3.7 is added to the assumptions of Theorem 3.4.2.

Then the conclusions of Theorem 3.4.2 hold for the feedback system (3.5.2), with x?k

and Xi replaced, respectively by x?k and Xi.

Theorem 3.5.3. Suppose A.3.7 is added to the assumptions of Theorem 3.4.3.

Then the conclusions of Theorem 3.4.3 hold, when Theorem 3.4.1 and Theorem 3.4.2

are replaced, respectively by Theorem 3.5.1 and Theorem 3.5.2.

Theorem 3.5.4. Suppose A.3.7. is added to the assumptions of Theorem 3.4.4.

Then the conclusions of Theorem 3.4.4 hold for the feedback system (3.5.2).

In [Kwo.2], Kwon and Pearson consider MH feedback laws for the LQRP and

treat the case where the horizons all have equal length (Mk = M,k ≥ 0). Their

stability results are a special case of Theorem 3.5.4. The proofs in [Kwo.2] utilize

the linearity of the feedback law and properties of a discrete-time Riccati equation;

stability of the feedback system is established in an indirect way, by demonstrating

the instability of its adjoint system. Our method of proof is based on Theorem 3.2.1,

with the IH cost associated with ηk serving as the Lyapunov function; linearity and

the Riccati equation are not used.

3.6 Convergence of the Moving-Horizon Cost

In this section, we state results which show that, as the horizons Mk are extended,

the MH feedback law becomes a good approximation of the optimal IH feedback law.

For i ≥ 0, a ∈ Xi, let: V ?(i, a) be the optimal cost of P (i, a), u?k(i, a) = ηk(x?k(i, a))

and y?k(i, a) = gk(x?k(i, a), u?k(i, a)), k ≥ i. Clearly

V ?(i, a) =∞∑k=i

hk(y?k(i, a), u?k(i, a)). (3.6.1)

28

Also, for i ≥ 0, a ∈ Xi, let: V (i, a) be the optimal cost for P (i, a;Mi), u?k(i, a) =

ηk(x?k(i, a)) and y?k(i, a) = gk(x

?k(i, a), u?k(i, a)), k ≥ i. The IH cost associated with

the MH feedback law is

V ?(i, a) =∞∑k=i

hk(y?k(i, a), u?k(i, a)). (3.6.2)

It has nice properties.

Theorem 3.6.1. Suppose the assumptions of either Theorem 3.5.2 or Theorem

3.5.3 hold. Then: (i) V ?(i, a) has the following bounds:

V ?(i, a) ≤ V ?(i, a) ≤ V (i, a) , i ≥ 0 , a ∈ Xi ; (3.6.3)

(ii) for all i ≥ 0 and a ∈ Xi , V?(i, a) → V ?(i, a) as Mi → ∞; (iii) there exists a

constant r > 0 and, given δ > 0 there exists a positive integer M(δ) such that

V ?(i, a) ≤ V ?(i, a) ≤ V ?(i, a)+δ , i ≥ 0 , Mi ≥M , a ∈ Xi , a ∈ N(r). (3.6.4)

Remark 3.6.1. Note that V ?(i, a) depends on Mkk≥i. Parts (ii) and (iii),

which depend only on Mi, appear to be inconsistent with this. Actually, A.3.7

requires Mk ≥Mi − (k − i) and thus Mi affects Mk for k > i.

The lower bound in (3.6.3) is obvious from the optimality for P (i, a). However, the

upper bound needs a more detailed proof. Part (iii) of Theorem 3.6.1 is stronger than

part (ii) in that the convergence is uniform with respect to i and a; it is weaker in that

it applies only locally. Under stronger hypotheses stronger convergence properties

can be obtained.

Theorem 3.6.2. Suppose (3.1.1) satisfies properties C and O , Z = Rn+m,

and A.3.7 holds. Then: (i) A.3.2-A.3.5 imply that, given any r > 0 and δ > 0 there

exists a positive integer M(r, δ) such that (3.6.4) holds; (ii) A.3.6 implies that, given

29

any δ > 0 there exists a positive integer M(δ) such that

V ?(i, a) ≤ V ?(i, a) ≤ (1 + δ)V ?(i, a) , i ≥ 0 , Mi ≥M , a ∈ Rn . (3.6.5)

Part (ii) of Theorem 3.6.2, specialized to the LQRP, was obtained by Kwon and

Pearson [Kwo.2].

3.7 Proofs

We now prove the results in Sections 3.4-3.6. Most of the details required for

proving Theorems 3.4.1-3.4.4 are omitted, since they parallel those which we use in

proving Theorems 3.5.1-3.5.4.

Proof of Theorem 3.5.1 : By A.3.1 there exists ε > 0 such that N(ε) ⊂ Z.

Choose λ > 0 such that Wc(λ) ≤ ε, where Wc is the C∞ function in Definition

3.3.1. Given i ≥ 0, a ∈ Rn, let (xk, yk, uk)i+Nc−1k=i be the sequence given by Defi-

nition 3.3.1 with yk = gk(xk, uk). We extend the sequence to (xk, yk, uk)k≥i by

setting (xk, yk, uk) = (0, 0, 0), k ≥ i + Nc. If ‖a‖ ≤ λ, it follows by A.3.1, A.3.7 and

(3.3.1) that (xk, yk, uk)i+Mi−1k=i is admissible for P (i, a;Mi). Since λ is independent

of i, N(λ) ⊂ X which proves parts (i) and (ii). Part (iii) follows immediately since

Z = Rn+m implies that ε, and hence λ can be arbitrarily large.

Note that P (i, a;Mi) is a finite dimensional problem that corresponds to minimiz-

ing a non-negative, lower semicontinuous function (Ji) with respect to the variables

(xk, uk)i+Mi−1k=i , on a closed set (

∏i+Mi−1k=i Zk) subject to equality constraints involv-

ing continuous functions ( (3.1.1), xi = a, xi+Mi= 0 ). It is not difficult to see that

this together with A.3.4 is sufficient to imply the existence of a solution. Alterna-

tively, part (iv) may be proved using Theorem 2.1.1 and Theorem 2.1.2 (condition

d3).

30

Proof of Theorem 3.4.1 : It is essentially the same as the proof of Theorem

3.5.1 except: P (i, a) replaces P (i, a;Mi); X replaces X; the results of chapter II,

which are needed because P (i, a) is an infinite dimensional problem, are used to

obtain existence.

To prove Theorems 3.5.2-3.5.4, we need the following lemma.

Lemma 3.7.1. For i ≥ 0, a ∈ Rn, let V (i, a) be defined by

V (i, a) =∞∑k=i

hk(yk, uk) , (3.7.1)

where (yk, uk)k≥i is the sequence defined in the proof of Theorem 3.5.1. Then: (i)

A.3.3 and A.3.5 imply the existence of a C∞ function ϕ such that

V (i, a) ≤ ϕ(‖a‖) , i ≥ 0, a ∈ Rn ; (3.7.2)

(ii) A.3.6 implies (3.7.2) and

ϕ(s) = psq , (3.7.3)

where p > 0 and q > 0 is the constant in A.3.6.

Proof : For i ≥ 0, a ∈ Rn, assumption A.3.3 and (3.3.1) yield ‖(yk, uk)‖ ≤

G(Wc(‖a‖)) + Wc(‖a‖) = W (‖a‖) , k ≥ i. By this and A.3.5, we get V (i, a) ≤i+Nc−1∑k=i

H2(‖ (yk, uk)‖) ≤ ϕ(‖a‖), where ϕ(s) = NcH2(W (s)). Clearly, ϕ ∈ C∞. This

proves part (i). By Theorem 3.3.1 and A.3.6, Wc(s) = pcs and H2(s) = p2sq. Also

G(s) = ds, where d = supk≥0‖[Ck Dk]‖. These results imply ϕ has the form of part (ii).

Proof of Theorem 3.5.2 : We begin by developing an inequality relating

the functions V and V ? defined in Section 6. Define Y = (i, a) : i ≥ 0, a ∈ Xi.

In what follows we use x?k, y?k, u

?k as abbreviations for x?k(i, a), y?k(i, a), u?k(i, a). For

(i, a) ∈ Y and k ≥ i, V (k, x?k) = hk(y?k, u

?k) + c?, where c? is the optimal cost for

31

P (k+ 1, x?k+1;Mk− 1). By A.3.7 and optimality for P (k+ 1, x?k+1;Mk+1), V (k, x?k)−

V (k+1, x?k+1) ≥ hk(y?k, u

?k). Successive use of this inequality yields V (i, a) ≥ V (i, a)−

V (j, x?j) ≥j−1∑k=i

hk(y?k, u

?k) , j > i. Since j can be arbitrarily large

V (i, a) ≥∞∑k=i

hk(y?k, u

?k) = V ?(i, a) . (3.7.4)

To prove part (i), let (i, a) ∈ Y . Since V ?(i, a) ≤ V (i, a) < ∞, A.3.4 implies

H1(‖(y?k, u?k)‖) → 0 as k → ∞. H1 ∈ C∞ implies ‖(y?k, u?k)‖ → 0 as k → ∞. It

follows from property O that Wo(‖x?k‖) → 0 as k → ∞. Since Wo ∈ C∞, ‖x?k‖ → 0

as k →∞, which proves part (i).

By part (i) the only equilibrium point for (3.5.2) is x = 0. To prove UAS of

x = 0, we use part (i) of Theorem 3.2.1 with V ? serving as the Lyapunov function.

Let i ≥ 0 and a ∈ N(λ), where λ > 0 is the constant defined in the proof of Theorem

3.5.1. By the definition of λ, the sequence (xk, yk, uk)i+Mi−1k=i used in the proof of

Theorem 3.5.1 is admissible for P (i, a;Mi). Optimality for P (i, a;Mi) together with

(3.7.2), (3.7.4) and Mi ≥ Nc yields

V ?(i, a) ≤ V (i, a) ≤ ϕ(‖a‖) , (i, a) ∈ Y , a ∈ N(λ) . (3.7.5)

Let (i, a) ∈ Y . Then A.3.4, H1 ∈ C∞ and property O imply

i+No−1∑k=i

hk(y?k, u

?k) ≥ H1( max

i≤k≤i+No−1‖(y?k, u?k)‖)

≥ H1(1

No

i+No−1∑k=i

‖(y?k, u?k)‖) ≥ ϕ(‖a‖), (3.7.6)

where ϕ : R+ → R+, defined by

ϕ(s) = H1(1

No

Wo(s)) , (3.7.7)

is in C∞ because H1 and Wo are in C∞. Thus,

V ?(i, a) ≥i+No−1∑k=i

hk(y?k, u

?k) ≥ ϕ(‖a‖) , (i, a) ∈ Y . (3.7.8)

32

Moreover,

V ?(i, a)− V ?(i+No, x?i+No

) =i+No−1∑k=i

hk(y?k, u

?k) ≥ ϕ(‖a‖) , (i, a) ∈ Y , (3.7.9)

and by the nonnegativity of hk,

V ?(i, a)− V ?(i+ 1, x?i+1) = hi(y?i , u

?i ) ≥ 0 , (i, a) ∈ Y . (3.7.10)

With α = γ = ϕ, β = ϕ, V = V ?, Y = Y and L = No it is seen that conditions (a)

and (b) of Theorem 3.2.1 hold. Thus the proof of part (ii) is complete.

Proof of Theorem 3.5.3 : A.3.4 is used in the proofs of Theorems 3.5.1 and

3.5.2 to show existence, prove part (i) of Theorem 3.5.2 and derive (3.7.8)-(3.7.10).

Condition (b) in Theorem 3.4.3 acts as an alternative to A.3.4 in proving the existence

of a solution to P (i, a;Mi). Condition (d2) of Theorem 2.1.2 may be used to show

this. The proof of part (i) of Theorem 3.5.2 is obvious from (a) and (c). Conditions

(a) and (c) also imply: V ?(i, a) ≥ V ?(i, a)−V ?(i+1, x?i+1) ≥ hi(y?i , u

?i ) ≥ H3(‖y?i ‖) ≥

H3(Go(‖a‖)) ≥ 0, (i, a) ∈ Y . This implies (3.7.8)-(3.7.10) with ϕ(s) = H3(Go(s))

and No = 1.

Proof of Theorem 3.5.4 : The proof of part (i) is omitted since by part (ii)

of Theorem 3.2.1 it is an obvious modification of the proof of part (ii) of Theorem

3.5.2. Part (ii) is established as follows. By A.3.6 and Theorem 3.3.1, H1(s) =

p1sq,Wo(s) = pos. Thus ϕ in (3.7.7) becomes

ϕ(s) = psq , (3.7.11)

where p = p1(po/No)q. This together with (3.7.3),(3.7.5), (3.7.8)-(3.7.10) and part

(iii) of Theorem 3.2.1 proves ES.

Proofs of Theorems 3.4.2-3.4.4 : The proofs are obtained from those of

Theorems 3.5.2-3.5.4 by: (i) defining V (i, a) = V ?(i, a) for i ≥ 0, a ∈ Xi; (ii) omit-

33

ting all the hats; (iii) replacing P (i, a;Mi) by P (i, a). Because of (i) the inequality

corresponding to (3.7.4) is immediate and eliminates the need for A.3.7.

Remark 3.7.1. It may seem surprising that simply removing the state-control

constraints changes UAS (part (ii) of Theorems 3.4.2 and 3.5.2) to UASL (part (i)

of Theorems 3.4.4 and 3.5.4). The change hinges on the fact that with constraints,

(3.7.5) holds only locally: the sequence triple xk, yk, uki+Mi−1k=i , which is used in

deriving (3.7.5), may not be admissible to P (i, a;Mi) for all i ≥ 0, a ∈ Xi. If (3.7.5)

can be made global (i.e., λ = ∞) by some other means, then part (ii) of Theorem

3.5.2 [3.4.2] and part (i) of Theorem 3.5.4 [3.4.4] merge to yield UASL.

Proof of Theorem 3.6.1 : The proof of part (i) follows directly from (3.7.4)

and the optimality for P (i, a). Let i ≥ 0, a ∈ Xi ⊂ Xi be given. Define the sequence

triple (xk, yk, uk)k≥i by (xk, yk, uk) = (x?k(i, a), y?k(i, a), u?k(i, a)), i ≤ k ≤ i + K −

1, (xk, yk, uk) = (xk, yk, uk), i + K ≤ k ≤ i + Mi − 1, (xk, yk, uk) = (0, 0, 0), k ≥

i + Mi, where K = Mi −Nc, yk = gk(xk, uk) and (xk, uk)i+Mi−1k=i+K is the sequence in

Definition 3.3.1 corresponding to the initial values (i, a) : i = i + K, a = x?i+K . By

(3.7.2), V (i, a), the cost given by the sequence triple (xk, yk, uk)k≥i to Ji in (3.1.3)

satisfies V (i, a) ≤ V ?(i, a) + ϕ(‖x?i+K‖). Whenever (xk, yk, uk)i+Mi−1k=i is admissible

to P (i, a;Mi), we have by the optimality for P (i, a;Mi),

V (i, a) ≤ V (i, a) ≤ V ?(i, a) + ϕ(‖x?i+K‖). (3.7.12)

By part (i) of Theorem 3.4.2 (or Theorem 3.4.3), (3.3.1) and A.3.1 there exists M

such that (xk, yk, uk)i+Mi−1k=i is admissible to P (i, a;Mi) whenever Mi ≥ M . Since

ϕ ∈ C∞ and ‖x?i+K‖ → 0 as K →∞, it is clear from (3.7.12) that V (i, a)→ V ?(i, a)

as Mi →∞. This together with (3.6.3) proves part (ii).

Given δ > 0 choose σ > 0 such that ϕ(σ) ≤ δ and σ ≤ λ, where λ is the constant

34

defined in the proof of Theorem 3.5.1. By part (ii) of Theorem 3.4.2 (or Theorem

3.4.3) there exists T (σ) > 0 such that (3.2.2) holds. Let M = T + Nc. If Mi ≥ M

then (3.2.2) implies ‖x?i+K‖ ≤ σ ≤ λ. By the definition of λ, (xk, yk, uk)i+Mi−1k=i

is admissible for P (i, a;Mi). Part (iii) then follows from (3.7.12), (3.6.3) and the

definition of σ.

Proof of Theorem 3.6.2 : The proof of part (i) is identical to that of part

(iii) of Theorem 3.6.1 if in the proof, we use part (i) of Theorem 3.4.4 instead of

part (ii) of Theorem 3.4.2. To prove part (ii) let δ > 0 be given. Choose a positive

integer K such that δ ≥ p(φζK)q/p, where φ, ζ, q, p and p are the constants in (3.2.3),

(3.4.4), (3.7.3) and (3.7.11). This is possible since 0 ≤ ζ < 1. Let M = K + Nc.

Corresponding to (3.7.8), we have: V ?(i, a) ≥ ϕ(‖a‖), i ≥ 0, a ∈ Xi. By this, (3.7.4),

(3.7.12), (3.7.3), (3.2.3) and (3.7.11), we have for i ≥ 0, a ∈ Rn,Mi ≥M ,

V ?(i, a) ≤ V (i, a) ≤ V ?(i, a) + p‖x?i+K‖q

≤ V ?(i, a) + p(φζK)q‖a‖q

≤ V ?(i, a) + δp‖a‖q ≤ (1 + δ)V ?(i, a). (3.7.13)

3.8 Stability of Delayed Feedback Laws

In deriving (3.4.7) and (3.5.2), it is ideally assumed that ηk(xk) and ηk(xk) can be

computed instantaneously. But because they take time to compute, they are usually

available at k + τ , where τ is a positive integer. To guarantee stability under delay,

we suggest a modification in the implementation of ηk and ηk which generalizes an

idea given by Mita in [Mit.1] for linear systems. Since the modification applies to a

rather general class of feedback laws, we discuss the details for a general feedback

law µk in this class and treat ηk and ηk as special cases.

Consider the feedback uk = µk(xk), µk : Xµk → Rm, k ≥ 0, for (3.1.1), where

35

Xµk ⊂ Rn and fk(x, µk(x)) ∈ Xµ

k+1 for all k ≥ 0, x ∈ Xµk . To be consistent with our

intended application, it is assumed that µk(0) = 0, k ≥ 0. When there is no delay,

the feedback system is

xk+1 = fk(xk, µk(xk)) , xk ∈ Xµk , k ≥ 0 . (3.8.1)

Assume xk ∈ Rn and Uk = (uk, · · · , uk+τ−1) ∈ Rτm are known. Then xk+τ can

be predicted by xj+1 = fj(xj, uj), j = k, · · · , k + τ − 1. Let f τk : Rn+τm → Rn

be the mapping that defines this prediction; specifically, f τk (xk, Uk) = xk+τ . To

allow for the computational delay, the control sequence is based on the prediction:

uk+τ = µk+τ (fτk (xk, Uk)). This leads to the modified implementation

xk+1 = fk(xk, uk) , uk+τ = µk+τ (fτk (xk, Uk)) , k ≥ 0. (3.8.2)

For meaningfulness, we require (xk, Uk) ∈ W µk , where, for k ≥ 0,W µ

k = (a, U) :

f τk (a, U) ∈ Xµk+τ. Defining µk : W µ

k → Rm by µk(a, U) = µk+τ (fτk (a, U)) and E =

[0 Im(τ−1)] ∈ Rm(τ−1)×mτ allows (3.8.2) to be written precisely and more compactly

as

xk+1 = fk(xk, uk) , Uk+1 = (E Uk, µk(xk, Uk)) , (xk, Uk) ∈ W µk , k ≥ 0 , (3.8.3)

Note that (x, Uk) ∈ W µk implies (fk(x, uk), (EUk, µk(x, Uk))) ∈ W µ

k+1.

For i ≥ 0, a ∈ Xµi , define xµk(i, a), k ≥ i, to be the solution of (3.8.1) given

xi = a, and uµk(i, a) = µk(xµk(i, a)), k ≥ i. To define the solution of (3.8.3) it is

necessary to specify (xi, Ui) = (a,A) = b ∈ W µi . The need for A is not surpris-

ing; it merely reflects our inability to determine ui, · · · , ui+τ−1 at the initial time i.

Let (xµk(i, b), Uµk (i, b)), k ≥ i, denote the solution of (3.8.3) given (xi, Ui) = b, and

uµk(i, b) be the first m components of the vector Uµk (i, b). In what follows, we will

36

use xµk , uµk , x

µk , U

µk and uµk as abbreviations for xµk(i, a), uµk(i, a), xµk(i, b), Uµ

k (i, b) and

uµk(i, b) respectively. The following remark relates these solutions.

Remark 3.8.1. It should be obvious that, for k ≥ i + τ, xµk is a solution of

(3.8.1); specifically, for b ∈ W µi ,

(xµk , uµk) = (xµk(i+ τ, f τi (b)), uµk(i+ τ, f τi (b))) , k ≥ i+ τ , (3.8.4)

For the special case fk(x, u) = Ax+ Bu and µk(x) = Kx, (3.8.4) has an interesting

interpretation: (3.8.1) is given by xk+1 = (A+ BK)xk, k ≥ 0, and (3.8.3) is a linear

system whose characteristic roots are those of (A+BK) together with τm roots at

the origin. This special result is discussed in detail by Mita in [Mit.1].

It is clear from (3.8.4) that the stability properties of (3.8.3) inherit those of

(3.8.1) provided there are suitable bounds on fk and µk. This statement is made

precise by the following aasumptions and theorem.

Assumption A.3.8. There exists a C∞ function F such that

‖fk(x, u)‖ ≤ F (‖(x, u)‖) , k ≥ 0 , (x, u) ∈ Rn+m. (3.8.5)

Assumption A.3.9. There exist λ > 0 and C∞ functions Ψ1 and Ψ2 such that

Ψ1(‖µk(x)‖) ≤ Ψ2(‖x‖) , k ≥ 0 , x ∈ Xµk , x ∈ N(λ) . (3.8.6)

Theorem 3.8.1. Suppose A.3.8 and A.3.9 hold. Then: (i) limk→∞‖xµk‖ → 0 for

all i ≥ 0, a ∈ Xµk implies lim

k→∞‖(xµk , U

µk )‖ → 0 for all i ≥ 0, b ∈ W µ

i ; (ii) UAS of

x = 0 for (3.8.1) implies UAS of (x, U) = (0, 0) for (3.8.3); (iii) N(λ) = Rn in

A.3.9 and UASL of x = 0 for (3.8.1) imply UASL of (x, U) = (0, 0) for (3.8.3); (iv)

N(λ) = Rn, F (s) = d s,Ψ1(s) = ψ1 s,Ψ2(s) = ψ2 s, d > 0, ψ1 > 0, ψ2 > 0, and ES of

x = 0 for (3.8.1) imply ES of (x, U) = (0, 0) for (3.8.3).

37

Proof : We first prove the case τ = 1. The proof of part (i) is obvious from

(3.8.4) and (3.8.6). Consider part (ii). To show US, let ε > 0 be given. Choose

ρ1 > 0 : Ψ2(ρ1) ≤ Ψ1(ε/2) and define ε = min(λ, ρ1, ε/2). Since x = 0 is US for

(3.8.1), there exists δ(ε) > 0 such that

‖xµk(j, a)‖ ≤ ε , j ≥ 0 , k ≥ j , a ∈ Xµj , a ∈ N(δ) . (3.8.7)

Choose δ > 0 : F (δ) ≤ δ and δ ≤ ε. Now let i ≥ 0, (xi, Ui) = b ∈ W µi , ‖b‖ ≤ δ.

Define j = i + 1, a = fi(b). By (3.8.5), ‖a‖ ≤ F (δ) ≤ δ. For k ≥ j, (3.8.6) and

(3.8.7) yield Ψ1(‖uµk(j, a)‖) ≤ Ψ2(‖xµk(j, a)‖) ≤ Ψ2(ρ1) ≤ Ψ1(ε/2). By this, (3.8.7),

(3.8.4) and ‖b‖ ≤ ε, ‖(xµk , Uµk )‖ ≤ ε, k ≥ i.

To complete the proof of UAS, let σ > 0 be given. Choose ρ2 > 0 : Ψ2(ρ2) ≤

Ψ1(σ/2) and define σ = min(λ, ρ2, σ/2). Since x = 0 is UAS for (3.8.1), there exist

r > 0 and T (σ) such that ‖xµk(j, a)‖ ≤ σ, j ≥ 0, k ≥ j + T , a ∈ Xµj , a ∈ N(r). Define

T = T+1 and choose r > 0 : F (r) ≤ r. As in the proof of US, it can be easily verified

that ‖(xµk , Uµk )‖ ≤ σ, i ≥ 0, k ≥ i+ T, b ∈ W µ

i , b ∈ N(r), which proves part (ii). Now

consider part (iii). Items (a) and (b) in the definition of UASL can be demonstrated

as in the proof of UAS. Also (b) can be easily established using (3.8.4)-(3.8.6) and

applying (b) to (3.8.1).

To prove part (iv), first note that the special forms of Ψ1 and Ψ2 yield, ‖uµk(j, a)‖ ≤

ψ3‖xµk(j, a)‖, j ≥ 0, k ≥ j, a ∈ Xµj , where ψ3 = ψ2/ψ1. Since x = 0 is ES for (3.8.1),

‖xµk(j, a)‖ ≤ φ(ζ)k−j‖a‖, j ≥ 0, k ≥ j, a ∈ Xµj , where φ > 0, 0 ≤ ζ < 1. By these and

(3.8.4), ‖(xµk , Uµk )‖ ≤ φ(ζ)k−i‖b‖, i ≥ 0, k ≥ i, b ∈ W µ

i , where ζ > 0, ζ ≤ ζ < 1 and

φ = max1, (1 + ψ3)φd/ζ.

The proofs for τ > 1 can be obtained inductively. Consider the case: τ = 2.

System (3.8.3) with τ = 1, for which we have just established stability properties, is

38

equivalent to the system

xk+1 = fk(xk, uk) , uk+1 = µk+1(fk(xk, uk)) = µk(xk, uk) , k ≥ 0 . (3.8.8)

Also, (3.8.3) with τ = 2 is the same as the feedback system obtained by applying an

1 - step prediction scheme to (3.8.8). If (3.8.1) satisfies A.3.8 and A.3.9, then it is

easily verified that (3.8.8) also satisfies those assumptions. Thus Theorem 3.8.1 for

τ = 1 applies to (3.8.8) and the proof of Theorem 3.8.1 is complete for τ = 2. The

argument can be repeated, so the induction is complete.

Now consider the relationship between the costs (Ji in (3.1.3)) corresponding to

(3.8.1) and (3.8.3). For i ≥ 0, a ∈ Xµi , let V µ(i, a) =

∞∑k=i

hk(yµk , u

µk), where yµk =

gk(xµk , u

µk), k ≥ i. Also, for i ≥ 0, b ∈ W µ

i , let V µ(i, b) =∞∑k=i

hk(yµk , u

µk), where yµk =

gk(xµk , u

µk), k ≥ i. By (3.8.4),

V µ(i, b) =i+τ−1∑k=i

hk(yk, uk) + V µ(i+ τ, f τi (b)) , i ≥ 0 , b ∈ W µi , (3.8.9)

where (xk, yk)i+τ−1k=i is the solution of (3.1.1) with (xi, ui, · · · , ui+τ−1) = b.

By applying Theorem 3.8.1 to ηk and ηk, we get the following theorem.

Theorem 3.8.2. Suppose A.3.8 is added to the hypotheses of Theorems 3.4.2,

3.4.4, 3.5.2 and 3.5.4. Then: (i) the conclusions of Theorems 3.4.2 and 3.4.4 hold for

the feedback system (3.8.3) corresponding to µk = ηk, with a,Xi and x?k(i, a) replaced,

respectively, by b,W ηi and (xηk(i, b), u

ηk(i, b)); (ii) the conclusions of Theorems 3.5.2

and 3.5.4 hold for the feedback system (3.8.3) corresponding to µk = ηk, with a, Xi

and x?k(i, a) replaced, respectively, by b,W ηi and (xηk(i, b), u

ηk(i, b)).

Proof : By (3.7.1), (3.7.2), the definition of λ in the proof of Theorem 3.5.1,

the optimality for P (i, a) and A.3.4,

H1(‖ηi(a)‖) ≤ V ?(i, a) ≤ ϕ(‖a‖) , i ≥ 0 , a ∈ Xi , a ∈ N(λ) . (3.8.10)

39

Thus ηk satisfies A.3.9 with Ψ1 = H1,Ψ2 = ϕ and Xηk = Xk. For ηk, the relation-

ship corresponding to (3.8.10) follows directly from (3.7.5) and A.3.4. The proof is

immediate from Theorems 3.4.2, 3.4.4, 3.5.2, 3.5.4 and the application of Theorem

3.8.1 to ηk and ηk.

The following theorem describes the convergence of V η to V η.

Theorem 3.8.3. Suppose A.3.8 is added to the hypotheses of Theorems 3.6.1

and 3.6.2. Then the conclusions of parts (ii) and (iii) of Theorem 3.6.1, and Theorem

3.6.2 hold, when V ?, V ?, a, Xi and Rn are replaced, respectively, by V η, V η, b,W ηi and

Rn+τm.

Proof : By applying (3.8.9) to ηk and ηk and noting V η = V ?, V η = V ?, we get

V η(i, b)− V η(i, b) = V ?(i+ τ, f τi (b))−V ?(i+ τ, f τi (b)) , i ≥ 0 , b ∈ W ηi , (3.8.11)

A.3.8 implies the existence of F ∈ C∞ such that ‖fiτ (b)‖ ≤ F (‖b‖), i ≥ 0, b ∈ Rn+τm.

The proof of the theorem is then obvious from (3.8.11) and Theorems 3.6.1 and 3.6.2.

The following example illustrates the usefulness of the τ - step prediction scheme.

Example 3.8.1. Let n = m = l = 1, Zk = R2, fk(x, u) = αx + u, gk(x, u) = x,

and hk be any lower semicontinuous function that satisfies A.3.4 and A.3.5. Con-

sider a moving-horizon feedback law with Mk = 1, k ≥ 0. Since P (i, a;Mi) requires

xi+Mi= 0, it is obvious that ηk(x) = −αx. When there is no delay, the feed-

back system is given by (3.5.2): xk+1 = 0, k ≥ 0. But suppose τ = 1. If the

moving-horizon feedback law is implemented with no corrections for the delay, then

uk+1 = ηk(xk) = −αxk. For α > 1, this, together with xk+1 = αxk + uk is a

second order linear system whose characteristic roots are both outside the unit

circle (unstable). On the other hand, using the τ - step prediction scheme gives

40

uk+1 = ηk+1(fk(xk, uk)) = −α(αxk + uk), which leads to a system that has both its

characteristic roots at the origin (2 step deadbeat) irrespective of the value of α.

3.9 Regulation Using Nonsummation Indices

The performance indices considered thus far in this chapter are summations over

time. In this section, we consider a modification and replace Ji and Ji in (3.1.3) and

(3.1.6) by

Ji = supk≥i

ξk−ihk(yk, uk) and Ji = maxi≤k≤i+Mi−1

ξk−ihk(yk, uk), (3.9.1)

respectively, where ξ ≥ 1. For the summation index in (3.1.3), the finiteness of Ji

implies that ‖(yk, uk)‖ → 0 as k →∞. However this is not true for the Ji in (3.9.1).

To drive ‖(yk, uk)‖ to zero, we require the following condition:

Assumption A.3.10. ξ > 1 .

For the new problem described by (3.9.1), the definitions of P (i, a), P (i, a;Mi),

Xi, Xi, ηi, ηi, (x?k, y

?k, u

?k) and (x?k, y

?k, u

?k) remain the same as before. Existence and

stability results, parallel to those in sections 3.4, 3.5 and 3.8 hold.

Theorem 3.9.1. The conclusions of Theorems 3.4.1-3.4.4, 3.5.1-3.5.4 and

3.8.2 hold for the modified problem described by (3.9.1) and A.3.10; also, in part (ii)

of Theorems 3.4.4 and 3.5.4, the degree of exponential stability is atleast ξ−1/q.

Proof : The main ideas of proof are as in sections 3.7 and 3.8. Therefore we

only indicate the necessary changes. First, the proofs of Theorems 3.4.1 and 3.5.1

go through as for the summation index. To prove the stability results, define, as in

section 3.6: V ?(i, a) = optimal cost for P (i, a), V (i, a) = optimal cost for P (i, a;Mi)

and V ?(i, a) = the IH cost associated with the MH feedback law. Thus,

V ?(i, a) = supk≥i

ξk−ihk(y?k(i, a), u?k(i, a)) , i ≥ 0 , a ∈ Xi , (3.9.2)

41

and

V ?(i, a) = supk≥i

ξk−ihk(y?k(i, a), u?k(i, a)) , i ≥ 0 , a ∈ Xi . (3.9.3)

It is easy to verify that Lemma 3.7.1 also holds if (3.7.1) is replaced by

V (i, a) = supk≥i

ξk−ihk(yk, uk) ; (3.9.4)

in the proof, we only have to define ϕ(s) = ξNcH2(W (s)). Now consider the re-

lationship parallel to (3.7.4). Let i ≥ 0, k ≥ i, a ∈ Xi and c? = optimal cost for

P (k + 1, x?k+1;Mi − 1). Then by A.3.7,

V (k, x?k) = max(hk(y?k, u

?k), ξc

?)

≥ max(hk(y?k, u

?k), ξV (k + 1, x?k+1)). (3.9.5)

Considering (3.9.5) for all k ≥ i yields

V (i, a) ≥ supk≥i

ξk−ihk(y?k, u

?k) = V ?(i, a). (3.9.6)

In the paragraph below (3.7.4), A.3.10 has to be used to show that ‖(y?k, u?k)‖ → 0

as k →∞.

The relation in (3.7.5) holds by Lemma 3.7.1, (3.9.6) and optimality for P (i, a;Mi).

The lower bound for V ?(i, a) corresponding to (3.7.8) can be established as follows.

V ?(i, a) ≥ maxi≤k≤i+No−1

H1(‖(y?k, u?k)‖)

= H1( maxi≤k≤i+No−1

‖(y?k, u?k)‖)

≥ ϕ(‖a‖) , (i, a) ∈ Y , (3.9.7)

where ϕ is as in (3.7.7). Now consider the relationships in (3.7.9) and (3.7.10). By

(3.9.3),

V ?(i, a) ≥ ξV ?(i+ 1, x?i+1). (3.9.8)

42

Then by A.3.10 and (3.9.7),

V ?(i, a)− V ?(i+ 1, x?i+1) ≥ (ξ − 1

ξ)V ?(i, a) ≥ (

ξ − 1

ξ)ϕ(‖a‖) , (i, a) ∈ Y . (3.9.9)

Thus Theorem 3.2.1 should be applied with α = ϕ, γ = (ξ − 1)ϕ/ξ , β = ϕ, V =

V ?, Y = Y and L = 1. With the above mentioned changes, the proofs in sections

3.7 and 3.8 extend to Theorem 3.9.1. To demonstrate the mentioned degree of

exponential stability, note that, by (3.9.8),

V ?(j + 1, x?j+1(i, a)) ≤ ψV ?(j, x?j(i, a)) , i ≥ 0 , j ≥ i , a ∈ Rn , (3.9.10)

where ψ = 1/ξ. This parallels the relationship in (3.2.4). By using (3.7.5), (3.7.3),

(3.9.7),(3.7.11) and the steps in (3.2.5) and (3.2.6), it is then easy to show that the

degree of exponential stability is atleast ξ−1/q.

Remark 3.9.1. Combining (3.9.6) with the optimality of V ?(i, a) for P (i, a),

it is easy to get

V ?(i, a) ≤ V ?(i, a) ≤ V (i, a) , i ≥ 0 , a ∈ Xi , (3.9.11)

which parallels (3.6.3). But the other results of section 3.6 that concern the conver-

gence of V ?(i, a) to V ?(i, a) as Mi → ∞, do not, in general, extend to the problem

considered here. The reason for this is the following. Parallel to (3.7.12), we can

derive

V (i, a) ≤ V (i, a) ≤ max(V ?(i, a), ξKϕ(‖x?i+K‖)) . (3.9.12)

Eventhough ‖x?i+K‖ → 0 as K →∞, the term ξKϕ(‖x?i+K‖) may not approach zero;

moreover, it may be larger than V ?(i, a) for all K. In other words, in (3.9.2), the

terms at the tail end may play a significant role in defining V ?(i, a), i.e., ξk−ihk(y?k(i, a), u?k(i, a)) <

V ?(i, a) for all k ≥ i. In spite of this remark, the MH feedback laws considered in

this section are useful because of the stability results in Theorem 3.9.1.

43

Example 3.9.1. Consider the linear time invariant case in (3.4.8) where (A,B)

is controllable and (C,A) is observable. In many applications, it is required that

the outputs and controls satisfy magnitude constraints: |yi| ≤ yimax, 1 ≤ i ≤ l

and |uj| ≤ ujmax, 1 ≤ j ≤ m; and that, the closed loop system have a prescribed

degree of exponential stability, ζ(0 < ζ < 1). One approach to this problem is to

consider optimal IH and MH feedback laws with Z = Rn+m, ξ = 1/ζ and hk(y, u) =

‖Q(y, u)‖∞, (y, u) ∈ Rl+m, where Q ∈ R(l+m)×(l+m) is a diagonal matrix whose

diagonal elements are 1/yimax, 1 ≤ i ≤ l and 1/ujmax, 1 ≤ j ≤ m. It is easy to verify

that Assumption A.3.6 is satisfied with q = 1. By Theorem 3.9.1 and the definition of

ξ, the IH and MH feedback systems are exponentially stable with degree of stability,

ζ. Also, given x0 = a, if V ?(0, a) ≤ 1, then y?k(0, a) and u?k(0, a) satisfy the required

magnitude bounds for all k ≥ 0. Correspondingly, if V ?(0, a) ≤ 1, then the output-

control sequence generated by the MH feedback law is also within the required limits.

It is quite difficult to solve for the optimal IH feedback law. However, closed form

expressions can be obtained for the MH feedback laws; algorithms for getting these

expressions are given in Chapter VII.

CHAPTER IV

ILLUSTRATIVE EXAMPLES

In this chapter we present some examples that illustrate the usefulness of consid-

ering problems with constraints and general cost functions.

4.1 Comparison of Performance Indices

Consider the first order system

xk+1 = αxk + uk , k ≥ 0 , x0 = a , (4.1.1)

where α > 0. Our aim is to compare the following three indices used for regulating

(4.1.1):

=1 =∞∑k=0

‖ξk(xk, ρuk)‖1 , =2 = (∞∑k=0

‖ξk(xk, ρuk)‖2)1/2 , =∞ = supk≥0‖ξk(xk, ρuk)‖∞ ,

(4.1.2)

where ρ > 0 and ξ ≥ 1. =1,=2 and =∞ can be considered as scaled L1, L2 and L∞

norms of the state-control sequence (xk, uk)k≥0; therefore, they have a common

basis for comparison. Depending on the index selected from (4.1.2), we will refer to

the resulting optimal feedback system as the optimal L1, L2 or the L∞ regulator.

The Optimal L1 Regulator. By letting xk = ξkxk, uk = ξk+1uk, k ≥ 0,

it is easy to see that minimizing =1 subject to (4.1.1) is equivalent to minimizing

44

45

=1 =∞∑k=0

(|xk|+ ρ|uk|) subject to

xk+1 = αxk + uk , k ≥ 0 , x0 = a , (4.1.3)

where ρ = ρ/ξ and α = αξ. The minimization leads to the following optimal feedback

system: xk+1 = λ1xk, k ≥ 0, where

λ1 =

α if ρ ≥ ξ/(1− αξ) and αξ < 1

0 otherwise

(4.1.4)

Since our aim here is only to compare the resulting optimal regulators, we will not

include the steps involved in deriving (4.1.4); for details, the reader is referred to

Example 6.2.3.

The Optimal L2 Regulator. Minimizing =2 subject to (4.1.1) is equivalent

to minimizing =22 =

∞∑k=0

(x2k + ρ2u2

k) subject to (4.1.3), which then is a special case

of the LQRP discussed in section 3.1. Thus the optimal feedback system is: xk+1 =

αxk−fxk, k ≥ 0, where f = pα/(ρ2 +p) and p is the positive solution of the discrete-

time algebraic Riccati equation, p− 1 = α2p− α2p2(ρ2 + p)−1. It is easy to solve the

Riccati equation and get the optimal L2 regulator as: xk+1 = λ2xk, k ≥ 0, where,

λ2 = ρ2α/(ρ2 + p) , p = (−b+√b2 + 4ρ2)/2 , b = (ρ2 − α2ρ2 − 1) . (4.1.5)

The Optimal L∞ Regulator. By a direct application of the results in Example

7.1.1, it is easy to solve for the optimal L∞ regulator: xk+1 = λ∞xk, k ≥ 0, where

λ∞ = min(αρ/(ρ+ ξ), 1/ξ) . (4.1.6)

The optimal closed loop eigenvalues, λi, i = 1, 2,∞, in (4.1.4)-(4.1.6) are indi-

cators of the quality of the three optimal regulators. For given values of α and ξ,

the variation of λi with respect to ρ are detailed in Figure 4.1.1. The following ob-

servations can be made. All the three performance indices in (4.1.2) lead to linear

46

Figure 4.1: 4.1.1. Optimal Poles for the First Order Example.

47

feedback laws. For =1 and =∞, this property does not, in general, extend to higher

order systems. When either α > 1, or ρ is sufficiently small, the optimal poles satisfy:

λ1 ≤ λ2 ≤ λ∞; also, when α ≤ 1, and ρ is large, we have λ∞ ≤ λ2 ≤ λ1. The solution

of the L1 problem is either uk = 0 (zero control) or uk = −αxk (deadbeat). Possible

extension of this behavior of the L1 regulator to higher order systems is discussed in

Chapter VI. For the optimal L1 and L2 regulators to be stable, we only need ξ ≥ 1.

For the L∞ regulator, however, Assumption A.3.10 is in general required to establish

stability (except, say, when α < 1, and ρ is small).

4.2 Need for Constraints

To illustrate the need for directly including constraints in the optimal control

problem, we consider a linear system with control constraints. Control magnitude

constraints are important elements of most practical control systems. One of the

usual design methods for handling these constraints is to find a linear feedback law

(say, by solving an LQRP) and then add an appropriate saturating element in the

control loop to satisfy the control constraints. There are two main objections to such

a feedback law. First, the state space region which can be stabilized by the saturation

feedback law may be smaller than the region which can be stabilized by admissible

controls. Gutman [Gut.2] gives a simple example where this may occur. Secondly,

even if an initial state can be driven to the origin by the saturation feedback law,

the quality of response may be poor. Here, we will show, by a second order example

that, a MH feedback law based on directly including the constraints in (3.1.2) gives

a much better quality of regulation compared to the saturation feedback law.

Consider the double integrator, y = u, with the control constraint: |u| ≤ 1. The

continuous-time system is discretized using a zero order hold (period T=1) to get:

48

x1k+1 = x1

k + x2k + 0.5uk, x

2k+1 = x2

k + uk, yk = xk, k ≥ 0. The saturation feedback

law (SFL) is developed as follows. Solve an LQRP with Qk = 1 and Rk = 10−3,

using a Riccati equation [Kwa.1] to get the linear feedback law, uk = gxk, where

g = (−1.962,−1.962). Then define

uk =

gxk if |gxk| ≤ 1

gxk/|gxk| otherwise.

(4.2.1)

The MH feedback law (MHFL) that we consider corresponds to: the same quadratic

cost function as in the LQRP, Mi = 30, i ≥ 0, and the constraint |uk| ≤ 1, included

in (3.1.2) by defining Zk appropriately. The finite-horizon problems P (i, a;Mi) were

solved by using an efficient quadratic programming code [Law.1]. For x0 = (15,−10),

the continuous-time responses corresponding to the two feedback laws are indicated

on the phase plane in Figure 4.2.1. While the moving-horizon feedback law leads to

a well damped response, (4.2.1) leads to a poor, slow one.

4.3 The Overhead Crane Example

To further illustrate the usefulness of nonquadratic cost functions and the need

for constraints, we take the discretized crane system (B.4) from Appendix B. Let T =

0.3, l = 4 and yk = xk. We consider l1 and l2 cost functions: h1(y, u) = ‖y‖1 + ρ|u|

and h2(y, u) = ‖y‖2+(ρu)2, where ρ = 10−3; also, problems with and without control

constraints. The following abbreviations will be used: UC = unconstrained, CC =

control constrained.

Our first computation concerns the solution of the UC-IH-l1 (Z = Rn+m, hk = h1)

problem by using MH approximations. An efficient l1 computer code [Bar.1] was

used to solve the UC-MH-l1 ( Z = Rn+m, hk = h1,Mk = M ) problems. Continuous-

time responses x1(t)(position of cart) and x3(t)(angle of cable), corresponding to

49

Figure 4.2: 4.2.1. Responses for the Saturation and MH Feedback Laws.

50

x0 = (−5, 0, 0, 0) are given in Figure 4.3.1 for M = 10, 15, 20 and 30. There was

no change in the solutions for M ≥ 20 and the solution of the UC-IH-l1 problem

appears to be deadbeat in 20 steps.

Now consider the convergence of the solutions of the UC-MH-l2 problems to that

of UC-IH-l2. The UC-IH-l2 problem is a time invariant LQRP and so its solution

is given by uk = K∞xk, where K∞ is obtained by solving a discrete-time algebraic

Riccati equation. For M ≥ n, the MH feedback law is given by uk = KMxk, where

KM is obtained by solving a modified Riccati equation [Kwo.1]. Continuous-time

responses are indicated in Figure 4.3.2.

Some indicators of solution properties are given in Table 4.3.1; ximax and umax

denote the maximum magnitudes of xi(t) and u(t). For the same M, the magnitudes

of variables for the l1 problem are larger than those for the l2 problem. The same

is true of the IH problems. The convergence of the MH solutions to the IH solution

as M → ∞, is much faster for the l1 problem (finite convergence). One other

observation is also clear from Table 4.3.1. Especially for small M , V ?(0, x0) is much

smaller than V (0, x0). Thus the control sequence generated by the MH feedback law

does much better (in terms of the regulation cost in (3.1.3) ) when compared to the

open loop control sequence that solves the first finite-horizon problem P (0, x0;M).

The control histories associated with the UC-IH-l1 and UC-IH-l2 solutions are

given in Figure 4.3.3. The values of umax for the two solutions (respectively, 7.67

and 4.562) are considerably larger than the desired value (1.0). Therefore, the con-

trol constraint |uk| ≤ 1, was imposed and solutions to the CC-IH-l1 and CC-IH-l2

problems were obtained by extending the MH. The responses x1(t), x3(t) and the

control histories u(t) are shown in Figures 4.3.4 and 4.3.5. Table 4.3.2 compares

solution properties of the UC and CC problems.

51

Figure 4.3: 4.3.1. Solutions of the UC-MH-l1 Problems.

52

Figure 4.4: 4.3.2. Solutions of the UC-MH-l2 Problems.

53

M umax x2max x4

max V (0, x0) V ?(0, x0)

l1 Problem

10 20.82 12.34 12.16 126.1 102

15 9.933 5.9 5.8 87.0 77.6

20 7.67 4.55 4.48 76.68 76.68

∞ 7.67 4.55 4.48 – 76.68

l2 Problem

10 19.14 11.34 11.17 701 391

15 6.40 3.8 3.74 181 156.64

20 4.47 2.65 2.6 156 150.6

∞ 4.562 2.7 2.66 – 149.3

Table 4.1: 4.3.1. Properties of Moving-Horizon Solutions.

Problem x1max x2

max x3max x4

max umax

UC-IH-l1 5.0 4.55 1.36 4.50 7.67

UC-IH-l2 5.0 2.70 1.29 2.66 4.56

CC-IH-l1 5.0 1.62 1.00 1.56 1.00

CC-IH-l2 5.0 1.61 1.20 1.40 1.00

Table 4.2: 4.3.2. Largest Magnitudes of Variables.

54

Figure 4.5: 4.3.3. Control Histories for UC-IH-l1 and UC-IH-l2 Solutions.

55

Figure 4.6: 4.3.4. Solutions of the CC-IH-l1 and CC-IH-l2 problems.

56

Figure 4.7: 4.3.5. Control Histories for CC-IH-l1 and CC-IH-l2 solutions.

57

The following observations can be made. Apart from keeping the force within

limits, the control constraint also causes the magnitudes of the velocities to be con-

siderably reduced, without much loss in the overall speed of response. It is also our

general experience that good response properties are obtained when the control term

is removed from the cost function (i.e., ρ = 0) and instead, a compactness constraint

is placed on the control. By Theorems 3.4.3 and 3.5.3, this problem has a sound

theoretical basis. The l1 cost offers a deadbeat solution even when the control con-

straint is imposed. Also, the magnitude values in Table 4.3.2 for the CC-IH-l1 and

CC-IH-l2 solutions are comparable. Therefore, with the control constraint, the l1

cost yields a better quality of regulation compared to the l2 cost.

APPENDICES

58

59

APPENDIX A

Notations

R The set of real numbers.

R+ The set of nonnegative reals.

Rn The real euclidean n dimensional space.

∈ a ∈ A means that a is an element of the set A.

3 a 3 A means that a is not an element of A.

zi The ith element of the vector z.

Ai The ith row of the matrix A.

A′ The transpose of the matrix A.

(y, z) For y ∈ Rp and z ∈ Rq, (y, z) ∈ Rp ×Rq

denotes the single column [y′, z′]′.

‖z‖ The l2 norm of the vector z, i.e., ‖z‖ = (∑i

(zi)2)1/2.

‖z‖1 The l1 norm of z, i.e., ‖z‖1 =∑i|zi|.

‖z‖∞ The l∞ norm of z, i.e., ‖z‖∞ = maxi|zi|.

[z]∞ Denotes maxizi.

‖A‖ For matrix A, ‖A‖ = sup‖Az‖ : ‖z‖ = 1.

N(ε) For ε ≥ 0, N(ε) = x : ‖x‖ ≤ ε.

∅ The empty set.

60

Table A.1: A Test Table

A+B For A ⊂ X and B ⊂ X, A+B denotes the set

x ∈ X : x = a+ b, a ∈ A, b ∈ B.

αA For α ∈ R and A ⊂ X, αA = x ∈ X : x = αa, a ∈ A.

A ⊥ B If A and B are subspaces of X, A ⊥ B means that

A and B are orthogonal.

A\B If A and B are subsets of X, then

A\B = x ∈ X : x ∈ A, x 3 B.

a := b Means that a is replaced by b.

61

APPENDIX B

Model of an Overhead Crane

The crane problem that we consider is described in [Hwa.1] and elsewhere. A

simplified model consists of an overhead trolley which runs on straight frictionless

rails, with its load suspended on an inextensible cable from its center of gravity (see

Figure B.1). The objective is to move the trolley from a given position to a reference

point with the load starting and finishing in a stationary position directly beneath

the trolley.

The linearized equations of motion are:

P (τ) + γψ(τ) = u(τ) , P (τ) + ψ(τ) + ψ(τ) = 0 (B.1)

where, P, ψ and u are dimensionless variables related to the physical variables by

P =g(m+M)

lFmaxq , ψ =

g(m+M)

Fmaxθ , u =

F

Fmax(B.2)

g is the acceleration due to gravity, γ = µ/(1 + µ), µ = m/M , τ = t√g/l and dot

denotes differentiation with respect to τ . Fmax is the maximum possible magnitude

of force and we assume that Fmax = g(m + M). Because of the way u is defined, it

is required that |u| ≤ 1.

62

Figure B.1: A Diagramatic Sketch of the Overhead Crane.

63

Letting x1 = P , x2 = P , x3 = ψ and x4 = ψ, we obtain the state space model

x(τ) =

0 1 0 0

0 0 µ 0

0 0 0 1

0 0 −(1 + µ) 0

x(τ) + (1 + µ)

0

1

0

−1

u(τ) . (B.3)

The continuous-time system in (B.3) can be discretized using a zero order hold

(period T) to get the difference equation

xk+1 =

1 T γc γ(T − s)

0 1 µs γc

0 0 (1− c) s

0 0 −ω2s (1− c)

xk +

(T 2/2) + γc

T + µs

−c

−ω2s

uk , (B.4)

where xk = x(kT ), uk = u(kT ), c = (1− cosωT ), s = (sinωT )/ω and ω =√

1 + µ.

BIBLIOGRAPHY

64

65

BIBLIOGRAPHY

[Bal.1] Balakrishnan, A.V., Applied Functional Analysis. New York:Springer-Verlag, 1981.

[Bar.1] Barrodale, I., and Roberts, F.D.K., “Solution of the constrained l1 linearapproximation problem,” ACM Trans. Math. Software, vol.6, No.2, June1980, pp.231-235.

[Bas.1] Bashein, G., “A simplex algorithm for on-line computation of time optimalcontrols,” IEEE Trans. Automat Control, Vol.AC-16, 1971, pp.479-482.

[Bau.1] Baum, R.F., “Existence theorems for Lagrange control problems with un-bounded time domain,” J. Opt. Theory and Appl., Vol.19, No.1, May1976, pp.89-116.

[Ber.1] Bertsekas, D., Dynamic Programming and Stochastic OptimalControl. New York: Academic, 1976.

[Cai.1] Caines, P.E., and Maine, D.Q., “On the discrete-time matrix Riccati equa-tion of optimal control,” Int. J. Control, vol.12, No.5, pp.785-794.

[Can.1] Canon,M.D., Cullum, C.D., and Polak, E., Theory of Optimal Controland Mathematical Programming. New York: McGraw-Hill, 1970.

[Cha.1] Chang, A., “An optimal regulator problem,” SIAM J. Control, Ser.A,Vol.2, No.2, 1965, pp.220-233.

[Che.1] Chen, C.C., and Shaw, L., “On receding horizon feedback control,” Au-tomatica, vol.18, No.3, 1982, pp.349-352.

[Cwi.1] Cwikel, M., and Gutman, P.O., “Convergence of an algorithm to findmaximal state constraint sets for discrete-time linear dynamical systemswith bounded controls and states,” Proc. 24th CDC, Fort Lauderdale, Dec.1985.

[Des.1] Desoer, C.A., and Wing, J., “An optimal strategy for a saturatingsampled-data system,” IRE Trans. Automat. Control, vol.AC-6, Feb.1961, pp.5-15.

66

[Des.2] Desoer, C.A., and Wing,J., “A minimal time discrete system,” IRE Trans.Automat. Control, vol.AC-6, May 1961, pp.111-125.

[Des.3] Desoer, C.A., and Wing, J., “Minimal time regulator problem for linearsampled-data systems (General theory),” J. Franklin Inst., vol.271, Sept.1961, pp.208-228.

[Dev.1] De Vlieger, J.H., Verbruggen, H.B., and Bruijn, P.M., “A time-optimalcontrol algorithm for digital computer control,” Automatica, 1982, pp.239-244.

[Dol.1] Dolezal, J., “Existence of optimal solutions in general discrete systems,”Kybernetica, vol.11, 1975, pp.301-312.

[Gra.1] Grattan-Guinness, I., “Joseph Fourier’s anticipation of linear program-ming,” Oper. Res. Quaterly, Vol.21, No.3, 1970, pp.361-364.

[Gut.1] Gutman, P.O., and Cwikel, M., “Admissible sets and feedback control fordiscrete-time linear dynamical systems with bounded controls and states,”Proc. of 23rd CDC, Las Vegas, Dec. 1984, pp.1727-1731.

[Gut.2] Gutman, P.O., and Per Hagander, “A new design of constrained con-trollers for linear systems,” IEEE Trans. Automat. Control, Vol.AC-30,No.1, Jan. 1985, pp.22-33.

[Han.1] Hanson, R.J., and Haskell, K.H., “Two algorithms for the linearly con-strained least squares problem,” ACM Trans. Math. Software, vol.8, No.3,Sept. 1982, pp.323-333.

[Hwa.1] Hwang, W.G., and Schmitendorf, W.E., “Controllability results for sys-tems with a non-convex target,” IEEE Trans. Automat. Control, vol.AC-29, No.9, Sept. 1984, pp.794-802.

[Kal.1] Kalman, R.E., and Bertram, J.E., “Control system analysis and designvia the second method of Lyapunov, II. Discrete-time systems,” J. BasicEngrg., Trans. ASME, vol.82, 1960, pp.394-400.

[Kal.2] Kalman, R.E., Ho, Y.C., and Narendra, K.S., “Controllability of lineardynamical systems,” Contributions to Differential Equations, vol.I, Inter-science, 1963.

[Kee.1] Keerthi, S.S., and Gilbert, E.G., “An existence theorem for discrete-timeinfinite-horizon optimal control problems,” IEEE Trans. Automat. Con-trol, vol.AC-30, No.9, Sept. 1985, pp.907-909.

[Kee.2] Keerthi, S.S., and Gilbert, E.G., “Optimal infinite-horizon control andthe stabilization of discrete-time systems: state-control constraints andnon-quadratic cost functions,” to be published in IEEE Trans. Automat.Control, vol.AC-31, March 1986.

67

[Kee.3] Keerthi, S.S., and Gilbert, E.G., “Optimal infinite-horizon feedback lawsfor a general class of constrained discrete-time systems: Stability andmoving-horizon approximations,” submitted to the SIAM J. Control andOptimization.

[Kle.1] Kleinman, D.L., “An easy way to stabilize a linear constant system,” IEEETrans. Automat. Control, vol.AC-15, Dec.1970, p.692.

[Koh.1] Kohler, D.A., “Translation of a report by Fourier on his work on linearinequalities,” OPSEARCH, Vol.10, 1973, pp.38-42.

[Knu.1] Knudsen, J.K.H., “Time-optimal computer control of a pilot plant evap-orator,” Proc. IFAC 6th World Congress, 1975, part 2, pp.45.3/1-6.

[Kwa.1] Kwakernaak, H., and Sivan, R., Linear Optimal Control Systems.New York: Wiley-Interscience, 1972.

[Kwo.1] Kwon, W.H., and Pearson, A.E., “A modified quadratic cost problem andfeedback stabilization of a linear system,” IEEE Trans. Automat. Control,vol.AC-22, No.5, Oct. 1977, pp.838-842.

[Kwo.2] Kwon, W.H., and Pearson, A.E., “On feedback stabilization of time-varying discrete linear systems,” IEEE Trans. Automat. Control, vol.AC-23, No.3, June 1978, pp.479-481.

[Law.1] Lawson, C.L., and Hanson, R.J., Solving Least Squares Problems.Prentice-Hall, Englewood Cliffs, New Jersey, 1974.

[Lee.1] Lee, K.Y., Chow, S.N., and Barr, R.O., “On the control of discrete-timedistributed parameter systems,” SIAM J. Control, vol.10, 1972, pp.361-376.

[Lev.1] Levis, A.H., Athans, M., and Schlueter, R.A., “On the behavior of optimallinear sampled-data regulators,” Int. J. Control,

[Man.1] Manson, G.A., “Time-optimal control of an overhead crane model,” Op-timal Control Appl. and Meth., vol.3, 1982, pp.115-120.

[Mit.1] Mita, T., “Optimal digital feedback control systems counting computationtime of control laws,” IEEE Trans. Automat. Control, vol.AC-30, No.6,June 1985, pp.542-548.

[Mur.1] Murty, K.G., Linear Programming. John Wiley, 1983.

[Nol.1] Noldus, E., “Effectiveness of state constraints in controllability andleast squares optimal control problems,” IEEE Trans. Automat. Control,vol.AC-22, No.4, April 1977, pp.627-631.

[Pay.1] Payne, H.J., and Silverman, L.M., “On the discrete-time algebraic Riccatiequation,” IEEE Trans. Automat. Control, vol.AC-18, 1973, pp.226-234.

68

[Pro.1] Propoi, A.I., “Use of LP methods for synthesizing sampled-data automaticsystems,” Aut. Remote Control, Vol.24, 1963, p.837.

[Ras.1] Rasmy, M.E., and Hamza, M.H., “Minimum-effort time-optimal controlof linear discrete systems,” Int. J. Control, vol.21, No.2, 1975, pp.293-304.

[Tho.1] Thomas, Y.A., “Linear quadratic optimal estimation and control withreceding horizon,” Electron. Lett., vol.11, Jan. 1975, pp.19-21.

[Wil.1] Willems, J.L., Stability Theory of Dynamical Systems. Thomas Nel-son and Sons, Ltd., London, 1970.

[Win.1] Wing, J., and Desoer, C.A., “The multiple-input minimum time regulatorproblem (General theory),” IEEE Trans. Automat. Control, vol.AC-8,April 1963, pp.125-136.

[Zad.1] Zadeh, L.A., and Whalen, B.H., “On optimal control and linear program-ming,” IRE Trans. Automat. Control, Vol. AC-7, 1962, p.45.

ABSTRACT

Title

by

Your Name

Chairperson: Name

Abstact starts here.

OPTIMAL FEEDBACK CONTROL CONTROL OF ...amadi/thesis.pdfdfkgjsklg sdfg dfsgdf dfgsdfg sfvgsdfg...

Documents

Transcript of OPTIMAL FEEDBACK CONTROL CONTROL OF ...amadi/thesis.pdfdfkgjsklg sdfg dfsgdf dfgsdfg sfvgsdfg...