Linear Optimal Control

Lecture 4 Daniela Iacoviello

Department of Computer and System Sciences “A.Ruberti”

Sapienza University of Rome

26/10/2015 Controllo nei sistemi biologici

Lecture 1

Pagina 2

Prof. Daniela Iacoviello

Department of computer, control and management

Engineering Antonio Ruberti

Office: A219 Via Ariosto 25

http://www.dis.uniroma1.it/~iacoviel

Prof.Daniela Iacoviello- Optimal Control

Grading

Project + oral exam

The exam must be concluded before the second

part of Identification that will be held by Prof. Battilotti

Grading

Project+ oral exam

Example of project:

- Read a paper on an optimal control problem

- Study: background, motivations, model, optimal control, solution,

results

- Simulations

You must give me, before the date of the exam:

- A .doc document

- A power point presentation

- Matlab simulation files

The exam must be concluded before the second

part of Identification that will be held by Prof. Battilotti

Some projects studied in 2014-15

Application of Optimal Control to malaria: strategies and simulations

Performance compare between LQR and PID control of DC Motor

Optimal Low-Thrust LEO (low-Earth orbit) to GEO (geosynchronous-Earth orbit)

Circular Orbit Transfer

Controllo ottimo di una turbina eolica a velocità variabile attraverso il metodo

dell'inseguimento ottimo a regime permanente

OptimalControl in Dielectrophoresis

On the Design of P.I.D. Controllers Using Optimal Linear Regulator Theory

Rocket Railroad Car

………

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM: YOU MUST STUDY ON THE BOOKS

Prof.Daniela Iacoviello- Optimal Control

Part of the slides has been taken from the References indicated below

References B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972 D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004 D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise Introduction", Princeton University Press, 2011 How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare: Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

Prof.Daniela Iacoviello- Optimal Control

R.F.Hartl, S.P.Sethi. R.G.Vickson, A Survey of the Maximum Principles for Optimal Control Problems with State Constraints, SIAM Review, Vol.37, No.2, pp.181-218, 1995

Prof.Daniela Iacoviello- Optimal Control

Pontryagin

Lev Semenovich Pontryagin (3 September 1908 – 3 May 1988) was a Soviet Russian mathematician.

Prof.Daniela Iacoviello- Optimal Control

The Pontryagin principle

Problem 1: Consider the dynamical system:

with:

tuxfx ,,

niURCx

ff

RUtuRtx

n

i

pn

,...,2,1,,

)(,)(

0

Prof.Daniela Iacoviello- Optimal Control

Assume fixed the initial control instant and the initial and final values :

Define the performance index :

with

fi

i xTxxtx )()(

))((),(),(,, TxGduxLTuxJ

T

ti

niURCx

LL n

i

,...,2,1,, 0

Prof.Daniela Iacoviello- Optimal Control

Determine the value

the control and the state

that satisfy the dynamical

system, the constrain t on the control, the

initial and final conditions and minimize the

cost index

,itT

)(0 RCuo

)(1 RCxo

Prof.Daniela Iacoviello- Optimal Control

Hamiltonian function

uxftuxLuxH T ,)(,,,, 00

Prof.Daniela Iacoviello- Optimal Control

The Pontryagin principle

Theorem 1 (necessary condition):

Assume the admissible solution is a minimum

there exist a constant

and a n-dimensional vector

not simultaneously null such that :

*** ,, Tux

00

*1* ,TtC i

0*

*

*

H

x

HT

U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Prof.Daniela Iacoviello- Optimal Control

The Pontryagin principle

Problem 2: Consider the dynamical system:

with:

tuxfx ,,

niURCx

ff

RUtuRtx

n

i

pn

,...,2,1,,

)(,)(

0

Prof.Daniela Iacoviello- Optimal Control

Assume fixed the initial control instant and the

initial state

while for final values assume:

where is a function of dimension

of C1 class.

Define the performance index :

with

i

i xtx )(

))((),(),(,, TxGduxLTuxJ

T

ti

niURCx

LL n

i

,...,2,1,, 0

0)( Tx nf

Prof.Daniela Iacoviello- Optimal Control

Determine the value

the control and the state

that satisfy the dynamical

system, the constrain on the control, the initial

and final conditions and minimize the cost

index .

,itT

)(0 RCuo

)(1 RCxo

Prof.Daniela Iacoviello- Optimal Control

Theorem 2 (necessary condition):

Consider an admissible solution such that

If it is a minimum there exist a constant

and an n-dimensional vector

not simultaneously null such that :

*** ,, Tux

00

*1* ,TtC i

fTdx

drank

*

)(

Prof.Daniela Iacoviello- Optimal Control

0*

*

*

H

x

HT

U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vector such that:

T

Tdx

dT

*

)()(

fR

Prof.Daniela Iacoviello- Optimal Control

The Pontryagin principle

Problem 3: Consider the dynamical system:

with:

tuxfx ,,

niURCx

ff

RUtuRtx

n

i

pn

,...,2,1,,

)(,)(

0

Prof.Daniela Iacoviello- Optimal Control

Assume fixed the initial control instant and the

initial state

while for final values assume:

where is a function of dimension

of C1 class.

Define the performance index :

with

i

i xtx )(

T

ti

duxLTuxJ ),(),(,,

niRURCt

L

x

LL n

i

,...,2,1,,, 0

0),( TTx 1 nf

Prof.Daniela Iacoviello- Optimal Control

Determine the value

the control and the state

that satisfy the dynamical

system, the constrain t on the control, the

initial and final conditions and minimize the

cost index .

,itT

)(0 RCuo

)(1 RCxo

Prof.Daniela Iacoviello- Optimal Control

Theorem 3:

Consider an admissible solution

such that

IF it is a minimum there exist a constant

and an n-dimensional vector

not simultaneously null such that :

*** ,, Tux

00

*1* ,TtC i

fTTx

rank

*

),(

Prof.Daniela Iacoviello- Optimal Control

RkkdH

Hx

HT

t

T

,,

**

**

*

U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vector such that:

T

T

T

TH

TxT

**

*

**

)()(

fR

Prof.Daniela Iacoviello- Optimal Control

The Pontryagin principle

Problem 4: Consider the dynamical system:

with:

fixed

tuxfx ,,

RURCt

f

x

ff

RUtuRtx

n

pn

0,,

)(,)(

Prof.Daniela Iacoviello- Optimal Control

i

i xtx )(

For the final values assume: where is a function of dimension of C1 class. Assume the constraint with Define the performance index : with

T

ti

duxLTuxJ ),(),(,,

RURCt

L

x

LL n

0,,

0),( TTx 1 nf

Prof.Daniela Iacoviello- Optimal Control

kduxh

T

ti

),(),(

niRURCt

h

tx

hh n ,...,2,1,,

)(, 0

Determine the value

the control and the state

that satisfy the dynamical

system, the constrain t on the control, the

initial and final conditions and minimize the

cost index .

,itT

)(0 RCuo

)(1 RCxo

Prof.Daniela Iacoviello- Optimal Control

Hamiltonian function

)),(),((,)(,,,, 00 ttutxhuxftuxLuxH TT

Prof.Daniela Iacoviello- Optimal Control

Theorem 4 (necessary condition):

Consider an admissible solution

such that

IF it is a local minimum

there exist a constant

and an n-dimensional vector

not simultaneously null such that :

*** ,, Tux

00

*1* ,TtC i

fTTx

rank

*

),(

Prof.Daniela Iacoviello- Optimal Control

,

*

*

T

x

H

U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vector such that: The discontinuities of may occur only in the instants in which u has a discontinuity and in these instant the Hamiltonian is conitnuous

T

T

T

TH

TxT

**

*

**

)()(

fR

Prof.Daniela Iacoviello- Optimal Control

*

The Pontryagin principle - convex case

Problem 5: Consider the dynamical linear

system:

with A and B of function of C1 class; assume fixed the initial and final instants and the initial state,

Assume

where U is a convex set.

utBxtAx )()(

n

T RTxorfixedxTx )()(

TttRUtu i

p ,)(

Prof.Daniela Iacoviello- Optimal Control

Define the performance index :

with

L convex function with respect to x(t), u(t) in

per ogni

G is a scalar function of C2 class and convex

with respect to x(T)

)(),(),(, TxGduxLuxJ

T

ti

niTtURCt

L

x

LL i

n

i

,...,2,1,,,, 0

URn Ttt i ,

Prof.Daniela Iacoviello- Optimal Control

Determine the control

and the state

that satisfy the dynamical system, the

constraint on the control, the initial

and final conditions and minimize the cost

index .

TtCu io ,0

TtCx io ,1

Prof.Daniela Iacoviello- Optimal Control

Theorem 5 (necessary and sufficient condition):

Consider an admissible solution

such that

It is a minimum normal (i.e.λ0 =1) if and only if

there exists an n-dimensional vector

such that :

oo ux ,

TtC i

o ,1

f

o

TTxrank

),(

Prof.Daniela Iacoviello- Optimal Control

U

ttutxHttxH ooooo

,)(),(),()(,),(

Moreover if

oT

o

Tdx

dGT

)()(

nRTx )(

oT

o

x

tuxH

),,,(

Prof.Daniela Iacoviello- Optimal Control

Remark

If the set U coincides with Rp the minimum

condition reduces to :

0

u

H

Prof.Daniela Iacoviello- Optimal Control

Example 3 (from L.C.Evans) Control of production and consumption Consider a factory whose output can be controlled. Let’s set: x(t) the amount of output produced at time t , 0 ≤ t. Assume we consume some fraction of the output at each time and likewise reinvest the remaining fraction u(t) . It is our control, subject to the constraints 0 ≤ u(t) ≤ 1

Prof.Daniela Iacoviello- Optimal Control

The corresponding dynamics are: The positive constant k represents the growth rate of our reinvestment. We will chose K=1. Assume as cost index the function: The aim is to maximize the total consumption of the output

0)0(

0),()()(

xx

ktxtkutx

T

dttxtuuJ

0

)()(1))((

Prof.Daniela Iacoviello- Optimal Control

We apply the Pontryagin Principle; the Hamiltonian is: The necessary conditions are:

xuxuuxH 1,,

1)()()()(max)(),(),(

)()()(

0)(1)()(1)(

10

ttxtutxttutxH

txtutx

Tttut

u

Prof.Daniela Iacoviello- Optimal Control

From the last equation we have, since x(t)>0: From the equation of the costate, since , by continuity we deduce for t<T, t close to T, that : thus for such values of t. Therefore and consequently: More precisely so long as and this holds for:

1)(0

1)(1)(

tif

tiftu

0)( T

0)( tu1)( t

1)( t

tTt )(tTt )( 1)( t

TtT 1Prof.Daniela Iacoviello- Optimal Control

For times with t near T we have Therefore the costate equation yields: Since we have for all And over this time interval there are no switchings

1Tt

1)( tu

)(1)(1)( ttt

1)1( T 1Tt

1)( 1 tTet

Prof.Daniela Iacoviello- Optimal Control

Therefore: For the switching time Homework: find the switching time

Tttif

ttiftu

*

*

*

0

01)(

1* Tt

Prof.Daniela Iacoviello- Optimal Control

*t T

Optimal solution: we should reinvest all

the output

(and therefore consume anything)

up to time t* and afterwards we should

consume everything

(and therefore reinvest nothing) Bang-bang control