Presentazione di PowerPoint - dis.uniroma1.itiacoviel/Materiale/Lecture_1_2018... · 21/09/2018...

52
Optimal Control Prof. Daniela Iacoviello Lecture 1

Transcript of Presentazione di PowerPoint - dis.uniroma1.itiacoviel/Materiale/Lecture_1_2018... · 21/09/2018...

Optimal Control

Prof. Daniela Iacoviello

Lecture 1

21/09/2018 Pagina 2

Prof. Daniela Iacoviello

Department of computer, control and management

engineering Antonio Ruberti

Office: A219 Via Ariosto 25

http://www.dis.uniroma1.it/~iacoviel

Prof.Daniela Iacoviello- Optimal Control Pagina 221/09/2018

21/09/2018 Pagina 3Prof.Daniela Iacoviello- Optimal Control Pagina 321/09/2018

Mailing list:

Send an e-mail to: [email protected]

Subject: Optimal Control 2018

21/09/2018 Pagina 4

Grading

Project+ oral exam

Example of project (1-3 Students):

-Read a paper on an optimal control problem

-Study: background, motivations, model, optimal control, solution, results

-Simulations

-Conclusions

-References

You must give me, before the date of the exam:

-A .doc document

-A power point presentation

-Matlab simulation files

Oral exam: Discussion of the project AND on the topics of the lectures

Pagina 421/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 5

Some projects proposed in 2014-15, 2015-16, 2016-17- 2017-18

Application Of Optimal Control To Malaria: Strategies And Simulations

Performance Compare Between Lqr And Pid Control Of Dc Motor

Optimal Low-thrust Leo (Low-earth Orbit) To Geo (Geosynchronous-earth Orbit)

Circular Orbit Transfer

Controllo Ottimo Di Una Turbina Eolica A Velocità Variabile Attraverso Il Metodo

dell'inseguimento Ottimo A Regime Permanente

Optimalcontrol In Dielectrophoresis

On The Design Of P.I.D. Controllers Using Optimal Linear Regulator Theory

Rocket Railroad Car

Optimal Control Of Quadrotor Altitude Using Linear Quadratic Regulator

Optimal Control Of An Inverted Pendulum

Glucose Optimal Control System In Diabetes Treatment ………

Pagina 521/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 6

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time optimal control of an automatic Cableway

Glucose Optimal Control System In Diabetes Treatment

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time Optimal Control Of An Automatic Cableway

Optimal Control Project On Arduino Managed Module For Automatic Ventilation Of

Vehicle Interiors

Optimal Control For A Suspension Of A Quarter Car Model

………

Pagina 621/09/2018

Some projects proposed in 2014-15, 2015-16, 2016-17- 2017-18

Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 7

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below

Pagina 721/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 8

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989

E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,

1993

C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993

A.Calogero, Notes on optimal control theory, 2017

L. Evans, An introduction to mathematical optimal control theory, 1983

H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004

D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise

Introduction", Princeton University Press, 2011

How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:

Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

……………………………………..

Pagina 821/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 9

Course outline

• Introduction to optimal control

• Nonlinear optimization

• Dynamic programming

• Calculus of variations

• Calculus of variations and optimal control

• LQ problem

• Minimum time problem

Pagina 921/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 10

Background

First course on linear systems

(free evolution, transition matrix, gramian matrix,…)

Pagina 1021/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 11

Notations

nRtx )(

pRtu )(

RRRRf pn → ::

kC

State variable

Control variable

Function (function with k-th derivative continuous a.e.)

Pagina 1121/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 12

Introduction

Optimal control is one particular branch of modern control that sets

out to provide analytical designs of a special appealing type.

The system, that is the end result of an optimal design, is supposed to

be the best possible system of a particular type

A cost index is introduced

Pagina 1221/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 13

Introduction

Linear optimal control is a special sort of optimal control:

✓ the plant that is controlled is assumed linear

✓ the controller is constrained to be linear

Linear controllers are achieved by working with quadratic cost

indices

Pagina 1321/09/2018Prof.Daniela Iacoviello- Optimal Control

21/09/2018 Pagina 14Prof.Daniela Iacoviello- Optimal Control

Introduction

Advantages of linear optimal control

✓ Linear optimal control may be applied to nonlinear systems

✓ Nearly all linear optimal control problems have computational

solutions

✓ The computational procedures required for linear optimal design

may often be carried over to nonlinear optimal problems

Pagina 1421/09/2018

21/09/2018 Pagina 15Prof.Daniela Iacoviello- Optimal Control Pagina 1521/09/2018

VARIATIONAL METHODS

❑ Early Greeks – Max. area/perimeter

❑ Hero of Alexandria – Equal angles of incidence /reflection

❑ Fermat - least time principle (Early 17th Century)

❑ Newton and Leibniz – Calculus (Mid 17th Century)

❑ Johann Bernoulli - brachistochrone problem (1696)

❑ Euler - calculus of variations (1744)

❑ Joseph-Louis Lagrange – Euler-Lagrange Equations (??)

❑ William Hamilton – Hamilton's Principle (1835)

❑ Raleigh-Ritz method – VA for linear eigenvalue problems (late 19th Century)

❑ Quantum Mechanics - Computational methods – (early 20th Century)

❑ Morse & Feshbach – technology of variational methods (1953)

❑ Solid State Physics, Chemistry, Engineering – (mid-late 20th Century)

❑ Personal computers – new computational power (1980’s)

❑ Technology of variational methods essentially lost (1980-2000)

❑ D. Anderson – VA for perturbations of solitons (1979)

❑ Malomed, Kaup – VA for solitary wave solutions (1994 – present)

21/09/2018 Pagina 16Prof.Daniela Iacoviello- Optimal Control

History

Pagina 1621/09/2018

21/09/2018 Pagina 17Prof.Daniela Iacoviello- Optimal Control

History

In 1696 Bernoulli posed the

Brachistochrone problem to his contemporaries:

“it seems to be the first problem which explicitely

dealt with optimally controlling the path or the behaviour of a dynamical

system»

Suppose a particle of mass M moves from A to B along smooth wire

joining the two fixed points (not on the same vertical line),under the

influence of gravity.

What shape the wire should be in order that the bead, when released

from rest at A should slide to B in the minimum time?

E.R.Pinch, Optimal control and calculus of variations, Oxford Science Publications, 1993

Pagina 1721/09/2018

Example 1 (Evans 1983)

Reproductive strategies in social insects

Let us consider the model describing how social insects (for example

bees) interact:

represents the number of workers at time t

represents the number of queens

represents the fraction of colony effort

devoted to increasing work force

lenght of the season

)(tw

)(tq

)(tu

T

Motivations

Prof.Daniela Iacoviello- Optimal Control Pagina 1821/09/2018

0)0(

)()()()()(

ww

twtutsbtwtw

=

+−= Evolution of the

worker population

Death rate

Known rate at which each worker

contributes to the bee economy

( )

0)0(

)()()(1)()(

qq

twtstuctqtq

=

−+−= Evolution of the

Population

of queens

Constraint for the control:1)(0 tu

Prof.Daniela Iacoviello- Optimal Control Pagina 1921/09/2018

The bees goal is to find the control that maximizes the number of

queens at time T:

The solution is a bang- bang control

( ) )()( TqtuJ =

Prof.Daniela Iacoviello- Optimal Control Pagina 2021/09/2018

Motivations

Example 2 (Evans 1983…and everywhere!) A moon lander

Aim: bring a spacecraft to a soft landing on the lunar surface, using

the least amount of fuel

represents the height at time t

represents the velocity =

represents the mass of spacecraft

represents thrust at time t

We assume

)(th

)(tv

)(tm

)(th

)(tu

1)(0 tu

Prof.Daniela Iacoviello- Optimal Control Pagina 2121/09/2018

Consider the Newton’s law:

We want to minimize the amount of fuel

that is maximize the amount remaining once we have

landed

where is the first time in which

ugmthm +−=)(

0)(0)()()(

)()(

)(

)()(

−=

=

+−=

tmthtkutm

tvth

tm

tugtv

)())(( muJ =

0)(0)( == vh

Pagina 22Prof.Daniela Iacoviello- Optimal Control 21/09/2018

21/09/2018 Pagina 23Prof.Daniela Iacoviello- Optimal Control Pagina 2321/09/2018

Analysis of linear control systems

Essential components of a control system

✓ The plant

✓ One or more sensors

✓ The controller

Controller Plant

Sensor

21/09/2018 Pagina 24Prof.Daniela Iacoviello- Optimal Control Pagina 2421/09/2018

Analysis of linear control systems

Feedback:

the actual operation of the control system is compared to the desired

operation and the input to the plant is adjusted on the basis of this

comparison.

Feedback control systems are able to operate satisfactorily despite

adverse conditions, such as disturbances and variations in plant

properties

21/09/2018 Pagina 25Prof.Daniela Iacoviello- Optimal Control Pagina 2521/09/2018

Definitions

Consider a function

and

denotes the Eucledian norm

A point is a local minimum of over

If

RRf n →::

nRD

Dx * f nRD

)()(

0

*

*

xfxf

xxsatisfyingDxallforthatsuch

21/09/2018 Pagina 26Prof.Daniela Iacoviello- Optimal Control Pagina 2621/09/2018

Definitions

Consider a function

and

denotes the Eucledian norm

A point is a strict local minimum of over

If

RRf n →::

nRD

Dx *f nRD

*)()(

0

*

*

xxxfxf

xxsatisfyingDxallforthatsuch

21/09/2018 Pagina 27Prof.Daniela Iacoviello- Optimal Control Pagina 2721/09/2018

Definitions

Consider a function

and

denotes the Eucledian norm

A point is a global minimum of over

If

RRf n →::

nRD

Dx * f nRD

)()( * xfxf

Dxallfor

21/09/2018 Pagina 28Prof.Daniela Iacoviello- Optimal Control Pagina 2821/09/2018

Definitions

The notions of a local/strict/global maximum are defined similarly

If a point is either a maximum or a minimum

is called an extremum

Unconstrained optimization - first order necessary conditions

All points sufficiently near in are in

Assume and its local minimum.

Let be an arbitrary vector.

Being in the unconstrained case:

Let’s consider:

0 is a minimum of g

x

nR

DnR

1Cf *x

)(:)( * += xfg

0

*

toenoughcloseR

Dx

+

*x

Prof.Daniela Iacoviello- Optimal Control 21/09/2018 Pagina 29

First order Taylor expansion of g around

Proof: assume

For these values of

If we restrict to have the opposite sign of

Contraddiction

0=

0)(

lim),()0(')0()(0

=++=→

ooggg

0)0(' =g

21/09/2018 Pagina 30Prof.Daniela Iacoviello- Optimal Control

Unconstrained optimization - first order necessary conditions

0)0(' g

)0(')(

0

gofor

thatsoenoughsmall

)0(')0(')0()( gggg +−

)0('g

)0(')0(')0()( gggg +− 0)0()( − gg

0)0(' =g

Δ is arbitrary

First order necessary condition for optimality

( )fofgradienttheis

fffwherexfgT

xx n

1:)()(' * =+=

0)()0(' * == xfg

0)( * = xf

Prof.Daniela Iacoviello- Optimal Control 21/09/2018 Pagina 31

Unconstrained optimization - first order necessary conditions

21/09/2018 Pagina 32Prof.Daniela Iacoviello- Optimal Control Pagina 3221/09/2018

A point satisfying this condition is a stationary point*x

Unconstrained optimization - first order necessary conditions

Assume and its local minimum. Let be

an arbitrary vector.

Second order Taylor expansion of g around

Since

nR2Cf *x

0=

0)(

lim),()0(''2

1)0(')0()(

2

2

0

22 =+++=→

oogggg

0)0(' =g

0)0('' g

Unconstrained optimization – second order conditions

21/09/2018 Pagina 33Prof.Daniela Iacoviello- Optimal Control

Proof: suppose

For these values of

Contraddiction

0)0('' g

22 )0(''2

1)(

0

gofor

thatsoenoughsmall

0)0()( − gg

21/09/2018 Pagina 34Prof.Daniela Iacoviello- Optimal Control

Unconstrained optimization – second order conditions

0)0('' g

By differentiating both sides with respect to

Second order necessary condition for optimality

=

+=

n

i

ix xfgi

1

* )()('

)()()0(''

)()(''

*2

1,

*

1,

*

xfxfg

xfg

T

j

n

ji

ixx

j

n

ji

ixx

ji

ji

==

+=

=

=

=

nnn

n

xxxx

xxxx

ff

ff

f

1

111

2

Hessian matrix

0)( *2 xf

21/09/2018 Pagina 35

Unconstrained optimization – second order conditions

Prof.Daniela Iacoviello- Optimal Control

Remark:

The second order condition distinguishes minima from maxima:

At a local maximum the Hessian must be negative semidefinite

At a local minimum the Hessian must be positive semidefinite

21/09/2018 Pagina 36

Unconstrained optimization – second order conditions

Prof.Daniela Iacoviello- Optimal Control

Unconstrained optimization- second order conditions

Let and

is a strict local minimum of f

0)( * = xf2Cf 0)( *2 xf

*x

21/09/2018 Pagina 37Prof.Daniela Iacoviello- Optimal Control

Global minimum – Existence result

Weierstrass Theorem

Let f be a continuous function and D a compact set

there exist a global minimum of f over D

21/09/2018 Pagina 38Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

Let

Equality constraints

Inequality constraints

Regularity condition:

1, CfRD n

1,:,0)( ChRRhxh p →=

1,:,0)( CgRRgxg q →

( )

a

a

x

a

qg

qpx

ghrank

dimensionwith

ofconstraintactivethearegwhere

,

a

*

+=

21/09/2018 Pagina 39Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

Lagrangian function

If the stationary point is called normal

and we can assume

( ) )()()(,,, 00 xgxhxfxL TT ++=

*x00

10 =

21/09/2018 Pagina 40Prof.Daniela Iacoviello- Optimal Control

From now on and therefore the Lagrangian is

If there are only equality constraints the are called

Lagrange multipliers

The inequality multipliers are called Kuhn – Tucker multipliers

i

10 =

( ) )()()(,, xgxhxfxL TT ++=

21/09/2018 Pagina 41

Constrained optimization

Prof.Daniela Iacoviello- Optimal Control

First order necessary conditions for constrained optimality:

Let and

The necessary conditions for to be a constrained local minimum are

If the functions f and g are convex and the functions h are linear these

conditions are necessary and sufficient

i

ixg

x

L

i

ii

T

T

x

=

=

0

,0)(

0

*

*

Dx * 1,, Cghf

*x

21/09/2018 Pagina 42Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

Second order sufficient conditions for constrained optimality:

Let and and assume the conditions

is a strict constrained local minimum if

ixgx

Liii

T

T

x

==

0,0)(0 *

*

Dx * 2,, Cghf

*x

pidx

xdhthatsuch

x

L

x

i

x

T ,...,1,0)(

0**

2

2

==

21/09/2018 Pagina 43Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

Definition:

21/09/2018 Pagina 44Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

2

2

0

j

i

Tj

i

hL

xx

h

x

Bordered Hessian

A point in which and is called

a non-degenerate critical point of the constrained problem.

*x 0L =

2

2

det 0

0

j

i

Tj

i

hL

xx

h

x

Theorem:

21/09/2018 Pagina 45Prof.Daniela Iacoviello- Optimal Control

Constrained optimization

A necessary and sufficient condition for the non-degenerate critical point

to minimize the cost subjects to the constraints

is that for all non-zero tangent vector

E.R.Pinch, Optimal control and calculus of variations, Oxford Science Publications, 1993

*x

f *( ) 0, 1,2,...,jh x j m= =

2

20T L

x

Let us consider the problem of minimizing function

with one constraint

The previous theorem implies that we should look for and

such that:

If the normal to the constraint curve

at has its

normal parallel to

the level surface of that passes through has the

same normal direction as the constraint curve at

In R2 this means that the two curves touch at

21/09/2018 Pagina 46Prof.Daniela Iacoviello- Optimal Control

A geometrical interpretation1 2( , )f x x

1 2( , )h x x c=

1 2( , )x x x=

( ) 0f h at x + = f h at x = −

0f 0 0h

1 2( , )h x x c= 1 2( , )x x x=

( )f x

1 2( , )f x x1 2( , )x x x=

1 2( , )x x x=

1 2( , )x x x=

Minimize the function

subject to

Introduce a Lagrange multiplier and consider:

There are three unknowns and we need another equation:

and

21/09/2018 Pagina 47Prof.Daniela Iacoviello- Optimal Control

A geometrical interpretation - Example

2 21 2 1 2( , ) 1f x x x x= − −

21 2 2 1( , ) 1 0h x x x x= − + =

( )2 2 21 2 2 1( ) 1 ( 1 ) 0f h x x x x + = − − + − + =

1 1 22 0 2 0x x x − + = − + =

22 11 0x x− + =

1 20, 1, 2x x = = = 1 21 , 1/ 2, 1

2x x = = =

21/09/2018 Pagina 48Prof.Daniela Iacoviello- Optimal Control

A geometrical interpretation - Example

1 20, 1, 2x x = = =

1 21 , 1/ 2, 1

2x x = = =

The minimum is at

You can find it evaluating

in the two points

Note that the level curves of f

are centered in the origin O

and f decreases

as you move away from the

origin.

1 20, 1x x= =

2 21 2 1 2( , ) 1f x x x x= − −

Function spaces

Functional

Vector space V,

is a local minimum of J over A if there exists an

RVJ →:

VA

Az *

)()(

0

*

*

zJzJ

zzsatisfyingAzallforthatsuch

21/09/2018 Pagina 49Prof.Daniela Iacoviello- Optimal Control

Consider function in V of the form

The first variation of J at z is the linear function

such that

First order necessary condition for optimality:

For all admissible perturbation we must have:

RVz + ,,

RVJz

→: and

)()()()( oJzJzJ z ++=+

0* =z

J

Function spaces

21/09/2018 Pagina 50Prof.Daniela Iacoviello- Optimal Control

A quadratic form is the second variation of J at z if

we have:

second order necessary condition for optimality:

If is a local minimum of J over

for all admissible perturbation we must have:

RVJz

→:2

and

)()()()()( 222 oJJzJzJ zz +++=+

0)(*

2 z

J

Az *VA

Function spaces

21/09/2018 Pagina 51Prof.Daniela Iacoviello- Optimal Control

The Weierstrass Theorem is still valid

If J is a convex functional and is a convex set a local minimum is

automatically a global one and the first order condition are necessary

and sufficient condition for a minimum

VA

Function spaces

21/09/2018 Pagina 52Prof.Daniela Iacoviello- Optimal Control