Linear Optimal control - uniroma1.itiacoviel/Materiale/OptimalControl-Lecture7.pdfD. Liberzon,...
Transcript of Linear Optimal control - uniroma1.itiacoviel/Materiale/OptimalControl-Lecture7.pdfD. Liberzon,...
Optimal Control
Lecture
Prof. Daniela Iacoviello
Department of Computer, Control, and Management Engineering Antonio Ruberti
Sapienza University of Rome
10/11/2015 Controllo nei sistemi biologici
Lecture 1
Pagina 2
Prof. Daniela Iacoviello
Department of computer, control and management
Engineering Antonio Ruberti
Office: A219 Via Ariosto 25
http://www.dis.uniroma1.it/
Prof.Daniela Iacoviello- Optimal Control
References B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972 Prof.Daniela Iacoviello- Optimal Control
References
D. E. Kirk, "Optimal Control Theory: An Introduction, New
York, NY: Dover, 2004 D. Liberzon, "Calculus of Variations and Optimal Control
Theory: A Concise Introduction", Princeton University
Press, 2011 How, Jonathan, Principles of optimal control, Spring
2008. (MIT OpenCourseWare: Massachusetts Institute of
Technology). License: Creative Commons BY-NC-SA.
Prof.Daniela Iacoviello- Optimal Control
The linear time-optimal control
Let us consider the problem of optimal control
of a linear system
with fixed initial and final state,
constraints on the control variables and
with cost index equal to the length of the
time interval Prof.Daniela Iacoviello- Optimal Control
Problem: Consider the linear dynamical system:
With and the constraint
Matrices A and B have entries with elements of class
Cn-2 and Cn-1 respectively.
The initial instant ti is fixed and also:
)()()()()( tutBtxtAtx pn RtuRtx )(,)(
Rtpjtu j ,,...,2,1,1)(
0)(,)( Txxtx i
i
Prof.Daniela Iacoviello- Optimal Control
The aim is to determine the final instant
the control and the state
that minimize the cost index:
)(RCu oo
RT o
)(1 RCxo
i
T
t
tTdtTJ
i
)(
Prof.Daniela Iacoviello- Optimal Control
Theorem: Necessary conditions for
to be an optimal solution are that there exists a
constant and an n-dimensional
function , not simultaneously
null and such that:
Possible discontinuities in can appear only in
the points in which has a discontinuity.
*** ,, Tux
0*
0 TtC i ,
1*
pjRBuB
A
j
pTT
T
,...,2,1,1:***
**
*
*uProf.Daniela Iacoviello- Optimal Control
Moreover the associated Hamiltonian is a
continuous function with respect to t and it
results:
Proof: The Hamiltonian associated to the problem is:
Applying the minimum principle the theorem is proved.
0*
* TH
BuAxtuxH TT 00 ,,,,
Prof.Daniela Iacoviello- Optimal Control
Notation:
Let us indicate with the j-th column of the matrix B . The strong controllability corresponds to the condition:
where :
)(tb j
i
n
jjjj
ttpj
tbtbtbtG
,,...,2,1
0)()()(det)(det )()2()1(
nktbtAtbtb
tbtb
k
j
k
j
k
j
jj
,...,3,2)()()()(
)()(
)1()1()(
)1(
Prof.Daniela Iacoviello- Optimal Control
Remark:
Strong controllability corresponds to the controllability in any instant ti, in any time interval and by any component of the control vector
Prof.Daniela Iacoviello- Optimal Control
Remark:
In the NON-steady state case the above condition is sufficient for strong controllability.
In the steady case it is necessary and sufficient and may be written in the usual way:
Prof.Daniela Iacoviello- Optimal Control
,,...,2,10det 1 pjbAAbb j
n
jj
The Pontryagin principle- remind
Problem 3: Consider the dynamical system:
with:
tuxfx ,,
niURCx
ff
RUtuRtx
n
i
pn
,...,2,1,,
)(,)(
0
Prof.Daniela Iacoviello- Optimal Control
Assume fixed the initial control instant and the
initial state
while for final values assume:
where is a function of dimension
of C1 class.
Define the performance index :
with
i
i xtx )(
T
ti
duxLTuxJ ),(),(,,
niRURCt
L
x
LL n
i
,...,2,1,,, 0
0),( TTx 1 nf
Prof.Daniela Iacoviello- Optimal Control
Determine the value
the control and the state
that satisfy the dynamical
system, the constrain t on the control, the
initial and final conditions and minimize the
cost index .
,itT
)(0 RCuo
)(1 RCxo
Prof.Daniela Iacoviello- Optimal Control
Theorem 3- remind:
Consider an admissible solution
such that
IF it is a minimum there exist a constant
and an n-dimensional vector
not simultaneously null such that :
*** ,, Tux
00
*1* ,TtC i
fTTx
rank
*
),(
Prof.Daniela Iacoviello- Optimal Control
RkkdH
Hx
HT
t
T
,,
**
**
*
U
ttutxHttxH
,)(,),(),()(,,),( **
0
****
0
*
Moreover there exists a vector such that:
T
T
T
TH
TxT
**
*
**
)()(
fR
Prof.Daniela Iacoviello- Optimal Control
SINGULAR SOLUTIONS
Definition: Let be an optimal solution of the above problem, and the corresponding multipliers.
The solution is singular if there exists a subinterval
in which the Hamiltonian
Is independent from at least one component of in
Prof.Daniela Iacoviello- Optimal Control
ooo Tux ,,oo 0
''','',' tttt
tttxH ooo ),(,,),( 0
'',' tt
SINGULAR SOLUTIONS
Theorem:
Assume the Hamiltonian of the form
Let be an extremum and the corresponding multipliers such that
Is dependent on any compenent of in any subinterval of
A necessary condition for to be a singular extremum is that there exists a subinterval
such that:
Prof.Daniela Iacoviello- Optimal Control
*** ,, Tux **
0
),,,,(),,,(),,,(),,,,( 002010 tuxNtxHtxHtuxH
),,,,( *
0** txN
],[ Tti
*** ,, Tux
'''],,[]'','[ ttTttt i
]'','[,0),,,( 02 ttttxH
Characterization of the optimal solution
Theorem: Let us consider the minimal time optimal problem .
If the strong controllability condition is satisfied and if an optimal solution exists
it is non singular.
Moreover every component of the optimal control is a piecewise constant assuming only the extreme values and the number of discontinuity instant is limited.
1
Prof.Daniela Iacoviello- Optimal Control
Proof: the non singularity of the solution is proved by contradiction arguments.
If the solution were singular, from the necessary condition of the previous theorem:
There would exist a value
and a subinterval such that:
pjR
BuB
j
p
TT
,...,2,1,1:
***
pj ,...,2,1
],[]'','[ o
i Tttt
]'','[,0)()( ttttbt j
oT Prof.Daniela Iacoviello- Optimal Control
Deriving successively this expression we obtain:
On the other hand must be different from zero in every instant of the interval , otherwise, from the necessary condition
it should be null in the whole interval and in particular .
From the necessary condition:
it would result that also which is absurd.
Therefore, from the condition:
]'','[,0)()( ttttGt j
oT
)(to
** TA0)( To
0*
* TH
00 o
]'','[,0)()( ttttGt j
oT Prof.Daniela Iacoviello- Optimal Control
it results that which contradicts the hypothesis of strong controllability . From the non singularity of the optimal solution it results that the quantity can be null only on isolated points . Therefore the necessary condition implies
]'','[0)(det ttttG j
)()( tbt j
oT
pjRBuB j
pTT ,...,2,1,1:***
],[,...,2,1
,)()()(
o
i
j
oTo
j
Tttpj
tbtsigntu
Prof.Daniela Iacoviello- Optimal Control
To demonstrate that the number of discontinuity instants is finite, we can proceed by contradiction argument that would
Easily imply
which contradicts the hypothesis of strong controllability
]'','[0)(det ttttG j
Prof.Daniela Iacoviello- Optimal Control
A control function that assumes only
the limit values is called bang-bang
control and
the relative instants of discontinuity
are called commutation instants
Prof.Daniela Iacoviello- Optimal Control
Remark
This result holds also to the case in which the control is in the space of measurable function defined on the space of real number R, M(R).
Prof.Daniela Iacoviello- Optimal Control
Theorem (UNIQUENESS):
If the hypothesis of strong controllability is satisfied, if an optimal solution exists it is unique.
Proof: the theorem is proved by contradiction arguments.
Prof.Daniela Iacoviello- Optimal Control
Existence of the optimal solution
Theorem: if the condition of strong controllability is satisfied and an admissible solution exists, then there exists a unique optimal solution nonsingular and the control is bang-bang.
Proof: the existence theorem is proved on the basis of the following results.
Prof.Daniela Iacoviello- Optimal Control
The following result holds:
Theorem: Let assume that the control functions belong to the space M(R).
If an admissible solution exists then the optimal solution exists.
Proof: let us consider the non trivial case in which the number of admissible solution is infinite and let us indicate with the minimum extremum of the instants T corresponding to the admissible solutions
oT
Prof.Daniela Iacoviello- Optimal Control
It is possible to built a sequence of admissible solutions
such that
Let us indicate with the transition matrix of the linear system. It results:
o
kkkk TTTux ,,,o
kk
TT
lim
,t
0)()(,lim
)()(,,lim
,,lim)()(lim
dttutBtT
dttutBtTtT
xtTtTTxTx
k
T
T
kk
k
T
t
o
kk
i
i
o
ikk
o
kkkk
k
o
o
i
Prof.Daniela Iacoviello- Optimal Control
Since it results:
Let us indicate by the function
truncated on the interval . This
function is limited and L2 . The sequence of
functions belongs to the space of
measurable function M , L2 and such that
,....2,1,0)( kTx kk
0)(lim
o
kk
Tx
ku ku
o
i Tt ,
ku
o
ij Tttpjtu ,,,...,2,1,1)(
Prof.Daniela Iacoviello- Optimal Control
Therefore this sequence admits a subsequence
weakly converging to a function .
This implies that for any measurable L2 function it results:
If we indicate by the evolution of the state corresponding to the input from the initial condition we have:
ku Muo
h
o
i
o
i
T
t
oT
T
t
K
T
kdttuthdttuth )()()()(lim
oxou
Prof.Daniela Iacoviello- Optimal Control
Therefore, taking into account the expression
, we deduce :
An admissible measurable control L2
exists able to transfer the initial
state to the origin.
The optimal solution is
)()()(,,
)()(,lim,)(lim
oo
T
t
ooi
i
o
T
t
k
o
k
i
i
oo
kk
TxdttutBtTxtT
dttutBtTxtTTx
o
i
o
i
0)(lim
o
kk
Tx 0)( oo Tx
ooo Tux ,,Prof.Daniela Iacoviello- Optimal Control
Therefore it results in:
If the strong controllability condition is satisfied and if an optimal control in the space of measurable function exists then this solution is unique, non singular and the control is bang bang type
Remark
The existence of the optimal solution is guaranteed only for the couples
for which the admissible solution exists
ii xt ,
Prof.Daniela Iacoviello- Optimal Control
If the strong controllability
condition is satisfied and if an
optimal control in the space of
measurable function exists
then this solution is unique,
non singular and the control
is bang bang type
If the strong controllability
condition is satisfied and if an
admissible solution exists then
this solution is unique,
non singular and the control is
bang bang type
Necessary condition for with u in M(R) to be an optimal solution; its characterization; unicity results
*** ,, Tux
Summary
Prof.Daniela Iacoviello- Optimal Control
Minimum time problem for steady state system
Let us consider the minimum time problem in case of steady state system.
In this case it is possible to deduce results about the number of commutation points
Prof.Daniela Iacoviello- Optimal Control
Problem: Consider the linear dynamical system:
With and the constraint
Matrices A and B have entries with elements of class
Cn-2 and Cn-1 respectively.
The initial instant ti is fixed and also:
)()()( tButAxtx pn RtuRtx )(,)(
Rtpjtu j ,,...,2,1,1)(
0)(,)( Txxtx i
i
Prof.Daniela Iacoviello- Optimal Control
The aim is to determine the final instant
the control and the state
that minimize the cost index:
)(RCu oo
RT o
)(1 RCxo
i
T
t
tTdtTJ
i
)(
Prof.Daniela Iacoviello- Optimal Control
Theorem:
Let us consider the above time minimum problem and
assume that the control function belong to the space
of measurable function M(R); if the system is
controllable there exists a neighbor Ω of the origin
such that for any there exists an optimal
solution.
ix
Prof.Daniela Iacoviello- Optimal Control
Proof:
We will show that it is possible to determine a neighbor
Ω of the origin such that for any there exists
an admissible solution. On the basis of the previous
results this implies the existence of the optimal
solution.
Let us fix any T>0. The existence of a control able to
transfer the initial state to the origin at the
instant T corresponds to the possibility of solving :
ix
ix
Prof.Daniela Iacoviello- Optimal Control
with respect to the control function u.
Let assume for the control the expression:
and substitute it in the preceeding equation:
0)(
T
t
TAitTA
i
i dBuexe
p
i
AT RTteBuT
,,)(
T
t
ATAiAt
i
Ti deBBexe
Gramian matrix
Prof.Daniela Iacoviello- Optimal Control
From the controllability hypothesis, the Gramian
matrix is not singular, therefore:
a measurable admissible control exists if the
initial state is sufficiently near the origin.
Tt
xedeBBeeBu
i
iAt
T
t
ATAAT i
i
TT
,)(
1
Prof.Daniela Iacoviello- Optimal Control
Remark
From results obtained in the non steady state case it results that an optimal non singular solution exists and the control is bang-bang type.
Prof.Daniela Iacoviello- Optimal Control
Theorem: Let us consider the above time
minimum problem in the steady state case
and assume that the control function belongs
to the space of measurable function M(R).
If the system is controllable and the eigenvalues
of matrix A have negative real part there
exists an optimal solution whatever the initial
state is.
Prof.Daniela Iacoviello- Optimal Control
Proof: Whatever the initial state is chosen, an
admissible solution can be defined as follows:
• Let Ω be a neighbor of the origin constituted by the
original state for which an admissible solution exists;
• Let Ω’ be a closed subset in Ω
• From the asymptotic stability of the system it is
possible to reach Ω’ in a finite time with null input,
from any initial state
Prof.Daniela Iacoviello- Optimal Control
Remark
From results obtained in the non steady state case it results that an optimal non singular solution exists and the control is bang-bang type.
Question: how many commutation instants are possible in the optimal time control problem?
Prof.Daniela Iacoviello- Optimal Control
Theorem: If the strong controllability condition is satisfied and if all the eigenvalues of the matrix A are real non positive, then the number of commutation instants for any components of control is less or equal than
n-1, whatever the inititial state is.
Proof: from the previous results, we have:
o
i
j
oTo
j
Tttpj
btsigntu
,,,...,2,1
,)()(
Prof.Daniela Iacoviello- Optimal Control
where the costate in the steady state case is:
Denoting by and the eigenvalues of A and their molteplicity, the r-th component of
can be expressed as follows:
Where prs is a polynomial function of degree less than ms.
0
i
ttA iT
et )(0 )(
k km
0
nretpt
k
s
tt
rsris ,...,2,1,)()(
1
)(0
Prof.Daniela Iacoviello- Optimal Control
Substituting this expression in the bang bang control we obtain:
o
i
k
s
tt
js
k
s
tt
n
r
rsjr
o
j
Tttpj
etPsign
etpbsigntu
is
is
,;,...,2,1
,)(
)()(
1
)(
1
)(
1
Polynomial function of degree less than ms
Prof.Daniela Iacoviello- Optimal Control
The argument of the sign function has at most
m1 + m12+ …. + mk-1 =n-1
real solutions.
Therefore the control has at most n-1 roots.
o
ju
Prof.Daniela Iacoviello- Optimal Control
References
B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972
Prof.Daniela Iacoviello- Optimal Control