OptimalControl_1
description
Transcript of OptimalControl_1
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
INTRODUCTION TO OPTIMAL CONTROL
SYSTEMS
Harry G. Kwatny
Department of Mechanical Engineering & MechanicsDrexel University
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
OUTLINEINTRODUCTION
PreliminariesBibliography
THE OPTIMAL CONTROL PROBLEMDefinitionExamples
OPTIMIZATION BASICSLocal ExtremaConstraints
VARIATIONAL CALCULUSClassical Variational CalculusFree Terminal TimeSystems with Constraints
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
PRELIMINARIES
WHAT IS OPTIMAL CONTROL?
I Optimal control is an approach to control systems design thatseeks the best possible control with respect to a performancemetric.
I The theory of optimal control began to develop in the WW IIyears. The main result of this period was the Wiener-Kolmogorovtheory that addresses linear SISO systems with Gaussian noise.
I A more general theory began to emerge in the 1950’s and 60’sI In 1957 Bellman published his book on Dynamic ProgrammingI In 1960 Kalman published his multivariable generalization of
Wiener-KolmogorovI In 1962 Pontryagin et al published the maximal principleI In 1965 Isaacs published his book on differential games
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
PRELIMINARIES
COURSE CONTENT
I Introduction & Preliminary ExamplesI Optimization ConceptsI Reachability, Controllability & Time Optimal ControlI The Maximum PrincipleI Principle of Optimality, Dynamic programming, & the HJB
EquationI Game Theory
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
BIBLIOGRAPHY
BIBLIOGRAPHY
I E. B. Lee and L. Markus, Foundations of Optimal ControlTheory. New York: John Wiley and Sons, Inc., 1967.
I A. E. Bryson and Y.-C. Ho, Applied Optimal Control.Waltham: Blaisdell, 1969.
I S. J. Citron, Elements of Optimal Control. New York: Holt,Rinehart and Winston, Inc., 1969.
I D. E. Kirk, Optimal Control Theory: An Introduction.Englewood Cliffs, NJ: Prentice-Hall, 1970. (now availablefrom Dover)
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
DEFINITION
PROBLEM DEFINITION
We will define the basic optimal control problem:I Given system dynamics
x = f (x, u) , x ∈ X ⊂ Rn, u ∈ U ⊂ Rm
I Find a control u (t) , t ∈ [0,T] that steers the system froman initial state x (0) = x0 to a target set G and minimizes thecost
J (u (·)) = gT (x (T)) +
∫ T
0g (x (t) , u (t)) dt
REMARKgT is called the terminal cost and g is the running cost. The terminaltime T can be fixed or free. The target set can be fixed or moving.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
DEFINITION
OPEN LOOP VS. CLOSED LOOP
I If we are concerned with a single specified initial state x0, thenwe might seek the optimal control control u (t), u : R→ Rm thatsteers the system from the initial state to the target. This is anopen loop control.
I On the other hand, we might seek the optimal control as afunction of the state u (x), u : Rn → Rm. This is a closed loopcontrol; sometimes called a synthesis.
I The open loop control is sometimes easier to compute, and thecomputations are sometimes performed online – a methodknown as model predictive control.
I The closed loop control has the important advantage that it isrobust with respect to model uncertainty, and that once the(sometimes difficult) computations are performed off-line, thecontrol is easily implemented online.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A MINIMUM TIME PROBLEM
Consider steering a unit mass, with bounded applied controlforce, from an arbitrary initial position and velocity to rest at theorigin in minimum time. Specifically,
x = vv = u, |u| ≤ 1
The cost function is
J =
∫ T
0dt ≡ T
REMARKThis is an example of a problem with a control constraint.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A MINIMUM FUEL PROBLEMConsider the decent of a moon lander.
h = vv = −g + u
mm = −ku
The thrust u is used to steer the system to h = 0, v = 0. Inaddition we wish to minimize the fuel used during landing, i.e.
J =
∫ t
0kudt
Furthermore, u is constrained, 0 ≤ u ≤ c, and the stateconstraint h ≥ 0 must be respected.
REMARKThis problem has both control and state constraints.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A LINEAR REGULATOR PROBLEM
Consider a system with linear dynamics
x = Ax + Bu
We seek a feedback control that steers the system from anarbitrary initial state x0 towards the origin in such a way as tominimize the cost
J = xT (T) QTx (T) +1
2T
∫ T
0
{xT (t) Qx (t) + uT (t) Ru (t)
}dt
The final time T is considered fixed.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A ROBUST SERVO PROBLEMConsider a system with dynamics
x = Ax + B1w + B2uz = C1x + D11w + D12uy = C2x + D21w + D22u
where w is an external disturbance. The goal is to find an output (y)feedback synthesis such that the performance variables (processerrors) z remain close to zero. Note that w (t) can be characterized inseveral ways, stochastic (the H2 problem)
J = E[∫ ∞−∞
zT (t) z (t) dt]
or deterministic (the H∞ problem)
J = max‖w‖2=1
∫ ∞−∞
zT(t)z(t)dt
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
DEFINITIONS
Consider the scalar function
f (x) , x ∈ Rn
which is defined and smooth on a domain D ⊂ Rn. We furtherassume that the region D is defined by a scalar inequalityψ (x) ≤ 0, i.e.,
intD = {x ∈ Rn |ψ (x) < 0}∂D = {x ∈ Rn |ψ (x) = 0}
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
DEFINITIONS
DEFINITION (LOCAL MINIMA & MAXIMA)An interior point x∗ ∈ intD is a local minimum if there exists aneighborhood U of x∗ such that
f (x) ≥ f (x∗) ∀x ∈ U
It is a local maximum if
f (x) ≤ f (x∗) ∀x ∈ U
Similarly, for a point x∗ ∈ ∂D, we use a neighborhood U of x∗
within ∂D. With this modification boundary local minima andmaxima are defined as above.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
OPTIMAL INTERIOR POINTS
I Necessary conditions. A point x∗ ∈ intD is an extremalpoint (minimum or maximum) only if
∂f∂x
(x∗) = 0
I Sufficient conditions. x∗ is a minimum if
∂2f∂x2 (x∗) > 0
a maximum if∂2f∂x2 (x∗) < 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
OPTIMIZATION WITH CONSTRAINTS – NECESSARY
CONDITIONSWe need to find extremal points of f (x), with x ∈ ∂D. i.e., findextremal points of f (x) subject to the constraint ψ (x) = 0.
Consider a more general problem where there are m constraints, i.e.,ψ : Rn → Rm.
Let λ be an m-dimensional constant vector (called Lagrangemultipliers) and define the function
H (x, λ)∆= f (x) + λTψ (x)
Then x∗ is an extremal point only if
∂H (x∗, λ)
∂x= 0,
∂H (x∗, λ)
∂λ≡ ψ (x) = 0
Note there are n + m equations in n + m unknowns x, λ
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
SIGNIFICANCE OF THE LAGRANGE MULTIPLIERConsider extremal points of f (x1, x2) subject to the single constraintψ (x1, x2) = 0. At an extremal point (x∗1 , x
∗2 ) we must have
df (x∗1 , x∗2 ) =
∂f (x∗1 , x∗2 )
∂x1dx1 +
∂f (x∗1 , x∗2 )
∂x2dx2 = 0 (1)
but dx1 and dx2 are not independent. They satisfy
dψ (x∗1 , x∗2 ) =
∂ψ (x∗1 , x∗2 )
∂x1dx1 +
∂ψ (x∗1 , x∗2 )
∂x2dx2 = 0 (2)
From (1) and (2) it must be that∂f (x∗1 , x
∗2 )/∂x1
∂ψ (x∗1 , x∗2 )/∂x1
=∂f (x∗1 , x
∗2 )/∂x2
∂ψ (x∗1 , x∗2 )/∂x2
∆= −λ
Accordingly, (1) and (2) yield∂f (x∗1 , x
∗2 )
∂x1+ λ
∂ψ (x∗1 , x∗2 )
∂x1= 0,
∂f (x∗1 , x∗2 )
∂x2+ λ
∂ψ (x∗1 , x∗2 )
∂x2= 0
REMARKNote that these are the form of the previous slide.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
OPTIMIZATION WITH CONSTRAINTS – SUFFICIENT
CONDITIONS
df =∂H∂x
dx +12
dxT ∂2H∂x2 dx− λTdψ + h.o.t.
butdψ =
∂ψ
∂xdx = 0⇒ dx ∈ ker
∂ψ
∂x⇒ dx = Ψdα
Ψ = span ker∂ψ (x)
∂xThus, for x∗ extremal
df (x∗) =12
dαTΨT ∂2H (x∗)∂x2 Ψdα+ h.o.t.
ΨT ∂2H (x∗)∂x2 Ψ > 0⇒ min
ΨT ∂2H (x∗)∂x2 Ψ < 0⇒ max
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
EXAMPLE
f (x1, x2) = x1
(x2
1 + 2x22 − 1
), ψ (x1, x2) = x2
1 + x22 − 1
Interior:(x1, x2, f ) = (0,−0.707, 0) ∨ (0, 0.707, 0) ∨ (−0.577, 0, 0.385) ∨ (0.577, 0.− 0.385)
Boundary:
(x1, x2, λ, f ) = (−0.577,−0.8165, 1.155,−0.385) ∨ (−0.577, 0.8165, 1.155,−0.385)∨ (0.577,−0.8165,−1.155, 0.385) ∨ (0.577, 0.8165,−1.155, 0.385)∨ (−1, 0, 1, 0) ∨ (1, 0,−1, 0)
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
OPTIMIZING A TIME TRAJECTORY
I We are interested in steering a controllable system along atrajectory that is optimal in some sense.
I Three methods are commonly used to address suchproblems:
I The ‘calculus of variations’I The Pontryagin ‘maximal Principle’I The ‘principle of optimality’ and dynamic programming
I The calculus of variations was first invented to characterizethe dynamical behavior of physical systems governed by aconservation law.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS: LAGRANGIAN SYSTEMSA Lagrangian System is characterized as follows:
I The system is define in terms of a vector of configurationcoordinates, q, associated with velocities q.
I The system has kinetic energy T (q, q) = qTM (q) q/2, andpotential energy V (q) from which we define the Lagrangian
L (q, q) = T (q, q)− V (q)
I The system moves along a trajectory q (t), between initialand final times t1, t2 in such a way as to minimize theintegral
J (q (t)) =
∫ t2
t1L (q (t) , q (t)) dt
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
EXAMPLES OF LAGRANGIAN SYSTEMS
T = 12 qTM (q) q
M (q) =(`2
1 (m1 + m2) + `22m2 + 2`1`2m2 cos θ2 `2m2 (`2 + `1 cos θ2)
`2m2 (`2 + `1 cos θ2) `22m2
)V (q) = m1g (g`1 (m1 + m2) sin θ1 + g`2m2 sin (θ1 + θ2))
m1
m2
�1
� 2
1θ
2θ
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS –1: NECESSARY
CONDITIONSI A real-valued, continuously differentiable function q (t) on the
interval [t1, t2] will be called admissible.I Let q∗ (t) be an optimal admissible trajectory and q (t, ε) a not
necessarily optimal trajectory, with
q (t, ε) = q∗ (t) + εη (t)
where ε > 0 is a small parameter and η (t) is arbitrary.I Then
J (q (t, ε)) =
∫ t2
t1L (q∗ (t) + εη (t) , q∗ (t) + εη (t)) dt
I An extremal of J is obtained from
δJ (q (t)) =∂J (q (t, ε))
∂ε
∣∣∣∣ε=0
= 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS – 2
δJ (q (t)) =
∫ t2
t1
(∂L (q∗ (t) , q∗ (t))
∂q∗η (t) +
∂L (q∗ (t) , q∗ (t))q∗
η (t))
dt
δJ ⇒∫∂L∂q
dη +
∫∂L∂q
dt
We can apply the ‘integration by parts’ formula∫udv = uv−
∫vdu
to the first term to obtain
δJ (q (t)) =∫ t2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ + ∂L(q∗(t),q∗(t))q∗
)η (t) dt
+ ∂L(q∗(t),q∗(t))∂q∗ η (t)
∣∣∣t2t1
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS – 3Now, set δJ (q (t)) = 0 to obtain:
I the Euler-Lagrange equations
ddt∂L (q∗ (t) , q∗ (t))
∂q∗− ∂L (q∗ (t) , q∗ (t))
q∗= 0
I the transversality (boundary) conditions
∂L (q∗ (t1) , q∗ (t1))
∂q∗δq (t1) = 0,
∂L (q∗ (t2) , q∗ (t2))
∂q∗δq (t2) = 0
REMARKThese results allow us to treat problems in which the initial and terminal times are fixedand individual components of q(t1) and q(t2) are fixed or free. Other cases of interestinclude: 1) the terminal time is free, and 2) the terminal time is related to the terminalconfiguration, e.g., by a relation ϕ (q (t2) , t2) = 0.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 1Consider the case of fixed initial state and free terminal time. Let q∗(t) be theoptimal trajectory with optimal terminal time t∗2 . The perturbed trajectoryterminates at time t∗2 + δt2. Its end state is q∗ (t∗2 ) + εη (t∗2 + ετ). Theperturbed cost is
J (q∗, δq, δt) =∫ t∗2 +δt
t1
L (q∗ (t) + δq (t) , q∗ (t) + δq (t)) dt
From this we obtain:
δJ (q∗ (t)) =∫ t∗2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ +∂L(q∗(t),q∗(t))
q∗
)δq (t) dt
+∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ δq (t∗2 ) + L (q∗ (t∗2 ) , q
∗ (t∗2 )) δt
Now, we want to allow both the final time and the end point to vary. Theactual end state is:
δq2∆= δq (t∗2 + δt) = δq (t∗2 ) + q∗ (t∗2 ) δt
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 2
δJ (q∗ (t)) =∫ t∗2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ +∂L(q∗(t),q∗(t))
q∗
)δq (t) dt
+∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ δq2 +
(L (q∗ (t∗2 ) , q
∗ (t∗2 ))−∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ q∗ (t∗2 )
)δt
Thus, we have have the Euler-Lagrange Equations, as before, but thetransversality conditions become:
I the previous conditions:∂L (q∗ (t1) , q∗ (t1))
∂q∗δq (t1) = 0,
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗δq (t∗2 ) = 0
I plus, additional conditions:∂L (q∗ (t∗2 ) , q
∗ (t∗2 ))∂q∗
δq (t∗2 ) = 0(L (q∗ (t∗2 ) , q
∗ (t∗2 ))−∂L (q∗ (t∗2 ) , q
∗ (t∗2 ))∂q∗
q∗ (t∗2 ))δt = 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 3
Ordinarily, the initial state and time are fixed so that the transversalityconditions require
δq (t1) = 0
For free terminal time δt is arbitrary. Thus from the transversality conditions:
1. If the terminal state is fixed, δq (t2) = 0 the terminal condition is:
L (q∗ (t∗2 ) , q∗ (t∗2 ))−
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗q∗ (t∗2 ) = 0
2. If the terminal state is free, δq2 arbitrary the terminal condition is:
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗= 0, L (q∗ (t∗2 ) , q
∗ (t∗2 )) = 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FREE ENDPOINT, FIXED TERMINAL TIMEFirst note that in the elementary variational calculus, we couldsimply replace q by x, and add the definition x = u, so that thecost function becomes
J (x (t)) =
∫ t2
t1L (u (t) , x (t)) dt
Hence, we have a simple control problem. Now suppose x ∈ R,t1 = 0, t2 = 1, and L =
(x2 + u2
)/2:
J (x (t)) =
∫ 1
0
12(x2 + u2) dt
The Euler-Lagrange equations become simplyddt
(∂L∂u
)− ∂L∂x
= 0⇒ u− x = 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1, CONT’DThus, we have the equations[
xu
]=
[0 11 0
] [xu
]
x (0) = x0,∂L (x (1) , u (1))
∂u= 0⇒ u (1) = 0
Consequently,
[x (t)u (t)
]= e
0 11 0
t [ab
]⇒[
x (t)u (t)
]=
[a cosh (t) + b sinh (t)b cosh (t) + a sinh (t)
]
a = x0, b =x0 − e2x0
1 + e2
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FIXED ENDPOINT, FIXED TERMINAL TIME
Suppose we consider a fixed end point, say x (t2) = 0 with theterminal time still fixed at t2 = 1. Then the Euler-Lagrangeequations remain the same, but the boundary conditionschange to:
x (0) = x0, x (1) = 0
Thus, we compute
a = x0, b = −x0 + e2x0
−1 + e2
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FIXED ENDPOINT, FREE TERMINAL TIME
Suppose we consider a fixed end point, say x (t2) = xf with theterminal time t2 free. Then the Euler-Lagrange equationsremain the same, but the boundary conditions change to:
x (0) = x0, x (t2) = xf , (L− Luu)|t2 = 0
From this we compute
a = x0, a2 = b2, a cosh (t2) + b sinh (t2) = xf
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1, CONT’D
The figures below show the optimal control and statetrajectories from the initial state x0 = 1,xf = 0, with t2 = 1 for thefixed time case. For the free time case xf = 0.01.
Free terminal state, Fixedtime:
0.2 0.4 0.6 0.8 1.0t
-0.5
0.5
1.0
x,u
Fixed terminal state,Fixed Time:
0.2 0.4 0.6 0.8 1.0t
-1.0
-0.5
0.5
1.0
x,u
Fixed terminal state,Free time:
1 2 3 4 5t
-1.0
-0.5
0.5
1.0
x,u
REMARKIn the free time case, with xf = 0.01, the final time is t2 = 4.60517. With xf = 0 the finaltime is t2 =∞. Note that the case of free terminal state and free terminal time (notshown) is trivial with t2 = 0.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
VARIATIONAL CALCULUS WITH DIFFERENTIAL
CONSTRAINTS
Systems with ‘nonintegrable’ differential , i.e., nonholonomic,constraints have been treated by variational methods. Asbefore, we seek extremals of the functional:
J (q (t)) =
∫ t2
t1L (q (t) , q (t) , t) dt
But now subject to the constraints:
ϕ (q (t) , q (t) , t) = 0
where ϕ : Rn × Rn × R→ Rm.
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
DIFFERENTIAL CONSTRAINTS, CONT’D
The constraints are enforce for all time, so introduce the mLagrange multipliers λ (t) and consider the modified functional
J (q (t)) =
∫ t2
t1L (q (t) , q (t) , t) + λT (t)ϕ (q (t) , q (t) , t) dt
We allow variations in both q and λ and t2, Taking the variationand integrating by parts yields
δJ =∫ t2
t1
({− d
dt
[Lq + λTϕq
]+[Lq + λTϕq
]}δq (t) + ϕTδλ
)dt
+[Lq + λTϕq
]t2δq2 +
([L + λTϕq
]−[Lq + λTϕq
]q)
t2δt2
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
NECESSARY CONDITIONS WITH CONSTRAINTSAgain we have the Euler-Lagrange equations, now in the form:
ddt
[Lq + λTϕq
]−[Lq + λTϕq
]= 0
The differential constraints:
ϕ (q, q, t) = 0
and the boundary conditions[Lq + λTϕq
]t2δq2 = 0
With free terminal time, we also have([L + λTϕq
]−[Lq + λTϕq
]q)
t2= 0
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLEOnce again consider the problem
x = u, J (x (t)) =∫ t2
0
12
(x2 + u2
)dt
But now, add the constraint
ϕ (x, x) = x2 − 1 ≡ u2 − 1 = 0→ u = ±1
The Euler-Lagrange equations now become
ddt
[Lu + λTϕu
]−[Lx + λTϕx
]= 0⇒ λ =
x2u
REMARKI Note that u = ±1→ u = 0 almost everywhere,I Also,
∫ t0 u2dt = t
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE CONTINUED
We now consider three cases:I fixed terminal time, fixed end pointI fixed terminal time, free end pointI free terminal time fixed end point
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FIXED END POINT
Steer from x (0) = x0 to x (1) = x1.
I system x = u
I Euler equation λ = x2u
I constraint u = ±1
I boundary conditions x (0) = x0, x (1) = x1
Assume single switch at time T
x0 + kT − k(1− T) = x1 ∧ (k = ±1) ∧ 0 ≤ T ≤ 1⇒ −1 + x1 ≤ x0 ≤ 1 + x1 ∧ (k = ±1) ∧ T = k−x0+x1
2k
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FREE END POINT
Steer from x (0) = x0.
I system x = u
I Euler equation λ = x2u
I constraint u = ±1
I initial condition x (0) = x0
I terminal condition x (1) = x1
I terminal condition[Lu + λTϕu
]t2= 0⇒ λ (t2) = − 1
2
The necessary conditions are satisfied with
(k = ±1) ∧(−3
4≤ λ0 ≤ −
12
)∧(
T =12−√
1− 2λ0√2
)
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FREE END POINT–2
Only the case λ0 = − 12 ,T = 1
2 is optimal.
λ0 = −12
:
0.0 0.2 0.4 0.6 0.8 1.0-1.0
-0.5
0.0
0.5
t
x,Λ,J
λ0 = −3
4
:
0.0 0.2 0.4 0.6 0.8 1.0-1.0
-0.5
0.0
0.5
t
x,Λ,J
OPTIMAL CONTROL
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FREE TERMINAL TIME, FIXED END POINT
Steer from x (0) = x0 to x (t2) = 0I system x = uI Euler equation λ = x
2uI constraint u = ±1I initial condition x (0) = x0
I terminal condition x (t2) = 0I terminal condition([
L + λTϕq]−[Lu + λTϕu
]u)
t2= 1
2 − 1− 2λ (t2) = 0⇒ λ (t2) = − 14
(x0 ≥ 0) ∧ (k = −1) ∧ (t2 = x0) ∧(λ0 =
14(1 + x2
0
))∨
(x0 ≤ 0) ∧ (k = +1) ∧ (t2 = −x0) ∧(λ0 =
14(1 + x2
0
))
OPTIMAL CONTROL