Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. ·...
Transcript of Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. ·...
![Page 1: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/1.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
Pontryagin’s maximum principle for optimalcontrol of SPDEs
AbdulRahman Al-Hussein
Department of Mathematics, College of Science, Qassim University,P. O. Box 6644, Buraydah 51452, Saudi Arabia
E-mail: [email protected]
Workshop on Stochastic Control Problems forFBSDEs and Applications
Marrakesh, December 13-18, 2010
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 2: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/2.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Outline
1 Problem formulationStatementAssumptionsThe adjoint equation
2 Pontryagin stochastic maximum principle
3 Plan of the proofEstimatesVariational inequality
ProofTwo more lemmasProof of main result
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 3: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/3.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Consider the SPDE on a sep. Hilbert space K :dX u(·)(t) = (A(t) X u(·)(t) + F (X u(·)(t),u(t)) )dt + G(X u(·)(t))dM(t),X u(·)(0) = x .
(1)
M is a continuous martingale, << M >>t =∫ t
0 Q(s) ds,for a predictable process Q(·) s.t. Q(t) ∈ L1(K ) is symmetric,positive definite, Q(t) ≤ Q, where Q ∈ L1(K ) (positive definite).
F : K ×O → K (O is a sep. Hilbert space).
G : K → L2(K0,K ), K0 = Q−1/2(K ).
A(t , ω) is a predictable unbounded linear operator on K .
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 4: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/4.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
. We shall derive a stochastic maximum principle for this controlproblem by using the adjoint equation (BSPDE).
u(·) : [0,T ]× Ω→ O is admissible if u(·) ∈ L2F (0,T ;O) and
u(t) ∈ U a.e., a.s. ( U is a nonempty convex subset of O ) .
The set of admissible controls Uad .
L2F (0,T ; E) := ψ : [0,T ]× Ω→ E ,predictable,
E [∫ T
0 |ψ(t)|2Edt ] <∞ .
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 5: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/5.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
. We shall derive a stochastic maximum principle for this controlproblem by using the adjoint equation (BSPDE).
u(·) : [0,T ]× Ω→ O is admissible if u(·) ∈ L2F (0,T ;O) and
u(t) ∈ U a.e., a.s. ( U is a nonempty convex subset of O ) .
The set of admissible controls Uad .
L2F (0,T ; E) := ψ : [0,T ]× Ω→ E ,predictable,
E [∫ T
0 |ψ(t)|2Edt ] <∞ .
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 6: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/6.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Definethe cost functional:
J(u(·)) := E [
∫ T
0g(X u(·)(t), u(t) ) dt + φ(X u(·)(T )) ],
J∗ := infJ(u(·)) : u(·) ∈ Uad.
The control problem for this SPDE (1) is to find a control u∗(·) and thecorresponding solution X u∗(·) of (1) s.t.
J∗ = J(u∗(·)).
u∗(·) is an optimal control.
X u∗(·) (or briefly X ∗) is an optimal solution.
The pair (X ∗ ,u∗(·)) is an optimal pair.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 7: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/7.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Definethe cost functional:
J(u(·)) := E [
∫ T
0g(X u(·)(t), u(t) ) dt + φ(X u(·)(T )) ],
J∗ := infJ(u(·)) : u(·) ∈ Uad.
The control problem for this SPDE (1) is to find a control u∗(·) and thecorresponding solution X u∗(·) of (1) s.t.
J∗ = J(u∗(·)).
u∗(·) is an optimal control.
X u∗(·) (or briefly X ∗) is an optimal solution.
The pair (X ∗ ,u∗(·)) is an optimal pair.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 8: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/8.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
(H1) F ,G,g, φ are C1 w.r.t. x , F is C1 w.r.t. u,the derivatives Fx , Fu, Gx ,gx are uniformly bounded.
Also |φx |K ≤ C1 (1 + |x |K ), some constant C1 > 0.
(H2) gx satisfies Lipschitz condition with respect to u uniformly inx .
(H3) A(t , ω) is a predictable linear operator on K , belongs toL(V ; V ′) ( (V ,K ,V ′) is a Gelfand triple ),
1 2⟨A(t , ω) y , y
⟩+ α |y |2
V≤ λ |y |2 a.e. t ∈ [0,T ] , a.s. ∀ y ∈ V ,
for some α, λ > 0.
2 ∃ C2 ≥ 0 s.t. |A(t , ω) y |V ′ ≤ C2 |y |V ∀ (t , ω), ∀ y ∈ V .
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 9: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/9.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Define the Hamiltonian:
H : [0,T ]× Ω× K ×O × K × L2(K )→ R,
H(t , x ,u, y , z) := −g(x ,u) +⟨F (x ,u) , y
⟩+⟨G(x)Q1/2(t) , z
⟩2 .
The (adjoint) BSPDE:
−dY u(·)(t) = [ A∗(t) Y u(·)(t) +∇xH(X u(·)(t),u(t),Y u(·)(t),Z u(·)(t)Q1/2(t)) ]dt−Z u(·)(t)dM(t)− dNu(·)(t), 0 ≤ t ≤ T ,
Y u(·)(T ) = −∇φ(X u(·)(T )),
A∗(t) is the adjoint operator of A(t).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 10: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/10.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
Define the Hamiltonian:
H : [0,T ]× Ω× K ×O × K × L2(K )→ R,
H(t , x ,u, y , z) := −g(x ,u) +⟨F (x ,u) , y
⟩+⟨G(x)Q1/2(t) , z
⟩2 .
The (adjoint) BSPDE:
−dY u(·)(t) = [ A∗(t) Y u(·)(t) +∇xH(X u(·)(t),u(t),Y u(·)(t),Z u(·)(t)Q1/2(t)) ]dt−Z u(·)(t)dM(t)− dNu(·)(t), 0 ≤ t ≤ T ,
Y u(·)(T ) = −∇φ(X u(·)(T )),
A∗(t) is the adjoint operator of A(t).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 11: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/11.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
M2,c[0,T ](K ) the space of continuous square integrable
martingales in K .
Two elements M and N ofM2,c[0,T ](K ) are very strongly orthogonal
(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],
for all [0,T ] - valued stopping times τ.
In fact: M and N are VSO⇔ << M,N >> = 0.
Λ2(K ;P,M) the space of integrands w.r.t. M s.t.
Φ(t , ω)Q1/2(t , ω) ∈ L2(K ), for every h ∈ K
the K - valued process Φ Q1/2(h) is predictable,
E [∫ T
0 ||(Φ Q1/2)(t)||22 dt ] <∞.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 12: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/12.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
M2,c[0,T ](K ) the space of continuous square integrable
martingales in K .
Two elements M and N ofM2,c[0,T ](K ) are very strongly orthogonal
(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],
for all [0,T ] - valued stopping times τ.
In fact: M and N are VSO⇔ << M,N >> = 0.
Λ2(K ;P,M) the space of integrands w.r.t. M s.t.
Φ(t , ω)Q1/2(t , ω) ∈ L2(K ), for every h ∈ K
the K - valued process Φ Q1/2(h) is predictable,
E [∫ T
0 ||(Φ Q1/2)(t)||22 dt ] <∞.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 13: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/13.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
M2,c[0,T ](K ) the space of continuous square integrable
martingales in K .
Two elements M and N ofM2,c[0,T ](K ) are very strongly orthogonal
(VSO) ifE [M(τ)⊗ N(τ)] = E [M(0)⊗ N(0)],
for all [0,T ] - valued stopping times τ.
In fact: M and N are VSO⇔ << M,N >> = 0.
Λ2(K ;P,M) the space of integrands w.r.t. M s.t.
Φ(t , ω)Q1/2(t , ω) ∈ L2(K ), for every h ∈ K
the K - valued process Φ Q1/2(h) is predictable,
E [∫ T
0 ||(Φ Q1/2)(t)||22 dt ] <∞.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 14: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/14.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
DefinitionA solution of a BSPDE: −dY (t) =
(A(t)Y (t) + f (t ,Y (t),Z (t)Q1/2(t))
)dt
−Z (t)dM(t)− dN(t), 0 ≤ t ≤ T ,Y (T ) = ξ,
is (Y ,Z ,N) ∈ L2F (0,T ; V )× Λ2(K ;P,M)×M2,c
[0,T ](K ) s.t. ∀ t ∈ [0,T ] :
Y (t) = ξ +
∫ T
t
(A(s) Y (s) + f (s,Y (s),Z (s)Q1/2(s))
)ds
−∫ T
tZ (s)dM(s)−
∫ T
tdN(s),
N(0) = 0, N is VSO to M.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 15: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/15.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
. The following theorem gives the unique solution to the adjointequation.
Theorem 1 (Existence & uniqueness of the solution of the adjointBSPDE)
Assume (H1)–(H3). There exists a unique solution (Y u(·),Z u(·),Nu(·) )
to the adjoint BSPDE in L2F (0,T ; V )× Λ2(K ;P,M)×M2,c
[0,T ](K ).
The proof of this theorem can be found in:[ Al-Hussein, A., Backward stochastic partial differential equations driven byinfinite dimensional martingales and applications, Stochastics, Vol. 81, No. 6,2009, 601-626 ].
Denote the solution of the (adjoint) BSPDE corresponding to u∗(·) by(Y ∗,Z ∗,N∗).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 16: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/16.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
. The following theorem gives the unique solution to the adjointequation.
Theorem 1 (Existence & uniqueness of the solution of the adjointBSPDE)
Assume (H1)–(H3). There exists a unique solution (Y u(·),Z u(·),Nu(·) )
to the adjoint BSPDE in L2F (0,T ; V )× Λ2(K ;P,M)×M2,c
[0,T ](K ).
The proof of this theorem can be found in:[ Al-Hussein, A., Backward stochastic partial differential equations driven byinfinite dimensional martingales and applications, Stochastics, Vol. 81, No. 6,2009, 601-626 ].
Denote the solution of the (adjoint) BSPDE corresponding to u∗(·) by(Y ∗,Z ∗,N∗).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 17: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/17.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
StatementAssumptionsThe adjoint equation
. The following theorem gives the unique solution to the adjointequation.
Theorem 1 (Existence & uniqueness of the solution of the adjointBSPDE)
Assume (H1)–(H3). There exists a unique solution (Y u(·),Z u(·),Nu(·) )
to the adjoint BSPDE in L2F (0,T ; V )× Λ2(K ;P,M)×M2,c
[0,T ](K ).
The proof of this theorem can be found in:[ Al-Hussein, A., Backward stochastic partial differential equations driven byinfinite dimensional martingales and applications, Stochastics, Vol. 81, No. 6,2009, 601-626 ].
Denote the solution of the (adjoint) BSPDE corresponding to u∗(·) by(Y ∗,Z ∗,N∗).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 18: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/18.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
Outline
1 Problem formulationStatementAssumptionsThe adjoint equation
2 Pontryagin stochastic maximum principle
3 Plan of the proofEstimatesVariational inequality
ProofTwo more lemmasProof of main result
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 19: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/19.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
Theorem 2 (Pontryagin stochastic maximum principle)
Suppose (H1)–(H3). Assume (X ∗,u∗(·)) is an optimal pair for ourcontrol problem.
Then there exists a unique solution (Y ∗,Z ∗,N∗) to the correspondingBSPDE s.t. the following inequality holds:
⟨∇uH(t ,X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),u − u∗(t)
⟩O ≤ 0,
∀ u ∈ U, a.e. t ∈ [0,T ], a.s.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 20: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/20.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Outline
1 Problem formulationStatementAssumptionsThe adjoint equation
2 Pontryagin stochastic maximum principle
3 Plan of the proofEstimatesVariational inequality
ProofTwo more lemmasProof of main result
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 21: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/21.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
. Let u∗(·) be an optimal control and X ∗ be the corresponding
solution of SPDE (1). Let u(·) be s.t. u∗(·) + u(·) ∈ Uad .
For 0 ≤ ε ≤ 1 consider the variational control:
uε(t) = u∗(t) + εu(t), t ∈ [0,T ].
The convexity of U ⇒ uε(·) ∈ Uad .
. Get the corresponding Xε of (1).
. Let p be the solution of the linear equation: dp(t) = (A(t)p(t) + Fx (X ∗(t),u∗(t))p(t))dt+ Fu(X ∗(t),u∗(t))u(t)dt + Gx (X ∗(t))p(t)dM(t),
p(0) = 0.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 22: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/22.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
. Let u∗(·) be an optimal control and X ∗ be the corresponding
solution of SPDE (1). Let u(·) be s.t. u∗(·) + u(·) ∈ Uad .
For 0 ≤ ε ≤ 1 consider the variational control:
uε(t) = u∗(t) + εu(t), t ∈ [0,T ].
The convexity of U ⇒ uε(·) ∈ Uad .
. Get the corresponding Xε of (1).
. Let p be the solution of the linear equation: dp(t) = (A(t)p(t) + Fx (X ∗(t),u∗(t))p(t))dt+ Fu(X ∗(t),u∗(t))u(t)dt + Gx (X ∗(t))p(t)dM(t),
p(0) = 0.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 23: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/23.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
. We obtain:
Lemma 1Assume (H1), (H3). Let
ηε(t) =Xε(t)− X ∗(t)
ε− p(t), t ∈ [0,T ].
Then:
1 supt∈[0,T ]
E [ |p(t)|2 ] <∞,
2 supt∈[0,T ]
E [ |Xε(t)− X ∗(t)|2 ] = O(ε2),
3 limε→0+
supt∈[0,T ]
E [ |ηε(t)|2 ] = 0.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 24: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/24.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Lemma 2 (Variational inequality)
Suppose (H1)–(H3). Then ∀ ε > 0,
J(uε(·))− J(u∗(·)) = ε E [φx (X ∗(T )) p(T ) ]
+ ε E [
∫ T
0gx (X ∗(s),u∗(s)) p(s) ds ]
+ E [
∫ T
0
(g(X ∗(s),uε(s))− g(X ∗(s),u∗(s))
)ds ]
+ o(ε).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 25: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/25.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Sketch proof of Lemma 2
Note
J(uε(·))− J(u∗(·)) = I1(ε) + I2(ε),
where
I1(ε) = E [φ(Xε(T ))− φ(X ∗(T )) ],
I2(ε) = E [
∫ T
0
(g(Xε(s),uε(s))− g(X ∗(s),u∗(s))
)ds ].
Then apply Lemma 1 (2), (3).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 26: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/26.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Lemma 3If (H1)–(H3) hold, then
− ε E⟨
Y ∗(T ),p(T )⟩
+ ε E [
∫ T
0gx (X ∗(s),u∗(s)) p(s) ds ]
+ E [
∫ T
0(−δεH(s) +
⟨Y ∗(s) , δεF (s)
⟩) ds ] ≥ o(ε),
where
δεF (s) = F (X ∗(s),uε(s))− F (X ∗(s),u∗(s)),
δεH(s) = H(X ∗(s),uε(s),Y ∗(s),Z ∗(s)Q1/2(s))
−H(X ∗(s),u∗(s),Y ∗(s),Z ∗(s)Q1/2(s)).
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 27: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/27.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Proof of Lemma 3
. Since u∗(·) is an optimal control, J(uε(·))− J(u∗(·)) ≥ 0.
. Next apply Lemma 2 and the definition of the Hamiltonian H.
Next need the following relation:
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 28: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/28.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Proof of Lemma 3
. Since u∗(·) is an optimal control, J(uε(·))− J(u∗(·)) ≥ 0.
. Next apply Lemma 2 and the definition of the Hamiltonian H.
Next need the following relation:
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 29: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/29.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Lemma 4
E⟨
Y ∗(T ),p(T )⟩
= E [
∫ T
0gx (X ∗(s),u∗(s)) p(s) ds ]
+ E [
∫ T
0
⟨Y ∗(s),Fu(X ∗(s),u∗(s))u(s)
⟩ds ].
Proof of Lemma 4
. Use Ito’s formula together with⟨∇xH(X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),p(t)
⟩= − gx (X ∗(t),u∗(t)) p(t) +
⟨Fx (X ∗(t),u∗(t)) p(t) ,Y ∗(t)
⟩+⟨
G(X ∗(t))Q1/2(t) ,Z ∗(t)Q1/2(t)⟩
2 a.s. ∀ t ∈ [0,T ].
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 30: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/30.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Lemma 4
E⟨
Y ∗(T ),p(T )⟩
= E [
∫ T
0gx (X ∗(s),u∗(s)) p(s) ds ]
+ E [
∫ T
0
⟨Y ∗(s),Fu(X ∗(s),u∗(s))u(s)
⟩ds ].
Proof of Lemma 4
. Use Ito’s formula together with⟨∇xH(X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),p(t)
⟩= − gx (X ∗(t),u∗(t)) p(t) +
⟨Fx (X ∗(t),u∗(t)) p(t) ,Y ∗(t)
⟩+⟨
G(X ∗(t))Q1/2(t) ,Z ∗(t)Q1/2(t)⟩
2 a.s. ∀ t ∈ [0,T ].
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 31: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/31.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Proof of main resultConsider the adjoint equation. From Lemma 3, Lemma 4 get
E [
∫ T
0
⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s))u(s)
⟩ds ]
− E [
∫ T
0δεH(s)ds ] ≥ o(ε).
The continuity, boundedness of Fu in (H1) and the DCT give
1εE [
∫ T
0
⟨Y ∗(s), δεF (s)− ε Fu(X ∗(s),u∗(s)) u(s)
⟩ds ]
= E[ ∫ T
0
⟨Y ∗(s),
∫ 1
0
(Fu(X ∗(s),u∗(s) + θ(uε(s)− u∗(s)))
−Fu(X ∗(s),u∗(s)))
u(s) dθ⟩
ds]→ 0,
as ε→ 0+.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 32: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/32.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Proof of main resultConsider the adjoint equation. From Lemma 3, Lemma 4 get
E [
∫ T
0
⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s))u(s)
⟩ds ]
− E [
∫ T
0δεH(s)ds ] ≥ o(ε).
The continuity, boundedness of Fu in (H1) and the DCT give
1εE [
∫ T
0
⟨Y ∗(s), δεF (s)− ε Fu(X ∗(s),u∗(s)) u(s)
⟩ds ]
= E[ ∫ T
0
⟨Y ∗(s),
∫ 1
0
(Fu(X ∗(s),u∗(s) + θ(uε(s)− u∗(s)))
−Fu(X ∗(s),u∗(s)))
u(s) dθ⟩
ds]→ 0,
as ε→ 0+.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 33: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/33.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
continued proof
In particular
E [
∫ T
0
⟨Y ∗(s), δεF (s)− εFu(X ∗(s),u∗(s)) u(s)
⟩ds ] = o(ε).
∴ − E [
∫ T
0δεH(s) ds ] ≥ o(ε).
∴ by dividing this inequality by ε and letting ε→ 0+ :
E [
∫ T
0∇uH(t ,X ∗(t),u∗(t),Y ∗(t),Z ∗(t)Q1/2(t)),u(t)
⟩dt ] ≤ 0.
The theorem then follows.
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs
![Page 34: Pontryagin's maximum principle for optimal control of SPDEs talks workshop... · 2012. 1. 9. · Problem formulation Pontryagin stochastic maximum principle Plan of the proof Pontryagin’s](https://reader033.fdocuments.us/reader033/viewer/2022051919/600c1fc2a992e755585819f2/html5/thumbnails/34.jpg)
Problem formulationPontryagin stochastic maximum principle
Plan of the proof
EstimatesVariational inequalityTwo more lemmasProof of main result
Thank you
AbdulRahman Al-Hussein Pontryagin’s maximum principle for optimal control of SPDEs