Two cases of stochastic maximum principle in the...
Transcript of Two cases of stochastic maximum principle in the...
Two cases of stochastic maximum principle
in the optimal control of SPDEs
Marco Fuhrman
Politecnico di Milano
Ying Hu
Universite de Rennes 1
Gianmario Tessitore
Universita di Milano-Bicocca
Rennes 24th of May 2013
Structure of the talk
We prove Pontryagin maximum principle (necessary conditions for optimality)for a controlled stochastic PDE in the following situations:
1. Part I: Infinite dimensional (white) noise - Special case (No second variationneeded)
• Stochastic parabolic equations in an interval [0,1]
• driven by space-time white noise (cylindrical Wiener process) (Wt)
• with convex set of controls (control affects noise)
• non-linearities are Nemytskii operators F (x) = f(·, x(·)), x ∈ Lp(O)
2. Part II: Finite dimensional noise, General case
• Stochastic parabolic equations in a domain O ⊂ Rd
• driven by a finite dimensional Wiener process Wt = (β1t , . . . , β
mt )
• with non convex set of controls (control affects noise)
• non-linearities are Nemytskii operators:
1
Very incomplete history of SMP in ∞ dimensions
• Bensoussan, J. Frank. Inst. (1983) and Hu-Peng, Stochastics (1990):Special case, State ∞-dim., noise has trace class covariance.
• Peng SICON (1990): General case, state and noise fin. dim.,
• Zhou, SICON (1993): State ∞-dim. noise fin. dim., Linear state equat.and cost.,
• Tang-Li LNPAM (1994): General case, state ∞-dim. noise fin. dim., noisecan have jumps, second derivatives of the coefficients are Hilbert-Schmidt.
• Fuhrman-Hu-T. CRAS 2012, AMO 2012 (electronic): General case, state∞-dim., noise fin. dim., Specific framework to cover stochastic parabolicPDEs
• Lu-Zhang, Preprint 2012: General case, state ∞-dim., noise fin. dim., Pt
characterized as “transposition solution” of a BSEE. Nonlinearities regularin functional spaces.
• Du-Meng, Preprints 2012: General case, state ∞-dim. noise either fin.dim. or trace class, Leading operator A can depend on t; unbounded linearterm affecting noise. Some regularity required for the nonlinarities.
• Fuhrman-Hu-T. : Special case, state ∞-dim. noise ∞-dim. and cylindrical.
2
PART I: INFINITE DIMENSIONAL - (WHITE) NOISE - Special Case
Formulation of the optimal control problem
Let (W(t, x)), t ≥ 0, x ∈ [0,1] be a space time white noise(Ft)t≥0 denotes its natural (completed) filtration.
The set of admissible control actions U is a convex subset of L∞([0,1]).A control u is a (progressive) process with values in U .
The controlled state equation is the following SPDE: for t ∈ [0, T ], x ∈ [0,1],dXt(x) =
∂2
∂x2Xt(x) dt+ b(x,Xt(x), ut(x)) dt+ σ(x,Xt(x), ut(x))dW(t, x),
Xt(0) = Xt(1) = 0, t ∈ [0, T ]
X0(x) = x0(x), x ∈ [0,1]
where b(x, r, u), σ(x, r, u) : [0,1]× R× R → R are given,we assume they are C1 and Lipschitz with respect to r and u;for fixed r and u we suppose b(·, r, u) ∈ L2([0,1]), σ(·, r, u) ∈ L∞([0,1]) bdd.
We also introduce the cost functional:
J(u) = E∫ T
0
∫Ol(x,Xt(x), ut(x)) dx dt+ E
∫Oh(x,XT(x)) dx
where l(x, r, u) : O×R×R → R, h(x, r) : O×R → R are given bounded functions,we assume that they are C1 with bounded derivatives with respect to r and u;
3
Abstract reformulation
The noise is reformulated as a L2([0,1]) valued cylindrical Wiener process (Wt)
E < Wt, x >L2< Ws, y >L2= (t ∧ s) < x, y >L2, ∀x, y ∈ H = L2([0,1])
A is the realization of the second derivative operator in H with Dirichlet boundaryconditions. So it is an unbounded operator with domain H2
0([0,1]) ⊂ H =L2([0,1]).
For all X,V ∈ H, x ∈ [0,1] the non linearities are defined by
F (X,u)(x) = b(x,X(x), u(x)), [G(X,u)V ](x) = σ(x,X(x), u(x))V (x),
L(X,u)(x) =
∫Ol(X(x), u(x))dx, Φ(X)(x) =
∫Oh(X(x))dx
The state equation written in abstract form becomes
dtXt = AXtdt+ F (Xs, us)ds+G(Xs, us)dWs, X0 = x0
where x0 ∈ H and the solution will evolve in H.
The cost becomes
J(x, u) = E∫ T
0L(Xs, us)ds+ EΦ(XT)
4
Standing Framework
(i) A is the generator of a C0 semigroup etA, t ≥ 0, in H. Moreover ∀s > 0:
esA ∈ L2(H) with |esA|L2(H) ≤ Ls−γ; for suitable L > 0, γ ∈ [0,1/2).
where L2(H) is the (Hilbert) space of Hilbert Schmidt operators in H.
(ii) U is a bounded convex subset of a separable Banach space U0
(iii) F : H × U → H is lipschitz in both variables
(iv) G : H × U → L(H) verifies for all s > 0, t ∈ [0, T ], X,Y ∈ H, u, v ∈ U ,
|esAG(t,0, u)|L2(H) ≤ L s−γ,
|esAG(t,X, u)− esAG(t, Y, v)|L2(H) ≤ L s−γ(|X − Y |+ |u− v|), (1)
for some constants L > 0 and γ ∈ [0,1/2).
(v) F (·, ·) is Gateaux differentiable H × U → H,for all s > 0, esAG(·, ·) is Gateaux differentiable H × U → L2(H).
(vi) L(·, ·) and Φ(·) are bounded lipschitz and differentiable
(vii) For all Ξ ∈ H the map u → G(X,u)Ξ is Gateaux differentiable and
|∇uG(X,u)Ξ|L(U0,H) ≤ cost|Ξ|H recall (U0 ⊂ L∞)
5
Under the above assumptions the state equation (formulated in mild sense):
Xt = etAx0 +
∫ t
0e(t−s)AF (Xs, us)ds+
∫ t
0e(t−s)AG(Xs, us)dWs
admits a unique solution X ∈ LpW(Ω, C([0, T ], H)) see [Da Prato, Zabczyk ’92]
Remark: If we perturb the control by spike variation that is we consider solutionof
Xϵt = etAx0 +
∫ t
0e(t−s)AF (Xs, u
ϵs)ds+
∫ t
0e(t−s)AG(Xs, u
ϵs)dWs
where uϵs = usI[t0,t0+ϵ]c(s) + v0I[t0,t0+ϵ](s) for fixed t0 ∈ [0, T ], v0 ∈ U then
|Xϵ(t0 + δ)−X(t0 + δ)|L2(Ω,P,H) ≈ δ(1/2−γ)
6
First Variation Equation
Let (X, u) be an optimal pair, fix any other bdd. U-valued progressive control v
Let uϵt = ut + ϵ(vt − ut) and Xϵ
t the corr. solution of the state equation.
Finally denote (δu)t = vt − ut
Since we are not considering spike variations things are easy at this level
Xϵt = Xt + ϵYt + o(ϵ).
dYt =[AYt +∇XF (Xt, ut)Yt +∇uF (Xt, ut)(δu)t
]dt
∇XG(Xt, ut)Yt dWt +∇uG(Xt, ut)(δu)t, dWt
Y ϵ0 = 0
By [Da Prato Zabczyk] the above equation admits an unique mild solution with
E( supt∈[0,T ]
|Yt|2) < +∞,
Moreover
J(x, uϵ) = J(x, u) + ϵI(v) + o(ϵ)
with
I(v) = E∫ T
0
[⟨∇XL(Xt, ut), Yt⟩+ ⟨∇uL(Xt, ut), (δu)t⟩
]dt+ E⟨∇XΦ(XT), YT ⟩
7
We fix a basis (ei)i∈N ∈ H and assume that for all i ∈ N the map X → G(X,u)eiis Gateaux differentiable H → H.
We notice that in our concrete case for all V ∈ H
[∇X(G(X,u)ei)V ](x) =∂
∂Xσ(x,Xt(ξ), ut(ξ))ei(ξ)V (ξ)
So it is enough to choose ei ∈ L∞([0,1])
We denote ∇X(G(Xt, ut)ei)V = Ci(t)V .
Recall that gradients ∇X are with respect to variables X ∈ H = L2([0,1])
For simplicity we let F = 0 from now on:.
The equation for the first variation becomesdYt(x) = AYtdt+
∑∞i=1Ci(t)Yt dβi
t +∇uG(Xt, ut)(δu)t dWt
Y0 = 0
where βit = ⟨ei,Wt⟩ and we have:
• |Ci(t)|L(H) ≤ c, P− a.s. for all t ∈ [0, T ]
•∑∞
i=1 |etACi(s)v|2 ≤ ct−2γ|v|2H for all t ≥ 0, s ≥ 0, (γ < 1/2)
•∑∞
i=1 |etAei|2 ≤ ct−2γ for all t ≥ (γ < 1/2),
We also take into account that A and Ci are self adjoint (although not essential).
8
Adjoint equation
The adjoint equation is (at least formally)−dpt(x) =
[Apt +∇XL(Xt, ut) +
∑∞i=1Ci(t)Qtei
]dt+QtdWt
pT = ∇xΦ(XT)
We expect a solution with pt ∈ H and Qt ∈ L2(H) but we notice that the term∑∞i=1Ci(t)Qtei does not converge for Qt ∈ L2(H).
We can rewrite the above equation in the mild form
pt = e(T−t)A∇XΦ(XT) +
∫ T
t
e(s−t)A∇XL(Xs, us)ds+
+
∫ T
t
∞∑i=1
e(s−t)ACi(s)Qseids+
∫ T
t
QsdWs
but still∑∞
i=1 e(s−t)ACi(s)Qei doesn’t converge if Q ∈ L2(H). Indeed if V ∈ H
∞∑i=1
⟨e(s−t)ACi(s)Qei, V ⟩ =∞∑i=1
⟨Qei, Ci(s)e(s−t)AV ⟩ ≤ |Q|L2(H)
( ∞∑i=1
|Ci(s)e(s−t)AV |2
)1/2
?
On he contrary
∞∑i=1
⟨e(s−t)ACi(s)Qei, V ⟩ ≤
( ∞∑i=1
|Qei|
)supi∈N
|Ci(s)e(s−t)AV |
9
Easy Facts on Schatten - von Neumann classes
We denote by L2(H) the Hilbert space of Hilbert Schmidt operators H → Hendowed with the scalar product ⟨L,M⟩2 =
∑∞i=1⟨Lei,Mei⟩H
Given L ∈ L2(H) there exists a sequence (aLn)n∈N ∈ ℓ2 and a couple of orthonormalbases (eLn)n∈N, (fL
n )n∈N in H such that
L =∞∑
n=1
aLnfLn ⟨eLn, ·⟩ and |L|2 =
∑n
(aLn)2.
If t → Lt is a L2 valued process then the above objects can be selected with thesame measurability properties as L.
Define L1(H) = L ∈ L2(H) : |L|1 < ∞ where
|L|1 := sup⟨B,L⟩2 : B ∈ L2(H), |B|L(H) ≤ 1
• If B ∈ L(H) and L ∈ L1(H) then LB, BL are in L1(H) moreover
|LB|1 ≤ |L|1|B|L(H), |BL|1 ≤ |L|1|B|L(H)
• If L ∈ L1(H) the trace Tr(L) :=∑∞
i=1⟨ei, Lei⟩ converges absolutely and itsvalue is independent on the choice of the basis (ei)i∈N
• |L|1 =∑∞
n=1 |aLn|, Tr(L) =∑∞
n=1 aLn consequently |Tr(L)| ≤ |L|1
10
Coming back to our bad term if Q ∈ L1(H) there exist two ONB such thatQ =
∑j ajfj⟨·, gj⟩, and
∑i
∣∣∣e(s−t)ACi(s)Qei
∣∣∣ =∑i
∣∣∣∣∣∣∑j
e(s−t)ACi(s)ajfj⟨ei, gj⟩
∣∣∣∣∣∣≤
∑j
|aj|∑i
|e(s−t)ACi(s)fj| |⟨ei, gj⟩| by Cauchy
≤∑j
|aj|c(s− t)−γ = c|Q|L1(s− t)−γ.
Moreover we formally compute dt⟨Yt, pt⟩ we get
E⟨YT ,∇XΦ(XT)⟩+ E∫ T
0⟨Ys,∇XL(Xs, us)⟩ds = E
∫ T
0Tr[(∇uG(Xs, us)(δu)s
)Qs
]ds
• The multiplication operator ∇uG(Xs, us)(δu)s is at most bounded in H thus
Tr[(∇uG(Xs, us)(δu)s
)Qs
]is not well defined for Q ∈ L2(H) but is well defined for Q ∈ L1(H) .
• We can not bypass the above term since it will remain in the final formulationof the maximum principle.
Conclusion The non-hilbertian space L1(H) has something to do here
11
Existence of a solution by approximations
Let us denote η := ∇XΦ(XT) ∈ L2(Ω,FT ,P, H) and f := ∇XL(X, u) ∈ L2W(Ω ×
[0, T ], H).
Consider the approximating BSDEs
dpNt = e(T−t)Aη +
∫ T
t
e(s−t)Afsds+
∫ T
t
N∑i=1
e(s−t)ACi(s)QNs eids+
∫ T
t
e(s−t)AQNs dWs
By the standard theory (see [Hu-Peng 91]) there exists a unique solution with
supt∈[0,T ]
E|pNt |2H + E
∫ T
0|QN
t |2L2(H)dt ≤ c
(E|η|2 + E
∫ T
0|ft|2dt
)The idea is to exploit the duality relation with a forward equation in order toobtain good estimates in the L1 norm.
First we show that the same duality relation easily implies weak convergence ofthe solutions of the sequence (pN , QN)
12
The perturbed forward equation
Consider the perturbed (forward) equationdY Γ,ξ
t = AY Γ,ξt dt+
[∑∞i=1Ci(t)Y
Γ,ξt +Γt
]dWt
Y Γ,ξs = ξ ∈ L2(Ω,Fs,P, H)
Proposition 1 Given Γ ∈ L2W(Ω × [0, T ], L2(H)) the above equation admits a
unique mild solution that verifies
E|Y Γ,ξt |2 ≤ cE
∫ t
s
|Γℓ|2L2(H)dℓ+ Eξ2
E|Y Γ,ξt |2 ≤ cE
∫ t
s
(t− ℓ)−2γ|Γℓ|2L(H)dℓ+ Eξ2
The same estimates hold (with independent constant) for the solutions Y Γ,N ofthe approximating equations
dY Γ,ξ,Nt = AY Γ,ξ,N
t dt+[∑N
i=1Ci(t)YΓ,ξ,Nt +Γt
]dWt
Y Γ,ξ,Ns = ξ ∈ L2(Ω,Fs, H)
Moreover E∫ T
s|Y Γ,ξ,N
t − Y Γ,ξt |2dt → 0 and E|Y Γ,ξ,N
t − Y Γ,ξt |2 → 0 for all t ∈ [s, T ]
13
If we compute (by introducing Yosida approximations of A) dt⟨Y Γ,Nt , pNt ⟩ we obtain
E∫ T
s
⟨Γt, QNt ⟩2dt+ E⟨ξ, pNs ⟩2 = E
∫ T
s
⟨ft, Y Γ,ξ,Nt ⟩dt+ E⟨η, Y Γ,ξ,N
T ⟩H
and taking into account the convergence of Y Γ,ξ,N towards Y Γ,ξ
Corollary 2 There exists a couple of adapted processes Q ∈ L2(Ω×[0, T ], L2(H)),p ∈ C([0, T ], L2(Ω, H)), such that
QN Q in L2(Ω× [0, T ], L2(H)) pNt pt in L2(Ω, H) ∀t ∈ [0, T ]
Moreover since for all t ∈ [0, T ] the stochastic integral∫ T
te(s−t)AQN
s dWs converges
weakly to∫ T
te(s−t)AQsdWs we immediately deduce that, by difference, for all
t ∈ [0, T ] there exists Ξt in L2(Ω,Ft, H) such that
N∑i=1
∫ T
t
e(s−t)ACi(s)QNs eids Ξt weakly in L2(Ω,FT , H)
The mild BSDE for the couple (p,Q) at this point reeds:
pt = Ξt + e(T−t)Aη +
∫ T
t
e(s−t)Afsds+
∫ T
t
e(s−t)AQsdWs
14
The above is not satisfactory in the sense that we want at least to obtain a mildBSDE. To start with we notice that the convergence of the bad term holds forthe limit process Q itself namely
Proposition 3
N∑i=1
∫ T
t
e(s−t)ACi(s)Qseids Ξt in L2(Ω,FT ,P, H)
Proof: Given ξ ∈ L2(Ω,FT ,P, H)
E⟨N∑
i=1
∫ T
t
e(s−t)ACi(s)Qseids, ξ⟩ =N∑
i=1
E∫ T
t
⟨Qsei, Ci(s)e(s−t)AE(ξ|Fs)⟩ds
= limM→∞
N∑i=1
E∫ T
t
⟨QMs ei, Ci(s)e
(s−t)AE(ξ|Fs)⟩ds
If γi(s) = Ci(s)e(s−t)AE(ξ|Fs) and Y M,N is the solution of the forward mild SDE
Y M,Nζ =
M∑i=1
∫ ζ
t
e(s−t)ACi(s)YM,Ns dβi
s +N∑
i=1
∫ ζ
t
e(s−t)Aγi(s)dβis
then
E⟨N∑
i=1
∫ T
t
e(s−t)ACi(s)QMs eids, ξ⟩ = E⟨η, Y M,N
ζ ⟩+ E∫ T
t
E⟨fζ, Y M,Nζ ⟩dζ
15
Noticing that for all ρ > t
∞∑i=1
E∫ ρ
t
|e(ρ−s)Aγi(s)ds|2 =∞∑i=1
E∫ ρ
t
|e(ρ−s)ACi(s)e(s−t)AE(ξ|Fs)ds|2 ≤ cE|ξ|2
we can show that, for all fixed t ∈ [0, T ], as N,M → ∞
E|Y M,Nζ − Y ∞
ζ |2 → 0,
∫ T
t
E|Y M,Nζ − Y ∞
ζ |2dζ → 0
where Y ∞ is the mild solution of the forward SDEdY ∞
t = AY ∞ζ dζ +
∑∞i=1Ci(ζ)Y
M,Nζ dβi
ζ +∑∞
i=1 γi(ζ)dβiζ
Y ∞t = 0
In conclusion
E⟨N∑
i=1
∫ T
t
e(s−t)ACi(s)Qseids, ξ⟩ → E⟨η, Y ∞t ⟩+ E
∫ T
t
E⟨fζ, Y ∞ζ ⟩dζ
In an identical way we can show that
E⟨N∑
i=1
∫ T
t
e(s−t)ACi(s)QNs eids, ξ⟩ → E⟨η, Y ∞
t ⟩+ E∫ T
t
E⟨fζ, Y ∞ζ ⟩dζ
and this concludes the proof
16
Estimates of Q in the L1 norm
Proposition 4 E∫ T
0(T − s)2γ|Qs|2L1(H)ds ≤ cE|η|2 + cE
∫ T
0|fs|2ds
Remembering the representation QNt =
∑∞n=1 an(t)fn(t)⟨en(t), ·⟩ we choose
ΓNt := α(t)
N∑n=1
sgn(an(t))fn(t)⟨en(t), ·⟩ with α : [0, T ] → R.
We notice that |ΓNt |L(H) ≤ α(t) and that ⟨Qt,ΓN
t ⟩L2(H) = α(t)∑N
n=1 |aQn (t)| so
that:
E∫ T
0|Qt|L1(H)α(t)dt = E
∫ T
0supN
⟨Qt, γNt ⟩2dt ≤ sup
N
[E⟨η, Y ΓN
T ⟩H +
∫⟨fs, Y ΓN
s ⟩Hds
]where
dY Γt = AY Γ
t dt+[∑∞
i=1Ci(t)Y Γt +Γt
]dWt
Y Γs = 0
recalling the estimate of Y Γ with respect to the L(H) norm of Γ we get.
E
∫ T
0|Qt|1α(t)dt ≤ cη,f
(∫ T
0(T − s)−2γα2(s)ds
)1/2
and the proof follows letting α(s) = (T − s)−γα(s) and rewriting the above as
E
∫ T
0|Qt|1(T − t)γα(t)dt ≤ cη,f |α|L2[0,T ]
17
Corollary 5 The sequence∑∞
i=1
∫ T
te(s−t)ACi(s)Qseids converges in L1(Ω,P, H)
and the BSDE is satisfied in proper sense that is for all t ∈ [0, T ] it holds P-a.s.
pt = e(T−t)Aη +
∫ T
t
e(s−t)Afsds+∞∑i=1
∫ T
t
e(s−t)ACi(s)Qseids+
∫ T
t
e(s−t)AQsdWs
Proof: Recall the estimate∞∑i=1
∣∣∣e(s−t)ACi(s)Qei
∣∣∣ ≤ C|Q|L1(s− t)−γ.
Then for all N
EN∑
i=1
∣∣∣∣∫ T
t
e(s−t)ACi(s)Qseids
∣∣∣∣ ≤ cE∫ T
t
|Qs|L1(s− t)−γds
≤(E∫ T
t
|Qs|2L1(T − s)2γds
)1/2(∫ T
t
(T − s)−2γ(s− t)−2γds
)1/2
and the claim follows since this last integral converges.
18
Conclusion
Passing to the limit the duality relation holding for (pN , QN) we get (recallingthe expansion of the cost)
J(x, uϵ)− J(x, u) = ϵE∫ T
0⟨(δu)s,
[∇uF (Xs, us)
]∗ps⟩ds+ ϵE
∫ T
0⟨∇uL(Xs, us), (δu)s⟩ds
+ϵE∫ T
0Tr[(∇uG(Xs, us)(δu)s
)Qs
]ds+ o(ϵ)
And we now know that all the terms in the above formula are well defined.
Recall the we are assuming that |∇uG(Xs, us)vs|L(H) ≤ cost and we have justproved that Q ∈ L1(H), P⊗ dt, a.s..
So we con conclude (by the usual localization - Lebesgue differentiation proce-dure) that ∀v ∈ U it holds P⊗ dt a.s.
⟨∇uL(Xs, us), v − us⟩+Tr[(∇uG(Xs, us)vs
)Qs
]≥ 0
Uniqueness of the mild BSDE ?
19
PART II: FINITE DIMENSIONAL NOISE
Formulation of the optimal control problem
Let (β1t , . . . , β
mt ), t ≥ 0, be a standard m-dimensional Wiener process.
(Ft)t≥0 denotes its natural (completed) filtration.
The set of control actions U is a separable metric space not necessarily convex.A control u is a process in U .
O ⊂ Rn is a bounded open set with regular boundary. The controlled stateequation is an SPDE of the following semi abstract form: for t ∈ [0, T ], x ∈ O, dXt(x) = AXt(x) dt+ b(x,Xt(x), ut) dt+
m∑j=1
σj(x,Xt(x), ut) dβjt ,
X0(x) = x0(x),
where
b(x, r, u), σj(x, r, u) : O × R × U → R are given (all difficulties are already presentif b and σj are very regular in r and independent on x).
H = L2(O) is the state space, with usual scalar product ⟨·, ·⟩.We assume x0 ∈ H. The solution Xt, t ∈ [0, T ], will be a process in H.
A is the realization of a partial differential operator, with appropriate boundaryconditions.
20
Standing assumptions
1) Regular coefficients
The functions b(x, r, u), σj(x, r, u), l(x, r, u), h(x, r) are measurable anda) continuous in u;b) of class C2 in r ∈ R;c) bounded together with their first and second derivative w.r.t. r,
2) Lp-boundedness of the semigroup
A is a generator of a strongly continuous semigroup etA, t ≥ 0, in H = L2(O).Moreover, for every p ∈ [2,∞) and t ∈ [0, T ],
etA(Lp(O)) ⊂ Lp(O), ∥etAf∥Lp(O) ≤ Cp∥f∥Lp(O)
for some constants Cp independent of t and f .
3) Compactness in L4 of the semigroup
the restriction of etA, t ≥ 0, to L4(O) is an analytic semigroup with domain ofthe infinitesimal generator compactly imbedded in L4(O).
21
Statement of the stochastic maximum principle
For u ∈ U and X, p, q1, . . . , qm ∈ H = L2(O) denote
H(u,X, p, q1, . . . , qm) =
∫O
[l(x,X(x), u)+b(x,X(x), u)p(x)+σj(x,X(x), u)qj(x)
]dx
Theorem. Let (Xt, ut) be an optimal pair. Then there are (suitably charac-terized):
1) (m+1) L2(O)-valued adapted processes pt, q1t, . . . , qmt, t ∈ [0, T ],2) one operator-valued process Pt, t ∈ [0, T ];
for which the following inequality holds P-a.s. for a.e. t ∈ [0, T ]:for every v ∈ U ,
H(v, Xt, pt, q1t, . . . , qmt)−H(ut, Xt, pt, q1t, . . . , qmt)
+1
2⟨Pt[σj(·, Xt(·), v)− σj(·, Xt(·), ut)], σj(·, Xt(·), v)− σj(·, Xt(·), ut)⟩ ≥ 0.
The first adjoint processes pt, qjt are characterized as the unique solutions inH of an appropriate BSPDE and satisfy
supt∈[0,T ]
E∥pt∥2H + E∫ T
0∥qjt∥2H dt < ∞.
The second adjoint process Pt takes values in the space of linear bounded op-erators L4(O) → L4(O)∗ = L4/3(O) and also admits a suitable unique character-ization.
22
Preliminaries to the proof of the maximum principle
Let (X, u) be an optimal pair. We introduce the spike variation:we fix an arbitrary interval [t, t+ ϵ] ⊂ (0, T ) and an arbitrary v ∈ U and define
uϵt =
ut if t /∈ [t, t+ ϵ],
v if t ∈ [t, t+ ϵ].
Letδlt(x) = l(x, Xt(x), uϵ
t)− l(x, Xt(x), ut)δbt(x) = b(x, Xt(x), uϵ
t)− b(x, Xt(x), ut)δσjt(x) = σj(x, Xt(x), uϵ
t)− σj(x, Xt(x), ut)δb′t(x) = b′(x, Xt(x), uϵ
t)− b′(x, Xt(x), ut)δσ′
jt(x) = σ′j(x, Xt(x), uϵ
t)− σ′j(x, Xt(x), ut)
Let (X, u) be an optimal pair, uϵt the spike variation, and Xϵ
t the corr. solution:dXϵ
t(x) = AXϵt(x) dt+ b(x,Xϵ
t(x), uϵt) dt+ σj(x,Xϵ
t(x), uϵt) dβ
jt ,
Xϵ0(x) = x0(x)
We wish to represent in the form
Xϵt = Xt + Y ϵ
t + Zϵt + remainder term
where the remainder has to be o(ϵ).
23
Equation for Y ϵt (to be understood in a mild sense):
dY ϵt (x) =
[AY ϵ
t (x) + b′(x, Xt(x), ut) · Y ϵt (x)
]dt
+σ′j(x, Xt(x), ut) · Y ϵ
t (x) dβjt + δbt(x) dt+ δσjt(x) dβ
jt
Y ϵ0(x) = 0
Equation for Zϵt (to be understood in a mild sense):
dZϵt(x) =
[AZϵ
t(x) + b′(x, Xt(x), ut) · Zϵt(x)
]dt+ σ′
j(x, Xt(x), ut) · Zϵt(x) dβ
jt
+[12b′′(x, Xt(x), ut) · Y ϵ
t (x)2 + δb′t(x) · Y ϵ
t (x)]dt
+[12σ′′j (x, Xt(x), ut) · Y ϵ
t (x)2 + δσ′
jt(x) · Y ϵt (x)
]dβj
t
Zϵ0(x) = 0
Proposition. For all p ≥ 2,
supt∈[0,T ]
(E∥Y ϵ
t ∥pLp(O)
)1/p= sup
t∈[0,T ]
(E∫O|Y ϵ
t (x)|pdx)1/p
≤ Cp√ϵ.
supt∈[0,T ]
(E∥Zϵ
t∥pLp(O)
)1/p= sup
t∈[0,T ]
(E∫O|Zϵ
t(x)|pdx)1/p
≤ Cp ϵ.
supt∈[0,T ]
(E∥Xϵ
t − Xt − Y ϵt − Zϵ
t∥2H)1/2
= supt∈[0,T ]
(E∫O|Xϵ
t(x)− Xt(x)− Y ϵt (x)− Zϵ
t(x)|2dx)1/2
= o(ϵ).
24
Expansion of the cost functional
Let (X, u) be an optimal pair for the cost
J(u) = E∫ T
0
∫Ol(x,Xt(x), ut) dx dt+ E
∫Oh(x,XT(x)) dx
Let uϵt be the spike variation, and J(uϵ) the corresponding cost. Then clearly
J(uϵ)− J(u) ≥ 0.
Recall
δlt(x) = l(x, Xt(x), uϵt)− l(x, Xt(x), ut)
Proposition. We have
0 ≤ J(uϵ)− J(u) = E∫ T
0
∫Oδlt(x) dx dt+∆ϵ
1 +∆ϵ2 + o(ϵ),
where
∆ϵ1 = E
∫ T
0
∫Ol′(x, Xt(x), ut)(Y
ϵt (x) + Zϵ
t(x)) dx dt
+E∫Oh′(x, XT(x))(Y
ϵT(x) + Zϵ
T(x)) dx,
∆ϵ2 =
1
2E∫ T
0
∫Ol′′(x, Xt(x), ut)Y
ϵt (x)
2 dx dt+1
2E∫Oh′′(x, XT(x))Y
ϵT(x)
2 dx.
25
The first adjoint processes
Consider the backward SPDE −dpt(x) = −qjt(x) dβjt +
[A∗pt(x) + b′(x, Xt(x), ut) · pt(x)
+σ′j(x, Xt(x), ut) · qjt(x) + l′(x, Xt(x), ut)
]dt
pT(x) = h′(x, XT(x))
By Hu-Peng, Stoch Anal Appl (’91) there exists of a unique (m + 1)-uple ofadapted processes (p, q1, ..., qm) solving the above in a mild sense and verifying
supt∈[0,T ]
E∫O|pt(x)|2Hdx+ E
∫ T
0
∫O|qjt(x)|2Hdx dt < ∞
Computing d∫O Y ϵ
t (x)pt(x) dx , d∫O Zϵ
t(x)pt(x) dx, and joining what one obtainswith the expression for ∆ϵ
2 we get
0 ≤ J(uϵ)−J(u) = E∫ T
0
∫O
[δlt(x)+δbt(x)pt(x)+δσjt(x)qjt(x)
]dx dt+
1
2∆ϵ
3+o(ϵ),
where ∆ϵ3 =
E∫ T
0
∫O
[l′′(x, Xt(x), ut) + b′′(x, Xt(x), ut)pt(x) + σ′′
j (x, Xt(x), ut)qjt(x)]Y ϵt (x)
2dxdt
+E∫Oh′′(x, XT(x))Y
ϵT(x)
2dx.
26
The second adjoint processes
Consider again
∆ϵ3 = E
∫ T
0
∫OHt(x)Y
ϵt (x)
2dxdt+E∫Oh(x)Y ϵ
T(x)2 dx = E
∫ T
0⟨HtY
ϵt , Y
ϵt ⟩ dt+E⟨hY ϵ
T , YϵT ⟩
where
Ht(x) = l′′(x, Xt(x), ut) + b′′(x, Xt(x), ut)pt(x) + σ′′j (x, Xt(x), ut)qjt(x),
h(x) = h′′(x, XT(x)).
Here and below, by Ht and h we denote multiplication operators by Ht(·) andh(·), acting on H:
Ht : f(·) 7→ Ht(·)f(·), h : f(·) 7→ h(·)f(·), f ∈ H = L2(O).
Note that
|h(x)| ≤ C := sup |h′′| < ∞, E∫ T
0
∫O|Ht(x)|2dx dt < ∞,
due to the occurrence of qjt(x), so
h(·) ∈ L∞(O) P− a.s., Ht(·) ∈ L2(O), P× dt− a.e.
In particular, h is bounded but Ht is not a (bounded) linear operator on H.
To finish our argument we have to compute limϵ→0 ϵ−1∆ϵ3
27
Characterization of P
For fixed t ∈ [0, T ] and f ∈ L4, we consider the equationdYt,f
s (x) = AYt,fs (x) ds+ b′(x, Xs(x), us)Yt,f
s (x) ds+ σ′j(x, Xs(x), us)Yt,f
s (x) dW js ,
Yt,ft (x) = f(x),
We define a progressive process (Pt)t∈[0,T ] with values in the space of bounded
linear operators L4 → (L4)∗ = L4/3 setting for t ∈ [0, T ], f, g ∈ L4
⟨Ptf, g⟩ = EFt
∫ T
t
⟨HsYt,fs ,Yt,g
s ⟩ ds+ EFt⟨hYt,fT ,Yt,g
T ⟩, P− a.s.
The process (Pt)t∈[0,T ] enjoys the following properties
Boundedness supt∈[0,T ] E∥Pt∥2L < ∞,
Continuity E|⟨Pt+ϵ − Pt)f, g⟩| → 0, as ϵ → 0, f, g ∈ L4(O)
Regularization For every η ∈ (0,1/4) there exists a constant Cη such that
E supf,g
|⟨Pt(−A)ηf, (−A)ηg⟩|2 ≤ Cη(T − t)−4ηE[∫ T
0∥Hs∥2L2(O)ds+ ∥h∥2L2(O)
].
where D(−A)η is the domain of the fractional power of A in L4(O) and thesup is taken over all f, g ∈ D(−A)η, ∥f∥L4(O) ≤ 1, ∥g∥L4(O) ≤ 1.
28
Conclusion of the proof
By the Markov property and suitable estimates(recalling that, for all p ≥ 1, E∥Y ϵ
t ∥2pLp(O) ≤ Cp ϵp for all t ∈ [0, T ]. )
∆ϵ3 = E
∫ T
0⟨HsY
ϵs , Y
ϵs ⟩ ds+ E⟨hY ϵ
T , YϵT ⟩ = E
∫ T
t0
⟨HsYϵs , Y
ϵs ⟩ ds+ E⟨hY ϵ
T , YϵT ⟩
= o(ϵ) + E∫ T
t0+ϵ
⟨HsYt0+ϵ,Y ϵ
t0+ϵ
s ,Yt0+ϵ,Y ϵt0+ϵ
s ⟩ ds+ E⟨hYt0+ϵ,Y ϵt0+ϵ
T ,Yt0+ϵ,Y ϵt0+ϵ
T ⟩
= o(ϵ) + E⟨Pt0+ϵYϵt0+ϵ, Y
ϵt0+ϵ⟩,
The argument is then concluded by proving the two following two relations:
E⟨(Pt0+ϵ − Pt0)Yϵt0+ϵ, Y
ϵt0+ϵ⟩ = o(ϵ),
E⟨Pt0Yϵt0+ϵ, Y
ϵt0+ϵ⟩ = E
∫ t0+ϵ
t0
⟨Psδϵσj(s, ·), δϵσj(s, ·)⟩ ds+ o(ϵ)
since in that case we obtain
∆ϵ3 = E
∫ t0+ϵ
t0
⟨Psδϵσj(s, ·), δϵσj(s, ·)⟩ ds+ o(ϵ)
29