Post on 14-Apr-2018
7/27/2019 Cours Controle Marrakech
1/77
Lectures on
Stochastic control and
applications in finance
Huyen PHAM
University Paris Diderot, LPMA
Institut Universitaire de France and CREST-ENSAE
pham@math.jussieu.fr
http://www.proba.jussieu.fr/pageperso/pham/pham.html
Autumn school on Stochastic control problems for FBSDEs and Applications
Marrakech, December 1-11, 2010
7/27/2019 Cours Controle Marrakech
2/77
Stochastic Control and applications in finance
Abstract. The aim of these lectures is to present an introduction to stochastic
control, a classsical topic in applied mathematics, which has known important devel-
opments over the last years inspired especially by problems in mathematical finance.We give an overview of the main methods and results in this area.
We first present the standard approach by dynamic programming equation and
verification, and point out the limits of this method. We then move on to the vis-
cosity solutions approach: it requires more theory and technique, but provides the
general mathematical tool for dealing with stochastic control in a Markovian context.
The last lecture is devoted to an introduction to the theory of Backward stochasticdifferential equations (BSDEs), which has emerged as a major research topic with
significant contributions in relation with stochastic control beyond the Markovian
framework. The various methods presented in these lectures will be illustrated by
several applications arising in economics and finance.
Lecture 1 : Classical approach to stochastic control problemLecture 2 : Viscosity solutions and stochastic control
Lecture 3 : BSDEs and stochastic control
7/27/2019 Cours Controle Marrakech
3/77
References for these lectures:
H. Pham (2009): Continuous-time stochastic control and optimization with fi-
nancial applications, Series SMAP, Springer.
I. Kharroubi, J. Ma, H. Pham and J. Zhang (2010): Backward stochastic dif-
ferential equations with constrained jumps and quasi-variational inequalities,
Annals of Probability, Vol. 38, 794-840.
7/27/2019 Cours Controle Marrakech
4/77
Lecture 1 : Classical approach to stochastic control problem
Introduction
Controlled diffusion processes
Dynamic Programming Principle (DPP)
Hamilton-Jacobi-Bellman (HJB) equation
Verification theorem
Applications : Merton portfolio selection (CRRA utility functions and general
utility functions by duality approach), Merton portfolio/ consumption choice
Some other classes of stochastic control
7/27/2019 Cours Controle Marrakech
5/77
I. Introduction
Basic structure of stochastic control problem
Dynamic system in an uncertain environment:
- filtered probability space (, F,F = (Ft),P): uncertainty and information
- state variables X = (Xt): F-adapted stochastic process representing the evolu-
tion of the quantitative variables describing the system
Control: a process = (t) whose value is decided at time t in function of
the available information Ft, and which can influence the dynamics of the state
process X.
Performance/criterion: optimize over controls a functional J(X, ), e.g.
J(X, ) = E
T0
f(Xt, t)dt + g(XT)
on a finite horizon
or
J(X, ) = E
0
etf(Xt, t)dt on an infinite horizon Various and numerous applications in economics and finance
In parallel, problems in mathematical finance new developments in the theory
of stochastic control
7/27/2019 Cours Controle Marrakech
6/77
Solving a stochastic control problem
Basic goal: find the optimal control(which achieves the optimum of the objective
functional) if it exists and the value function (the optimal objective functional)
Tractable characterization of the value function and optimal control
- if possible, explicit solutions
- otherwise: qualitative description and quantitative results via numerical solu-
tions
Mathematical tools
Dynamic programming principle and stochastic calculus
- PDE characterization in a Markovian context
- BSDE in general
Stochastic control is a topic at the interface between probability, stochastic analysis
and PDE.
7/27/2019 Cours Controle Marrakech
7/77
II. Controlled diffusion processes
Dynamics of the state variables in Rn:
dXs = b(Xs, s)ds + (Xs, s)dWs, (1)
W d-dimensional Brownian motion on (, F,F = (Ft),P).
- The control = (t) is an F-adapted process, valued in A subset of Rm, and
satisfying some integrability conditions and/or state constraints A set ofadmissible
controls.
- Given A, (t, x) [0, T] Rn, we denote by Xt,x = Xt,x, the solution to (1)
starting from x at t.
Performance criterion (on finite horizon)
Given a function f from Rn A into R, and a function g from Rn into R, we define
the objective functional:
J(t,x,) = E Tt f(Xt,xs , s)ds + g(Xt,xT ) , (t, x) [0, T] Rn, A,and the value function:
v(t, x) = supA
J(t,x,).
A is an optimal control if: v(t, x) = J(t,x, ).
A process in the form s = a(s, Xt,xs ) for some measurable function a from
[0, T] Rn into A is called Markovian or feedback control.
7/27/2019 Cours Controle Marrakech
8/77
III. Dynamic programming principle
Bellmans principle of optimality
An optimal policy has the property that whatever the initial state and initial decision
are, the remaining decisions must constitute an optimal policy with regard to the
state resulting from the first decision
(See Bellman, 1957, Ch. III.3)
Mathematical formulation of the Bellmans principle or Dynamic Pro-
gramming Principle (DPP)
The usual version of the DPP is written as
v(t, x) = s u pA
E
t
f(Xt,xs , s)ds + v(, Xt,x )
, (2)
for any stopping time Tt,T (set of stopping times valued in [t, T]).
7/27/2019 Cours Controle Marrakech
9/77
Stronger version of the DPP
In a stronger and useful version of the DPP, may actually depend on in (2). This
means:
v(t, x) = s u pA supTt,TE
t f(Xt,xs , s)ds + v(, X
t,x
)= sup
AinfTt,T
E
t
f(Xt,xs , s)ds + v(, Xt,x )
.
(i) For all A and Tt,T:
v(t, x) E
t
f(Xt,xs , s)ds + v(, Xt,x )
.
(ii) For all > 0, there exists A such that for all Tt,T
v(t, x) E
t
f(Xt,xs , s)ds + v(, Xt,x )
.
7/27/2019 Cours Controle Marrakech
10/77
Proof of the DPP. (First part)
1. Given A, we have by pathwise uniqueness of the flow of the SDE for X, the
Markovian structure
Xt,xs = X
,Xt,x
s , s ,
for any Tt,T. By the law of iterated conditional expectation, we then get
J(t,x,) = E
t
f(s, Xt,xs , s)ds + J(, Xt,x , )
,
Since J(. , . ,) v, and is arbitrary in Tt,T, this implies
J(t,x,) infTt,T
E
t
f(s, Xt,xs , s)ds + v(, Xt,x )
sup
AinfTt,T
E
t
f(s, Xt,xs , s)ds + v(, Xt,x )
.
=
v(t, x) supA
infTt,T
E
t
f(s, Xt,xs , s)ds + v(, Xt,x )
. (3)
7/27/2019 Cours Controle Marrakech
11/77
Proof of the DPP. (Second part)
2. Fix some arbitrary control A and Tt,T. By definition of the value functions,
for any > 0 and , there exists , A, which is an -optimal control for
v((), Xt,x()()), i.e.
v((), Xt,x()
()) J((), Xt,x()
(), ,). (4)
Let us now define the process
s() =
s(), s [0, ()]
,s (), s [(), T].
Delicate measurability questions! By measurable selection results, one can showthat is F-adapted, and so A.
By using again the law of iterated conditional expectation, and from (4):
v(t, x) J(t,x, ) = E
t
f(s, Xt,xs , s)ds + J(, Xt,x ,
)
E
tf(s, Xt,xs , s)ds + v(, Xt,x ) .
Since , and are arbitrary, this implies
v(t, x) supA
supTt,T
E
t
f(s, Xt,xs , s)ds + v(, Xt,x )
.
7/27/2019 Cours Controle Marrakech
12/77
IV. Hamilton-Jacobi-Bellman (HJB) equation
The HJB equation is the infinitesimal version of the dynamic programming principle:
it describes the local behavior of the value function when we send the stopping time
in the DPP (2) to t. The HJB equation is also called dynamic programming equation.
Formal derivation of HJB
We assume that the value function is smooth enough to apply It os formula, and we
postpone integrability questions.
For any A, and a controlled process Xt,x, apply Itos formula to v(s, Xt,xs
)
between s = t and s = t + h:
v(t + h, Xt,xt+h) = v(t, x) +
t+ht
v
t+ Lsv
(s, Xt,xs )ds + (local) martingale,
where for a A, La is the second-order operator associated to the diffusion X with
constant control a:
Lav = b(x, a).Dxv +1
2tr (x, a)(x, a)D2xv .
7/27/2019 Cours Controle Marrakech
13/77
7/27/2019 Cours Controle Marrakech
14/77
V. Verification approach
Theorem
Let w be a function in C1,2([0, T] Rn), solution to the HJB equation:
wt
(t, x) + supaA
Law(t, x) + f(x, a) = 0, (t, x) [0, T) Rn,w(T, x) = g(x), x Rn.
(and satisfying eventually additional growth conditions related to f and g). Suppose
there exists a measurable function a(t, x), (t, x) [0, T) Rn, valued in A, attaining
the supremum in HJB i.e.
supaA
Law(t, x) + f(x, a) = La(t,x)w(t, x) + f(x, a(t, x)),such that the SDE
dXs = b(Xs, a(s, Xs))ds + (Xs, a(s, Xs))dWs
admits a unique solution, denoted by Xt,xs , given an initial condition Xt = x, and the
process = {a(s,
Xt,x
s ) t s T} lies in A. Then,
w = v,
and is an optimal feedback control.
7/27/2019 Cours Controle Marrakech
15/77
Proof of the verification theorem. (First part)
1. Suppose that w is a smooth supersolution to the HJB equation:
w
t(t, x) sup
aA
Law(t, x) + f(x, a)
0, (t, x) [0, T) Rn, (6)
w(T, x) g(x), x Rn. (7)
For any A, and a controlled process Xt,x, apply Itos formula to w(s, Xt,xs )
between s = t and s = T n, and take expectation:
E
w(T n, X
t,xTn
)
= w(t, x) + E
Tnt
w
t+ Lsw
(s, Xt,xs )ds
where (n) is a localizing sequence of stopping times for the local martingale appearingin Itos formula.
Since w is a supersolution to HJB (6), this implies:
E
w(T n, Xt,xTn
)
+ E
Tnt
f(Xt,xs , s)ds
w(t, x).
By sending n to infinity, and under suitable integrability conditions, we get:
E
w(T, Xt,xT )
+ ET
t
f(Xt,xs , s)ds
w(t, x).
Since w(T, .) g, and is arbitrary, we obtain
v(t, x) = s upA
E
Tt
f(Xt,xs , s)ds + g(Xt,xT )
w(t, x).
7/27/2019 Cours Controle Marrakech
16/77
Proof of the verification theorem. (Second part)
2. Suppose that the supremum in HJB equation is attained:
w
t(t, x) La(t,x)w(t, x) + f(x, a(t, x) ) = 0, (t, x) [0, T) Rn, (8)
w(T, x) = g(x), x Rn. (9)
Apply Itos formula to w(s, Xt,xs ) for the feedback control . By same arguments
as in the first part, we have now the equality (after an eventual localization):
w(t, x) = E
Tt
w
t+ Lsw
(s, Xt,xs )ds + w(T, X
t,xT )
= E T
tf(Xt,xs , s)ds + g(Xt,xT ) ( v(t, x)).
Together with the first part, this proves that w = v and is an optimal feedback
control.
7/27/2019 Cours Controle Marrakech
17/77
Probabilistic formulation of the verification approach
The analytic statement of the verification theorem has a probabilistic formulation:
Suppose that the measurable function w on [0, T] Rn satisfies the two properties:
for any control A with associated controlled process X, the process
w(t, Xt) +
t0
f(Xs, s)ds is a supermartingale (10)
there exists a control A with associated controlled process X, such that the
process
w(t, Xt) + t
0
f(Xs, s)ds is a martingale. (11)
Then, w = v, and is an optimal control.
Remark. Notice that in the probabilistic verification approach, we do not need
smoothness of w, but we require a supermartingale property. In the analytic verifi-
cation approach, the smoothness of w is used for applying Itos formula to w(t, Xt).
This allows us to derive the supermartingale property as in (10), which is in fact the
key feature for proving that w v, and then w = v with the martingale property
(11).
7/27/2019 Cours Controle Marrakech
18/77
VI. Applications
1. Merton portfolio selection in finite horizon
An agent invests at any time t a proportion t of his wealth X in a stock of price S
and 1 t in a bond of price S0
with interest rate r. The investor faces the portfolioconstraint that at any time t, t is valued in A closed convex subset ofR.
Assuming a Black-Scholes model for S(with constant rate of return and volatility
> 0), the dynamics of the controlled wealth process is:
dXt =Xtt
StdSt +
Xt(1 t)
S0tdS0t
= Xt (r + t( r)) dt + XttdWt.
The preferences of the agent is described by a utility function U: increasing and con-
cave function. The performance of a portfolio strategy is measured by the expected
utility from terminal wealth Utility maximization problem at a finite horizon T:
v(t, x) = s u pA
E[U(Xt,xT )], (t, x) [0, T] (0, ).
Standard stochastic control problem
7/27/2019 Cours Controle Marrakech
19/77
HJB equation for Mertons problem
vt + rxvx + supaA
a( r)xvx +1
2x2a22vxx
= 0, (t, x) [0, T) (0, )
v(T, x) = U(x), x > 0.
The case of CRRA utility functions:
U(x) =xp
p, p < 1, p = 0
Relative Risk Aversion: xU(x)/U(x) = 1 p.
We look for a candidate solution to HJB in the form
w(t, x) = (t)U(x).
Plugging into HJB, we see that should satisfy the ODE:
(t) + (t) = 0, (T) = 1,
where
= rp +p supaA
a( r)
1
2a2(1 p)2
,
(t) = e(Tt).
7/27/2019 Cours Controle Marrakech
20/77
The value function is equal to
v(t, x) = e(Tt)U(x),
and the optimal control is constant (in proportion of wealth invested)
a = argmaxaA a( r) 12a2(1 p)2.When A = R (no portfolio constraint), the values of and a are explicitly given by
=( r)2
22p
1 p+ rp.
and
a = r2(1 p)
,
7/27/2019 Cours Controle Marrakech
21/77
General utility functions:
U is C1, strictly increasing and concave on (0, ), and satisfies the Inada conditions:
U(0) = , U() = 0.
Convex conjugate of U:
U(y) := supx>0
[U(x) xy] = U(I(y)) yI(y), y > 0,
where I := (U)1 = U.
Assume that A = R (no portfolio constraint and complete market) and for simplicity
r = 0 so that HJB is also written as
vt 1
2
2
2v2xvxx
= 0,
with a candidate for the optimal feedback control:
a(t, x) =
2vx
x2vxx.
Recall the terminal condition:
v(T, x) = U(x).
Fully nonlinear second order PDE
But remarkably, it can be solved explicitly by convex duality!
7/27/2019 Cours Controle Marrakech
22/77
Introduce the convex conjugate of v, also called dual value function:
v(t, y) = s u px>0
[v(t, x) xy], y > 0.
change of variables: y = vx and x = vy.
v satisfies the linear parabolic Cauchy problem:
vt +1
2
2
2y2vyy = 0
v(T, y) = U(y).
From Feynman-Kac formula, v is represented as
v(t, y) = E
U(yYtT)
,
where Yt is the solution to
dYts = Yts
dWs, Y
tt = 1.
Remark. YtT = EdQ/dPFt is the density of the risk-neutral probability measureQ, under which S is a martingale:
dSt = StdWQt ,
7/27/2019 Cours Controle Marrakech
23/77
The primal value function is obtained by duality relation:
v(t, x) = inf y>0
v(t, y) + xy
, x > 0.
From the representation of v, we get:
v(t, x) = inf y>0
E
U(yYtT)
+ xy
(12)
Recalling that U = (U)1 =: I, the infimum in (12) is attained at y = y(t, x) s.t.
E
YtTI(yYtT)
= x, (saturation budget constraint) (13)
and we have
v(t, x) = E
U(yYtT) + yYtTI(yY
tT)
.
Recalling that the supremum in U is attained at x = I(y), i.e. U(y) = U(I(y))yI(y),
we obtain:
v(t, x) = EU(Xt,xT ), with Xt,xT = I(yYtT). (14)
7/27/2019 Cours Controle Marrakech
24/77
Consider now the strictly positive Q-martingale process:
Xt,xs := EQ
I(yYtT)Fs, t s T.
- From the saturation budget constraint (13), we have Xt,xt = x.
- From the martingale representation theorem (or since the market is complete), there
exists A s.t.
dXt,xs = Xt,xs sdW
Qs =
Xt,xs sSs
dSs,
which means that Xt,x is a wealth process controlled by the proportion , and starting
from initial capital x at time t.
From the representation (14) of the value function, this proves that Xt,x is the
optimal wealth process:
v(t, x) = E
U(Xt,xT )
.
7/27/2019 Cours Controle Marrakech
25/77
2. Merton portfolio/consumption choice on infinite horizon
In addition to the investment in the stock, the agent can also consume from his
wealth:
(ct)t0 consumption per unit of wealth
The wealth process, controlled by (, c) is governed by:
dXt = Xt (r + t( r) ct) dt + XttdWt.
The preferences of the agent is described by a utility U from consumption, and the
goal is to maximize over portfolio/consumption the expected utility from intertem-
poral consumption up to a random time horizon:
v(x) = sup(,c)
E
0
etU(ctXxt )dt
, x > 0.
We assume that is independent of F (market information), and follows E().
Infinite horizon stochastic control problem:
v(x) = s u p(,c)
E
0
e(+)tU(ctXxt )dt
, x > 0.
7/27/2019 Cours Controle Marrakech
26/77
HJB equation
(+ )v rxv supaA
[a( r)v +1
2a2x22v] sup
c0[U(cx) cxv] = 0, x > 0.
Explicit solution for CRRA utility function: U(x) = xp/p.
Under the condition that + > , we have
v(x) = K U(x), with K =
1 p
+
1p.
The optimal portfolio/consumption strategies are:
a = arg maxaA
[a( r) 1
2a2(1 p)2]
c =1
x(v(x))
1p1 = K
1p1 .
7/27/2019 Cours Controle Marrakech
27/77
VII. Some other classes of stochastic control problems
Ergodic and risk-sensitive control problems
- Risk-sensitive control problem:
lim supT
1
T lnE exp T
0 f(Xt, t)dt Applications in finance: Bielecki, Pliska, Fleming, Sheu, Nagai, Davis, etc ...
- Large deviations control problem:
lim supT
1
TP
XTT
x
Dual of risk-sensitive control problem
Applications in finance: Pham, Sekine, Nagai, Hata, Sheu
7/27/2019 Cours Controle Marrakech
28/77
Optimal stopping problems:
The control decision is a stopping time where we decide to stop the process
Value function of optimal stopping problem (over a finite horizon):
v(t, x) = suptT
E
t
f(Xt,xs )ds + g(Xt,x )
.
The HJB equation is a free boundary or variational inequality:
min
v
t Lv f , v g
= 0,
where L is the infinitesimal generator of the Markov process X.
Typical applications in finance in American option pricing
7/27/2019 Cours Controle Marrakech
29/77
Impulse and optimal switching problems:
The control is a sequence of increasing stopping times (n)n associated to a sequence
of actions (n)n: n represents the time decision when we decide to intervene on the
state system X by using an action n Fn-measurable: Xn (Xn , n)
Value function:
v(t, x) = sup(nn)
E
Tt
f(Xt,xs )ds + g(Xt,xT ) +
n
c(Xn , n)
.
The HJB equation is a quasi-variational inequality:
min
v
t Lv f , v Hv
= 0,
where L is the infinitesimal generator of the Markov process X, and H is a nonlocal
operator associated to the jump and cost induced by an action:
Hv(t, x) = s u peE
v(t, (x, e)) + c(x, e)
.
Various applications in finance:
Transaction costs and liquidity risk models, where trading times take place dis-
cretely
Real options and firm investment problems, where decisions represent change of
regimes or production technologies
7/27/2019 Cours Controle Marrakech
30/77
Lecture 2 : Viscosity solutions and stochastic control
Non smoothness of value functions: a motivating financial example
Introduction to viscosity solutions
Viscosity properties of the dynamic programming equation
Comparison principles
Application: Super-replication in uncertain volatility models
7/27/2019 Cours Controle Marrakech
31/77
I. Non smoothness of value functions: a motivating financial example
Consider the controlled diffusion process
dXs = sXsdWs,
with an unbounded control valued in A = R+: Uncertain volatility model.
Consider the stochastic control problem
v(t, x) = s u pA
E[g(Xt,xT )], (t, x) [0, T] (0, ),
Superreplication cost of an option payoff g(XT).
If v were smooth, it would be a classical solution to the HJB equation:
vt + supaR+
12
a2x2vxx
= 0, (t, x) [0, T) (0, ). (1)
But, for the supremum in a R to be finite and HJB equation (1) to be well-posed,
we must have
vxx 0, i.e. v(t, .) is concave in x, for any t [0, T).
7/27/2019 Cours Controle Marrakech
32/77
Now, by taking the zero control in the definition of v, we get
v(t, x) g(x),
which combined with the concavity of v(t, .), implies:
v(t, x) g(x), t < T ,
where g is the concave envelope of g: the smallest concave function above g.
Moreover, since g g, and by Jensens inequality and martingale property of X,
we have
v(t, x) supA
E[g(Xt,xT )]
supA
g
E[Xt,xT ]
= g(x).
Therefore,
v(t, x) = g(x), (t, x) [0, T) (0, ).
There is a contradiction with the smoothness of v, whenever g is not smooth!,for example when g is concave (hence equal to g) but not smooth.
Need to consider the case where the supremum in HJB can explode (singular case)
and to define weak solutions for HJB equation
Notion of viscosity solutions (Crandall, Ishii, P.L. Lions)
7/27/2019 Cours Controle Marrakech
33/77
II. Introduction to viscosity solutions
Consider nonlinear parabolic second-order partial differential equations:
F(t,x,w,w
t, Dxw, D
2xxw) = 0, (t, x) [0, T) O, (2)
where O is an open subset ofRn
and F is a continuous function of its arguments,satisfying the ellipticity condition: for all (t, x) [0, T) O, r R, (q, p) R Rn,
M, M Sn,M M = F(t,x,r,q,p,M) F(t,x,r,q,p, M), (3)
and the parabolicity condition: for all t [0, T), x O, r R, q, q R, p Rn,
M Sn,
q q = F(t,x,r,q,p,M) F(t,x,r, q ,p,M ). (4)
Typical example: HJB equation
F(t,x,r,q,p,M) = q H(x,p,M),
where H is the Hamiltonian function of the stochastic control problem:
H(x,p,M) = s u paA
b(x, a).p +
1
2tr ( (x, a)M) + f(x, a)
7/27/2019 Cours Controle Marrakech
34/77
Intuition for the notion of viscosity solutions
Assume that w is a smooth supersolution to (2). Let be a smooth test function
on [0, T) O, and (t, x) [0, T) O be a minimum point of w :
0 = (w )(t, x) = min(w ).
In this case, the first and second-order optimality conditions imply
(w )
t(t, x) 0 (= 0 ift > 0)
Dxw(t, x) = Dx(t, x) and D2xw(t, x) D
2x(t, x).
From the ellipticity and parabolicity conditions (3) and (4), we deduce that
F(t, x, (t, x), t
(t, x), Dx(t, x), D2x(t, x))
F(t, x, w(t, x),w
t(t, x), Dxw(t, x), D
2xw(t, x)) 0,
Similarly, if w is a classical subsolution to (2), then for all test functions , and
(t, x) [0, T) O such that (t, x) is a maximum point of w , we have
F(t, x, (t, x),
t(t, x), Dx(t, x), D
2x(t, x)) 0.
7/27/2019 Cours Controle Marrakech
35/77
General definition of (discontinuous) viscosity solutions
Given a locally bounded function w on [0, T] O, we define its upper-semicontinuous
(usc) envelope w and lower-semicontinuous (lsc) envelope w by
w(t, x) = lim s upt
7/27/2019 Cours Controle Marrakech
36/77
III. Viscosity properties of the DPE
We turn back to the stochastic control problem:
v(t, x) = supA
E
Tt
f(Xt,xs , s)ds + g(Xt,xT )
, (t, x) [0, T] Rn,
with Hamiltonian function H on Rn
Rn
Sn:
H(x,p,M) = s u paA
b(x, a).p +
1
2tr ((x, a)M) + f(x, a)
.
We introduce the domain of H as
dom(H) = {(x,p,M) Rn Rn Sn : H(x,p,M) < } ,
and make the following hypothesis (DH):
H is continuous on int(dom(H))
and there exists a continuous function G on Rn Rn Sn such that
(x,p,M) dom(H) G(x,p,M) 0.
Example. In the example considered at the beginning of this lecture:
H(x,p,M) = s u paR
12
a2x2M
,
and so
G(x,p,M) = M.
7/27/2019 Cours Controle Marrakech
37/77
Viscosity property inside the domain
Theorem .1 The value function v is a viscosity solution to the HJB variational
inequality
min vt
H(x, Dxv, D2xv) , G(x, Dxv, D2xv) = 0, on [0, T) Rn.Remark. In the regular case when the Hamiltonian H is finite on the whole domain
RnRn Sn (this occurs typically when the control space is compact), the condition
(DH) is satisfied with any choice of strictly positive continuous function G. In this
case, the HJB variational inequality is reduced to the regular HJB equation:
v
t(t, x) H(x, Dxv, D
2xv) = 0, (t, x) [0, T) R
n,
which the value function satisfies in the viscosity sense. Hence, the above Theorem
states a general viscosity property including both the regular and singular case.
7/27/2019 Cours Controle Marrakech
38/77
Proof of viscosity supersolution property
Let (t, x) [0, T) Rn and let C2([0, T) Rn) be a test function such that
0 = (v )(t, x) = min[0,T)Rn
(v ). (5)
By definition of v(t, x), there exists a sequence (tm, xm)m in [0, T) R
n
such that
(tm, xm) (t, x) and v(tm, xm) v(t, x),
when m goes to infinity. By the continuity of and by (5) we also have that
m := v(tm, xm) (tm, xm) 0.
Let A, a constant process equal to a A, and Xtm,xms the associated controlled
process. Let m = inf{s tm : |Xtm,xms xm| }, with > 0 a fixed constant. Let
(hm) be a strictly positive sequence such that
hm 0 andmhm
0.
We apply the first part of the DPP for v(tm, xm) to m := m (tm + hm) and get
v(tm, xm) E mtm
f(s, Xtm,xms , a)ds + v(m, X
tm,xmm ) .
Equation (5) implies that v v , thus
(tm, xm) + m E
mtm
f(s, Xtm,xms , a)ds + (m, Xtm,xmm
)
.
7/27/2019 Cours Controle Marrakech
39/77
Apply Itos formula to (s, Xtm,xms ) between tm and m:
mhm
+ E
1
hm
mtm
t La f
(s, Xtm,xms , a)ds
0. (6)
Now, send m to infinity: by the mean value theorem, and the dominated convergence
theorem, we get
t(t, x) La(t, x) f(t, x, a) 0.
Since a is arbitrary in A, and by definition of H, this means:
t(t, x) H(x, Dx(t, x), D
2x(t, x)) 0.
In particular, (x, Dx(t, x), D2x(t, x)) dom(H), and so
G(x, Dx(t, x), D2x(t, x)) 0.
Therefore,
min
t(t, x) H(x, Dx(t, x), D
2x(t, x)) , G(x, Dx(t, x), D
2x(t, x))
0,
which is the required supersolution property.
7/27/2019 Cours Controle Marrakech
40/77
Proof of viscosity subsolution property
Let (t, x) [0, T) Rn and let C2([0, T) Rn) be a test function such that
0 = (v )(t, x) = max[0,T)Rn
(v ). (7)
As before, there exists a sequence (tm, xm)m in [0, T) Rn
s.t.
(tm, xm) (t, x) and v(tm, xm) v(t, x),
m := v(tm, xm) (tm, xm) 0.
We will show the result by contradiction, and assume on the contrary that
t(t, x) H(x, Dx(t, x), D
2x(t, x)) > 0,
and G(x, Dx(t, x), D2x(t, x)) > 0.
Under (DH), there exists > 0 such that
t(t, y) H(y, Dx(t, y), D
2x(t, y)) > 0, for (t, x) B(t, ) B(x, ). (8)
Observe that we can assume w.l.o.g. in (7) that (t, x) achieves a strict maximum so
that
maxpB((t,x),)
(v ) =: < 0, (9)
where pB((t, x), ) = [t, t + ] B (x, ) {t + } B(x, ).
7/27/2019 Cours Controle Marrakech
41/77
We apply the second part of DP: there exists m A s.t.
v(tm, xm)
2 E
mtm
f(Xtm,xms , ms )ds + v(m, X
tm,xmm
)
, (10)
where m = inf{s tm : (s, Xtm,xms ) / B(t, ) B(x, )}. Observe by continuity of
the state process that (m, Xtm,xm
m) pB((t, x), ) so that from (9)-(10):
(tm, xm) + m
2 E
mtm
f(Xtm,xms , ms )ds + (m, X
tm,xmm
)
.
Apply Itos formula to (s, Xtm,xms ) between tm and m, we then get after noting that
the stochastic integral vanishes in expectation:
m
2+ E
m
tm t Lms f (s, Xtm,xms , ms )ds . (11)Now, from (8) and definition of H, we have
t(s, Xtm,xms ) L
ms (s, Xtm,xms ) f(Xtm,xms ,
ms )
t(s, Xtm,xms ) H(s, Dx(s, X
tm,xms ), D
2x(s, X
tm,xms ))
> 0, for tm s m.
Plugging into (11), this implies
m
2 , (12)
and we get the contradiction by sending m to infinity: /2 .
7/27/2019 Cours Controle Marrakech
42/77
Terminal condition
Due to the singularity of the Hamiltonian H, the value function may be discontinuous
at T, i.e. v(T, x) may be different from g(x). The right terminal condition is given
by the relaxed terminal condition:
Theorem .2 The value function v is a viscosity solution to
min
v g , G(x, Dxv, D2xv)
= 0, on {T} Rn. (13)
This means that v(T, .) is a viscosity supersolution to
min
v(T, x) g(x) , G(x, Dxv(T, x), D
2xv(T, x))
0, on Rn. (14)
and v(T, .) is a viscosity subsolution to
min
v(T, x) g(x) , G(x, Dxv(T, x), D2xv
(T, x))
0, on Rn. (15)
Remark. Denote by g the upper G-envelope of g, defined as the smallest function
above g, and viscosity supersolution to
G(x, Dg, D2g) 0.
Then v(T, x) g(x). On the other hand, since g is a viscosity supersolution to (14),
and if a comparison principle holds for (13), then v(T, x) g(x). This implies
v(T, x) = v(T, x) = v(T, x) = g(x).
In the regular case, we have g = g, and v is continuous at T.
7/27/2019 Cours Controle Marrakech
43/77
IV. Strong comparison principles and uniqueness
Consider the DPE satisfied by the value function
min
v
t H(x, Dxv, D
2xv) , G(x, Dxv, D
2xv)
= 0, on [0, T) Rn. (16)
min v(T, x) g(x) , G(x, Dxv, D2xv) = 0, on {T} Rn. (17) We say that a strong comparison principle holds for (16)-(17) when the follow-
ing statement is true:
Ifu is an usc viscosity subsolution to (16)-(17) and w is a lsc viscosity supersolution
to (16)-(17), satisfying some growth condition, then u w.
Remark. The arguments for proving comparison principles are:
- dedoubling variables technique
- Ishiis Lemma
Standard reference: users guide of Crandall, Ishiis Lions (92).
7/27/2019 Cours Controle Marrakech
44/77
Consequence of strong comparison principles
Uniqueness and continuity
Suppose that v and w are two viscosity solutions to (16)-(17). This means that v is
a viscosity subsolution to (16)-(17), and w is a viscosity supersolution to (16)-(17),
and vice-versa. By the strong comparison principle, we get:
v w and w v.
Since w w, v v
, this implies:
v = v = w = w.
Therefore,
v = w, i.e. uniqueness
v = v, i.e. continuity of v on [0, T) Rn.
Conclusion
The value function of the stochastic control problem is the unique continuous viscosity
solution to (16)-(17) (satisfying some growth condition).
A li i li i i i l ili d l
7/27/2019 Cours Controle Marrakech
45/77
V. Application: superreplication in uncertain volatility model
Consider the controlled diffusion
dXs = sXsdWs, t s T,
with the control process valued in A = [a, a], where 0 a a . Given a
continuous function g on R+, we consider the stochastic control problem:
v(t, x) = s u pA
E
g(Xt,xT )
, (t, x) [0, T] (0, ).
Financial interpretation
represents the uncertain volatility process of the stock price X, and the functiong represents the payoff of an European option of maturity T. The value function
v is the superreplication cost for this option, that is the minimum capital required
to superhedge (by means of trading strategies on the stock) the option payoff at
maturity T whatever the realization of the uncertain volatility.
Th H il i f hi h i l bl i
7/27/2019 Cours Controle Marrakech
46/77
The Hamiltonian of this stochastic control problem is
H(x, M) = supa[a,a]
12
a2x2M
, (x, M) (0, ) R.
We shall then distinguish two cases: a finite or not.
Bounded volatility: a < .
In this regular case, H is finite on the whole domain (0, ) R, and is given by
H(x, M) =1
2a2(M)x2M,
with
a(M) = a if M 0a if M < 0. v is continuous on [0, T] (0, ), and is the unique viscosity solution with linear
growth condition to the so-called Black-Scholes-Barenblatt equation
vt +1
2a2 (vxx) x
2vxx = 0, (t, x) [0, T) (0, ),
satisfying the terminal condition
v(T, x) = g(x), x (0, ).
Remark. If g is convex, then v is equal to the Black-Scholes price with volatility a,
which is convex in x, so that a(vxx) = a.
U b d d l tilit
7/27/2019 Cours Controle Marrakech
47/77
Unbounded volatility: a = .
In this singular case, the Hamiltonian is given by
H(x, M) =
12
a2x2M if G(M) := M 0
if M < 0.
v is the unique viscosity solution to the HJB variational inequality
min
vt 1
2a2x2vxx , vxx
= 0, on [0, T) (0, ), (18)
min
v g , vxx
= 0, on {T} (0, ). (19)
Explicit solution to (18)-(19)
Denote by g the concave envelope of g, i.e. the solution to
min[g g , gxx] = 0.
Let us consider the Black-Scholes price with volatility a of the option payoff g, i.e.
w(t, x) = E
g
Xt,xT
,
where
dXs = aXsdWs, t s T, Xt = x.
Then,
v = w, on [0, T) (0, ).
P f
7/27/2019 Cours Controle Marrakech
48/77
Proof.
Indeed, the function w is solution to the Black-Scholes equation:
wt +1
2a2x2wxx = 0, on [0, T) (0, )
w(T, x) = g(x), x (0, ).
Moreover, w inherits from g the concavity property, and so
wxx 0, (t, x) [0, T) (0, ).
(This holds true in the viscosity sense)
Therefore, w satisfies the same HJB variational inequality as v:
min
wt 1
2a2x2wxx , wxx
= 0, on [0, T) (0, ),
min
w g , wxx
= 0, on {T} (0, ).
We conclude by uniqueness result.
Remark. When a = 0, we have w = g, and so v(t, x) = g(x) on [0, T) (0, ).
L t 3 B k d St h ti Diff ti l E ti d
7/27/2019 Cours Controle Marrakech
49/77
Lecture 3 : Backward Stochastic Differential Equations and
stochastic control
Introduction
General properties of BSDE
The Markov case : nonlinear Feynman-Kac formula. Simulation of BSDE
Application: CRRA utility maximization
Reflected BSDE and optimal stopping problem
BSDE with constrained jumps and quasi-variational inequalities
I Introduction
7/27/2019 Cours Controle Marrakech
50/77
I. Introduction
BSDEs first introduced by Bismut (73): adjoint equation in Pontryagin maximum
principle (linear BSDEs)
Emergence of the theory since the seminal paper by Pardoux and Peng (90): general
BSDEs
BSDEs widely used in stochastic control and mathematical finance
Hedging and pricing problems linear and reflected BSDE
Portfolio optimization, risk measure nonlinear BSDE, reflected and constrained
BSDEs
Improve existence and uniqueness of BSDEs, especially quadratic BSDEs
BSDE provide a probabilistic representation of nonlinear PDEs: nonlinear Feynman-
Kac formulae
Numerical methods for nonlinear PDEs
II General results on BSDEs
7/27/2019 Cours Controle Marrakech
51/77
II. General results on BSDEs
Let W = (Wt)0tT be a standard d-dimensional Brownian motion on (, F,F, P)
where F = (Ft)0tT is the natural filtration of W, and T is a fixed finite horizon.
Notations
P: set of progressively measurable processes on [0, T]
S2(0, T): set of elements Y P such that
E
sup0tT
|Yt|2
< ,
H2(0, T)d: set of elements Z P, Rd-valued, such that
E T0
|Zt|2dt < .
Definition of BSDE
A (one-dimensional) Backward Stochastic Differential Equation (BSDE in short) is
written in differential form as
dYt = f(t, Yt, Zt)dt Zt.dWt, YT = , (1)
where the data is a pair (, f), called terminal condition and generator (or driver):
L2(, FT,P) , f(t ,,y,z ) is P B(R Rd)-measurable.
A solution to (1) is a pair (Y, Z) S2(0, T) H2(0, T)d such that
Yt = + T
t
f(s, Ys, Zs)ds T
t
Zs.dWs, 0 t T.
Under some specific assumptions on the generator f there is existence and uniqueness
7/27/2019 Cours Controle Marrakech
52/77
Under some specific assumptions on the generator f, there is existence and uniqueness
of a solution to the BSDE (1).
Standard Lipschitz assumption (H1)
f is uniformly Lipschitz in (y, z), i.e. there exists a positive constant C s.t. for
all (y , z , y, z):
|f(t ,y,z ) f(t, y, z)| C
|y y| + |z z|
, dt dP a.e.
The process {f(t, 0, 0), t [0, T]} H2(0, T)
Theorem (Pardoux and Peng 90) Under (H1), there exists a unique solution (Y, Z)
to the BSDE (1).Proof. (a) Assume first the case where f does not depend on (y, z), and consider
the martingale
Mt = E
+
T0
f(t, )dt
Ft ,which is square-integrable under (H1), i.e. M S2(0, T). By the martingale
representation theorem, there exists a unique Z H2
(0, T)d
s.t.
Mt = M0 +
t0
Zs.dWs, 0 t T.
Then, the process
Yt := E
+
Tt
f(s, )dsFt
= Mt
t0
f(s, )ds, 0 t T,
satisfies (with Z) the BSDE (1).
Proof (b) Consider now the general Lipschitz case As in the deterministic case
7/27/2019 Cours Controle Marrakech
53/77
Proof. (b) Consider now the general Lipschitz case. As in the deterministic case,
we give a proof based on a fixed point method. Let us consider the function on
S2(0, T)m H2(0, T)d, mapping (U, V) S2(0, T) H2(0, T)d to (Y, Z) = (U, V)
defined by
Yt = + Tt f(s, Us, Vs)ds T
t Zs.dWs.
This pair (Y, Z) exists from Step (a). We then see that (Y, Z) is a solution to the
BSDE (1) if and only if it is a fixed point of .
Let (U, V), (U, V) S2(0, T)H2(0, T)d and (Y, Z) = (U, V), (Y, Z) = (U, V).
We set (U , V) = (U U, V V), (Y , Z) = (Y Y, Z Z) and ft = f(t, Ut, Vt)
f(t, U
t, V
t ). Take some > 0 to be chosen later, and apply Itos formula to es
|Ys|
2
between s = 0 and s = T:
|Y0|2 =
T0
es
|Ys|2 2Ys.fs
ds
T0
es|Zs|2ds 2
T0
esYs Zs.dWs.
By taking the expectation, we get
E|Y0|2 + E
T0
es
|Ys|2 + |Zs|
2
ds
= 2E T
0
esYs.fsds
2CfE T
0
es|Ys|(|Us| + |Vs|)ds
4C2fE T
0
es|Ys|2ds
+
1
2E T
0
es(|Us|2 + |Vs|
2)ds
Proof continued (b) Now we choose = 1 + 4C2 and obtain
7/27/2019 Cours Controle Marrakech
54/77
Proof continued. (b) Now, we choose = 1 + 4Cf, and obtain
E T
0
es
|Ys|2 + |Zs|
2
ds
1
2E T
0
es(|Us|2 + |Vs|
2)ds
.
This shows that is a strict contraction on the Banach space S2(0, T) H2(0, T)d
endowed with the norm
(Y, Z)
=E T
0
es
|Ys|2 + |Zs|
2
ds 1
2
.
We conclude that admits a unique fixed point, which is the solution to the BSDE
(1).
Non-Lipschitz conditions on the generator
f is continuous in (y, z) and satisfies a linear growth condition on (y, z). Then,
there exists a minimal solution to the BSDE (1). (Lepeltier and San Martin 97)
f is continuous in (y, z), linear in y, and quadratic in z, and is bounded. Then,
there exists a unique bounded solution to the BSDE (1) (Kobylanski 00).
III The Markov case: non-linear Feynman-Kac formulae
7/27/2019 Cours Controle Marrakech
55/77
III. The Markov case: non linear Feynman Kac formulae
Linear Feynman-Kac formula
Consider the linear parabolic PDE
v
t
(t, x) + Lv(t, x) + f(t, x) = 0, on [0, T) Rd (2)
v(T, .) = g, on Rd, (3)
where L is the second-order differential operator
Lv = b(x).Dxv +1
2tr( (x)D2xv).
Consider the (forward) diffusion process
dXt = b(Xt)dt + (Xt)dWt.
Then, by Itos formula to v(t, Xt) between t and T, with v smooth solution to (2)-(3):
v(t, Xt) = g(XT) +
Tt
f(s, Xs)ds
Tt
Dxv(s, Xs)(Xs)dWs.
It follows that the pair (Yt, Zt) = (v(t, Xt), (Xt)Dxv(t, Xt)) solves the linear BSDE:
Yt = g(XT) + Tt
f(s, Xs)ds Tt
ZsdWs.
Remark
We can compute the solution v(0, X0) = Y0 by the Monte-Carlo expectation:
Y0 = E
g(XT) +
T0
f(s, Xs)ds
.
Non linear Feynman-Kac formula
7/27/2019 Cours Controle Marrakech
56/77
Non linear Feynman Kac formula
Consider the semilinear parabolic PDE
v
t+ Lv + f(t,x, v, Dxv) = 0, on [0, T) R
d (4)
v(T, .) = g, on Rd, (5)
The corresponding BSDE is
Yt = g(XT) +
Tt
f(s, Xs, Ys, Zs)ds
Tt
ZsdWs, (6)
in the sense that:
the pair (Yt, Zt) = (v(t, Xt), (Xt)Dxv(t, Xt)) solves (6)
Conversely, if (Y, Z) is a solution to (6), then Yt = v(t, Xt) for some deterministic
function v, which is a viscosity solution to (4)-(5).
The time discretization and simulation of the BSDE (6) provides a numerical
method for solving the semilinear PDE (4)-(5)
Simulation of BSDE: time discretization
7/27/2019 Cours Controle Marrakech
57/77
Simulation of BSDE: time discretization
Time grid = (ti) on [0, T] : ti = it, i = 0, . . . , N , t = T /N
Forward Euler scheme X for X : starting from Xt0 = x,
Xti+1 := X
ti + b(X
ti)t + (X
ti)Wti+1 Wti
Backward Euler scheme (Y, Z) for (Y, Z) : starting from YtN = g(XtN
),
Yti = Yti+1
+ f(Xti , Yti+1
, Zti)t Zti
.
Wti+1 Wti
(7)
and take conditional expectation:
Yti = EYti+1 + f(Xti , Yti+1, Zti)tXti
To get the Z-component, multiply (7) by Wti+1 Wti and take expectation:
Zti =1
tE
Yti+1(Wti+1 Wti)Xti
Simulation of BSDE: numerical methods
7/27/2019 Cours Controle Marrakech
58/77
S S s
How to compute these conditional expectations! several approaches:
Regression based algorithms (Longstaff, Schwartz)
Choose q deterministic basis functions 1, . . . , q, and approximate
Zti = E
Yti+1(Wti+1 Wti)Xti q
k=1
kk(Xti
)
where = (k) solve the least-square regression problem:
arg infRq
E
Yti+1(Wti+1 Wti)
q
k=1kk(X
ti
)
2
Here E is the empirical mean based on Monte-Carlo simulations of Xti , Xti+1
, Wti+1
Wti.
Efficiency enhanced by using the same set of simulation paths to compute all
conditional expectations.
Other methods:
Malliavin Monte-Carlo approach (P.L. Lions, Regnier)
Quantization methods (Pages)
Important literature: Kohatsu-Higa, Pettersson (01), Ma, Zhang (02), Bally and
Pages (03), Bouchard, Ekeland, Touzi (04), Gobet et al. (05), Soner and Touzi (05),
Peng, Xu (06), Delarue, Menozzi (07), Bender and Zhang (08), etc ...
IV. Application: CRRA utility maximization
7/27/2019 Cours Controle Marrakech
59/77
pp y
Consider a financial market model with one riskless asset S0 = 1, and n stocks of
price process
dSt = diag(St)tdt + tdWt
,
wher W is a d-dimensional Brownian motion (with d n), b, bounded adapted
processes, of full rank n.
Consider an agent investing in the stocks a fraction of his wealth X at any time:
dXt = Xtt diag(St)
1dSt = Xt(ttdt +
ttdWt) (8)
A0: set ofF-adapted processes valued in A closed convex set ofRn
, and satisfying:T0 |
tt|dt +
T0 |
tt|
2dt < , (8) is well-defined.
Given a utility function U on (0, ), and starting from initial capital X0 > 0, the
objective of the agent is:
V0 := supA
E[U(XT)]. (9)
Here, X is the solution to (8) controlled by A0, and starting from X0 at time 0,
and A is the subset of elements A0 s.t. {U(X ), T0,T} is uniformly integrable.
We solve (9) by dynamic programming and BSDE.
Value function processes:
7/27/2019 Cours Controle Marrakech
60/77
p
For t [0, T], and A, we denote by:
At() = { A : .t = .t} ,
and define the family ofF-adapted processes
Vt() := ess supAt()
E
U(XT)Ft, 0 t T.
Dynamic programming (DP)
For any A, the process {Vt(), 0 t T} is a supermartingale
There exists an optimal control A to V0 if and only if the martingale property
holds, i.e. the process {Vt(), 0 t T} is a martingale.
In the sequel, we exploit the DP in the case of CRRA utility functions: U(x) =
xp/p, p < 1. The key observation is the property that the F-adapted process
Yt :=Vt()
U(Xt )> 0 does not depend on A, and YT = 1.
We adopt a BSDE verification approach: we are looking for (Y, Z) solution to
Yt = 1 +
Tt
f(s ,,Y s, Zs)ds
Tt
ZsdWs, (10)
for some generator f to be determined such that
For any A, the process {U(Xt )Yt, 0 t T} is a supermartingale
There exists A for which {U(Xt )Yt, 0 t T} is a martingale.
By applying Itos formula to U(Xt )Yt, the supermartingale property for all
7/27/2019 Cours Controle Marrakech
61/77
y pp y g ( t ) t, p g p p y
A, and the martingale property for some imply that f should be equal to
f(t, Yt, Zt) = p supaA
(tYt + tZt).a
1 p
2Yt|ta|
2
, (11)
with a candidate for the optimal control given by
t arg maxaA
(tYt + tZt).a
1 p2
Yt|ta|2, 0 t T. (12)
Existence and uniqueness of a solution to the BSDE (10)-(11):
Change of variables Y = ln Y, Z = Z/Y
(Y , Z) satisfy a quadratic BSDE. Then, we rely on results by Kobylanski (00)
Existence and uniqueness of (Y, Z) S(0, T) H2(0, T)d
Verification argument: let (Y, Z) be the solution to (10)-(11)
By construction U(Xt )Yt is a (local)-supermartingale + integrability conditions
on A: it is a supermartingale supAE[U(XT)] U(X0)Y0.
By BMO techniques, we show that defined in (12) lies in A U(Xt )Yt is amartingale E[U(XT)] = U(X0)Y0
We conclude that V0 := supAE[U(XT)] = U(X0)Y0, and is an optimal control.
Markov cases
7/27/2019 Cours Controle Marrakech
62/77
Merton model: the coefficients of the stock price (t) and (t) are deterministic
In this deterministic case, the BSDE (10)-(11) is reduced to an ODE:
Y(t) = 1 + T
t
f(s, Y(s))ds, f (t, y) = y p supaA (t).a
1 p
2|(t)a|2 =: y(t)
and the solution is given by: Y(t) = eTt(s)ds we find again the solution to the
Merton problem:
V0 = U(X0)exp T
0
(s)ds
.
Factor model: the coefficients of the stock price (t, Lt) and (t, Lt) depend on
7/27/2019 Cours Controle Marrakech
63/77
( ) ( )
a factor process
dLt = (Lt)dt + dWt.
In this case, the BSDE for (Y , Z) = (ln Y,Z/Y) is written as:
Yt =Tt
f(s, Ls, Ys, Zs)ds Tt
ZsdWs
with a quadratic (in z) generator
f(t,,y,z) = p supaA
((t, ) + (t, )z).a
1 p
2|(t, )a|2
+
1
2z2.
Yt
= (t, Lt), with a corresponding semilinear PDE for :
t+ ().D +
1
2tr(D2) + f(t,,,D) = 0, (T, ) = 0.
Value function:
V0 = U(X0)exp
(0, L0)
.
Remark The BSDE approach and dynamic programming is also well-suitable for
exponential utility maximization:
Many papers: El Karoui, Rouge (00), Hu, Imkeller, Muller (04), Sekine (06),
Becherer (06), etc ...
V. Reflected BSDE and optimal stopping problem
7/27/2019 Cours Controle Marrakech
64/77
We consider a class of BSDEs where the solution Y is constrained to stay above a
given process L, called obstacle. An increasing process K is introduced for pushing
the solution upwards, above the obstacle Notion of reflected BSDE:
Given pair of terminal condition/generator (, f) and a continuous obstacle process
(Lt) s.t. LT, find a triple of adapted processes (Y ,Z,K) with K nondecreasing
s.t.
Yt = +
Tt
f(s, Ys, Zs)ds
Tt
ZsdWs + KT Kt, 0 t T, (13)
Yt Lt, 0 t T, (14)T0
(Yt Lt)dKt = 0. (15)
Connection with optimal stopping problem: in the case where f(t, ) does
not depend on (y, z), there exists a unique solution to (13)-(14)-(15) given by
Yt = esssupTt,T E
tf(s)ds + L1
7/27/2019 Cours Controle Marrakech
65/77
Snell envelope of Ht :=t0
f(s)ds + Lt1t
7/27/2019 Cours Controle Marrakech
66/77
rator, and L is a continuous obstacle in S2(0, T).
Existence and approximation by penalization
For each n N, we consider the (unconstrained) BSDE
Ynt = +Tt
f(s, Yns , Zns )ds + K
nT K
nt
Tt
Zns .dWs, (17)
where Knt = nTt
(Ysn Ls)ds existence and uniqueness of (Yn, Zn).
State a priori uniform estimates on the sequence (Yn, Zn, Kn): there exists ome
positive constant C s.t.
E sup0tT
|Ynt |2 + T0
|Znt |2dt + |KnT|2 C, n N. By comparison principle for BSDE, (Yn)n is an increasing sequence, and it con-
verges to some Y S2(0, T), and the convergence also holds in H2(0, T), i.e.
limnE T
0 |Ynt Yt|
2dt
= 0. Moreover, Yt Lt.
Prove that (Zn, Kn)n is a Cauchy sequence in H2(0, T)d S2(0, T): use Itos
formula to |Ynt Ymt |
2, and inequality 2ab a2+ 1
b2 for suitable . Consequently,
(Zn, Kn)n converges to some (Z, K) in H2(0, T)d S2(0, T). Pass to the limit in
(17) in order to obtain the existence of (Y ,Z,K) solution to the reflected BSDE.
Remark: Alternative formulation of the Skorokhod condition.
7/27/2019 Cours Controle Marrakech
67/77
The definition of a solution to the reflected BSDE (13)-(14) with the Skorokhod
condition (15)T0 (YtLt)dKt = 0 can be formulated equivalently in terms of minimal
solution:
We say that (Y ,Z,K) is a minimal solution to (13)-(14) if for any other solution(Y , Z, K) solution to (13)-(14), we have Yt Yt, 0 t T.
Any solution to the reflected BSDE (13)-(14)-(15) is a minimal solution to (13)-
(14), and the converse is also true.
Connection with variational inequalities in the Markov case
7/27/2019 Cours Controle Marrakech
68/77
Consider the case where:
= g(XT), f(t ,,Y t, Zt) = f(t, Xt, Yt, Zt), Lt = h(Xt), 0 t T,
with g h, and where X is a diffusion process on Rn
dXt = b(Xt)dt + (Xt)dWt.
Then, the solution to the reflected BSDE is given by Yt = v(t, Xt) for some deter-
ministic function v, viscosity solution to the variational inequality:
min
v
t Lv f(t,x,v,Dxv) , v h
= 0, on [0, T) Rn
v(T, .) = g on Rn.
VI. BSDE with constrained jumps and quasi-variational inequalities
7/27/2019 Cours Controle Marrakech
69/77
Consider the impulse control problem:
v(t, x) = supE
g(XT) +
Tt
f(Xs )ds +
t
7/27/2019 Cours Controle Marrakech
70/77
a continuation region C in which v(t, x) > Hv(t, x) and
v
t Lv f = 0
an action region D in which:
v(t, x) = Hv(t, x) = supeE
v(t, x + (x, e)) + c(x, e).
Various applications of impulse control problems:
Financial modelling with discrete transaction dates, due e.g. to fixed transaction
costs or liquidity constraints
Optimal multiple stopping: swing options
Projects investment and real options: management of power plants, valuation
of gas storage and natural resources, forest management, ...
...
Impulse control: widespread economical and financial setting with many prac-
tical applications More generally to models with control policies that do not
accumulate in time.
Main theoretical and numerical difficulty in the QVI (18) :
7/27/2019 Cours Controle Marrakech
71/77
The obstacle term contains the solution itself
It is nonlocal
Classical approach : Decouple the QVI (18) by defining by iteration the sequence
of functions (vn)n:
min
vn+1
t Lvn+1 f , vn+1 Hvn
= 0 , vn+1(T, .) = g
associated to a sequence of optimal stopping time problems (reflected BSDEs)
Furthermore, to compute vn+1, we need to know vn on the whole domain
heavy computations, especially in high dimension (state space discretization): nu-
merically challenging!
Idea of our approach
7/27/2019 Cours Controle Marrakech
72/77
Instead of viewing the obstacle term as a reflection ofv onto Hv (or vn+1 into Hvn),
consider it as a constraint on the jumps of v(t, Xt) for some suitable forward jump
process X:
Let us introduce the uncontrolled jump diffusion X :
dXt = b(Xt)dt + (Xt)dWt +
E
(Xt, e)(dt, de),
where is a Poisson random measure whose intensity is finite and supports the
whole space E.
We randomize the state space!
Take some smooth function v(t, x) and define:
Yt := v(t, Xt), Zt := (Xt)Dxv(t, Xt),
Ut(e) := v(t, Xt + (Xt, e)) v(t, Xt) + c(Xt, e)
= (Hev v)(t, Xt)
Apply Itos formula to Yt = v(t, Xt) between t and T:
7/27/2019 Cours Controle Marrakech
73/77
Yt = YT +
Tt
f(Xs)ds + KT Kt
Tt
Zs.dWs
+
Tt
E
[Us(e) c(Xs, e)](ds, de),
where
Kt :=
t0
(v
t Lv f)(s, Xs)ds
Now, suppose that min[v
t Lv f, v Hv] 0, and v(T, .) = g :
Then (Y ,Z,U,K) satisfies
Yt = g(XT) + Tt
f(Xs)ds + KT Kt Tt
Zs.dWs
+
Tt
E
[Us(e) c(Xs, e)](ds, de), (19)
K is a nondecreasing process, and U satisfies the nonpositivity constraint :
Ut(e) 0, 0 t T, e E. (20)
View (19)-(20) as a Backward Stochastic Equation with jump constraints
We expect to retrieve the solution to the QVI (18) by solving the minimal solution
to this constrained BSE.
General definition of BSDEs with constrained jumps
7/27/2019 Cours Controle Marrakech
74/77
Minimal Solution : find a solution (Y ,Z,U,K) S2 H2(0, T)d L2() A2 to
Yt = g(XT) +
Tt
f(Xs, Ys, Zs)ds + KT Kt
Tt
Zs.dWs
T
t E(Us(e) c(Xs, Ys, Zs, e))(ds, de) (21)with
h(Ut(e), e) 0, dP dt (de) a.e. (22)
such that for any other solution (Y , Z, U , K) to (21)-(22) :
Yt Yt, 0 t T, a.s.
Assumptions on the coefficients:
Forward SDE : b and Lipschitz continuous, bounded and Lipschitz contin-
uous w.r.t. x uniformly in e:
|(x, e) (x, e)| k|x x| e E
Backward SDE : f, g and c have linear growth, f and g Lipschitz continuous,
c Lipschitz continuous w.r.t. y and z uniformly in x and e
|c(x ,y,z,e) c(x, y, z, e)| kc(|y y| + |z z|)
Constraint : h Lipschitz continuous w.r.t. u uniformly in e:
|h(u, e) h(u, e)| kh|u u|
and
u h(u, e) nonincreasing. (e.g. h(u, e) = u)
Existence and approximation via penalization
7/27/2019 Cours Controle Marrakech
75/77
Consider for each n the BSDE with jump:
Ynt = g(XT) +
Tt
f(Xs, Yns , Z
ns )ds + K
nT K
nt
Tt
Zns .dWs
TtE
[Uns (e) c(Xs, Yns, Zns , e)](ds, de) (23)
with a penalization term
Knt = n
t0
E
h(Uns (e), e)(de)ds
where h = max(h, 0).
For each n, existence and uniqueness of (Yn, Zn, Un) solution to (23) from Tang
and Li (94), and Barles et al. (97).
Convergence of the penalized solutions
Theorem .1 Under (H1), there exists a unique minimal solution
(Y ,Z,U,K) S2 H2(0, T)d L2() A2
with K predictable, to (21)-(22). Y is the increasing limit of (Yn) and also in
S2(0, T), K is the weak limit of (Kn) inS2(0, T), and for any p [1, 2),
Zn ZHp(0,T)
+ Un ULp()
0,
as n goes to infinity.
Sketch of proof.
7/27/2019 Cours Controle Marrakech
76/77
Convergence of (Yn): by comparison results (under the nondecreasing property of
h) Yn Yn+1
Convergence of (Zn, Un, Kn) : more delicate!
A priori uniform estimates on (Yn, Zn, Un, Kn)n in L2
weak convergence of (Zn, Un, Kn) in L2
Moreover, in general, we need some strong convergence to pass to the limit in
the nonlinear terms f(X, Yn, Zn), c(X, Yn, Zn) and h(Un(e), e).
Control jumps of the predictable process K via a random partition of the
interval (0,T) and obtain a convergence in measure of (Zn, Un, Kn)
Convergence of (Zn, Un, Kn) in Lp, p [1, 2)
Related semilinear QVI
7/27/2019 Cours Controle Marrakech
77/77
By Markov property, the minimal solution to the constrained BSDE with jumps is
Yt = v(t, Xt) for some deterministic function v.
The function v is the unique viscosity solution to the QVI:
min
v
t Lv f(. ,v ,Dxv), inf
eEh(Hev v, e)
= 0 on [0, T) Rn, (24)
together with the relaxed terminal condition:
min
v g , infeE
h(Hev v, e)
= 0 on {T} Rn. (25)
Probabilistic representation of semilinear QVIs, and in particular ofimpulse control problems by means of BSDEs with constrained jumps.
Numerical implications for the resolution of QVIs by means of simulation of the pe-
nalized BSDE: PhD thesis of M. Bernhart, in partnership with EDF for the valuation
of swing options and gas storage contacts.