Some aspects of Mean Field Games - Part III · PDF fileSome aspects of Mean Field Games - Part...

Some aspects of Mean Field Games - Part III

P. Cardaliaguet

Paris-Dauphine

GNAMPA School “DIFFERENTIAL EQUATIONS AND DYNAMICALSYSTEMS"

Serapo (Latina), June 11-15, 2012

P. Cardaliaguet (Paris-Dauphine) MFG 1 / 90

We investigate the long-time behavior of the solution (uT ,mT ) to the MeanField Game system

(MFG)

(i) −∂tuT −∆uT + 1

2 |DuT |2 = F (x ,mT )

(ii) ∂tmT −∆mT − div(mT DuT ) = 0

(iii) mT (0) = m0, uT (x ,T ) = uf (x)

Motivation : Hope to reduce the system to a stationary equation.


Outline

1 Case of Hamilton-Jacobi equations

2 Discussion for the MFG system

3 Convergence for local equationsConvergence under mild monotonicity conditionsThe convergence rateApplication to the optimal control of Kolmogorov equation

4 Convergence for nonlocal equationsConvergenceThe convergence rate

5 Other differential games on measure spacesA game with incomplete informationA differential game with lack of observation


Case of Hamilton-Jacobi equations

Outline








Asymptotic behavior of solutions for HJ equations

For a single HJ equation

(HJ)

(i) ∂tu −∆u + 12 |Du|2 = F (x) in Rd × (0,+∞)

(iii) u(0, x) = G(x) in Rd

long time average : limit of u(t , ·)/t as t → +∞.

long time behaviour : limit of (u(t , ·)− long time average) as t → +∞.

Some references on long time behavior of HJ equations :Lions-Papanicolau-Varadhan, Evans, Arisawa-Lions, Fathi,Namah-Roquejoffre, Barles-Souganidis, Capuzzo Dolcetta-Ishii,Fathi-Siconolfi, Alvarez-Bardi, Fujita-Ishii-Loreti...



Assume that F ,G : Rd → R are smooth and Zd−periodic.

Theorem (Arisawa-Lions, ’98)

u(t , ·)t

converges uniformly to a constant λ as t → +∞.

u(t , ·)− λt converges uniformly to a solution u of

λ−∆u +12|Du|2 = F (x) in Rd

Remark : Barles-Souganidis ’01 shows that this result holds under for moregeneral time-dependent problems of the form

∂tu − Tr(A(t , x ,Du)D2u) + H(t , x ,Du) = 0



Main step of the proof :

Construction of a corrector,

Use of the corrector to show the long time average,

Use of some dissipativity to show the long time behavior



Corrector equationThere is a unique constant λ for which the problem

λ−∆u +12|Du|2 = F (x) in Rd

has a continuous, periodic solution u. This solution is unique up to constants.

Proof of uniqueness : If (λ1, u1) and (λ2, u2) are two solutions, let x be amaximum point of u1 − u2. Then

Du1(x) = Du2(x), D2u1(x) ≤ D2u2(x) .

Soλ1 = ∆u1(x)− 1

2|Du1(x)|2 + F (x)

≤ ∆u2(x)− 12|Du2(x)|2 + F (x) = λ2



Proof of existence : For δ > 0 small, let vδ be the unique bounded solution to

δvδ −∆vδ +12|Dvδ|2 = f (x) in Rd .

Then

δvδ is uniformly bounded,

vδn − vδn (0) is bounded and, hence, C2,α uniformly in δ,

Hence ∃δn → 0 s.t.

δnvδn (0) converges to a constant λvδn − vδn (0) converges uniformly to a periodic map u.

u satisfiesλ−∆u +

12|Du|2 = F (x) in Rd



Convergence of u(t , ·)/t

Note that w(t , x) := u(x) + λ t solves : ∂tw −∆w +12|Dw |2 = F (x).

Since u is the solution to

(HJ)

(i) ∂tu −∆u + 12 |Du|2 = F (x) in Rd × (0,+∞)

(iii) u(0, x) = G(x) in Rd

by comparison, we get

u(x) + λ t − C ≤ u(t , x) ≤ u(x) + λ t + C

for some C. Dividing by t gives

limt→+∞

u(t , x)

t= λ uniformly in x .



Convergence of u(t , ·)− λ t

Recall that u := u − λt is bounded :u(x)− C ≤ u(t , x)− λ t ≤ u(x) + C

By maximum principle, the map t → maxx

(u(t , x)− u(x)) isnonincreasing and thus converge to some `.

Uniform estimates⇒ ∃tn → +∞ s.t. u(tn + ·, ·) converges to a solution wof

∂tw −∆w +12|Dw |2 = F (x) in Rd × (−∞,+∞)

But maxx (w(t , x)− u(x)) = ` for all t .

Strong maximum principle⇒ w(t , x) = u(x) + `.



Link with stochastic optimal control

Let uT be the solution to the backward equation

(HJ)

(i) −∂tuT −∆uT + 12 |DuT |2 = F (x) in Rd × (0,+∞)

(iii) uT (T , x) = GT (x) in Rd

Then uT is the value function of the optimal control problem :

uT (t , x) = infα

E

[∫ T

t

12|αs|2 + F (Xs) ds + G(XT ,m(T ))

].

where (Xs) solves

dXs = αsds +√

2dBs, Xt = x

and (Bs) is a standard B.M.



The optimal control is given by

αT ,∗(t , x) = −DuT (t , x) .

This means that

uT (t , x) = E

[∫ T

t

12∣∣−Du(t , Xs)

∣∣2 + F (Xs) ds + G(XT ,m(T ))

].

where (Xs) solves

dXs = −Du(t , Xs)ds +√

2dBs, Xt = x

PropositionThe optimal control αT ,∗ converges to the constant in time controlα(x) = −Du(x) uniformly on compact sets.


Discussion for the MFG system

Outline








We come back to the Mean Field Game system

(MFG)

(i) −∂tuT −∆uT + 1

2 |DuT |2 = F (x ,mT )


(iii) mT (0) = m0, uT (x ,T ) = uf (x)

−→ Very few results in that direction : Gomes-Mohr-Souza ’10 for discretetime, discrete space MFG.



The ergodic system

In the MFG framework, the limit system “should" be

(MFG − ergo)

(i) λ−∆u + 12 |Du|2 = F (x , m) in Rd

(ii) −∆m − div(mDu) = 0 in Rd

(introduced in Lasry-Lions 06)

Note that

m = e−u/(∫

Q1e−u

)solves (MFG-ergo)(ii)

the map(x , t)→ (u(x) + λt , m(x))

satisfies (MFG)(i-ii).



In view of the HJ case, one expects :

The convergence of uT/T to λ,

the convergence of mT to m.

the convergence of uT − T λ to u+constant



The analogy with the HJ equation is misleading :

(HJ) is a Cauchy problem

... while (MFG) system has an initial and a terminal condition.

−→Where does the convergence take place ?

(HJ) is a single equation with a comparison principle

... no comparison for the MFG system.



The Hamiltonian structure (local case)Set Φ(x ,m) =

∫ m0 F (x , ρ) dρ and

E(u,m) =

∫Q1

m12|Du|2 + 〈Du,Dm〉 − Φ(x ,m) dx

Lemma(uT ,mT ) solution of (MFG)⇔ (uT ,mT ) satisfies

(i) −∂tuT = − ∂E∂m

(uT ,mT )

(ii) ∂tmT = −∂E∂u

(uT ,mT )

(iii) mT (0) = m0, uT (x ,T ) = G(x ,mT (T ))

In particular the energy E(uT (t),mT (t)) is constant along the flow.



Main energy equality

Lemma (Lasry-Lions, 06)For any t ∈ [0,T ]

− ddt

∫Q1

(uT (t)− u)(mT (t)− m)dx =

∫Q1

(mT (t) + m)

2|DuT (t)− Du|2 + (F (x ,mT (t))− F (x , m))(mT (t)− m)

Proof : Multiply (MFG)(i)-(MFG-ergo)-(i) by (mT − m) and substract to(MFG)(ii)-(MFG-ergo)(ii) multiplied by (uT − u).



Why the convergence ?

We define the scaled functions on Rd × [0,1] :

υT (x , t) := uT (x , tT ) ; µT (x , t) := mT (x , tT )

Integrate in time the main energy equality :∫ 1

0

∫Q1

(µT + m)

2|DυT − Du|2 + (F (x , µT )− F (x , m))(µT − m) dxdt

= − 1T

[∫Q1

(υT − u)(µT − m)dx]1

0

Assume F is increasing. If the RHS→ 0 as T → +∞, then

DυT → Du,

which implies that DυT → Du



Why the convergence ? (continued)

However,

In the RHS = − 1T

[∫Q1


0the quantity υT (0) is of order λT

The main energy equality alone does not explain the behavior ofuT

T.


Convergence for local equations

Outline







Convergence for local equations

The MFG system in the local case

We work on the system

(MFG)

(i) −∂tuT −∆uT + 1

2 |DuT |2 = F (x ,mT (x , t))


(iii) mT (0) = m0, uT (x ,T ) = G(x)

where F : Rd × [0,+∞) is local and increasing in m.


Convergence for local equations Convergence under mild monotonicity conditions

Outline








Assumptions on the data

F : Rd × R→ R is locally Lipschitz continuous, Zd−periodic in x , andincreasing with respect to m.

m0 : Rd → R is smooth, Zd−periodic, m0 > 0 and∫

Q1m0 = 1.

G : Rd → R is smooth, Zd−periodic.



Convergence under mild monotony condition

Recall the definition of the scaled functions on Rd × [0,1] :

υT (t , x) := uT (tT , x) ; µT (t , x) := mT (tT , x)

Theorem (C.-Lasry-Lions-Porretta, ’12)As T → +∞,

1 υT (t , ·)/T converges to t → (1− t)λ in L2(Q1) for any t ∈ [0,1],

2 υT −∫

Q1

υT (t) converges to u in L2(Q1 × (0,1)),

3 µT converges to m in Lp(Q1 × (0,1)),for any p < N+2

N if N > 2 and for any p < 2 if N = 2.



Ideas of the proof

Without loss of generality, we assume that F ≥ 0.

From the Hamiltonian structure, there is MT such that

MT =

∫Q1

〈DuT (t),DmT (t)〉+mT (t)

2|DuT (t)|2 − Φ(x ,mT (t)) ∀t ∈ [0,T ] .

where Φ(x ,m) =∫ m

0 F (x , ρ) dρ.

We claim

Lemma 1

(i) MT is bounded with respect to T .

(ii) |DuT (0)| is bounded in L2(Q1).



Proof of Lemma 1 : Since F ≥ 0 and uT (T ) = G(x), we have

MT =

∫Q1

〈DuT (T ),DmT (T )〉+mT (T )

2|DuT (T )|2 − Φ(x ,mT (T ))

≤ −∫

Q1

mT (T )∆G +mT (T )

2|DG|2 ≤ C‖mT (T )‖L1(Q1) = C .

On the other hand we have

MT =

∫Q1

〈DuT (0),Dm0〉+m0

2|DuT (0)|2 − Φ(x ,m0)

where, since m0 > 0,

∣∣〈DuT (0),Dm0〉∣∣ ≤ m0

4|DuT (0)|2 dx + 4

|Dm0|2

m0.

HenceMT ≥ 1

4

∫Q1

m0|DuT (0)|2 dx − C .



Lemma 2

∫ T

0

∫Q1

(mT + m)

2|DuT − Du|2 + (F (x ,mT )− F (x , m))(mT − m) dxdt ≤ C

Proof : From the main energy inequality we have∫ 1

0

∫Q1

(µT + m)

2|DυT − Du|2 + (F (x , µT )− F (x , m))(µT − m) dxdt

= − 1T

[∫Q1


0



Our aim is to bound

RHS = −[∫

Q1

(uT (t)− u)(mT (t)− m)dx]T

0.

Since uT (T ) = G,∣∣∣∣∫Q1

(uT (T )− u)(mT (T )− m)dx∣∣∣∣ ≤ C(‖mT (T )‖1 + 1) ≤ C .

Note that∣∣∣∣∫

Q1

u(mT (t)− m)dx∣∣∣∣ ≤ 2‖u‖∞.

For the last term, we have by Cauchy-Schwarz and Poincaré∫Q1

uT (0)(m0 − m)dx =

∫Q1

(uT (0)− 〈uT (0)〉)(m0 − m)dx

≤ C (‖m0 − m‖2) ‖DuT (0)‖L2(Q1)

We use Lemma 1 to get |RHS| ≤ C.



Proof of the convergence theorem

We have proved that∫ T

0

∫Q1

(mT (t) + m)

2|DuT (t)−Du|2 + (F (x ,mT (t))−F (x , m))(mT (t)−m)

= RHS ≤ C .

This implies that DυT converges to Du in L2(Q1 × (0,1)).

Using extra estimates for mT , we get the convergence of µT to m andF (·, µT ) to F (·, m) in L1((0,1)×Q1).



We integrate the equation satisfied by υT on Q1 × (t ,1) :∫Q1

(υT (t)

T− G

T) +

∫ 1

t

∫Q1

|DυT |2

2=

∫ 1

t

∫Q1

F (x , µT (s))

Since DυT → Du in L2 and F (·, µT (·))→ F (·, m) in L1((0,1×Q1),

limT→+∞

1T

∫Q1

υT (x , t)dx = (1− t)∫

Q1

[−1

2|Du|2 + F (x , m)

]dx = (1− t)λ .

Since ‖DµT‖2 ≤ C, we get that υT (t , ·)/T converges to t → (1− t)λ inL2(Q1).


Convergence for local equations The convergence rate

Outline








Assumptions on the data (strong monotony)

F : Rd × R→ R is locally Lipschitz continuous, Zd−periodic in x , andthere is γ > 0 with

F (x , s)− F (x , t) ≥ γ(s − t) ∀s ≥ t , ∀x ∈ Rd .

m0 : Rd → R is smooth, Zd−periodic, m0 > 0 and∫

Q1m0 = 1.

G : Rd → R is smooth, Zd−periodic.



The convergence result (strong monotony)

Set uT (t , x) = uT (t , x)−∫

Q1

uT (t , y)dy .

Theorem (C.-Lasry-Lions-Porretta, ’12)There is κ > 0 such that

1 ‖uT (t)− u‖L1(Q1) ≤C

T − t

(e−κ(T−t) + e−κt

)2 ‖mT (t)− m‖L1(Q1) ≤

Ct

(e−κ(T−t) + e−κt

)3

∥∥∥∥uT (t)T− λ

(1− t

T

) ∥∥∥∥L1(Q1)

≤ CT



Remarks

We have exponential convergence for uT (t)− 〈uT (t)〉 and mT .

The convergence estimate foruT

Tis of order

CT

.



Main ingredient of proof

LemmaThere exists σ > 0 such that∫ (1−δ)T

δT

∫Q1

|DuT (x , t)− Du(x)|2 + |mT (x , t)− m(x)|2 dxdt ≤ Ce−σ δT

for every δ > 0.



Proof of the LemmaProof. Recall uT (x , t) = uT (x , t)−

∫Q1

uT (t). Set

ϕ(t) =

∫Q1

(uT (t)− u(t))(mT (t)− m(t)) dx

From the main energy equality :ddtϕ(t) =

−∫

Q1

mT + m

2|D(uT − u)|2 +

(F (x ,mT )− F (x , m)

)(mT − m)

dx

where, since m > 0 and by Poincaré–Wirtinger inequality :∫Q1

mT + m2

|D(uT − u)|2 ≥ c0

∫Q1

(uT − u)2 dx

and by our assumption on F :(F (x ,mT )− F (x , mT )

)(mT − m) ≥ γ(mT − m)2



Proof of the Lemma

Soddtϕ(t) ≤ −σ

∫Q1

(uT − u)2 + (mT − m)2 dx

≤ −2σ∣∣∣∣∫

Q1

(uT − u)(mT − m)

∣∣∣∣ = −2σ |ϕ(t)|

On concludes by using the Hamiltonian structure of the problem which gives

ϕ(0) ≤ C and ϕ(T ) ≥ −C .


Convergence for local equations Application to the optimal control of Kolmogorov equation

Outline








The optimal control of Kolmogorov equation

We consider the optimal control problem :

(Kopt ) infα

∫ T

t0

∫Q1

Ψ(x ,m) +12

m|α|2 dxdt +

∫Q1

G(x)m(T , x)dx

where α : [t0,T ]× Rd → Rd is a distributed control and m is the solution to(i) ∂tm −∆m − div(mα) = 0 in [t0,T ]× Rd

(ii) m(t0, x) = m0(x) in Rd

We assume that the data are periodic in space and DmΨ = F (i.e., Ψ∗ = Φ).



Recall that α(t , x) = −Du(t , x) is the optimal control for the problem (Kopt ).

CorollaryThe optimal control α(t , x) = −Du(t , x) converge exponentially fast to theconstant in time control α(x) := −Du(x).

Indeed, we have seen that there exists σ > 0 such that∫ (1−δ)T

δT

∫Q1

|DuT (x , t)− Du(x)|2 ≤ Ce−σ δT

for every δ > 0.


Convergence for nonlocal equations

Outline








The MFG system

We work on the system

(MFG)

(i) −∂tuT −∆uT + 1

2 |DuT |2 = F (x ,mT )


(iii) mT (0) = m0, uT (x ,T ) = G(x)

where F is nonlocal, non-decreasing and smoothing.

Namely, F : Rd ×M → R, where M is the set of Borel probabilitymeasures on Td .



The ergodic system

The limit system “should" be

(MFG − ergo)

(i) λ−∆u + 12 |Du|2 = F (x , m) in Rd

(ii) −∆m − div(mDu) = 0 in Rd

where

m = e−u/(∫

Q1e−u

)solves (MFG-ergo)(ii)

the map(x , t)→ (u(x) + λt , m(x))

satisfies (MFG)(i-ii).


Convergence for nonlocal equations Convergence

Outline








Assumptions

1 (Regularity) F (·,m) is bounded in C2loc unif. in m,

2 (Monotonicity)∫Q1

(F (x ,m)− F (x ,m′))d(m −m′)(x) ≥ 0 ∀m,m′ ∈ M

with an equality iff F (x ,m) = F (x ,m′),

3 (Periodicity) F (·,m) and G(·) are Zd−periodic and G is smooth.

4 (initial data) m0 : Rd → R is smooth, Zd−periodic, with m0 > 0 and∫Q1

m0 = 1.



Example

LetF (x ,m) = f (x , (ρ ∗m) ∗ ρ(x))

where ρ : Rd → R is smooth and Zd−periodic and f : R× R→ R is smoothand increasing w.r.t. the last variable.

Then F satisfies the above conditions.



Well-posedness of (MFG) and (MFG-ergo)

Under the above assumptions :

Theorem (Lasry-Lions, 2006)

(MFG) has a unique solution (uT ,mT ).

Moreover (uT ,mT ) is smooth, Zd−periodic in space, with mT > 0.

Symmetrically, (MFG-ergo) has a unique solution (λ, u, m)(up to a constant for u).

Moreover (u, m) is smooth, Zd−periodic, with m = e−u/(∫

Q1e−u) > 0.



The convergence result

Recall the definition of the scaled functions on Rd × [0,1] :


Theorem (C.-Lasry-Lions-Porretta, in preparation)As T → +∞,

υT

Tconverges uniformly to (1− t)λ in Rd × [0,1].

µT converges to m in Lp((0,1)×Q1), for any p ≥ 1.



Ingredients of proof

LemmaThe map uT is uniformly semi-concave in space.

In particular, uT is uniformly Lipschitz continuous in space.

Proof : Comes from the regularity of F (·,mT ).



Recall the main energy inequality :∫ T

0

∫Q1

(mT + m)

2|DuT − Du|2 + (F (x ,mT )− F (x , m))(mT − m) dxdt

= − 1T

[∫Q1

(uT − u)(mT − m)dx]T

0

We can now bound the RHS.



LemmaFor any p ≥ 1 there is a constant Cp such that∫ T

0

∫Q1

(mT )p ≤ CpT ∀T ≥ 1 .

Idea of proof : Multiply (MFG)(ii) by (mT )p and integrate :∫ T

0

∫Q1

p(mT )p−1|DmT |2

= −[∫

Q1

(mT )p+1

p + 1

]T

0−∫ T

0

∫Q1

p(mT )p〈DmT ,DuT 〉

≤ C + C∫ T

0

∫Q1

(mT )p|DmT |



Proof of the Convergence TheoremRecall the notations :


Step 1 : Convergence of µT

Lp bounds on mT ⇒ weak convergence of µT to some µ.(up to subsequences)

Since DυT → Du in L2 and µT solves

∂tµT

T−∆µT − div(µT DυT ) = 0 ,

and µ satisfies

−∆µ− div (µDu) = 0 in Rd × (0,1) .

Uniqueness of the solution of MFG-ergo(ii)⇒ υ = m.



Proof of the Convergence Theorem

Step 2 : convergence of υT (·, t)/T

Fix t and integrate (MFG)(i) over Q1 × [t ,1] :

1T

(∫Q1

υT (t)dx −∫

Q1

Gdx)

+12

∫ 1

t

∫Q1

|DυT |2dxds =

∫ 1

t

∫Q1

Fdxds

where∫ 1

t

∫Q1

F (x , µT )− |DυT |2

2dxds →

∫ 1

t

∫Q1

F (x , m)− |Du|2

2dxds = (1− t)λ

So1T

∫Q1

υT (t)dx → (1− t)λ

Conclusion by Lipschitz estimates on υT (·, t).


Convergence for nonlocal equations The convergence rate

Outline








Preliminary remarks

In the local case, convergence arises because F = F (x ,m) is strictlyincreasing in m : this implies

ddtϕ(t) ≤ −2σ |ϕ(t)| where ϕ(t) =

∫Q1

(uT (t)− u(t))(mT (t)− m(t))

Nothing of the sort can be expected in the nonlocal case : for instance, ifF (x ,m) = (m ? ρ) ? ρ, then inequality∫

Q1

(F (x ,m)− F (x ,m′))d(m −m′)(

=

∫Q1

((m −m′) ? ρ)2)≥ γ‖m −m′‖2

2

for any m,m′ cannot hold.

−→ convergence must rely on the smoothing properties of the equations.



We assume that∫Q1

(F (m)− F (m′))(m −m′)dx ≥∫

Q1

(F (m)− F (m′))2dx ∀m,m′ .

Example : F (x ,m) = f (x , (m ? ρ) ? ρ) where f is smooth and where∂f∂s≥ γ > 0. In this case, the Hamiltonian is

E(u,m) =

∫Q1

m12|Du|2 + 〈Du,Dm〉 − Φ(x ,m) dx

where

Φ(x ,m) = F (x , ρ ?m(x)) (with∂F∂s

(x , s) = f (x , s))



The convergence result

Set uT (t , x) = uT (t , x)−∫

Q1

uT (t , y)dy .

Theorem (C.-Lasry-Lions-Porretta, in preparation)There is κ > 0 such that

1 ‖uT (t)− u‖∞ ≤ C(

e−κ(T−t) + e−κt)

2 ‖mT (t)− m‖L1(Q1) ≤ C(

e−κ(T−t) + e−κt)

3

∥∥∥∥uT (t)T− λ

(1− t

T

) ∥∥∥∥∞≤ C

T



Ideas of proof

Main difficulty : Find a quantity which “sees" both equations.

For simplicity, we present the proof for the linearized system.

For this, we assume that F is linear w.r.t. m.



The linearized system of MFG at the solution (u + λ(T − t), m) is given by(i) −∂tvT −∆vT + 〈Du,DvT 〉 = F (x , µT )(ii) ∂tµ

T −∆µT − div(µT Du)− div(mDvT ) = 0(iii) µT (0) = m0 − m, vT (T ) = uf − u

Setting µ0 = m0 − m, vf = uf − u,

Ωµ = −∆µ− div(µDu) , Ω∗v = −∆v + 〈Du,Dv〉 ,

Av = −div(mDv) , B = F (x , µT (t)) ,

the system can be rewritten as(i) vT = Ω∗vT − B(ii) µT = −ΩµT − AvT

(iii) µT (0) = µ0, vT (T ) = vf



Lemma 1

We have A = mΩ∗ = Ωm. In particular, AΩ∗ = ΩA.

Proof :

For any smooth map v

Av = −div(mDv) = −m∆v − 〈Dm,Dv〉

Since m = e−u/(∫

Q1e−u), we have Dm = −mDu.

So A = mΩ∗.

As A is symmetric, this implies that A = (mΩ∗)∗ = Ωm.

So AΩ∗ = ΩmΩ∗ = ΩA.



Lemma 2

µT = Ω2µT + AB

Proof : Recall (i) vT = Ω∗vT − B(ii) µT = −ΩµT − AvT

(iii) µT (0) = µ0, vT (T ) = vf

HenceµT = −ΩµT − AvT = −ΩµT − A

[Ω∗vT − B

]= −ΩµT − ΩAvT + AB= Ω2µT + AB



Lemma 3There is a constant ω > 0 such that, if we set wT = A−

12µT , then

d2

dt2 ‖wT (t)‖2

2 ≥ ω2‖wT (t)‖22

Proof : We have

d2

dt212‖wT (t)‖2

2 = ‖wT (t)‖22 + 〈wT (t),wT (t)〉 ≥ 〈A−1µT (t), µT (t)〉

≥ 〈A−1(Ω2µT (t) + AB), µT (t)〉 (Lemma 2)≥ 〈A−1Ω2µT (t), µT (t)〉+ 〈BµT (t), µT (t)〉

where〈A−1Ω2µT (t), µT (t)〉 ≥ 1

2ω2‖wT (t)‖2

2

while〈B, µT (t)〉 =

∫Q

F (x , µT (t))µT (t) ≥ 0



One can show that

‖wT (0)‖2 + ‖wT (T )‖2 ≤ C where C = C(‖µ0‖2, ‖vF‖2)

Sinced2

dt2 ‖wT (t)‖2

2 ≥ ω2‖wT (t)‖22

we get‖wT (t)‖2

2 ≤ C(

eωt + eω(T−t))

... This yields the convergence result.



Conclusion and open problems

We have explained that, as T → +∞

uT (·, t)T

converges to λ(T − t),

mT converges to m.

uT −∫

Q1uT converges to u.

We have found an exponential rate for this convergence.

However

the behavior of uT − λ(T − t) is not understood,

the approach has to be extended to more general MFG...


Other differential games on measure spaces

Outline







Other differential games on measure spaces A game with incomplete information

Outline








A continuous-time game with finite horizon

We now consider a two-player zero-sum game on [0,T ], in which

Player 1 plays with u = u(t) to minimize

J(t0,u, v) :=

∫ T

t0`(t , x ,u(t), v(t)) dt

where t0 ∈ [0,T ] and ` : [0,T ]× Rd × U × V → R,

Player 2 plays with u = v(t) to maximize J(t0,u, v),

Player 1 knows x but not Player 2. Player 2 only knows the law of x ,denoted by m0 ∈ P2,

Players observe each other.



The value functions of the game

V+(t0,m0) = infα∈AX (t0)

supβ∈B(t0)

∫RN

∫ T

t0E[`(t , αx (t), β(t))] dt dm0(x),

and

V−(t0,m0) = supβ∈B(t0)

infα∈AX (t0)

∫RN

∫ T

t0E[`(t , αx (t), β(t))] dt dm0(x).

Remarks :

Continuous-time and space version of Aumann-Maschler, 95.

Player 2 learns about the unknown parameter x along the time−→ no (standard) dynamic programming.



If Player 1 did not know about x , the value functions would be

V+(t0,m0) = infα∈A(t0)

supβ∈B(t0)

∫RN

∫ T

t0E[`(t , α(t), β(t))] dt dm0(x),

and


infα∈AX (t0)

∫RN

∫ T

t0E[`(t , α(t), β(t))] dt dm0(x).

and it is clear that

V+(t0,m0) = V−(t0,m0) =

∫ T

t0supv∈V

infu∈U

∫RN`(t , x ,u, v) dm0(x)

=∫ T

t0H(t ,m0) dt

Hence V+ = V− should solve the ODE

−∂tw −H(t ,m) = 0



LemmaThe value functions and V− and V+ are convex with respect to m.

Proof : For V−, this is obvious because


infα∈AX (t0)

∫RN

∫ T

t0E[`(t , αx (t), β(t))] dt dm0(x)

= supβ∈B(t0)

∫RN

infα∈A(t0)

[∫ T

t0E[`(t , α(t), β(t))] dt

]dm0(x).

For V+, Aumann-Maschler splitting method.



We work on the set of Borel probability measures

P2 := µ/∫RN|x |2dµ(x) <∞

endowed with the Wasserstein distance :

d2(µ, ν) = minπ∈Π(µ,ν)

∫R2N|x − y |2dπ(x , y)

where Π(µ, ν) is the set of transports with marginals µ and ν.

We consider the “Hamiltonian"

H(t ,m) = supv∈V

infu∈U

∫RN`(t , x ,u, v) dm(x)

We assume that Isaacs condition holds :

H(t ,m) = infu∈U

supv∈V

∫RN`(t , x ,u, v) dm(x)



Theorem (C.-Rainer, 2012)For all (t ,m) :

V+(t ,m) = V−(t ,m)

Moreover V := V+ = V− is the unique viscosity solution of

(HJ)

max

−∂tw −H(t ,m) , −λmin

(∂2w∂m2

)= 0 in (0,T )× P2

w(T ,m) = 0 in P2



One also proves that

(Representation formula)

V(t0,m0) = minM∈M(t0,m0)

E

[∫ T

t0H(s,Ms)ds

],

whereM(t0,m0) is the set of martingale measures starting at 0.

(Optimal strategy) Let u∗(t ,m) = argminu∈U maxv∈V

∫RN`(t , x ,u, v) dm(x)

and M be optimal in the above problem. Then Players’1 optimalstrategy consists in playing the random control u∗(t , Mt ) at time t .

(Characterization of the optimal martingale) M is optimal “iff"

−∂tV(t , Mt )−H(t , Mt ) = 0 a.s., ∀t ∈ [0,T ] .


Other differential games on measure spaces A differential game with lack of observation

Outline








Deterministic differential game with finite horizon

We now consider a deterministic differential gamedXt = f (Xt ,ut , vt )dtxt0 = x0

The trajectory associated to (u, v) is denoted by X t0,x0,u,v· .

Main assumption on the game : Player II does not observe anything.



Rules of the game

At time t0, the initial state x0 is drawn at random according to aprobability µ0 on RN .

Player I is informed on the initial state x0, Player II just knows µ0.

Player I observes x(t) and v(t). He minimizes g(X t0,x0,u,vT ).

Player II observes nothing but has perfect recall about his own control v .He maximizes g(X t0,x0,u,v

T ).



The value functions

The lower value function is :

V−(t0, µ0) = supv∈Vr (t0)

inf(αx )∈(Ar (t0))RN

∫RN

E[g(X t0,x,αx ,v

T )]

dµ0(x)

The upper value function is :

V+(t0, µ0) = inf(αx )∈(Ar (t0))RN

supv∈Vr (t0)

∫RN

E[g(X t0,x,αx ,v

T )]

dµ0(x)



We work on the set of Borel probability measures

P2 := µ/∫RN|x |2dµ(x) <∞

endowed with the Wasserstein distance :

d2(µ, ν) = minπ∈Π(µ,ν)

∫R2N|x − y |2dπ(x , y)

We consider the Hamiltonian

H(µ,p) = supv∈∆(V )

∫RN

infu∈∆(U)

∫U×V〈f (x ,u, v),p(x)〉du(u)dv(v)dµ(x)

(for p ∈ L2µ(RN ,RN), µ ∈ P2)



Existence of the value

Theorem (C.-Souquière, 2012)For all (t , µ) :

V+(t , µ) = V−(t , µ)

Moreover V+ = V− is the unique viscosity solution of

(HJ)

−∂tw −H(µ,Dµw) = 0 in (0,T )× P2

w(T , µ) =

∫RN

g(x)dµ(x) in P2



Solution of the HJ Equation

Definition (Subsolution of the HJ Equation)V : [t0,T ]× P2 → R, Lipschitz continuous, is a subsolution to (HJ) if, for anytest function φ(t , µ) of the form

φ(t , µ) =α

2d2(µ, µ) + ηd(ν, µ) + ψ(t)

(where ψ ∈ C1(R,R), α, η > 0, ν, µ ∈ P2) such thatV− φ has a local maximum at (ν, t), one has :

−ψ′(t)−H(ν,−αpy ) ≤ ‖f‖∞η

where, for a fixed π ∈ Πopt(µ, ν), py ∈ L2ν(RN ,RN) is defined by :∫

RN 〈ξ(y), x − y〉d π(x , y) =∫RN 〈ξ(y),py (y)〉d ν(y) ∀ξ ∈ L2

ν




Definition (Supersolution of the HJ Equation)V : [t0,T ]× P2 → R, Lipschitz continuous, is a supersolution to (HJ) if, for anytest function φ(t , µ) of the form

φ(t , µ) = −α2

d2(µ, µ)− ηd(ν, µ) + ψ(t)

(where ψ ∈ C1(R,R), α, η > 0 and µ, ν ∈ P2) such thatV− φ has a local minimum at (ν, t) ∈ (0,T )× P2, one has :

−ψ′(t)−H(ν, αpy ) ≥ −‖f‖∞η .

A solution of (HJ) is a subsolution and a supersolution.




Lemma (Comparison principle)Let w1 be some subsolution of (HJ) and w2 some supersolution such thatw2(T , µ) ≥ w1(T , µ).Then for all (t , µ) ∈ [t0,T ]× µ ∈ P2 :

w2(t , µ) ≥ w1(t , µ)

The definition comes from Cardaliaguet-Quincampoix (2007) (cf. alsoGangbo-Nguyen-Adrian (2008), Feng-Katsoulakis (2009), Lasry-Lions).

The proof of the comparison principle is an adaptation of Crandall, Lions(1986).



Idea of proof for the theorem :

First step (Dynamic programming principle)For 0 ≤ t0 ≤ t1 ≤ T ,

V+(t0, µ0) = inf(αx )∈(Ar (t0))RN

supv∈Vr (t0)

V+(t1, µt1 ) .

where µt1 is is the information of player II on the state of the system,knowing the strategy of his opponent :∫

RNϕ(x)dµt1 (x) =

∫RN

E[ϕ(X t0,x,α)x ,v

t1 )]

dµ0(x)

for any ϕ ∈ Cb(RN ,R).

P.D.E. characterization of V+.

Sion’s min-max Theorem for the equality V+ = V−.



Conclusion

Differential games with imperfect information :- well understood for simple information structure.- a lot remains to be done in more general settings.

Differential games with lack of observation : almost completely open.

Nonzero sum differential games with lack of information : open.



Some references on differential games

Existence and characterization of the value

Fleming W. H. (1967)Evans L.C. and Souganidis P.E. (1984)Fleming W.H. and Souganidis P.E. (1989)Bardi M. and Capuzzo Dolcetta (1996)

Other approaches for information issues

Baras J. and James M. (1996)Bernhard, P. Systems Control Lett., 24 (1995)Chernousko F. and Mellikyan A. (1975)Quincampoix M. and Veliov V. (2005)Rapaport A. and Bernhard P., (1995)



Some references on differential games withinformation issues

C., (2006) SIAM J. Control Optim. 46, no. 3, 816–838.

C., (2008) JOTA, 138, no. 1, 1–16. .

C., Annals of ISDG, 2009.

C., J. Math.Anal. Appl. 360 (2009), no. 1, 95-107.

C. and Rainer, Appl. Math. Optim. (2009) 59 : 1-36.

C. and Rainer, Math. Oper. Res. 34 (2009), no. 4, 769-794.

C. and Souquière, To appear in SIAM Contr. Opti.

Grün, pre-print 2011.

Souquière, IJGT, 2010.


Some aspects of Mean Field Games - Part III · PDF fileSome aspects of Mean Field Games - Part...

Documents

Transcript of Some aspects of Mean Field Games - Part III · PDF fileSome aspects of Mean Field Games - Part...