Games with exhaustible resources

Sam Howison

OCIAM/MCFG, Mathematical Institute and

Oxford-Man Institute for Quantitative Finance

Oxford University

Joint with Chris Harris (Cambridge), Ronnie Sircar (Princeton)

& Jeff Dewynne (Oxford)

Oxford, February 2010

1

1: Monopoly games

2

One player static game with costs

• Monopoly player has infinite reserve of resource

• Cost CQ to extract Q units

• Sells into Cournot market: quantities produced determineprices

• Price function P (Q) = 1−Q

• Wants to maximise the cash flow

Q(1−Q)− CQ

3

Solution is Q∗ = 12(1−C) provided 0 ≤ C < 1; otherwise, Q = 0.

12(1− C)

14(1− C)2

Q

Profit

Quantity Q

Price

1−Q Q(1−Q)

4

Continuous time version

• Produce at rate q dt, cost c to produce.

• Cash flow (q(1− q)− cq) dt

• Maximise lifetime discounted profit (value function)

v =∫ ∞0

e−rt(q(1− q)− cq

)dt

• As above q∗ = max(12(1− c),0) and

rv =(1− c)2

4= q∗2.

5

Exhaustibility: when it’s gone it’s gone

Now suppose the resource is free to extract but player startswith a finite amount x0 at time t = 0.

• Time-t resource xt; runs out at T so xT = 0.

• Extraction strategy q(xt): (depends only on xt) and so

dxt

dt= −q(xt).

• Choose q to maximise

v(x0) =∫ T

0e−rsq(xs)

(1− q(xs)

)ds.

6

Take any strategy q and write

v(x0) =∫ t

0e−rsq(xs)

(1− q(xs)

)ds + e−rtv(xt).

Now d /dt:

0 = e−rtq(xt)(1− q(xt)

)+ e−rt

(v′(xt)

dxt

dt− rv(xt)

).

Usedxt

dt= −q(xt)

and rearrange to get an ODE for v(x) given q(x):

rv = q(1− q)− qv′.

Then ....

7

... maximise over q a la Bellman/Pontryagin: for each x,

rv = maxq

[q(1− q)− qv′

]

so the optimal strategy is

q∗(xt) = 12

(1− v′(xt)

)

and then v(x) satisfies the HJB equation (Hotelling equation)

rv =1

4(1− v′)2, v(0) = 0.

Compare the static game with extraction cost, Q∗ = 12(1 − C).

Here v′ is a shadow cost of depleting the reserve. Note also

rv = q∗2.

8

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.05

0.1

0.15

0.2

0.25

x

v

The solution starts at zero and tends to 1/4r at infinity (here

r = 1).

The strategy at infinity is to produce 12 (to maximise q(1− q)),

decreasing as you run out; at exhaustion x = 0, q = 0.

9

The replacement technology

• Suppose there is a way to produce the resource at cost c asan alternative to extracting it for free.

• Clearly need 0 ≤ c < 1.

• Player can now produce when resource is exhausted.

• No production in interior x > 0 (same ODE).

• Value at x = 0 now (1 − c)2/4r: boundary condition forHotelling equation.

10

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.05

0.1

0.15

0.2

0.25

x

v

The new possibility gives greater interior value.

2: Zero- and one-dimensional duopolyproblems

11

Nash equilibria

Competitive equilibrium:

• Two players each seeking to maximise their own reward func-tion. Strategies q1 and q2.

• Each assumes the other acts optimally.

• Nash equilibrium strategy choices q∗1 and q∗2.

• Player 1 maximises f(q1, q∗2) over q1, player 2 maximisesg(q∗1, q2) over q2.

12

q1

q2

Optimal q2 given q1

Optimal q1 given q2

Hope for a unique Nash equilibrium ...

13

q1

q2

Optimal q2 given q1

Optimal q1 given q2

... and can find by iteration.

14

Two players, infinite resource

• In continuous time, no extraction costs:

• Player 1 produces at rate q1, player 2 at q2.

• Each player maximises their own value function:

q∗1 : maximises∫ ∞0

e−rtq1(1− q1 − q∗2) dt over q1

q∗2 : maximises∫ ∞0

e−rtq2(1− q∗1 − q2) dt over q2

• 1− 2q∗1 − q∗2 = 0, 1− q∗1 − 2q∗2 = 0, so q∗1 = q∗2 = 1/3.

(Competition increases production and cuts costs!)

15

Blockading

• One player has infinite resources.

• The other has run out but can use the alternative technologyat cost c.

• Player 1 maximises∫ ∞0

e−rtq1(1− q1 − q∗2) dt over q1.

• Player 2 maximises∫ ∞0

e−rtq2(1− q∗1 − q2)− cq2 dt over q2.

• Note that 0 ≤ q1,2 < 1.

16

q1

1

q2

12(1− c)

12

1− c 1

Player 2 choice

Player 1 choice

Case 1: 0 ≤ c < 12

Unique equilibrium, both produce. Value functions are

v = (1 + c)2/9r, w = (1− 2c)2/9r.

17

q1

1

q2

1

Player 2 choice

Player 1 choice

Case 2: 12≤ c < 1

12

12(1− c)

1− c

Now player 2’s best choice is not to play at all: q∗2 = 0. Player 2is blockaded, player 1 has a monopoly and value functions are

v = 1/4r, w = 0.

18

Two players + exhaustibility for player 1

• Player 1 has finite resource x, value function v(x). Strategy

q1(x) and dxt/dt = −q1(xt).

• Player 2 has no resource but can produce at cost c (as can

player 1). Value function w(x), strategy q2(x).

• Each player maximises their own value function:

q∗1 : maximises∫ ∞0

e−rtq1(1− q1 − q∗2) dt over q1

q∗2 : maximises∫ ∞0

e−rtq2(1− q∗1 − q2)− cq2 dt over q2

19

• Write down ODEs:

rv = q1(1− q1 − q∗2)− v′q1,

rw = q2(1− q∗1 − q2)− w′q∗1 − cq2

• Nash maximisation: solve

1− 2q∗1 − q∗2 = v′, 1− q∗1 − 2q∗2 = c

for q∗1(v′, c), q∗2(v′, c).

• Hotelling equations:

rv = q∗12, rw = q∗2

2 − q∗1w′︸ ︷︷ ︸competition

with initial condition at x = 0.

20

Case 1: 0 ≤ c < 12.

Both compete and

q∗1 =1

3(1− 2v′ + c),

q∗2 =1

3(1 + v′ − 2c)

1

12(1− c)

11− c

Player 1Player 21− v′

12(1− v′)

q2

q1

21

0 1 2 3 4 50

0.05

0.1

0.15

0.2

x

value functions

22

Case 2: 12 ≤ c < 1.

Player 2 is blockaded for large x,

but waits until it is worth play-

ing, at x = xb. For x < xb,

q∗1 =1

3(1− 2v′ + c),

q∗2 =1

3(1 + v′ − 2c).

For x > xb,

q∗1 =1

2(1− v′) (monopoly),

q∗2 = 0.

Continuous value functions at

xb.

1

1

Player 1Player 2

q2

q1

12(1− c)

1− v′

1− c 12(1− v′)

23

0 0.5 1 1.5 2 2.5 3 3.5 40

0.05

0.1

0.15

0.2

0.25

x

Value functions

xb

24

3: Full duopoly games

25

• Two players:

– Resources x and y

– Strategies q∗1, q∗2

– Value functions v(x, y), w(x, y).

• Game trajectory

dx

dt= −q∗1(x, y),

dy

dt= −q∗2(x, y)

• Game changes when one player runs out (can use alternative

technology).

x

(x, y)y

26

• Given any strategies, d /dt of

v(x0, y0) =∫ t

0e−rsq1(xs, ys)

(1−q1(xs, ys)−q2(xs, ys)

)+e−rtv(xt, yt)

gives PDEs for v, w:

rv = q1(1− q1 − q2)− q1vx − q2vy (1)

rw = q2(1− q1 − q2)− q1wx − q2vy. (2)

• Nash-optimise (1) over q1, (2) over q2:

q∗1 =1

3(1− 2vx + wy) q∗2 =

1

3(1− 2wy + vx)

• PDEs

rv = q∗12 − q∗2vy, rw = q∗2

2 − q∗1wx.

27

The duopoly PDEs

rv =1

9(1− 2vx + wy)

2 − 1

3(1− 2wy + vx)vy,

rw =1

9(1− 2wy + vx)

2 − 1

3(1− 2vx + wy)wx.

To be solved with Dirichlet boundary conditions

v(x,0), w(x,0), v(0, y), w(0, y)

from the 1-D games above.

Note: there is an exact quadratic solution (no economic use!).

28

Neumann BCs

If

• The q∗i are continuous at the

boundary

• There is no blockading

we can find Neumann BCs on

the axes.

x

y

vy = 0wy = c

vx = 0wx = c

29

4: Numerics

Global solve of discretised PDEs (optimisation or FEM).

Left: wiggles in vx, vy for exact quadratic solution, 20× 20 grid.Right: 50× 50 grid, full BCs.

30

Marching out from axes (exploit structure).

0

2

4

6

0123450

0.1

0.2

0.3

0.4

xy

v

Oscillations get worse as grid is refined.

31

‘Cobweb’ iteration of games

• Assume a strategy for player 2

• Find player 1’s optimal response (1st order PDE, Charpit)

• Find player 1 strategy and repeat for player 2.

• Continue until converged (??).

32

0 1 2 3 4 5

0

1

2

3

4

50

0.1

0.2

V

Left: Rays for v after 4 iterations (refined grid near origin).

Right: Surface of v(x, y).

33

Write equations as dynamical system

∂

∂τ

(vw

)=

(equation 1equation 2

)

and drive towards equilibrium from suitable starting guess.

34

Usual result is NaN ...

35

• Write equations as dynamical system

• Smoothing to suppress grid-scale oscillations

• Non-uniform grid (ξ, η) to focus near axes

• Neumann data (not Dirichlet).

∂

∂τ

(vw

)= h

(equation 1equation 2

)+ Dh2

(∆ξηv

∆ξηw.

)

With care it works . . .

36

. . . but only for 0 < c < 12.

37

102

103

10−3

10−2

10−1

Number of grid points

Err

or

c=0.05

c=0.25

c=0.45

Convergence as N−1/3 (log-log plot error vs N).

38

5: Properties of the equations

39

Equation type?

System is fully nonlinear. Differentiate to get a quasilinear second-

order system. Then ask

• Assuming that the first derivatives are continuous

• Can there be jumps in the second derivatives across a real

curve in the (x, y) plane?

If so, call the system hyperbolic; if not, call it elliptic.

40

Criterion depends on the solution itself. If the quadratic

q1(4q1 + vy)λ2 + λ(vywx − 7q1q2) + q2(4q2 + wx) = 0.

where

q1 =1

3(1− 2vx + wy), q2 =

1

3(1− 2wy + vx).

has real roots, then hyperbolic.

Case 1: 0 ≤ c ≤ 12.

• elliptic on the axes and at infinity

• (numerically found to be) elliptic everywhere else.

• Explicitly shown elliptic for the small-c linearisation.

41

Case 2: 12 < c < 1 (blockading on axes).

• Elliptic on axes near origin, and at (∞,∞) (static solution).

• Hyperbolic on blockaded parts of x axis x > xb, and y axis,

y > yb.

• Also hyperbolic on small segments of x axis, x0 < x < xb and

v.v. on y axis.

• Hyperbolic near x axis as x →∞, and v.v. y axis.

42

Probably this is the picture:

Hyperbolic

Elliptic

xbx

yb

y0

y

x0

but what does it mean?43

6: Final remarks

44

Open issues

• How to interpret ‘elliptic/hyperbolic’???

• Local expansions near (eg) origin, blockade point.

• Blockading in interior?

• Reliable numerics.

45

Future directions

• Other price/supply functions (eg isoelastic P (q) = 1/q).

• Stochasticity in reserves or price function.

• Different cost structures.

• Large number of players: efficient limit.

• Control theory approach.

• Other market models eg Bertrand, Kreps-Scheinkman.

46