Week 7: Games in Extensive Form - University of Warwick · Battle of the SexesStackelbergEssential...

Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection

Week 7:Games in Extensive Form

Dr Daniel Sgroi

Reading: 1. Osborne chapter 5 (and also 6, 7);2. Snyder & Nicholson, pp. 255–260.

With thanks to Peter J. Hammond.

EC202, University of Warwick, Term 2 1 of 33


Sequential BoS, I

b�� XXXXX

1B Sr

��

@@@

2B Sr

2, 1r

0, 0

r�

��

@@@

2B Sr

0, 0r

1, 2

Consider the BoS game, with the key differencethat player 1 who prefers (B,B) to (S ,S) moves first,and player 2 observes player 1’s move before replying.This difference affects the normal or strategic form,because player 2 now has four instead of two strategies, as follows:

1. B always (which we denote by b);

2. S always (which we denote by s);

3. B if B and S if S (which we denote by m for match);

4. S if B and B if S (which we denote by x for cross).



Sequential BoS, II

Here is the normal form of sequential BoS,with (multiple) best responses starred as usual.

P2

b s m x

B 2∗ 0 2∗ 0∗1∗ 0 1∗ 0

P1 S 0 1∗ 1 0∗0 2∗ 2∗ 0

There are three pure strategy Nash equilibria,and many mixed strategy equilibria(for example, B with any mixture of b and m).



Sequential BoS, III

P2

b s m x

B 2∗ 0 2∗ 0∗1∗ 0 1∗ 0

P1 S 0 1∗ 1 0∗0 2∗ 2∗ 0

Player P2’s strategy x is strictly dominated by m,while both b and s are weakly dominated by m.

If we eliminate all weakly dominated strategies once,then eliminate 1’s strictly dominated strategy, only (B,m) remains.

This is a refinement of the set of Nash equilibria.



Sequential BoS, IV

An alternative way to justify P2 choosing strategy mconsiders the two different subgamesthat arise after player 1 has chosen either B or S .

In each of these player 2 can also choose B or S :

after B, player 2’s best response is B;

after S , player 2’s best response is S .

Thus player 2 matches by choosing m in the full game.

Player 1’s best response is B.

In effect, player 1’s move anticipates player 2’s response.

The result is player 1’s favoured outcome,so moving first confers an advantage in this game.

Player 1 has a “first-mover advantage”.



Sequential Cornot, I

Consider the Cournot duopoly model where firms have zero cost,and inverse demand is p = max{0, 100− q}, with q = q1 + q2.

Firm i ’s best response, taking the other firm’s qj as fixed,solves maxqi (100− qi − qj)qi , so qi = 50− 1

2qj .

After allowing for the possibility of a corner solution, we derivethe best response (BR) function bi (qj) = max{0, 50− 1

2qj}.

As in sequential BoS, suppose that before firm 2 chooses q2,it will have observed firm 1’s choice of q1.Firm 2’s strategy space becomes the setof all response functions q1 7→ r2(q1) from R+ to R+.

A strategy for firm 2 that weakly dominates all othersis to choose r2(·) to be the best response function b2(·).When making its move, firm 2 not only has the deterministic belief,but the certain knowledge that q1 has been played.



Sequential Cornot, II

Assuming common knowledge of rationality,what should firm 1 do?

A deterministic belief about q2 would be naıve,since firm 1 knows that a rational firm 2will choose b2(q1) = max{0, 50− 1

2q1}in response to its choice of q1.

This, in turn, means that a rational firm 1would replace the “fixed” q2 in its profit functionwith firm 2’s best response b2(q1) to 1’s choice of q1.

That is, firm 1 now chooses q1 to solve

maxq1

[100− q1 − b2(q1)] q1.



Stackelberg, I

Firm 1 chooses q1 to solve

maxq1

[100− q1 − b2(q1)] q1,

where b2(q1) = max{0, 50− 12q1}.

In the interior case, the maximandis [100− q1 − (50− 1

2q1)] q1 = (50− 12q1) q1,

with derivative 50− q1, which is ≷ 0 as q1 ≶ 50.

So, anticipating firm 2’s response b2(q1) to q1,firm 1’s optimal strategy is q1 = 50.

After this firm 2 responds by choosing q2 = b2(50) = 25.

The price is 100− q1 − q2 = 100− 75 = 25.

This is the Stackelberg equilibrium.



Stackelberg, II

Given q1 = 50, q2 = 25, and p = 25,the resulting profits (revenues)are π1 = 25 · 50 = 1, 250 and π2 = 25 · 25 = 625.

Recall that in the original Cournot equilibrium(with simultaneous moves)the quantities were q1 = q2 = 100/3 = 331

3 ,the price was p = 100/3 = 331

3 ,and the profits were π1 = π2 = 10, 000/9 = 1, 1111

9 .

So firm 1 has a first-mover advantage,relative to both firm 2 and the Cournot equilibrium.

Total profits, however, are lower.



Extensive Form Games

One obvious drawback (sometimes an advantage)of the normal form of a game is its failure to capture time.

A player’s strategy describes one complete plan for the whole game.Payoff functions represent how players valuethe consequences of strategy profiles.But how is the order of moves captured?

A game in extensive form captures sequential strategic situationswhen there is a well defined order of moves.

Our analyses of sequential BoS and sequential Cournot suggestthat the order of moves can affectwhat we would regard as a reasonable outcome of the game.

It illustrates a solution concept that capturesan important idea of sequential rationality.



Players and Payoffs

The extensive form formally captures situations where:

1. players may move sequentially;

2. what players know, when it is their turn to move,may depend on players’ previous choices.

Like the normal or strategic form,any extensive form game has two key ingredients in its description:

(EF1) The set of players, N

(EF2) The players’ payoffs (ui (·))i∈N as functions ofeverybody’s actions.



Move Order and Feasible Sets

To allow sequential play to be represented, we extendthe notion of pure strategy by specifying two separate parts:

first, as before, what players can do;

second, when (and with what information)they can do it.

In the Stackelberg example, as in the Cournot game,players can choose any quantity they like.But we also needed to specify that player 1 moves first,and only then does player 2 move (after observing 1’s prior choice).Thus, in general we need two extra componentsin order to model allowable move sequences:

(EF3) The order of moves.

(EF4) What choices players have when they do move(their feasible sets).



Players’ Knowledge

Simultaneous moves in the normal form represent the situationwhere players know nothing about other players’ prior moves.In the extensive form,it is not the chronological order of play that matters,but what players know when they choose.In both the Cournot and Stackelberg models of duopoly,firm 1 may indeed move some time before firm 2.But if firm 2 must choose without observing firm 1’s choice,then the Cournot model of simultaneous choice still applies.

In contrast, if firm 1’s choice is revealed before firm 2 moves,then we should use the sequential Stackelberg model.In general, we need to specifyhow information and knowledge change over time:

(EF5) Players’ information when making their moves.



Chance Moves

Exogenous random events vastly enrichthe strategic decision problems we can describe.

Example

Suppose firm A embarks ona research and development (R&D) projectthat may succeed or fail.

Competing firm B can adapt its strategyto the outcome of firm A’s R&D project,by waiting to see if it succeeds or not.We may think of the outcome of the R&D project as random— either success or failurewith fixed probabilities p and 1− p respectively.



Time Line

The time line of this scenario is as follows:

1. firm A chooses its R&D project;

2. the project succeeds with probability pand fails with probability 1− p;

3. after observing the outcome of firm A’s project,firm B will make its own choice.



Probabilities of Chance Moves

We will generally regard uncertainty outcomesas determined by “chance moves” or moves of nature.We think of nature (or chance) as a capricious playerwith a fixed random strategy, but no strategic objective.The probabilities attached to different chance moves are treatedas exogenous — i.e., as not chosen by any player, but fixed,like the strategy sets and payoff functions.We represent chance/nature as:

(EF6) Probabilities over exogenous events.

Finally, to use the tools and concepts we have already learned,we add the final familiar requirement:

(EF7) The structure of the extensive form game,represented by (EF1)–(EF6) above,is common knowledge.



Game Tree Representation

Components (EF1)–(EF7) capture all we needto represent a sequential game.

But a formal notation will be required.

For this, we introduce the idea of a game treeas a figurative way of representingthe unfolding of an extensive form game.



A Biped or Trust Game

rr

rr r

?

-

?

-1 2

NE

DC

0, 0 −1, 2

1, 1

Example

Player 1 first chooses whether to enter (E ) the game,or not to enter (N).The latter gives both players a payoff of zero.If player 1 plays E , player 2 can choose cooperate (C ),giving both players a payoff of 1.Or player 2 can defect (D) and get a payoff of 2,while leaving player 1 with a payoff of −1.



Trees

DefinitionA tree T = 〈X , <〉 is a set of nodes x ∈ X , together witha binary precedence relation < on X , which satisfy:

1. The relation < is transitive(i.e., x < x ′ and x ′ < x ′′ imply x < x ′′),asymmetric (x < x ′ implies not x ′ < x),irreflexive (x < x ′ implies x ′ 6= x),and generally incomplete(not every pair of nodes x , x ′ can be ordered).

2. There is a unique initial node x0 that precedesevery other node, but has none preceding it.

An interpretation is that, apart from the initial node x0,every node x in the tree can be reached as a resultof specific actions chosen at all the nodes x ′ < x that precede it.



Terminal Nodes and Game Trees

Let Z ⊆ X denote the set of terminal nodes,defined as those that precede no others.Each branch of a tree will have a unique terminal nodeunless that branch has infinitely many nodes.

DefinitionA game tree is a tree Tsupplemented by a set of players i ∈ Nand their payoff functions ui : Z → Rdefined on the terminal nodes of T .The order of play is given by a function i : X \ Z → Nthat identifies a player i(x) at each non-terminal node x ∈ X \ Z .Let A(x) denote the feasible set of actionsthat are possible for player i(x) at node x .



Sequential BoS . . .

b�� XXXXX

1O Fr

��

@@@

2o fr

2, 1r

0, 0

r�

��

@@@

2o fr

0, 0r

1, 2

Consider again sequential BoS,with O and o representing opera,and with F and f representing football.One equilibrium strategy profile is (F ,ff ),resulting in payoffs of (1, 2).



. . . Has Some Incredible Nash Equilibria

The equilibrium (F ,ff ) is supported by player 1’s beliefthat deviating to play O would lead to payoff 0because player 2 will play f at x1.

This “threat” to choose f off the equilibrium pathsupports player 1’s action on the equilibrium path.

But this “threat” is incredible because for player 2 to carry it outwould be irrational given the payoffs of the game.

So does the Nash equilibrium (F ,ff )make a reasonable prediction of player 1’s rational choice?

It does meet the definition of Nash equilibrium:strategy ff is a best response to F , and vice versa.But ff commits player 2 to f after 1’s choice of O,even though o would be better.



Refined Equilibria in the Sequential BoS, I

The normal form representation treats all strategic choices as“once-and-for-all” commitments.

Yet we expect rational players to play best responsesto their beliefs wherever they are called on to move.

In the normal form it is hard to analyse how credibilitylimits what can be accepted as “rational behavior”.

But this can be done in the extensive form.



Refined Equilibria in the Sequential BoS, II

Two of the three Nash equilibria in this game — namely (O, oo)and (F ,ff ) — commit player 2 to a strategy that,though a best response to 1’s equilibrium strategy,would not be best were player 1 to deviatefrom the equilibrium path. (“Counterfactual” analysis.)

The new structure we are about to set upwill result in more refined predictions for dynamic games.

It will rule out equilibria that involve such incredible threats.

In particular, it will admit only (O, of )as the unique “refined” Nash equilibrium.



Sequential Rationality

Equilibria (O, oo) and (F ,ff ) in sequential BoSrequire an implausible reaction by player 2when 1 departs from the presumed equilibrium.

We now insist that every player use strategiesthat are optimal at every information set in the game tree,even those not reached along the equilibrium path.

DefinitionGiven any player i and any strategy profile s−i ∈ S−i ,the strategy si is sequentially rational if and only ifwhat it prescribes at each information set h ∈ Hi

is a best response to s−i .

In sequential BoS, sequential rationality requires:if player 1 plays O, then player 2 should play o;if player 1 plays F , then player 2 should play f .



Sequential Rationality in Dynamic BoS

Now move back to initial node x0,where player 1 has to choose between O and F .Player 2’s sequential rationality should lead player 1to expect that playing O will result in payoffs (2, 1),whereas playing F will result in payoffs (1, 2).So sequential rationality implies player 1 chooses O.Sequential rationality makes the unique predictionthat the path of play is O followed by o.Furthermore, the process predicts what would happenif one or more players deviate from the path of play:if player 1 chooses F instead of O, then 2 will choose f .

ConclusionThe particular Nash equilibrium (O, of ) is the only strategy pair(and the only one of the three Nash equilibria)that meets the sequential rationality requirement.



Backward Induction

The procedure used in dynamic BoSto find sequentially rational Nash equilibria (or SRNE)(in pure strategies) starts at terminal nodes.It is a backward induction procedure that steps backnode by node, all the way to the initial node, where it ends.

At each node, the player to move responds bestto the other players’ anticipated actions at all later nodes.



The Centipede (or Quadruped) Game

rr

rr r

?

-

?

-1 2

N1

C1n1

c1

1, 1 0, 3rr

rr r

?

-

?

-1 2

N2

C2n2

c2

2, 2 1, 4

3, 3

The game goes from left to right as follows:

First, player 1 can stop the game at once by choosing N1,or continue by choosing C1.

Second, player 2 faces a choice between n1 and c1.If player 2 chooses c1, the next move is player 1’s,who chooses between N and C .If C2 is chosen, player 2 concludes the gameby choosing n2 or c2 a second time.



What the Players Would Like

rr

rr r

?

-

?

-1 2

N1

C1n1

c1

1, 1 0, 3rr

rr r

?

-

?

-1 2

N2

C2n2

c2

2, 2 1, 4

3, 3

If the players could reach a binding agreement,they would surely like to continue at each move,to reach the payoffs of (3, 3), the highest possible total.

However, backward induction indicatesthat this contradicts sequential rationality.



Appying Backward Induction . . .

rr

rr r

?

-

?

-1 2

N1

C1n1

c1

1, 1 0, 3rr

rr r

?

-

?

-1 2

N2

C2n2

c2

2, 2 1, 4

3, 3

At the last information set player 2 will choose n2 rather than c2,to get 4 instead of 3.

Anticipating this one step earlier,player 1 will choose N2 to get 2 instead of 1.

Anticipating this, player 2 will choose n1 on his first move.The same argument goes all the way back to the initial node,where player 1 will choose N1.So the backward induction payoffs are (1, 1).



The Curse of Rationality

In the quadruped game, every time a player continues,the sum of their payoffs goes up by 1.

We can continue increasing the payoffs like thiswith (2, 5), (4, 4), (3, 6), etc. after (3, 3).

The payoffs late in the game become extremely large.

Backward induction still unravels any form of cooperation,leading to the unique sequentially rational payoffs (1, 1).This is the “curse of rationality”.



Proper Subgames: Definition

DefinitionLet Γ be an extensive-form game with X as its set of nodes.A (proper) subgame G of Γ starts at some single node xG ,and includes all nodes belonging to information setsthat include one or more successors of xG in Γ.

The subgame G is itself an extensive-form game treewith its information sets and payoffs inherited from Γ.

DefinitionLet Γ be an n-player extensive form game.A behavioral strategy profile σ∗ = (σ∗1, σ

∗2, . . . , σ

∗n)

is a subgame-perfect Nash equilibrium (or SPE)if for every proper subgame G of Γ,the restriction of σ to G is a Nash equilibrium in G .



On and off the Equilibrium Path

Subgame perfection imposes sequential rationalityon Nash equilibrium by requiring best responsesnot only on the equilibrium path (by definition),but off the equilibrium path as well.

It does so by requiring the relevant restrictionof the equilibrium strategy profile σ∗

to be a Nash equilibrium in every proper subgame,including any subgame not reached on an equilibrium path.

By its definition, every SPE is a Nash equilibrium.

But not every Nash equilibria is an SPE,implying that SPE refines the set of Nash equilibria,and thus predicts behaviour more precisely.


Week 7: Games in Extensive Form - University of Warwick · Battle of the SexesStackelbergEssential...

Documents

Transcript of Week 7: Games in Extensive Form - University of Warwick · Battle of the SexesStackelbergEssential...