CD Lecture6 2014

8/29/2014

1

MAE 301 T2 2014MICROECONOMIC THEORY AND POLICY

LECTURE 6

GAME THEORY:Dynamic Games

(sequential and repeated)

Learning Objectives Analyze sequentially played games

Extensive-form games Subgame Perfect Nash Equilibrium

Discuss the implications of repeated interactions

8/29/2014

2

Dynamic Games

In dynamic games: players move either sequentially or repeatedly players are assumed to have complete

information about payoff functions at each move, players are assumed to have

perfect information about previous moves of all players

Extensive Form

Dynamic games are analyzed in their extensive form, which specifies the n players the sequence of their moves the actions they can take at each move the information each player has about players

previous moves the payoff function over all possible strategies

8/29/2014

3

Actions v Strategies

Consider a single period two-stage game: First stage: player 1 moves Second stage: player 2 moves

In games where players move sequentially, we distinguish between an action and a strategy An action is a move that a player makes at a

specified point in the game tree A strategy is a battle plan that specifies the action a

player will make condition on information available at each move

Return to the Airline Game to demonstrate these concepts Assume American chooses its output before United

does

The Airline Game

This game treeshows decision nodes:

indicates which players turn it is

branches:indicates all possible actions available

subgames: subsequent decisions available given previous actions

8/29/2014

4

SPNE

To predict the outcome of the Airline game (and, more generally, the outcome of any sequential game with a finite number of nodes), we use a strong version of Nash equilibrium, the Subgame Perfect Nash Equilibrium (SPNE)

SPNE requires players to believe that their opponents will act optimally (in their own best interest) at any decision node

A set of strategies forms a subgame perfect Nash equilibrium if the players strategies are a Nash equilibrium in every subgame The Airline game has four subgames; three subgames

at second stage where United makes a decision and an additional subgame at the time of the first-stage decision

Backward induction We can solve for the subgame perfect Nash equilibrium

using backward induction Once game is understood through backward induction,

players play it forward To apply backward induction, first determine optimal

actions at last decision nodes that result in terminal nodes

Then determine optimal actions at next-to-last decision nodes, assuming that optimal actions will follow at next decision nodes

8/29/2014

5

Backward induction Continue backward process until root node is reached Backward induction implicitly assumes that a players

strategy will consist of optimal actions at every node in the game tree

Principle of sequential rationality: At any point in the game tree, players strategy should consist of optimal actions from that point on given other players strategies

Backward induction Backward induction is where we determine:

the best response by the last player to move the best response for the player who made the

next-to-last move repeat the process until we reach the beginning of

the game Airline Game

If American chooses 48, United selects 64, Americans profit=3.8



Thus, American chooses 96 in the first stage

8/29/2014

6

How reasonable is backward induction as a behavioral principle?

May work to explain actual outcomes in simple games, with few players and moves

More difficult to use in complex sequential move games such as Chess We cant draw out the game tree because there are too

many possible moves, estimated to be on the order of 10120

Need a rule for assigning payoffs to non-terminal nodes an intermediate valuation function

May not always predict behavior if players are unduly concerned with fair behavior by other players and do not act so as to maximize their own payoff, e.g., they choose to punish unfair behavior

Sequential Battle-of-the-Sexes

As an example of a sequential game, consider the Battle-of-the-Sexes game

Husband prefers going to fights and wife prefers opera

However, they both prefer spending their leisure time together

In the simultaneously played game, there exist two pure-strategy Nash equilibria (both going to the opera or both to the fights)

Assume the husband take the lead

8/29/2014

7

Game tree for Battle-of-the- Sexes

Sequential Battle-of-the-Sexes

Suppose the husband picks opera If the wife picks opera as well, the husband will end up

with a payoff of 2 and the wife with a payoff of 5 If the husband picks fights, it will be optimal for the

wife to pick fights (resulting payoffs are 5 for husband and 2 for wife)

For husband (first player), 5 is greater than 2. Both going to the fights is not only an overall equilibrium, but also an equilibrium in each of the subgames, that is, a subgame perfect Nash equilibrium

8/29/2014

8

The Simultaneous-Move Bargaining Game Management and a union are negotiating, among other

things, a wage increase Strategies are wage offers & wage demands Successful negotiations lead to $600 million in surplus,

which must be split among the parties Failure to reach an agreement results in a loss to the

firm of $100 million and a union loss of $3 million An agreement is reached ONLY if wage offer=wage

demand (rationale: higher wage offered at worse conditions, such as less job security, or longer shifts)

Otherwise you can assume that an agreement is reached if (all other things being equal) wage offer is higher or equal to wage demand

Only one-shot in making a deal

The Simultaneous-Move Bargaining Game

Union

Man

agem

ent Strategy W = AU$10 W = AU$5 W = AU$1

W = AU$10 100, 500 -100, -3 -100, -3W=AU$5 -100, -3 300, 300 -100, -3W=AU$1 -100, -3 -100, -3 500, 100

8/29/2014

9

The Simultaneous-Move Bargaining Game

Union

Man

agem

ent

3 Nash Equilibria!

Strategy W = AU$10 W = AU$5 W = AU$1W = AU$10 100, 500 -100, -3 -100, -3

W=AU$5 -100, -3 300, 300 -100, -3W=AU$1 -100, -3 -100, -3 500, 100

The Sequential-Move Bargaining Game

Now suppose that the management moves first by making the union a take-it-or-leave-it offer

Firm

10

5

1

Union

Union

Union

Accept

Accept

Accept

Reject

Reject

Reject

100, 500

-100, -3

300, 300

-100, -3

500, 100

-100, -3

8/29/2014

10


Identify the firms feasible strategies: the management has three feasible strategies: (1) Offer AU$10; (2) Offer AU$5; (3) Offer AU$1

Identify the unions feasible strategies: the union has eight feasible strategies (23): (1) Accept AU$10, Accept AU$5, Accept AU$1; (2) Accept AU$10, Accept AU$5, Reject AU$1; (3) Accept AU$10, Reject AU$5, Accept AU$1; (4) Accept AU$10, Reject AU$5, Reject AU$1; (5) Reject AU$10, Accept AU$5, Accept AU$1; (6) Reject AU$10, Accept AU$5, Reject AU$1; (7) Reject AU$10, Reject AU$5, Accept AU$1; (8) Reject AU$10, Reject AU$5, Reject AU$1


Identify Nash Equilibrium Outcomes: outcomes such that neither the firm nor the union has an incentive to change its strategy, given the strategy of the other

Find the Sub-game Perfect Nash Equilibrium Outcomes (SPNE): outcomes where no player has an incentive to change its strategy, given the strategy of the rival, and the outcomes are based on credible actions, that is, they are not the result of empty threats by the rival

For a firms strategy to be a credible threat rivals must believe that the firms strategy is rational (it is in the firms best interest to use it)

8/29/2014

11


Accept $10, Accept $5, Accept $1Accept $10, Accept $5, Reject $1Accept $10, Reject $5, Accept $1Reject $10, Accept $5, Accept $1Accept $10, Reject $5, Reject $1Reject $10, Accept $5, Reject $1Reject $10, Reject $5, Accept $1Reject $10, Reject $5, Reject $1

Union's Strategy Firm's Best Response

Mutual Best Response?

$1 Yes$5$1$1

$10

YesYesYesYes

$5 Yes$1

NoYes

$10, $5, $1

Nash and Credible Nash Only Neither Nash Nor Credible


We have identified many (7) combinations of strategies that are mutual best response - Nash equilibrium strategies

In all but one the firm does something that isnt in its self interest (and thus entail threats that are not credible)

Only one Nash equilibrium outcome (out of 3 Nash equilibrium outcomes) is based on credible actions/threats

Empty threats will not succeed in inducing a player to select some action

8/29/2014

12


Firm

10

5

1

Union

Union

Union

Accept

Accept

Accept

Reject

Reject

Reject

100, 500

-100, -3

300, 300

-100, -3

500, 100

-100, -3

There are 3 Nash Equilibrium Outcomes!


Firm

10

5

1

Union

Union

Union

Accept

Accept

Accept

Reject

Reject

Reject

100, 500

-100, -3

300, 300

-100, -3

500, 100

-100, -3

Only 1 Subgame-Perfect Nash Equilibrium Outcome!

8/29/2014

13


In take-it-or-leave-it bargaining, there is a first-mover advantage

Management can gain by making a take-it-or-leave-it offer to the union. But...

Management should be careful; real world evidence suggests that people sometimes reject offers on the basis of principle instead of cash considerations

What would happen if the union moved first?

The Ultimatum Game

Player 1 moves first and proposes a division of $1.00. Suppose there are just 3 possible discrete divisions, limited to $0.25 increments: Player 1 can propose x=$0.25, x=$0.50, or x=$0.75

for himself, with the remainder, 1-x going to Player 2

Player 2 moves second and can either acceptor reject Player 1s proposal

If Player 2 accepts, the proposal is implemented

If Player 2 rejects, both players get $0 each. The $1.00 gains from trade vanish

8/29/2014

14

The Ultimatum Game

Problems with Take-it-or-Leave-it

Take-it-or-leave-it games are too trivial; there is no back-and-forth bargaining

Another problem is the credibility of take-it-or leave-it proposals If player 2 rejects player 1s offer, is it really

believable that both players walk away even though there are potential gains from trade?

Or do they continue bargaining? What about fairness? Is it really likely that Player

1 will keep as much of M as possible for himself?

8/29/2014

15

The Centipede Game

RepeatedGames Wenowexaminetheeffectofrepetition onstrategic

behavioringameswithperfectinformation. Ifagameisplayedrepeatedly,withthesameplayers,the

playersmaybehaveverydifferentlythanifthegameisplayedjustonce(a oneshotgame).

Twotypesofrepeatedgames: Finitelyrepeated:thegameisplayedforafiniteandknownnumberofrounds,forexample,2rounds/repetitions.

Infinitely orIndefinitelyrepeated:thegamehasnopredeterminedlength;playersactasthoughitwillbeplayedindefinitely,oritendsonlywithsomeprobability.

8/29/2014

16

Infinitely RepeatedGames Finitelyrepeatedgamesareinteresting,butrelativelyrare;

howoftendowereallyknowforcertainwhenagameweareplayingwillend?(Sometimes,butnotoften).

Someofthepredictionsforfinitelyrepeatedgamesdonotholdupwellinexperimentaltests: Theuniquesubgameperfectequilibriuminthefinitelyrepeated

ultimatumgameorprisonersdilemmagame(alwaysconfess)arenotusuallyobservedinallroundsoffinitelyrepeatedgames.

Ontheotherhand,weroutinelyplaymanygamesthatareindefinitelyrepeated (noknownend).Wecallsuchgamesinfinitelyrepeatedgames,andwenowconsiderhowtofindsubgameperfectequilibriainthesegames.

DiscountinginInfinitelyRepeatedGames Recallfromourearlieranalysisofbargaining,thatplayersmay

discount payoffsreceivedinthefutureusingaconstantdiscountfactor,d=1/(1+r),where0

8/29/2014

17

DiscountinginInfinitelyRepeatedGames,Cont. Theinfinitesum, convergesto

Simpleproof:Letx=Noticethatx=solve

Hence,thepresentdiscountedvalue ofreceiving$pineveryfutureroundis$p[d/(1d)]or $pd/(1d)

Notefurtherthatusingthedefinition,d=1/(1+r), d/(1d)=[1/(1+r)]/[11/(1+r)]=1/r,sothepresentvalueoftheinfinitesumcanalsobewrittenas$p/r.

Thatis,$pd/(1d)=$p/r,sincebydefinition,d=1/(1+r).

1

...32

...32 x ...)( 32

1 ;)1(:for xxxxx

ThePrisonersDilemmaGame Considerathefollowingprisonersdilemmagame,where

higherpayoffs arenowpreferredtolowerpayoffs.

Tomakethisaprisonersdilemma,wemusthave:b>c>d>a.Wewillusethisexampleinwhatfollows.

C D

C

D

c,c a,b

b,a d,d

C=cooperate,(dontconfess)D=defect(confess)

C D

C

D

4, 4 0, 6

6, 0 2, 2

Supposethepayoffsnumbersareindollars

8/29/2014

18

SustainingCooperationintheInfinitelyRepeatedPrisonersDilemmaGame

TheoutcomeC,Cforever,yieldingpayoffs(4,4)canbeasubgameperfectequilibriumoftheinfinitelyrepeatedprisonersdilemmagame,providedthat1)thediscountfactorthatbothplayersuseissufficientlylargeand2)eachplayerusessomekindofcontingent ortriggerstrategy.Forexample,thegrimtriggerstrategy: Firstround:PlayC. Secondandlaterrounds:solongasthehistoryofplayhasbeen(C,C)in

everyround,playC.OtherwiseplayDunconditionallyandforever. Proof:Consideraplayerwhofollowsadifferentstrategy,playing

CforawhileandthenplayingDagainstaplayerwhoadherestothegrimtriggerstrategy.

CooperationintheInfinitelyRepeatedPrisonersDilemmaGame,Continued

Considertheinfinitelyrepeatedgamestartingfromtheroundinwhichthedeviantplayerfirstdecidestodefect.Inthisroundthedeviantearns$6,or$2morethanfromC,$6$4=$2.

SincethedeviantplayerchoseD,theotherplayersgrimtriggerstrategyrequirestheotherplayertoplayDforeverafter,andsobothwillplayDforever,alossof$4$2=$2inallfuturerounds.

Thepresentdiscountedvalueofalossof$2inallfutureroundsis$2d/(1d)

Sotheplayerthinkingaboutdeviatingmustconsiderwhethertheimmediategainof2>2d/(1d),thepresentvalueofallfuturelostpayoffs,orif2(1d)>2d,or2>4d,or1/2>d.

If

8/29/2014

19

CooperationintheInfinitelyRepeatedPrisonersDilemmaGame,Continued

d* =1/2isthecriticaldiscountfactor. Thecriticaldiscountfactoristhethresholdofthediscount

factorabovewhichcollusioncanbesustainedasaSPNEoftheinfinitelyrepeatedgame.

Foranyd >d*thepresentvalueofallfuturelostpayoffsexceedstheimmediategainfromdeviating,implyingthatcollusionissustainable.

OtherSubgamePerfectEquilibriaarePossibleintheRepeatedPrisonersDilemmaGame

ThesetofsubgameperfectNashEquilibria,isthegreenarea,asdeterminedbyaveragepayoffsfromallroundsplayed(forlargeenoughdiscountfactor,d).

ColumnPlayerAvg.Payoff

Row

Player

Avg.

Payoff

TheFolktheoremofrepeatedgamessaysthatalmostanyoutcomethatonaverage yieldsthemutualdefectionpayofforbettertobothplayerscanbesustainedasasubgameperfectNashequilibriumoftheindefinitelyrepeatedPrisonersDilemmagame.

Theefficient,mutualcooperationinallroundsequilibriumoutcomeishere,at4,4.

ThesetoffeasiblepayoffsistheunionofthegreenandyellowregionsMutualdefectioninall

roundsequilibrium

8/29/2014

20

MustWeDiscountPayoffs? Answer1:Howelsecanwedistinguishbetweeninfinite

sums ofdifferentconstantpayoffamounts? Answer2:Wedonthavetoassumethatplayersdiscount

futurepayoffs.Instead,wecanassumethatthereissomeconstant,knownprobabilityq,0

8/29/2014

21

GrimTriggerStrategiesatwork:Anexample

TheAdvertisingGame: Twofirms(Kelloggs&GeneralMills)seektomaximizeprofits

Strategiesconsistofadvertisingcampaigns Simultaneousmoves Repeatedinteraction


Strategy None Moderate HighNone 12,12 1, 20 -1, 15

Moderate 20, 1 6, 6 0, 9High 15, -1 9, 0 2, 2

GeneralMills

Kello

ggs

Nash Equilibrium of the one shot advertising game

8/29/2014

22



Moderate 20, 1 6, 6 0, 9High 15, -1 9, 0 2, 2

GeneralMills

Kello

ggs

Collusive Equilibrium


Ifthegameisrepeatedafinitenumberoftimes,e.g.2times,collusioncannotwork(bybackwardsinduction)!

Inperiod2,thegameisaoneshotgame,soequilibriumentailsHighAdvertisinginthelastperiod=>period1isreallythelastperiod,sinceeveryoneknowswhatwillhappeninperiod2.

EquilibriumentailsHighAdvertisingbyeachfirminbothperiods.

Thesameholdstrueifwerepeatthegameanyknown,finitenumberoftimes.

8/29/2014

23


Cancollusionworkiffirmsplaythegameeachyear,forever? Considerthefollowinggrimtriggerstrategy byeachfirm:

Dontadvertise,providedtherivalhasnotadvertisedinthepast.Iftherivaleveradvertises,punishitbyengaginginahighlevelofadvertisingforeverafter.


Eachfirmagreestocooperateaslongastherivalhasntcheatedinthepast.Cheatingtriggerspunishmentinallfutureperiods.

SupposeGeneralMillsadoptsthistriggerstrategy.Kelloggsprofits?

Wehavetocomparethevalueofcooperationwiththevalueofcheating.

8/29/2014

24



Moderate 20, 1 6, 6 0, 9High 15, -1 9, 0 2, 2

GeneralMills

Kello

ggs

Consider a deviation by Kellogg. In the deviation period, Moderate is the best response against None.

Deviation (or Cheating) Payoff


Cooperate =12+12+12 2+123+=12+12/(1 ) = 12/(1 )

with =1/(1+r)beingthediscountfactorCheat =20+2+2 2+2 3 +

=20+2 /(1 )Cheat Cooperate = 20+2 /(1 ) 12/(1 ) ==(8 18)/(1 )ifr=0.05=> =0.95238Cheat Cooperate =191.67

8/29/2014

25


Ifr=5%,itdoesntpaytodeviate=>CollusionisaNashequilibriumintheinfinitelyrepeatedgame!

Benefits&CostsofCheating:ImmediateBenefit (20 12 today) versus PresentValueofFutureCost (12 2 foreverafter).

IfImmediateBenefit PresentValueofFutureCost>0=>Paystocheat;IfImmediateBenefit PresentValueofFutureCost 0=>Doesntpaytocheat.

ThecriticaldiscountfactorcanbecomputedbysolvingCooperate =12/(1 )Cheat =20+2 /(1 ).

*=4/9.Forany > *collusionissustainable.

KeyInsights CollusioncanbesustainedasaNashequilibriumwhen

thereisnocertainendtoagame. Collusionsustainabilityrequires:

Abilitytomonitoractionsofrivals Ability(andreputationfor)punishingdefectors Lowinterestrate Highprobabilityoffutureinteraction.

CD Lecture6 2014

Documents

Transcript of CD Lecture6 2014