Infinitely Repeated Games

Finitely Repeated Game

• Take any game play it, then play it again, for a specified number of times.

• A single play of the game that is repeated is known as the stage game.

• Let players observe all previous play.• For every history that you have observed, you

could have a different response.

Prisoners’ dilemma 3 times• How many strategies that vary with other players’s actions on previous move.First move: C or DSecond move:

– C always– C if C, D if D– D if C, C if D– D always

• Third move: There are 4 possible histories of other guy’s moves. For each move by other guy there are two things you can do on your next move. That gives you 2x2x2x2=16 possible 3d move strategies.

• Then you have 2 possible first move strategies, 4 possible 2d move strategies and 16 possible third move strategies

• That is 2x4x16=128 strategies that depend on observed behavior of other guy

Prisoners’ Dilemma

R, R S, T

T, S P, P

Cooperate Defect

Cooperate

Defect

PLAyER 1

Player 2

T > R > P > S “Temptation” “Reward” “Punishment” “Sucker”

Twice Repeated Prisoners’ Dilemma

Two players play two rounds of Prisoners’ dilemma. Before second round, each knows what other did on the first round. Payoff is the sum of earnings on the two rounds.

Two-Stage Prisoners’ DilemmaWorking back

Player 1

Cooperate Defect

Player 2

CooperateCooperateDefect Defect

Player 1 Player 1 Player 1 Player 1

C

C

C

C

C CD D D D

C C C D

Player 1Pl. 2 Pl 2

Pl 2 Pl 2

2R2R

D DC D C D C D DR+SR+T

R+TR+S

R+PR+P

S+RT+R

D

P+RP+R

P+SP+T

T+PP+S

2P2P

Etc…etc

Two-Stage Prisoners’ DilemmaWorking back further

Player 1

Cooperate Defect

Player 2

CooperateCooperateDefect Defect

Player 1 Player 1 Player 1 Player 1

C

C

C

C

C CD D D D

C C C D

Player 1Pl. 2 Pl 2

Pl 2 Pl 2D DC D C D C D D

1021

2110

1111

1021

022

1111

112

1111

D

220

121

1111

212

121

Longer Game

• What is the subgame perfect outcome if Prisoners’ dilemma is repeated 100 times?• Work backwards: In last round, nothing you do

affects future, so you play the dominant strategy for stage game: defect.

• Since last round is determined, nothing you do in next to last round affects future, so you play dominant strategy for stage game: defect

• Work your way back. Only subgame perfect outcome is “Defect always”.

In a repeated game that consists of four repetitions of a stage game that has a unique Nash equilibrium

A) There are four subgame perfect Nash equilibria

B) There are 24=16 subgame perfect Nash equilibria

C) There is only one subgame perfect Nash equilibrium.

D) The number of subgame perfect Nash equilibria varies, depending on the details of the game.

More generally

• In a subgame perfect equilibrium for a finitely repeated game where the stage game has a unique N.E, the moves in the last stage are determined for each person’s strategy. Given that the moves in the last stage don’t depend on anything that happened before, the Nash equilibrium in previous stage is uniquely determined to be the stage game equilibrium.

• And so it goes…All the way back to the beginning.

Games without a last round

• Two kinds of models– Games that continue for ever– Games that end at a random, unknown time

Infinitely repeated game

• Wouldn’t make sense to add payoffs.• You would be comparing infinities. • Usual trick. Discounted sums. • Just like in calculating present values.• We will see that cooperative outcomes can

often be sustained as Nash equilibria in infinitely repeated games.

Why consider infinite games?We only have finite lives.

• Many games do not have known end time.• Just like many human relationships.• Simple example—A favorite of game theorists• After each time the stage game is played there is

some probability d<1 that it will be played again and probability 1-d that play will stop.

• Expected payoff “discounts” payoffs in later rounds, because game is less likely to last until then.

Cleaning house as a Repeated Prisoners’ Dilemma

• Maybe a finite game if you have a fixed lease and don’t expect to see roommate again after lease expires.

• Most relationships don’t have a known last time.

• Usually some room for “residual good will.”

In a repeated game, after each round of play, a fair coin is tossed. If it comes up heads, the game continues to another

round. If it comes up tails, the game stops. What is the probability that the game is played for at least three rounds?

A) 1/3B) 2/3C) 1 /4D) 1 /2E) 1/8

Calculating sums

• In a repeated game, with probability d of continuation after each round, the probability that the game is still going at round k is dk-1

• Calculate expected winnings if you receive R so long as the game continues.R+dR+d2R+ d3R+ d4R + ….+=R(1+d +d2 + d3 + d4 + ….+ )• What is this infinite sum?

Adding forever

• The series (1+d +d2 + d3 + d4 + ….+ )Is known as a geometric series. When |d|<1, this series converges. That is, to say, the limit as n approaches infinity of 1+d +d2 + d3 + d4 + ….+ dn exists. Let S= 1+d +d2 + d3 + d4 + ….+ Then dS=d +d2 + d3 + d4 + ….+ And S-dS=1.So S(1-d)=1S=1/(1-d).

What is the limit as n approaches infinity of 1+3/4+(3/4)2+…+(3/4)n

A) 1/(4n)B) InfinityC) 2D) 4E) 16

Infinitely repeated prisoners’ dilemma and the “Grim Trigger Strategy”

• Suppose 2 players play repeated prisoners dilemma, where the probability is d<1 that you will play another round after the end of each round.

• The grim trigger strategy is to play cooperate on the first round and play cooperate on every round so long as the other doesn’t defect.

• If the other defects, the grim trigger strategy plays defect on all future rounds.

When is there a symmetric SPNE where all play Grim Trigger?

• Suppose that the other player is playing Grim Trigger.

• If you play Grim Trigger as well, then you will cooperate as long as the game continues and and you will receive a payoff of R.

Your expected payoff from playing Grim Trigger if the other guy is playing Grim Trigger is therefore R(1+d +d2 + d3 + d4 + ….+ )=R/(1-d)

What if you defect against Grim Trigger

• If you defect and the other guy is playing Grim Trigger, you will get a payoff of T>R the first time that you defect. But after this, the other guy will always play defect. The best you can do, then is to always defect as well.

• Your expected payoff from defecting is therefore T+ P(d +d2 + d3 + d4 + ….+ )

=T+Pd/1-d

Cooperate vs Defect

• If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing cooperate is R/(1-d)

• If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing defect is T+Pd/(1-d)

• Cooperate isR/(1-d) better for you if R/(1-d)>T+Pd/(1-d) which implies d>(T-R)/(T-P)• Example If T=10, R=5, P=2, then condition is d>5/8.• If d is too small, it pays to “take the money and run”

Other equilbria?

• Grim trigger is a SPNE if d is large enough.• Are there other SPNEs?• Yes, for example both play Always Defect is an

equilibrium.• If other guy is playing Always Defect, what is

your best response in any subgame?• Another is Play Defect the first 10 rounds,

then play Grim Trigger.

Tit for Tat

• What is both players play the following strategy in infinitely repeated P.D?

• Cooperate on the first round. Then on any round do what the other guy did on the previous round.

• Suppose other guy plays tit for tat.• If I play tit for tat too, what will happen?

Payoffs

• If you play tit for tat when other guy is playing tit for tat, you get expected payoff of

R(1+d +d2 + d3 + d4 + ….+ )=R/(1-d)• Suppose instead that you choose to play “Always

defect” when other guy is tit for tat.• You will get T+ P(d +d2 + d3 + d4 + ….+ ) =T+Pd/1-dSame comparison as with Grim Trigger. Tit for tat is a better response to tit for tat than always defect if d>(T-R)/(T-P)

Another try

• Sucker punch him and then get him to forgive you.

• If other guy is playing tit for tat and you play D on first round, then C ever after, you will get payoff of T on first round, S on second round, and then R for ever. Expected payoff is T+ Sd+d2R(1+d +d2 + d3 + d4 + ….+ )=T+ Sd+d2R/(1-d).

Which is better?• Tit for tat and Cheat and ask forgiveness give same

payoff from round 3 on.• Cheat and ask for forgiveness gives T in round 1 and S

in round 2. • Tit for tat give R in all rounds.• So tit for tat is better if R+dR>T+dS, which means d(R-S)>T-R or d>(T-R)(R-S) If T=10, R=6, and S=1, this would mean if d>4/5.But if T=10, R=5, and S=1, this would be the case only if d>5/4, which can’t happen. In this case, tit for tat could not be a Nash equilibrium.