Post on 31-Dec-2015
Evolution of cooperation in Stackelberg games
Raimo P. Hämäläinen
Ilkka Leppänen
Systems Analysis Laboratory
Aalto University
Main theme
• To learn about cooperative behavior in repeated interactions
• In a Stackelberg setting, the players are in asymmetric positions. How does this affect cooperation?
• How do people act in these games when they have possibility to cheap talk / cheat / second-play?
• We study cheap talk with a second-play model
Earlier results from repeated settings• Cournot duopoly: random pairs converge to Nash, fixed
pairs collude (Holt 1985) • Sequential prisoner’s dilemma: cooperation decreases
over repetition (Clark and Sefton 2001)• Market games (auctions): convergence to equilibrium
prices (Roth et al. 1991)• Ultimatum games: offers and rejections become lower
over time (Roth et al. 1991, Camerer 2003) • Public goods games: cooperation deteriorates and free-
riding increases, but cooperation is maintained with punishment (Fehr and Schmidt 1999)
Stackelberg game
Basic setting:1) Leader decides: knows both payoffs and takes into
account the best response reaction of the follower2) Follower reacts to the leader’s decision
Second-play Stackelberg setting (Hämäläinen 1981):1) Leader decides and announces his decision to the
follower2) Follower reacts to the leader’s decision3) Leader takes into account follower’s decision and decides
again (re-optimizes / cheats)
Stackelberg game
• Leader has the opportunity to better payoffs because he decides first
• Second-play is possible when follower does not have full information about leader’s payoffs
• Stackelberg games have not received much attention in the experimental games literature
Stackelberg game with second-play
• In the second-play Stackelberg game, the leader’s announcement should not affect the follower’s choice
• In theory, the leader is the follower
• Second-play announcement introduces a possibility of cheap talk and cheating
• Does this affect cooperation, and is this possibility used?
• Cheap talk literature: cheap talk signaling helps cooperation in social dilemma situations (Crawford, 1998)
Experiment of Huck, Müller, and Normann (2001)
• Stackelberg game in a market context• Players acting as quantity-choosing firms• 13 × 13 payoff matrix, pen and paper• 92 subjects• Repeated for 10 rounds, with
– fixed pairs where each player meets the same pair repeatedly– random pairs where each player meets a new randomly
selected pair every round
• Compensated from the payoffs of two randomly selected rounds out of the ten rounds (average DM 15.67)
JO = joint optimum (6,6) L = Stackelberg leader (12,6)LS = second-play Stackelberg leader (9,6) F = Stackelberg follower as leader (6,12)N = Cournot-Nash equilibrium (8,8)
FollowerLe
ader
Our experiment
• Same payoff matrix and market context (similar instructions) as Huck et al. (2001)
• Computerized: SAL experimental game platform• 210 student subjects from engineering faculties• Repeated for 20 to 24 rounds, players did not know the
number of rounds• Compensated from the payoffs of two randomly selected
rounds out of ten final rounds (average 6.77 €)
8 different games, only one for each subject
Game PairsFollower knows payoffs of leader
Number of pairs
Stackelberg fixed Yes (*) 11
fixed No 15
random Yes (*) 10
random No 9
Second-play Stackelberg fixed Yes 14
fixed No 14
random Yes 18
random No 14
(*) Games studied by Huck et al. (2001)Subjects remained in their roles leader/follower for the whole session
SAL experimental game platform
• Web-based to allow independence of location• Our experiments arranged in a regular computer classroom
Evolution of cooperation: Stackelberg fixed pairs
F does not know payoffs of L
F knows payoffs of L
Evolution of cooperation: Stackelberg random pairs
F does not know payoffs of L
F knows payoffs of L
Evolution of cooperation: second-play fixed pairs
F does not know payoffs of L
F knows payoffs of L
Evolution of cooperation: second-play random pairs
F does not know payoffs of L
F knows payoffs of L
Stackelberg games result: gains and losses in cooperation
• Reference outcome is the Stackelberg outcome (72,36), payoff difference 36
• Fixed pairs: joint optimum (72,72), leader’s loss 0, follower’s gain 36
• Random pairs: Cournot-Nash (64,64), leader’s loss 8, follower’s gain 28
Second-play games result: gains and losses in cooperation
• Reference outcome is the second-play Stackelberg outcome (81,54), payoff difference 27
• Fixed pairs: joint optimum (72,72), leader’s loss 9, follower’s gain 18
• Random pairs: Cournot-Nash (64,64), leader’s loss 17, follower’s gain 10
• In second-play games, the evolution to cooperation is driven by the leader
Comparison to Huck et al. (2001)
Fixed pairs Random pairs
Frequencies of leader choices over all rounds, Stackelberg games with complete information
Sta
ckel
berg
Cou
rnot
-Nas
h
Join
t-op
timum
Explanation for the different leader behavior?
Evolution: Stackelberg fixed pairs, F knows payoffs of L
Convergence to the joint optimum
Vertical axis: number of given outcomes
Evolution: Stackelberg fixed pairs, F does not know payoffs of L
Convergence to both the joint optimum and the Cournot-Nash
Vertical axis: number of given outcomes
Evolution: Stackelberg random pairs, F knows payoffs of L
Cournot-Nash
Vertical axis: number of given outcomes
Evolution: Stackelberg random pairs, F does not know payoffs of L
Cournot-Nash
Vertical axis: number of given outcomes
Evolution: Second-play fixed pairs, F knows payoffs of L
Joint optimum
Vertical axis: number of given outcomes
Evolution: Second-play fixed pairs, F does not know payoffs of L
Joint optimum
Vertical axis: number of given outcomes
Evolution: Second-play random pairs, F knows payoffs of L
Convergence to equal payoffs
Vertical axis: number of given outcomes
Evolution: Second-play random pairs, F does not know payoffs of L
Convergence to equal payoffs
Vertical axis: number of given outcomes
Outcomes from last five rounds
GameFollower better off
Equal payoffs
Leader better off
Questionnaire:Leader
cooperative
Questionnaire: Follower signals
cooperation
Stackelberg fixed, F knows 25,5 % 61,8 % 12,7 % 91 % 91 %
Stackelberg fixed, F does not know 10,0 % 42,9 % 47,1 % 60 % 73 %
Stackelberg random, F knows 48,0 % 18,0 % 34,0 % 55 % 60 %
Stackelberg random, F does not know 6,7 % 57,8 % 35,6 % 78 % 67 %
Second-play fixed, F knows 15,7 % 58,6 % 25,7 % 71 % 86 %
Second-play fixed, F does not know 7,1 % 68,6 % 24,3 % 71 % 71 %
Second-play random, F knows 25,3 % 50,5 % 24,2 % 72 % 56 %
Second-play random, F does not know 27,1 % 48,6 % 24,3 % 32 % 43 %
Average 20,7 % 50,8 % 28,5 %
Questionnaires
• Leaders drive cooperation– In stackelberg games, 10 out of 11 leaders answered that they tried
either to maximize the sum of both payoffs or aim for equal payoffs; in second-play, 8/14 leaders
• Followers signal for cooperation, but if leaders do not notice it, cooperation does not result– In stackelberg games, 10 out of 11 followers answered that with their
choices, they tried to signal the intention to get better payoffs for both; in second-play, 12/14 followers
– In pairs that do not converge to the joint optimum, only 1 leader out of 6 has noticed the follower’s signaling; in second-play 2/7 leaders
• Punishments are used only in pairs which are not cooperating
Summary
• Strong other-regarding behavior: fixed pairs converge to joint optimum, random pairs Cournot-Nash, both result in equal payoffs
• On average 51% of outcomes have equal payoffs
• No Stackelberg outcomes
• Leader drives cooperation in second-play games with cheap talk
• Leader has no threat of loss of payoffs (as in the ultimatum game) and still chooses cooperative strategy, even in random pairs
• Frequencies of cooperation:– Our leaders: 35,5%– Huck leaders: 21%– Typical ultimatum game fair offers: 71% (Fehr and Schmidt 1999)
Neural correlates to other regarding behavior?
• Are there differences in the neural areas activated for self regarding and other regarding players?
• Compare to Sanfey et al. (2003) fMRI observations from receiving unfair ultimatum proposals:– Activation in the bilateral anterior insula, ”emotional goal of
resisting unfairness”– Activation in the dorsolateral prefrontal cortex, ”cognitive goal of
accumulating money”– Activation in the anterior cingulate cortex, ”motivational conflict
between fairness and self-interest”
References• Camerer, C. 2003. Behavioral Game Theory, Princeton University Press.
• Crawford, V. 1998. A survey of experiments on communication via cheap talk. Journal of Economic Theory, Vol. 78.
• Fehr, E. and Schmidt, K.M. 1999. A theory of fairness, competition, and cooperation. Quarterly Journal of Economics, Vol. 114. No. 3.
• Holt, C. 1985. An experimental test of the consistent-conjectures hypothesis. The American Economic Review, Vol. 75, No. 3.
• Huck, S., Müller, W., and Normann, H-T. 2001. Stackelberg beats Cournot: on collusion and efficiency in experimental markets. The Economic Journal, Vol. 111.
• Hämäläinen, R. 1981. On the cheating problem in Stackelberg games. Int J Syst Sci, Vol. 12, No. 6.
• Roth, A., Prasnikar, V., Okuno-Fujiwara, M. and Zamir, S. 1991. Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study. The American Economic Review, Vol. 81, No. 5.
• Sanfey, A.G., Rilling, J.K., Aronson, J.A., Nystrom, L.E., Cohen, J.D. 2003. The neural basis of economic decision-making in the ultimatum game. Science, Vol. 300.
More Huck and Müller et al. references
• Huck, S., Müller, W. and Normann, H-T. 2002. To commit of not to commit: Endogenous timing in experimental duopoly markets. Games and Economic Behavior, Vol. 38, No. 2..
• Huck, S., and Wallace, B. 2002. Reciprocal strategies and aspiration levels in a Cournot-Stackelberg experiment. Economics Bulletin, Vol. 3., No. 3.
• Müller, W. 2006. Allowing for two production periods in the Cournot duopoly: experimental evidence. Journal of Economic Behavior & Organization, Vol. 60, No. 1.
• Müller, W., and Tan, F. 2010. Team versus individual play in a sequential market game. Manuscript, http://www.tilburguniversity.edu/research/institutes-and-research-groups/center/phd_stud/tan/Mueller_Tan.pdf