Heterogeneous Payoffs and Social Diversity in the Spatial Prisoner’s Dilemma game Dept Computer...
-
Upload
kiara-wheelock -
Category
Documents
-
view
215 -
download
0
Transcript of Heterogeneous Payoffs and Social Diversity in the Spatial Prisoner’s Dilemma game Dept Computer...
Heterogeneous Payoffs and Social Diversity in the Spatial
Prisoner’s Dilemma game
Dept Computer Science and Software Engineering
Golriz RezaeiDr. Michael Kirley
SEAL08 Conference8 Dec 2008
Evolution of cooperation
• Open ended question in many areas• Evolutionary Computation (IEEE Trans, CEC, GECCO)• Autonomous agents and multi agent systems (AAMAS)• Distributed Artificial Intelligence (DAI)• Physics (Statistical Physics)• Biology (Theoretical biology, Nature)
• Prisoner’s Dilemma (PD game)
• Different individual conditions (Heterogeneity) have impact
• In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game.
Good abstract Game theoretic approach Mathematical model Applied in many areas (biology, economics, and sociology)
Today’s Agenda
• Brief overview of Prisoner’s Dilemma game and different variations
• The challenge and related works• Proposed model• Evaluation by experiments• Conclusion• Questions
Prisoner’s DilemmaC
cooperateD
Defect
Ccooperate
R=3 R=3
T=5 S=0
DDefect
S=0T=5
P=1P=1
• 2 players / agents • 2 choices (C or D)• Actual values order
• Order change game change
• (D,D) Nash Equilibrium But
i) T > R > P > Sii) 2R >= (T + S)
Iteration reciprocal interaction Spatial local neighbourhood
Spatial Prisoner’s Dilemma• Limited to local neighbourhood interaction only• Accumulates received payoffs from games fitness• At the end of each round selection process
imitation of the most successful neighbour (MSN)• Clusters of cooperators
outweigh losses against defectors
1 2 3
4 5
6 7 8
The Challenge
• Typically
“Universal fixed payoff matrix”
• Hypothesis
Introducing “social diversity” alters trajectory of the population.
Related work
• Few studies investigated the impact of varying the magnitude of the payoff matrix values
1. Tomochi and Kono [Physical Review E 2002]:• Payoff matrix evolved based on the ratio of defectors
(considered R and P only) - Universal payoff matrix
2. Perc and Szolnoki [Physical Review E 2008]:• Random noise added to the individual payoff matrix at the
beginning of the game - Fixed matrix till the end
3. Fort [Physica A 2007]:• The payoff matrix was correlated with a spatial and temporal
zones (considered only T) - The Prisoner’s Dilemma inequality was relaxed.
Proposed model
• Idea Associated payoffs evolve based on individual experience.
• Each agent
• Dynamic payoffs each agent has its own version of payoff matrix and it gets updated at each time step based on the level of the agent’s experience
Age increases at each time step αi(t+1) = αi(t) + 1
Life-span expected life time (λi) randomly drawn from a uniform distribution αi(t) == λi die and replaced by a new random agent
Proposed model
Update
Where is the payoff values for agent i at time t is the default payoff matrix values T, R, P, S is the magnitude of the rescaled values is the age of agent i at time t is the expected life time of agent i is limiting factor and characterises the uncertainty related to
the environment
1)
2)
Three scenarios
1. Standard PD universal fixed Payoffs no Age
2. Homogeneous model universal fixed Payoffs Age
3. Heterogeneous model individual Payoffs Age
• What is the equilibrium state?
Experimental Setup
• Implemented in Netlogo4.0 [Wilensky 1999]• Underlying framework Standard Spatial
Iterated Prisoner’s Dilemma. Agents mapped on 2-D regular lattice (32*32 torus)
• Population initialized 20% cooperators• Each trial 1000 iterations• All configurations 30 times• Statistical results are reported
Experiment 1 sensitivity to the base payoff values
• Two different base level payoff values
T, R, P, S and K = 0.2
a) Big 5, 3, 1, 0
b) Small 1, 1, 0, 0
Experiment 2 sensitivity to the magnitude of K
• base level payoff values T, R, P, S 5, 3, 1, 0
• K was changed systematically
• K represents environmental constraint on social diversity
Snapshots
• Evolving population for homogeneous and heterogeneous model
• K = 0.1 and initial cooperation 20%• Varying size clusters of cooperators (black)
Homogeneous
Heterogeneous
Conclusion
• Results heterogeneous social diversity, promotes cooperation.
• Differences to previous work each agent is equipped with their own evolving payoff matrix. The evolving payoff matrix agents’ age or experience level.
• More realistic approach real world scenarios.
• Future work extend the model to distributed multiagent systems (P2P, MANET)
Questions?
Thank you
References
• H. Fort, On evolutionary spatial heterogeneous games, Physica A (2007).
• M. Perc and A. Szolnoki, Social diversity and promotion of cooperation in spatial prisoner's dilemma game, Physical Review E 77 (2008).
• M. Tomochi and M. Kono, Spatial prisoner's dilemma games with dynamic payoff matrices, Physical Review E 65 (2002), no. 026112.
• Wilensky, U.: NetLogo is a cross-platform multi-agent programmable modeling environment. In: Modeling Nature’s Emergent Patterns with Multi-agent Languages. Proceedings of EuroLogo 2002 (2002),http://ccl.northwestern.edu/netlogo/
Experiment 3 sensitivity to the life span (λ)
• base level payoff values T, R, P, S 5, 3, 1, 0• K = 0.2• λ from different range
Experiment 4 sensitivity to the replacement strategy
• base level payoff values T, R, P, S 5, 3, 1, 0• K = 0.2• Replacement with
random generated agent
and defector agent
Related work
• Few studies have examined the impact of varying the magnitude of the payoff matrix values in PD
• Tomochi and Kono:• Payoff matrix was designed to evolve based on the ratio of
defectors (cooperators) to the whole population. (considered R and P only)
• Universal payoff matrix applicable to all agents at time t. • The level of cooperation within population was directly related to
the payoff matrix values
Related work …
• Perc and Szolnoki:• Random noise drawn from alternative statistical
distributions was added to the payoff matrix at the beginning of the game. (fixed matrix till the end)
• They concluded that this correlated “social diversity mechanism” promoted higher-levels of cooperation in the spatial game examined.
• It was suggested that variable social status might play a crucial role in the evolution of cooperation.
Related work …
• Fort:• The payoff matrix was correlated with a spatial and temporal
zones. (considered only T) • It was possible that the payoffs for an agent and their opponent
were not equal – reminiscent of what happens in general in real life.
• The results reported suggested that the effect of asymmetries in the interactions between agents, which takes into account the effect of asymmetries in the costs and benefits on the evolution of cooperation, had a direct impact on the proportion of agents cooperating in the population.
• The Prisoner’s Dilemma inequality was relaxed, and when the payoff matrix values changed, the game oscillated between the Prisoner’s Dilemma game and Chicken game or the game becomes Stag Hunt game.
What is the idea?
• Ex./ You and your friend, colleague• Ex./ 2 countries punishment system for the same crime.
• Different individual conditions (Heterogeneity) have impact on the behaviour of two people/agents and may alter their interaction and their cooperation.
• In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game.
• Why?
Good abstract many real world scenarios. Famous game theoretic approach capture agents interaction Mathematical model study the evolution of cooperation Applied in many areas biology, economics, and sociology