Towards Realistic Models for Evolution of Cooperation

LIK MUI

… about procedure …

• Briefly go over the paper– Clarify major points

• Describe simulations (not in paper)

RoadMap

• Introduction

• Cooperation Models

• Simulations

• Conclusion

Evolution of Cooperation

• Animals cooperate

• Two questions:

– How does cooperation as a strategy becomes stable evolutionarily?

– How does cooperation arise in the first place?

Darwinian Natural Selection

“Survival of the fittest”

• If evolution is all about individual survival, how can cooperation be explained?

• Fittest what?

Fittest what ?

• Individual– Rational agency theory (Kreps, 1990)

• Group– Group selection theory (Wilson, 1980)

• Gene– Selfish gene hypothesis (Dawkins, 1979)

• Organization– Classic organizational theory (Simon,

1969)

RoadMap

• Introduction

• Cooperation Models• Group Selection• Kinship Theory• Direct Reciprocity• Indirect Reciprocity• Social Learning

• Simulations

• Conclusion

Group Selection

• Intuition: we ban cannibalism but not carnivorousness

• Population/species: basic unit of natural selection

• Problem: explain war, family feud, competition, etc.

Kinship Theory I

• Intuition: nepotism

• Hamilton’s Rule:

– Individuals show less aggression and more cooperation towards closer kin if rule is satisfied

– Basis for most work on kinship theory

• Wright’s Coefficient of Related: r– Self: r=1– Siblings: r=0.5– Grandparent-grandchild: r=0.25

cr

b

Kinship Theory II

• Cannot explain:– Competition in viscuous population– Symbioses– Dynamics of cooperation

Direct Reciprocity

• Intuition: being nice to others who are nice

• “Reciprocal Altruism”– Trivers (1971)

• Tit-for-tat and PD tournament– Axelrod and Hamilton (1981)

• Cannot explain:– We cooperate not only with people who cooperate

with us

Indirect Reciprocity

• Intuition: respect one who is famous

• Social-biological justifications– Biology: generalized altruism (Trivers, 1971, 1985)– Sociobiology: Alexandar (1986)– Sociology: Ostrom (1998)

• 3 types of indirect reciprocity:– Looped– Observer-based– Image-based

Indirect Reciprocity: Looped

• Looped Indirect Reciprocity– Boyd and Richerson (1989)

Indirect Reciprocity: Observers

• Observer-based Reciprocity– Pollock and Dugatkin (1992)

Indirect Reciprocity: Image

• Image (reputation) based Reciprocity– Nowak and Sigmund (1998, 2000)

Social Learning

• Intuition: imitate those who are successful

• Cultural transmission– Boyd and Richerson (1982)

• Docility– Simon (1990, 1991)

Critiques of Existing Models

• Many theories each explaining one or a few aspects of cooperation

• Unrealism of model assumptions

Unrealism for Existing Models

• asexual, non-overlapping generations • simultaneous play for every interaction

– c.f., Abell and Reyniers, 2000

• dyadic interactions• mostly predetermined behavior

– c.f., May, 1987 (lack of modeling stochasticity)

• discrete actions (cooperate or defect)• social structure and cooperation

– c.f., Simon, 1991; Cohen, et al., 2001

• extend social learning– c.f., Simon, 1990

RoadMap

• Introduction

• Cooperation Models

• Simulations• Nowak and Sigmund Game• Prisoner’s Dilemma Game• Simon’s Docility Hypothesis

• Conclusion

Nowak and Sigmund Game

• Payoff Matrix

C = 0.1

B = 1.0

• Image Adjustment

A = 1

Interact

Not interact

Donor -C 0

Recipient

B 0

Interact

Not interact

Donor A -A

Recipient

0 0

Using Global Image: 1 Run

t=0

0102030405060708090

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Ab

un

dan

ce

t=9

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

t=149

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

t=19

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Using Global Image: 100 Runs

average over 50 sim ulations t=0

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

average over 50 sim ulations t=0

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

average over 50 sim ulations t=19

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

average over 50 sim ulations t=999

0

10

20

30

40

50

60

70

80

90

100

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Dynamics using Global Reputation

50,000

-6

-4

-2

0

2

4

6

8

Number of Generations

Str

ateg

y, K

50,000

0

0.5

1

1.5

2

2.5

Number of Generations

Pay

off

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3 4 5 6 7 8 9 10 11 12

Strategy, K

Fre

qu

ency

Using 10 Observers/Interactions

n=20

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Strategy, K

Fre

qu

ency

n=50

0

0.05

0.1

0.15

0.2

0.25

0.3

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Strategy, K

Fre

qu

ency

n=100

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Strategy, K

Fre

qu

ency

n=200

0

0.05

0.1

0.15

0.2

0.25

0.3

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Strategy, K

Fre

qu

ency

Evolutionary PD Game

• Repeated Prisoners’ Dilemma Game

• Agent Actions:Action = { cooperate, defect }

• Payoff Matrix:C D

C 3/3 0/5

D 5/0 1/1

PD Game Agent Strategies

• All defecting (AllD)

• Tit-for-tat (TFT)

• Reputational Tit-for-tat (RTFT): using various notions of reputation

Base Case: PD GameGroup Reputation (base: min_gr >= 0)

0

20

40

60

80

100

120

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49

Generation

TF

T C

ou

nt10000 12000 12100 12200 12500

Simple Groups: social structures

• Group structure affects members– Interactions, observations, and knowledge– Persistent structure

• Groups actions– Observed indirectly through member's

actions

Group Membership

• Member agents– Have public group identity– Directly associated with one environment

• Group Structure is a Tree– Least common ancestral node (LCAN)– Events occur with respect to a shared

environment

Shared Environment Example

Agents Group

A1,A2 G1

A3,A4 G2

A5,A2 G1

A1,A3 G0

A5,A3 G0

A1

A2 A4A3

A5

A0

G0

G1 G2

G3

PD Game with Group Reputation(varying encounters per generation EPG)

Group Reputation (min_gr >= 0.5)

0

20

40

60

80

100

120

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49

Generation

TF

T C

ou

nt

100 200 500 1000 1200

PD Game with Group Reputation (100 EPG; varying Inter-group interaction probability)

Group Reputation (min_gr >= 0.5, varying ip)

0

20

40

60

80

100

120

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49

Generation

TF

T C

ou

nt

0.1 0.3 0.325 0.35 1.0

Groups/Organizations: bounded rationality explanation

• Docility– Cooperation (altruism) as an explanation for the

formation of groups/organizations

• Why individuals “identify” with a group?– boundedly rational individuals– increase their survival fitness

(Simon, 1969, 1990, 1991)

PD Game with Docility(50 cooperators and 50 defectors; 100 EPG; 1.0 IP)

Varying intergroup docility, intragroup docility = 1.0

0

20

40

60

80

100

120

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49

Generation

Co

op

erat

or

Co

un

t

0.0 0.4 0.41 0.425 1.0

Conclusion

• Reviewed 5 major approaches to study evolution of cooperation

• Provided 2 main critiques for existing models

• Constructed model extensions addressing the critiques

Implications for Computer Science

• Artificial intelligence– Benevolent agents are not good enough

(c.f., multi-agents systems)– Learning theory can be used to study evolution of

cooperation

• Systems– Improve system design by understanding the

dynamics of agents– Accountability substrate needed for distributed

systems

Future Plan

• Extend the simple group social structure

• Overlapping generations

• Sexual reproduction

• Extend social learning using realistic/robust learning model

Modeling Diploid Organisms

Modeling Diploid Organisms

Modeling Diploid Organisms

Parental Chromosomes One of 2 Child Chromosomes

Simulation Demo

• Recall PD payoff matrix: C D

C R/R S/T

D T/S P/P

• PD strategies: viewed as a probability vectors– Strategy: <PI, PT, PR, PP, PS>

– TFT: < 1, 1, 1, 0, 0 >– AllD: < 0, 0, 0, 0, 0 >– AllC: < 1, 1, 1, 1, 1 >– STFT: < 0, 1, 1, 0, 0 >

Simulation: a search problem

• Search Optimal PD Strategy– Search space: I, T, R, P, S probabilities