Introduction Indirect Reciprocity Gossip Results

Cooperation, Reputation & Gossiping

V.A. Traag1, P. Van Dooren1, Y.E. Nesterov2

1ICTEAMUniversite Catholique de Louvain

2COREUniversite Catholique de Louvain

15 April 2011

Introduction Indirect Reciprocity Gossip Results

Outline

1 Introduction

2 Indirect Reciprocity

3 Gossip

4 Results

Introduction Indirect Reciprocity Gossip Results

Cooperation

Cooperation (and defection)

• Organizations (also Wikipedia, open source software, . . . )◮ Why do people contribute?

• Worker ants in colonies◮ Why do workers help without individual benefit?

• Prudent parasites in hosts◮ Why do parasites not replicate faster?

• Human body◮ Why do cells not replicate faster?

Central question

If defecting (not cooperating) is a real option, why (and how) hascooperation evolved?

Introduction Indirect Reciprocity Gossip Results

Formal cooperation (and defection)

Prisoner’s Dilemma

• The game knows two options, donating or not donating.

• Donate at a cost c > 0 to benefit someone else with benefitb > c .

• Agents are paired, and play a round of donating or not.

• Cooperators C donate, defectors D do not donate.

This can be summarized in the payoff matrix

A =

(C D

C b − c −c

D b 0

)

Defectors dominate

Whatever strategy you encounter (C or D), always better to defect.

Introduction Indirect Reciprocity Gossip Results

Dynamical View

• Natural to model game dynamics in an evolutionary context.

• Survival of the fittest (fitness = payoff).

Definition (Replicator equation)

Population with i = 1, . . . , n different mixed strategies pi

xi Relative abundance (frequency)

p =∑

i pixi Average strategy

fi = p⊺

i Ap Expected payoff

f = p⊺Ap Average payoff

Evolution of the population given by

xi = xi (fi − f ) = xi ((pi − p)⊺Ap).

Introduction Indirect Reciprocity Gossip Results

Overview

What are possibly mechanisms to get cooperation?Payoff matrix

A =

(C D

C b − c −c

D b 0

)

Mechanisms

• Kin selection (r > cb)

Cooperate because offspring benefits of your cooperation. Basisof ‘selfish gene’, or ‘inclusive fitness’.

• Direct reciprocity (w > cb)

Cooperate because of possible future payoffs.

• Indirect reciprocity (q > cb)

Cooperate because someone else may cooperate with you in thefuture.

Nowak. Science (2006) vol. 314 (5805) pp. 1560-1563

Introduction Indirect Reciprocity Gossip Results

Some strategies

Example (Always)

Defect/cooperate on all rounds

Other CDDDDCC

AllD DDDDDDD

AllC CCCCCCCC

Example (Tit-for-tat)

Start cooperating, then repeatopponent.

Other CDDDDCC

TFT CCDDDDC

Example (Win-Stay, Lose-Shift)

Change strategy if losing, keep itotherwise.

Other CDDDDCC

WSLS CCDCDCC

Example (Generous Tit-for-tat)

As TFT, but cooperates afterdefection with probability p.

Other CDDDDCC

GTFT CCDDCDC

Introduction Indirect Reciprocity Gossip Results

Insufficient explanation

Why is kin selection and reciprocity not sufficient?

Insufficient explanation

• Humans cooperate also with non-kin.

• Humans cooperate in non-iterative situations.

Indirect reciprocity

• Cooperate if cooperated with others in the past.

• Brings reputation into play.

• How to respond to reputation?

• How to determine new reputation?

Nowak and Sigmund. Nature (2005) vol. 437 (7063) pp. 1291-1298

Introduction Indirect Reciprocity Gossip Results

Image score

Definition (Image score, reputation)

• Integer status −5 ≤ Si ≤ 5 known to all.

• If cooperate increase (with 1).

• If defect decrease (with 1).

Definition (Discriminator Strategy)

• Cooperative threshold −5 ≤ kj ≤ 6.

• If status Si ≥ kj cooperate, otherwise defect.

• Strategy kj = −5 corresponds to AllC.

• Strategy kj = 6 corresponds to AllD.

Nowak and Sigmund. Nature (1998) vol. 393 (6685) pp. 573-577.

Introduction Indirect Reciprocity Gossip Results

Other reputation dynamics

Morals

• Defecting a defector, good or bad?

• What action should be regarded as good?

• When to cooperate, when to defect?

GG GB BG BB

C ∗ ∗ ∗ ∗

D ∗ ∗ ∗ ∗

∗ ∗ ∗ ∗

Reputation of donor and recipientAction of donor

New reputation can beeither Good or Bad

Action can be eitherCooperate or Defect

Introduction Indirect Reciprocity Gossip Results

Some reputation dynamics

GG GB BG BB

C G G G G

D B B B BImage scoring

C G G G G

D B G B BStanding

C G B G B

D B G B BJudging

C G B G B

D B B B BShunning

Introduction Indirect Reciprocity Gossip Results

Leading eight

Best strategies

• In total 2, 048 different possible strategies.

• There are 8 strategies (leading eight) that perform best (highestpayoff, and ESS).

GG GB BG BB

C G ∗ G ∗

D B G B ∗

C D C ×

Maintainance of cooperation

Mark defectors

Punish defectors

Forgive defectors

Apologize

Ohtsuki and Iwasa. Journal of Theoretical Biology (2004) vol. 231 (1) pp. 107-120

Introduction Indirect Reciprocity Gossip Results

Subjective reputation

Subjective reputation

• Unrealistic that everybody knows the reputation of everybody.

• Introduce a subjective (private) reputation.

• ‘Observe’ only a few interactions.

Observing

• Probability q of observing an interaction.

• Cooperation declines with lower q.

• Diverging reputations cause further errors.

• Good may defect bad, but not all agree on who’s bad.

Introduction Indirect Reciprocity Gossip Results

Synchronize reputations

Synchronizing reputations

• Spread local information to synchronize reputations.

• Players ‘gossip’ about each other to share information.

• Start gossip, spread gossip and how to interpret gossip?

Lying, cheating and defecting

• Possibly ‘false’ gossips spread.

• Spread rumours unconditionally allows liars to invade.

• Liars cannot invade conditional rumour spreaders.

Introduction Indirect Reciprocity Gossip Results

Proposed model

Basics

1 Each agent has a reputation of the other: Rij(m)

2 Everybody plays and cooperates/defects based on localreputation

3 Everybody gossips the result of the interaction

4 New reputation Rij(m + 1) based on:◮ Own observation (∆Iij (m)),◮ Gossip (∆Sij (k ,m)).

Decision to cooperate

The decision to cooperate αij(m) =

{0 if Rij(m) < 01 if Rij(m) ≥ 0

Introduction Indirect Reciprocity Gossip Results

Gossiping

Consider all neighbours k when updating the reputation Rij

i j

k

The link tobe updated.

Does i ‘like’ k?

Will k gossip to i?

What actionhas j takento k?

Introduction Indirect Reciprocity Gossip Results

Social strategy

G B

C G B

D B G

Reputation of k , or αik(m).

Action of j , or αjk(m)

Action is considered aseither Good or Bad

Social strategy

• Cooperation vs. good agent and defection vs. bad agent is good

• Change in reputation due to gossiping with neighbour k

∆Sij(k ,m) = αki (m)(2αik(m)− 1)(2αjk(m)− 1)

Introduction Indirect Reciprocity Gossip Results

Individual strategy

C D

C + −

D − +

Action of j , or αji (m).

Action of j , or αij(m)

Action is considered aseither Good or Bad

Individual strategy

• +1 for ‘good’ actions, −1 for ‘bad’ actions to reputation

• We currently study WSLS-like: Consider CC and DD as good.

∆Iij(m) = (2αij(m)− 1)(2αji (m)− 1)

Introduction Indirect Reciprocity Gossip Results

Reputation dynamics

Combine individual & social strategies

Combine with social influence parameter 0 ≤ λ ≤ 1

∆Rij(m) = (1− λ)

Individual strategy︷ ︸︸ ︷

(2αij(m)− 1)(2αji (m)− 1)+

λ1

n − 2

∑

k 6=i ,j

αki (m)(2αik(m)− 1)(2αjk(m)− 1)

︸ ︷︷ ︸

Social strategy

Reputation dynamics

Rij(m + 1) = Rij(m) + ∆Rij(m)

Introduction Indirect Reciprocity Gossip Results

Cooperative fixed points

Fixed point

• For which networks do we have αij(m + 1) = αij(m)?

• Good reputation remains good, bad reputation remains bad

Undirected case

• If αij(m) = αji (m), fixed points are groups

• Cooperate within groups, defect between groups

• Implies it is (weakly) social balanced

• Can have q groups if

λ >q

q + 1

More social influence may lead to more fragmented cooperation.

Introduction Indirect Reciprocity Gossip Results

Evolutionary dynamics

Four different regimes (Cooperate with prob p on first round)

p < 1/2 p > 1/2

λ < 1/2 Individualistic prejudiced

• Defect vs. cooperators

• Cycles of cooperation vs.defectors

Individualistic trusting

• Cooperate vs.cooperators

• Cycles of cooperationvs. defectors

λ > 1/2 Social prejudiced

• Cooperate vs. cooperators(except second round)

• Defect vs. defectors(except second round)

Social trusting

• Cooperate vs.cooperators

• Defect vs. defectors

Introduction Indirect Reciprocity Gossip Results

Phase portraits Individual

C D

Gossipers

Individual PrejudicedC D

Gossipers

Individual Trusting

In ‘friendly’ environment, being individually prejudiced pays off.

Introduction Indirect Reciprocity Gossip Results

Phase portraits Social

C D

Gossipers

Social PrejudicedC D

Gossipers

Social Trusting

In ‘hostile’ environment, being socially trusting pays off.

Introduction Indirect Reciprocity Gossip Results

Conclusions

Proposed model

• Proposed model for gossiping and reputation dynamics

• Interesting possible cooperative network structure

• Evolutionary stable for some parameter range

• More socially oriented strategy could have developed fromindividual strategy

Shortcomings

• Actual convergence to fixed point not investigated

• Characterize directed fixed points

• Evolutionary dynamics investigated in limit of large n

• Interact all-to-all unrealistic, e.g. restrict to graph

• Gossip perhaps passed on further (cascades of gossip)

Introduction Indirect Reciprocity Gossip Results

Thank you for your attention.

Questions?