CR4 – Ecole Polytechnique Fédérale de Lausanne (EPFL) ex-Université de Lausanne (UNIL)
Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière...
-
Upload
brendan-burns -
Category
Documents
-
view
215 -
download
2
Transcript of Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière...
![Page 1: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/1.jpg)
Daniel Ariosa
Ecole Polytechnique Fédérale de Lausanne (EPFL)Institut de Physique de la Matière Complexe
CH-1015 Lausanne, Switzerland
and
Hugo Fort
Instituto de Física, Facultad de CienciasUniversidad de la República
Montevideo, Uruguay
Statistical MechanicsStatistical Mechanics Applied to Social Sciences Applied to Social Sciences
________________________________________________________________________
![Page 2: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/2.jpg)
Introduction
Estimating Utilities: Magnetic Systems and Games People Play
... and Adaptive Self-Interested Agents
Extended Estimator Approach for 2x2 Games and its Extended Estimator Approach for 2x2 Games and its Mapping to the Ising HamiltonianMapping to the Ising Hamiltonian
![Page 3: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/3.jpg)
Outline
Self-organization into cooperative equilibrium states
The extended estimator formulation
Mapping the iterated game into an Ising model
Classifying Markovian Strategies
Stability of cooperation-Asynchronous random dynamics-Synchronous fully connected system
Thermodynamics of Ising mappable strategies
Discussion
![Page 4: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/4.jpg)
Self-organization into cooperative equilibrium states
How populations of self interested agents cooperate, or manage in order to
satisfy this goal globally or collectively ?
A few examplesA few examples::
- electrons in a superconductor
- local magnetic moments in a ferromagnet
- molecules that cooperate to form cells, cells that cooperate to form
living creatures that in turn cooperate to form societies ...
Different approachesDifferent approaches (paradigms and extremal principles)::
- Biology Darwin’s evolution fitness maximization
- Economics « Homo economicus » profit maximization
- Physics - Physics statistical thermodynamics statistical thermodynamics free energy minimization free energy minimization
![Page 5: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/5.jpg)
Game theory and the Prisoner’s dilemma
- The Prisoner’s dilemma models the social behavior of "selfish" individuals
- 2x2 game in normal form:
i) 2 players, each confronting 2 choices : to cooperate (C) or to defect (D)
ii) each player makes his choice without knowing what the other will do
iii) there is a 22 matrix specifying the payoffs of each player for the 4 possible outcomes:
M R S
T P
T > R > P > S
2R > S + T
With the condition
and
![Page 6: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/6.jpg)
Elementary Markovian strategies and estimates (aspiration levels) for iterated games_____________________________________________
S
P
R
T
C ; D
D ; D
C ; C
D ; C-The player updates its behavior (C or D) according to the outcome of the preceding run.
- A simple updating rule (= strategy) consists in comparing the obtained utilities () with a
given estimate ()for the expected income.
- Example: The “PAVLOV” strategy
Behavior: “win-stay, lose-shift”
Estimate: P < < R
Character: ( pR
( 1
pT
0
pS
0
pP)
1 )
![Page 7: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/7.jpg)
-more examples:“Retaliator” strategy:
-Behavior: retaliates when the other player defects
- Estimate: S < < P
- Character
“Tit-for-Tat” strategy:
-Behavior: cooperate on the first move an then reproduce what the other player did
In the preceding move.
- Estimate (conditional):
S < S* ; P > P* ; R > R* ; T < T*- Character: ( pR
( 1
pT
1
pS
0
pP)
0 )
( pR
( 1
pT
0
pS
0
pP)
0 )
S
P
R
T
C ; D
D ; D
C ; C
D ; C
S*
T*
R*
P*Retaliator
![Page 8: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/8.jpg)
The extended estimator formulation
- Behavior (state): “SPIN”:
-Estimate Estimator payoff matrix(EPM):
- Updating rule: player(i) FLIPS
M* R * S *
T * P *
c
1 for C
0 for D
Sc
1 c
i SiTMSj
i Si
TM S j
i i
![Page 9: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/9.jpg)
Mapping the iterated game into an Ising model
______________________________• Story line: two-valued variable Ising spinIsing spin
updating rule for c Metropolis Metropolis algorithm algorithm (T=0)
i
12
if agent i is in the C state
1
2if agent i is in the D state
; with i ci 1
2
Flipping condition:
i i i i ; j E i E i
![Page 10: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/10.jpg)
H J i ji , j h i
i
The Ising Hamiltonian:
E i E i Ei 2 Jz nn h i
Energy density associated with the flip:
i i ; nn Ei
The link:
![Page 11: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/11.jpg)
Explicit form of the utilities:
i SiT MSj R S T P cic j S P ci T P c j P
- when playing against a single player (j):
- when playing against z nearest neighboring (nn) players:
i SiT MSnn Si
TM z Snnnn
- In terms of the Ising variables:
i R - S - T + P i j 1
2R + S - T - P i +
1
2R - S + T - P j +
1
4R + S + T + P
i z R - S - T + P i nn z
2R + S - T - P i +
z
2R - S + T - P nn +
z
4R + S + T + P
and
![Page 12: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/12.jpg)
The mapping:Ei 2 Jz i nn 2h i
i z R - S - T + P i nn z
2R + S - T - P i
+ z
2R - S + T - P nn +
z
4R + S + T + P
1 S * S = P P *
2 R * RT T *
i i ; nn Ei R *+T*R + T and S *+P* S + P
J = 2 1 and h z
21 2
S
P
R
T
C ; D
D ; D
C ; C
D ; C
1
2
![Page 13: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/13.jpg)
Classifying Markovian strategies
![Page 14: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/14.jpg)
Stability of cooperation__________________________________
• Average cooperation:
Fraction c of agents in the C-state
a) Steady state cooperation (asymptotic): c*
b) Ground state cooperation (equilibrium at T=0): ceq
Synchronous Dynamics:
All agents simultaneously update their states in one round.
Asynchronous Dynamics:The update is carried out by the subset of agents who just played.
ceq 1
2 i0
![Page 15: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/15.jpg)
c c c 1 pR 1 c 1 pS 1 c cpT 1 c pP 0
A pair of players is randomly chosen for each round.
• Stability in Asynchronous Random Dynamics (ARD):
Example for the PAVLOV (1 0 0 1)strategy:
c 1 c 1 c 1 c c 1
2 or c 1
Only c* = 1/2 is a stable solution (for all but one cooperating agents, the system is rapidly driven away from c = 1.)
![Page 16: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/16.jpg)
Asynchronous Random Dynamics (ARD):
![Page 17: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/17.jpg)
• Stability in the Synchronous Fully Connected System (SFC):
A) The FES case
The equilibrium sate strongly depends upon the initial configuration.
Obtained utilities as a function of c:
i SiT M Snn Si
T M z Snn N 1 c R 1 c S ; for Si Cc T 1 c P ; for Si D
nn
• A stable configuration is reached when all players get a payoff
greater than the estimate or, in other words, when is lower than the cooperator’s utilities.
• Marginal stability is reached also when all players defect and is
lower than the defector’s utilities.
![Page 18: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/18.jpg)
Phase diagram for the FES case
FROZEN
S
P
R
T
0 1c0
0 <-> 1OSCILLATING
OSCILLATING
c* = 1
RET
PAV
AMB
ALT
FR
c* = 0
c* = 1- c0
co <-> 1- co
c* = c0
0 <-> 1OSCILLATING
![Page 19: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/19.jpg)
CON AD
AC TFT
c* = 1
c* = 0c0
1
2
Phase diagram for the steady statesteady state cooperation in IMS
![Page 20: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/20.jpg)
Thermodynamics of Ising-mappable strategies
H J i ji , j h i
i
J
2N2 2 N i
2 hN
NJ
2N 2 h
J
8
U
F NJ
2N 2 h
J
8
F N 2
22 1 2 1 2
Fully connected system(z=N-1):
Free energy functional in terms of mapping parameters:
![Page 21: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/21.jpg)
Mean field approximation:
HMF J iz jij h i
i zJ h
| |
i i
˜ h
ZMF Tr exp HMF 2 cosh ˜ h
2
N
1
N
dFMF
dh
1
Nd ln ZMF
dh
1
2tanh
zJ h
2
Partition function:
Average magnetization:
![Page 22: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/22.jpg)
ceq = 0
ceq = 1
ceq =
1
1
2
2
1CON AD
AC TFT
Phase diagram for the ground stateground state cooperation in IMS
![Page 23: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/23.jpg)
Summary• All the relevant elementary Markovian strategies for the Prisoner’s
dilemma have been formulated in terms of an extended (conditional) estimate.
• Another subset (AD ; TFT ; CON ; AC) has been mapped on an Ising Hamiltonian with 2 parameters.
• The remaining (7) strategies can also be formulated in terms of estimators involving more than two parameters (3, 4).
• A straightforward application of the thermodynamic approach of IMS consists of finding the ground state cooperation of complex systems of interacting agents, which is clearly different from the steady state of the iterated game. Finite temperature leads to “generous” strategies allowing more efficient equilibrium states in its “fitness” landscape.
• A subset of these strategies (FR ; RET ; PAV ; AMB ; ALT) admits a fixed (single) estimate.
![Page 24: Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto.](https://reader038.fdocuments.us/reader038/viewer/2022110211/56649efd5503460f94c11804/html5/thumbnails/24.jpg)
Future work:The mapping on Hamiltonian systems can be exploited
in many ways:
- Replacing the two valued state [C or D] by a continuous variable (more rich systems as the XY model)
- Evolution (Darwinian) and learning models:
Instead of cooperation, consider the strategy (eg. the FES *) as the site variable (order parameter?). Within this context, thermal fluctuations represent spontaneous “mutations” and the Boltzman energy factor will operate the selection.
- Heterogeneous estimates varying from site to site (eg. spin glass model)