Advances in Evolutionary Games
description
Transcript of Advances in Evolutionary Games
![Page 1: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/1.jpg)
E. Altman INRIA, France
Advances in Evolutionary Games
BioneticsBioneticsDcember 2010
![Page 2: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/2.jpg)
Overview of the talk
1. Background on Evolutionary Games and Population Games and Examples
2. Adding time varying states
3. Adding controlled state transitions
4. Examples
5. Mathematical model
6. Computing Equilibria
![Page 3: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/3.jpg)
EGs and PGs in Biology and Engineering
BIOLOGY CONTEXT: Central tool defined by Meynard Smith (1972) for explaining and predicting dynamics of large competing populations with many limited local interactions. EG.
TELECOM CONTEXT: Competition between protocols, technologies. Can be used to design and regulate evolution
ROAD TRAFFIC CONTEXT: Competition between cars over routes. Introduced by Wardrop (1952). PG.
![Page 4: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/4.jpg)
Framework
Large population Several strategies (behavior of individuals). Call
all those who use a strategy a subpopulation EG: Competition between the strategies through
a very large number of interactions each involving a small number of individuals typically pairwize interactions
PG: interactions with an infinite number of players
![Page 5: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/5.jpg)
Evolutionary Games: Definitions
Evolutionary Stable Strategy (ESS): At ESS, the populations are immuned from being invaded by other small populations (mutations).
ESS more robust than standard Nash equilibrium.
“State” vector X: fractions of users that belong to different populations. Or fractions of strategies in a population
Fitness: J(p,q):= utility when playing pure strategy p and all others play q. J(x,y) fitness when using mixed strategies x and y, resp.
![Page 6: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/6.jpg)
Evolutionary Stable StrategyEvolutionary Stable Strategy (ESS):
p is ESS if for all q there is d(q) s.t. for all 0<e<d(q)
J(p,p) >= (1-e) J(q,p) + e J(q,q)
Equivalent condition:
J(p,p) > J(q,p) or
J(p,p) = J(q,p) and J(q,q) < J(p,q)
![Page 7: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/7.jpg)
Ex 1: Hawk and Dove Game
Large population of animals. Occasionally two animal find themselves in competition on the same piece of food. An animal can adopt an aggressive behavior (Hawk) or a peaceful one (Dove).
D-D: peaceful, equal-sharing of the food. fitness of 0.5 to each player.
H-D or D-H: 0 fitness to D and 1 for H that gets all the food no fight
![Page 8: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/8.jpg)
HD Game
H-H: fight in which with equal chances to obtain the food but also to be wounded. Then the fitness of each player is 0.5-d, -d is the expected loss of fitness due to being injured.
![Page 9: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/9.jpg)
Modeling competition: Generalized HD Game
•Generalized game: A11<A22<A12 and A21<A22.
•Simple conditions for H to be unique ESS and for mixed ESS
![Page 10: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/10.jpg)
Ex 2: Competition between protocols
There are various flow control protocols to regulate traffic in the Internet.
Huge number of file transfers every second Interactions occur between limited number of
connections that use the same bottleneck link The average speed of transfer, the delay etc
depend on the versions of the protocol involved in the interaction
![Page 11: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/11.jpg)
Competition between protocols
![Page 12: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/12.jpg)
Ex 3: Population Games (PG) in Wireless communications
• Cellular network contains many mobiles. One base station (BS) per cell
•CDMA: At each time an individual sends a packet it interacts with all mobiles in the same cell
•A mobile can transmit with different power levels q1 < … < qK. Higher power is more costly
•Objective of : max_k J(k,w) :=
where wk is the fraction of mobiles that use qk
![Page 13: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/13.jpg)
Replicator Dynamics
Delayed case: present growth rate depends on past fitness
K and tau : design parameters. Determine speed of convergence and stability
![Page 14: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/14.jpg)
Architecting evolution: impact of K
stability iff K tau< θ. Oscillations mean no convergence to ESS.
![Page 15: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/15.jpg)
Individual States in EG and PG
Different behaviors may be a result of different inherent characteristics – individual states
Example: weather conditions, age, The individual state can be random Description through a Markov chain EG: Local interactions with players chosen at
random; their state is unknown PG: Global interactions, the state can be known
![Page 16: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/16.jpg)
Indiv. states in HD Game
The decisions H or D determine whether a fight will occur
There is also a true identity -- Strong or Weak We call this the individual STATE
If there is a fight then the states determine the outcome.
Note: the decision H/D are taken without knowing the state of the other.
![Page 17: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/17.jpg)
Indiv. States in Networks
Flow control protocol: large end to end delay slows the protocol and decreases its throughput
Wireless:
- the power received may depend on the radio channel conditions
- the transmitted power may depend on the energy level of the battery
![Page 18: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/18.jpg)
MDEG: Markov Decision EG ASG: Anonymous Sequential G
Each player has a controlled Markov chain (MDP)
A player has finite or infinite life time. It has several interactions each time with another randomly selected player (MDEG) with a large population (ASG)
Each interaction results in an immediate fitness that depends on the actions and states of the players involved
The states and actions of a player determine also the probability distribution of the next state
![Page 19: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/19.jpg)
Assumptions, References
A player maximizes the total expected or average fitness
EG average fitness: EA & YH IEEE trans Autom Contr, June 2010 (theory) EG total expected fitness: Infocom 2008 (power control) Evolutionary Ecology Research, 2009 (the theory) (EA, YH, R El-Azouzi, H. Tembine) SAG: Jovanovic & Rosenthal, J Math. Econ, 1988 (disc cost)
Assume: The transition probabilities of the MDP of a player depend only on its own actions and states
![Page 20: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/20.jpg)
Ex 1 (MDEG): Hawk and Dove game
A bird that looses becomes weaker (less energy)
A very weak bird dies
State: Energy level
Would a weaker bird be more or less aggressive?
If the result of the fight are determined by the energy level then the transitions are determined by states and actions of both birds.
![Page 21: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/21.jpg)
Ex 2: (MDEG or ASG): Battery dependent power control
Transmitting at higher power empties faster the battery
A battery with little energy left is not able to support transmissions at high power
The state: remaining energy in the battery
The transitions do not depend on other mobiles
![Page 22: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/22.jpg)
Ex 3: channel dependent power control
The decision to transmit at power qk may depend on the channel state
Seems “degenerate”: the mobile does not control the transitions
Restriction: discrete power set; if a power level is chosen then the next power cannot differ by more than one unit.
This creates non-trivial transitions. The state = (Channel state, current power level)
![Page 23: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/23.jpg)
MDEG: Local interactions
Each local interaction is described by a stochashtic game with partial monitoring
The stochastic game has an equilibrium. The game is equivalent to a matrix game where the pure actions of a player are its pure stationary policies
Allows us to transform the problem into a standard EG with a huge action space (action=pure policy)
We show: equivalence to a polytope game in the space of marginal stationary occupation measures Cardinality: no. of states times number of actions
![Page 24: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/24.jpg)
Model of Individual player
Each player is associated with a MDP with
POWER CONTROL ASG
![Page 25: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/25.jpg)
The model for an individual
•State of an individual corresponds to the battery level.
•Set of actions available at state s:
• Qs decreases with the energy: smaller powers are available when the battery has less energy
•Transitions: the probability to stay at a state s if q is used is
•Recharging: P0N is the probability to move from 0 to N
![Page 26: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/26.jpg)
The model: Interactions
Global state: fraction of mobile in each individual state:
Proportion of mobiles using qk at time t is The Reward:
Stationary policy the ptob to choose qk in s
![Page 27: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/27.jpg)
Interactions and System model
In stationary regime:
![Page 28: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/28.jpg)
Interactions and System model
The expected reward:
A stationary policy u is an equilibrium if
![Page 29: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/29.jpg)
Results (1)
Define the interference of u:
Denote the probability distribution with mass 1 at q
![Page 30: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/30.jpg)
Results (2)
Threorem. An equilibrium exists within
![Page 31: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/31.jpg)
Results (3)
![Page 32: Advances in Evolutionary Games](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815a93550346895dc80c6f/html5/thumbnails/32.jpg)
Future work
Branching MDPs: a state-action pair of an individual determines the immediate fitness, the transition probabilities and the number of off-springs