The Globus Grid Programming Toolkit: A User-level Tutorial The Globus Project Team ANL and USC/ISI .
Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE...
-
date post
21-Dec-2015 -
Category
Documents
-
view
218 -
download
3
Transcript of Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE...
Oblivious AQM and Oblivious AQM and Nash EquilibriaNash EquilibriaD. Dutta, A. Goel and J. Heidemann
USC/ISI USC USC/ISI
IEEE INFOCOM 2003 - The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies
Presented By Sharon MendelGame Theory in Networks Seminar 25/01/2006
Oblivious AQM & Nash Equilibria 3
Active Queue Management ??
A congestion control protocol (e.g. TCP) operates at the end-points and uses the drops or marks received from the AActivective Q Queueueue MManagementanagement policies (e.g. Drop-tail, RED) at routers as feedback signals to adaptively modify the sending rate in order to maximize its own goodput.
Before we Begin…
Oblivious AQM & Nash Equilibria 4
Transport Control ProtocolTCP is the dominating transport layer protocol in the Internet and accounts for over 90% of the total traffic. The TCP Protocol is well defined, robust and congestion-reactive (thus stable).The end-to-end congestion control mechanisms of TCP have been a critical factor in the robustness of the Internet.It is widely believed that if all users deployed TCP, networks will rarely see congestion collapses and the overall utilization of the network will be high.
Before we Begin…
Oblivious AQM & Nash Equilibria 5
Today’s InternetThere are indications that the amount of non-congestion-reactive traffic is on the rise. Most of this misbehaving traffic does not use
TCP. e.g. Real media, network games, other real time
multimedia applications.
The unresponsive behavior can result in both unfairness and congestion collapse for the Internet.The network itself must now participate in controlling its own resource utilization.
Before we Begin…
* Some of the previous slides:“Promoting the Use of End-to-End Congestion Control in the
Internet”,S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7
1999.
Oblivious AQM & Nash Equilibria 6
The Paper’s MotivationTCP (and in fact, any transport protocol) does not guarantee good performance in the face of aggressive, greedy users (who are willing to violate the protocol to obtain better performance). Protocol Equilibrium – A protocol which leads to an efficient utilization and a somewhat fair distribution of network resources (like TCP does), and also ensure that no user can obtain better performance by deviating from the protocol.If protocol equilibrium is achievable, then it would be a useful tool in designing robust networks.
Before we Begin…
Oblivious AQM & Nash Equilibria 8
Oblivious AQMOblivious (stateless) AQM scheme – a router strategy that does not differentiate between packets belonging to different flows.Stateful schemes – e.g. Fair-Queuing.Stateful Schemes offer good performance, but oblivious schemes are easier to implement.
More Introduction
Oblivious AQM & Nash Equilibria 9
Popular AQM Schemes (1)Congestion avoidance is achieved through packet dropping.
1. Drop-Tail – Buffers as many packets as it can and drops the ones it can't buffer:
Distributes buffer space unfairly among traffic flows.
Can lead to global synchronization as all TCP connections "hold back" simultaneously, hence networks become under-utilized.
More Introduction
DDTT
Oblivious AQM & Nash Equilibria 10
Popular AQM Schemes (2)2. RED – Random Early Dedication - Monitors
the average queue size and drops packets based on statistical probabilities:
If the buffer is almost empty, all incoming packets are accepted; As the queue grows, the probability for dropping an incoming packet grows; When the buffer is full, the probability has reached 1 and all incoming packets are dropped.
Considered more fair than tail drop - The more a host transmits, the more likely it is that its packets are dropped.
Prevents global synchronization and achieves lower average buffer occupancies.
More Introduction
RREEDD
Oblivious AQM & Nash Equilibria 11
Oblivious AQM and Nash Equilibria
The paper studies the existence and quality of Nash equilibria imposed by oblivious AQM schemes on selfish agents:
1. Existence 2. Efficiency3. Achievability
More Introduction
Oblivious AQM & Nash Equilibria 12
ContentIntroduction
The ModelExistenceEfficiencyAchievabilitySummary and Future Work
Oblivious AQM & Nash Equilibria 14
The Internet GamePlayers – The end-to-end selfish traffic agents. Each player has a strategy which is to control the
average rate he tries to push through the network. Users’ Performance Metric – goodput.
Rules – set by the AQM policies (AQM schemes in routers).Nash Equilibrium – No selfish agent has any incentive to unilaterally deviate from it’s current state.Oblivious AQM scheme leads to a protocol equilibrium only if it imposes a Nash equilibrium on the selfish users.
Paper’s focusPaper’s focus – AQM schemes that guarantee bounded average buffer occupancy regardless of the total arrival rate.
The Model
Oblivious AQM & Nash Equilibria 15
The Markovian Internet Game
The agents generate Poisson traffic. Does not accurately model Internet traffic, yet a
reasonable first step. Each user controls its own offered load = the
average Poisson traffic rate.
The system is modeled as a M/M/1/K queue: Average service time - Without loss of generality
assumed to be unity. Buffers capacity - K=B.
No assumptions are made on the selfish protocol (i.e. TCP, AIMD etc’).
The Model
Oblivious AQM & Nash Equilibria 16
The M/M/1/K Internet Game
n – Number of users – players.λi – The Poisson average arrival rate of player i.
Ui – Utility function of player i.
μi – Goodput , Ui = μi .p – AQM router drop probability due an average aggregated load (offered load) of λ and an average service time of unity.A symmetric Nash equilibrium - Ensures that every agent has the same goodput at equilibrium. Unless mentioned otherwise, quantities such as the rates, goodput and throughput are averages (Poisson traffic sources).
The Model
Oblivious AQM & Nash Equilibria 17
Nash Equilibrium Conditions (1)1. No agent can increase their goodput, at Nash
equilibrium, by either increasing or decreasing their throughput:
2. A symmetric Nash equilibrium:
3. Oblivious AQM scheme, hence functions of router state (drop probability, queue length) are independent in i:
nandji jiji
,
The Model
0
i
ii
d
di
i
nd
p
dp
1
Nash Equilibrium Condition :Nash Equilibrium Satisfying Condition –Necessary and sufficient
pU iii 1Utility Function in Nash Equilibrium :
RREEDD
DDTT
Oblivious AQM & Nash Equilibria 18
Nash Equilibrium Conditions (2)
λNash , μNash , pNash - The aggregate throughput
(offered load), goodput, drop probability respectively.
The Nash equilibrium imposed by an AQM scheme is efficient if the goodput of any selfish agent is bounded below when the throughput (offered load) of the same agent is bounded above:
2
11
c
cpNash
NashNash
pNash is bounded
c1 c2 some constants
The Model
Nash Equilibrium Efficiency Condition :
Existence
Are there oblivious AQM schemes that impose Nash equilibria on selfish users?
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 20
Drop-Tail Queuingp - Drop Probability = Probability to find a full system.
From Queuing Theory:
Theorem 1: There is NO Nash Equilibrium for selfish agents and routes implementing Drop-Tail queuing.
Proof: applying the condition for Nash equilibrium:
11
1
B
B
p
Existence
ii
iii ppU 11
00
iii
ii
i
QED.
2
ii
i
11
11
B
B
p
B
21
11
Oblivious AQM & Nash Equilibria 21
Random Early DetectionApproximated steady state RED model:
From Queuing Theory, at steady state:RED Router, at steady state:
Steady State:
thth
thq
pl
minmaxmin max
p =
0
1
If lq <minth
If minth lq maxth
Otherwisw
“Faster network simulation with scenario pre-filtering” Tech. Rep., USC/ISI Tech Report 550, November 2001.
lq – Queue length
p – Drop Probability
λ – Aggregated Offered Load
p
plq
11
1
lq maxth
thth
th pp
p
pminmax
min11
1max
Existence
Oblivious AQM & Nash Equilibria 22
RED and Nash EquilibriaTheorem 2: RED Does NOT impose a Nash equilibrium on uncontrolled selfish agents.
Proof: applying the condition for Nash equilibrium:
Summary: RED punishes all flows with the same drop probability. The nature of the drop function is considerably gentle. Misbehaving flows can push more traffic and get less hurt (marginally). There is no incentive for any source to stop pushing packets.
q
qq l
lp
p
pl
1
11
11
1
q
qiii l
lp
11
011
q
q
i
ii
iq
q
i
i
l
l
l
l
QED.
0
i
i 01
1
1 2
d
dl
ll
l q
q
i
q
q
i
i
RED is oblivious:
Nash Equlibria does not exist.
Existence
Oblivious AQM & Nash Equilibria 23
Virtual Load REDVirtual Load RED model:
Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if
Proof:Throughput at Nash equilibrium is independent of minth:
thth
thvql
minmax
min
p =
0
1
If lvq <minth
If minth lvq maxth
Otherwise
lvq – M/M/1 Queue length when facing the same load :
1vql
1max1min thth
2
1
2
max41,1
1
2
nnnt
t
t thNash
Existence
EfficiencyIf an Oblivious AQM scheme can impose a Nash equilibria, is that equilibria efficient, in terms of achieving high goodput and low drop probability?
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 25
Number Of FlowsN
orm
aliz
ed D
ata Total Offered Load
Goodput
minth = 0
maxth =50
n – 1,..,50
Example - Efficiency of VLRED
Efficiency
Proof: Applying Nashequilibrium satisfying andefficiency conditions:
Theorem 4: VLRED is not efficientimposing a Nash Equilibriumon selfish agents
1p α, β - some constants
pn
d
dp
1
11
2
2
lvq2 nμ/α
nμ = θ(lvq2)
th
thvql
max1
max
The goodput falls to zero
asymptotically.QED.
Oblivious AQM & Nash Equilibria 26
Efficient Nash AQM scheme
Assume – Totaldesirable offered loadat Nash equilibrium:
Oblivious mechanism can ensure an efficient Nash equilibria under selfish behavior of users.
1214
11
2
d
p
dp
n
Efficiency
Goodput
Number Of Flows
Nor
mal
ized
Dat
a
Total Offered Load
Drop Probability
11
11
3
11p
ENAQM drop probability is bounded
AchievabilityHow easy is it for players (users) to reach the equilibrium point? or How can we ensure that agents actually reach the Nash equilibrium state?
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 28
Ensuring a Nash equilibrium by an Oblivious AQM
λ*i – Offered load at equilibria when the number of agents is i.
Δi = λ*i - λ*
i-1 = iα α some constant
p = ƒ(λ*i) non decreasing and convex.
Applying Nash equilibrium satisfying and efficiency conditions:
From convexity of p*i :
From efficiency:
The equilibrium imposed by any oblivious AQM strategy is (very) sensitive to the number of agents, thus making it impractical to deploy in the Internet.Agents need the help of the router to reach the equliibria.
i
ii
icf
ic
2'
1
Achievability
c1, c2 - some constants
*1
'1
*2
'2
*1
'1
*1
* ... iii fffpp
ciiii
i
1
1
1
c - constant
2ii The sensitivity coefficient falls faster than the inverse quadric.
Summary and Future Work
OverviewNow what? – Future WorkSome further work
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 30
OverviewIntroduction – Today’s Internet. The proposed model – Markovian (M/M/1/K) GameExistence – Drop tail and RED cannot impose a Nash equilibra. VLRED imposes a Nash equilibra.
But the equilibrium points do not have a very high utilization.
Efficiency - ENAQM imposes an efficient Nash equilibra.Achievability - Equilibrium points in oblivious AQM strategies are very sensitive to the change in the number of users. It may be hard to deploy oblivious schemes that do have
Nash equilibria without the explicit help of a protocol.
Summary and Future Work
Oblivious AQM & Nash Equilibria 31
Now What ? - Future WorkVLRED – Explore why the Nash equilibria do not result in good network utilization. Conjecture – VLRED’s drop function becomes
very harsh as we reach equilibria.
Study gentler versions of VLRED and determine whether such modification can still impose Nash equilibria.Design protocols which lead to efficient network operation, such that no user has any incentive to unilaterally deviate from the protocol – can it be done ? – The Protocol Equilibrium Question.
Summary and Future Work
Oblivious AQM & Nash Equilibria 32
Some Further work (1)
“Towards Protocol Equilibrium with Oblivious Routers” D. Dutta, A. Goel and J. Heidemann, IEEE INFOCOM 2004. “…In this paper, we show that if routers used
EWMA to measure the aggregate rate, then the best strategy for a selfish agent to minimize its losses is to arrive at a constant rate. Even though the protocol space is arbitrary, our scheme ensures that the best greedy strategy is simple, i.e. send with CBR. Then, we show how we can use the results of an earlier paper to enforce simple and efficient protocol equilibria on selfish traffic agents…”
Summary and Future Work
Oblivious AQM & Nash Equilibria 33
Some Further work (2)“Pricing Differentiated Services A Game -Theoretic Approach”, E. Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for publication in Computer Networks, 2005). “…The goal of this paper is to study pricing of differentiated services
and its impact on the choice of service priority at equilibrium. We consider both TCP connections as well as non controlled (real time) connections…. We first study the performance of the system as a function of the connections’ parameters and their choice of service classes. We then study the decision problem of how to choose the service classes. We model the problem as a noncooperative game. We establish conditions for an equilibrium to exist and to be uniquely defined. We further provide conditions for convergence to equilibrium from non equilibria initial states. We finally study the pricing problem of how to choose prices so that the resulting equilibrium would maximize the network benefit…”
“…The paper (“Oblivious AQM and Nash Equilibria”)(“Oblivious AQM and Nash Equilibria”) restricted itself to symmetric users and symmetric equilibria and the pricing issue was not considered. In this framework, with a common RED buffer, it was shown that an equilibrium does not exist. An equilibrium was obtained and characterized for an alternative buffer management that was proposed, called VLRED. We note that in contrast to (Oblivious AQM and Nash Equilibria”)(Oblivious AQM and Nash Equilibria”), since we also include in the utility of CBR traffic a penalty for losses we do obtain an equilibrium when using RED…”
Summary and Future Work
AppendixesStateful Schemes – Fair QueuingFrom Queuing TheoryNash Equilibrium ConditionsApproximated Steady State RED ModelVLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof Efficient Nash AQM Drop Probability
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 36
Stateful Schemes – Fair Queuing - Nagle’s Algorithm
Gateways maintain separate queues for packets from each individual source. The queues are serviced in a round-robin manner. Nagle’s algorithm, by changing the way packets from different sources interact, does not reward, nor leave others vulnerable to, anti-social behavior.
Introduction - Appendix
“Analysis and Simulation of a Fair Queuing Algorithm”,A.Demers, S. Keshav and S.J. Shenker, ACM SIGCOMM,
1989.
Oblivious AQM & Nash Equilibria 37
From Queuing TheoryM/M/1/K Queuing system - Poisson arrivals, Exponentially distributed service times, one server and finite capacity buffer:
PASTA - Poisson Arrivals See Time Averages – For a queuing system, when the arrival process is Poisson and independent of the service process:The probability that an arriving customer finds i customers in the system Equals The probability that the system is at state i.πi – Probability that the system is in state i, Using birth-death model:
Block Probability -
Kii
i
0
,1
11K
K
KBP
2 KK-1
10
λ
μ
λ
μ
λ
μ
λ
μ
λ
μ
Oblivious AQM & Nash Equilibria 38
Nash Equilibrium Condition
1.
2.
3.
4.
5.
Substituting 2,3 in 5 we obtain:
nandji jiji
,
d
di
i
pU iii 1
01
ii
i
i pp
nd
p
dp
1Nash Equilibrium Satisfying Condition :
The Model – Appendix
0
i
ii
Oblivious AQM & Nash Equilibria 39
Approximated Steady State RED Model
Standard RED Transfer Function
Avg. Buffer Occupancy
Dro
p p
rob
.
Approximated RED Transfer Function
Dro
p p
rob
.
Normalized Offered Load
Approximated RED Buffer Occupancy
Normalized Offered Load
Avg
. B
uff
er
Occ
up
an
cy
“Faster network simulation with scenario
pre-filtering”Tech. Rep., USC/ISI Tech Report 550, November
2001.
Existence - Appendix
Example:minth = 10 maxth =20
n – 1,..,50 Pmax = 0.3
Oblivious AQM & Nash Equilibria 40
VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (1)
Existence - Appendix
Virtual Load RED model:
Proof of Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if Assume the drop probability can be written as a
continues function for all lvq<maxth:
Nash equilibrium condition:
thth
thvql
minmax
min
p =
0
1
If lvq <minth
If minth lvq maxth
Otherwise
lvq – M/M/1 Queue length when facing the same load :
1vql
1max1min thth
2
11minmax
1
minmax
min1
thththth
th
d
dpp
thth
th
thth
nminmax
min11
11minmax
12
Oblivious AQM & Nash Equilibria 41
VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (2)
Substituting and simplifying we obtain:
This is true if (by Substituting n=1)
1
t
Existence - Appendix
0max)1(2 thnnt
thth t
nnnt min,
2
1
2
max41 2
1max1min thth
Oblivious AQM & Nash Equilibria 42
Efficient Nash AQM Drop Probability
Total desirable offered load at Nash equilibrium:
Substituting y2=1-λ and then integrating:
k is determined so when there is one user, the drop probability is zero as long as his offered load is less than unity
The offered load at Nash equilibria is boundeddrop probability at equilibria is bounded (proved by
substitution).
y
ykpk
y
yp
y
dy
y
dy
p
dp
1
11log
1
1log1log
112
1
1
Efficiency - Appendix
1214
11
2
d
p
dp
n
11
11
3
11p
k - arbitrary constant to be determined
31k