Robust Price of Anarchy Bounds via Smoothness Arguments
description
Transcript of Robust Price of Anarchy Bounds via Smoothness Arguments
1
Robust Price of Anarchy Bounds via
Smoothness Arguments
Tim RoughgardenStanford University
2
Price of AnarchyDefinition: price of anarchy (POA) of a
game (w.r.t. some objective function, eq concept):
optimal obj fn valueequilibrium objective fn value
the closer to 1 the better
3
Price of AnarchyDefinition: price of anarchy (POA) of a
game (w.r.t. some objective function, eq concept):
optimal obj fn valueequilibrium objective fn value
the closer to 1 the better
s t2x 12
5x5cost = 14+10 = 24
cost = 14+14 = 28
s t2x 12
5x500
4
The Need for RobustnessMeaning of a POA bound: if the game is at
an equilibrium, then outcome is near-optimal.
5
The Need for RobustnessMeaning of a POA bound: if the game is at
an equilibrium, then outcome is near-optimal.
Problem: what if can’t reach equilibrium?• (pure) equilibrium might not exist• might be hard to compute, even
centrally– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/
Goldbeg/Papadimitriou], [Chen/Deng/Teng], etc.• might be hard to learn in distributed
way
Worry: are our POA bounds “meaningless”?
6
Fix #1: Provable Convergence
One Good Research Agenda: prove that “natural learning dynamics” converge (quickly) to equilibrium.
• [Hart/Mas-Collell], [Even-Dar/Kesselman/Mansour], [Fisher/ Racke/Voecking], [Chien/Sinclair], [Awerbuch et al], [Even-Dar/Mansour/Nadav], [Kleinberg/Piliouras/Tardos], etc.
Problem: this “best-case scenario” has limited reach
• non-existence/complexity issues• lower bounds on convergence time of natural
dynamics (e.g., [Skopalik/Voecking STOC 08])
7
Fix #2: Robust POA BoundsHigh-Level Goal: worst-case bounds that
apply even to non-equilibrium outcomes!
• best-response dynamics, pre-convergence– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],
[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]• correlated equilibria
– [Christodoulou/Koutsoupias 05]• coarse correlated equilibria aka “price
of total anarchy” aka “no-regret players”– [Blum/Even-Dar/Ligett 06],
[Blum/Hajiaghayi/Ligett/Roth 08]
8
Abstract Setup• n players, each picks a strategy si
• player i incurs a cost Ci(s)
Important Assumption: objective function is cost(s) := i Ci(s)
Key Definition: A game is (λ,μ)-smooth if, for every pair s,s* outcomes (λ > 0; μ < 1):
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
[(*)]
9
Smooth => POA BoundNext: “canonical” way to upper bound
POA (via a smoothness argument).• notation: s = a Nash eq; s* = optimal
Assuming (λ,μ)-smooth:
cost(s) = i Ci(s) [defn of cost]
≤ i Ci(s*i,s-i) [s a Nash
eq] ≤ λ●cost(s*) + μ●cost(s)
[(*)]
Then: POA (of pure Nash eq) ≤ λ/(1-μ).
10
Why Is Smoothness Stronger?
Key point: to derive POA bound, only needed
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
[(*)]
to hold in special case where s = a Nash eq and s* = optimal.
Smoothness: requires (*) for every pair s,s* outcomes.– even if s is not a pure Nash equilibrium
11
Some Smoothness Bounds• atomic (unweighted) selfish routing
[Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]
• nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]
• weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06],
[Bhawalkar/Gairing/Roughgarden 10]• submodular maximization games [Vetta 02], [Marden/Roughgarden 10]• coordination mechanisms [Cole/Gkatzelis/Mirrokni 10]
Application: Out-of-Equilibria
Definition: a sequence s1,s2,...,sT of outcomes is no-regret if:
• for each player i, each fixed action qi:– average cost player i incurs over sequence
no worse than playing action qi every time– simple hedging strategies can be used by
players to enforce this (for suff large T)
Theorem: [Roughgarden STOC 09] in a (λ,μ)-smooth game, average cost of every no-regret sequence at most [λ/(1-μ)] x cost of optimal outcome.
12
13
Smooth => POTA Bound• notation: s1,s2,...,sT = no regret; s* =
optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
14
Smooth => POTA Bound• notation: s1,s2,...,sT = no regret; s* =
optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
15
Smooth => POTA Bound• notation: s1,s2,...,sT = no regret; s* =
optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
16
Smooth => POTA Bound• notation: s1,s2,...,sT = no regret; s* =
optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st) [defn of cost]
= t i [Ci(s*i,st
-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st
-
i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
No regret: t ∆i,t ≤ 0 for each i.To finish proof: divide through by T.
17
Why Important?
pureNash
mixed Nashcorrelated eq
no regret• bound on no-regret
sequences implies bound on inefficiency of mixed and correlated equilibria
• bound applies even to sequences that don’t converge in any sense• no regret much weaker than reaching
equilibrium• e.g., if every player uses “multiplicative
weights” then get o(1) regret in poly-time
Tight Game ClassesTheorem: [Roughgarden STOC 09] for every set
C, unweighted congestion games with cost functions restricted to C are tight:maximum [pure POA] = minimum [λ/(1-μ)]congestion games
w/cost functions in C(λ ,μ): all such gamesare (λ ,μ)-smooth
18
Tight Game ClassesTheorem: [Roughgarden STOC 09] for every set
C, unweighted congestion games with cost functions restricted to C are tight:maximum [pure POA] = minimum [λ/(1-μ)]
• weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense
congestion gamesw/cost functions in C
(λ ,μ): all such gamesare (λ ,μ)-smooth
19
20
Further Applications
pureNash
mixed Nash
correlated eq
no regret
best-responsedynamics
approximate Nash
Theorem: in a (λ,μ)-smooth game, everything in these sets costs (essentially) λ/(1-μ) x OPT.
21
More Applications?Theorem: [Nadav/Roughgarden 10] Consider a (λ,μ)-smooth game for optimal choices of λ,μ. Then precisely the “aggregate” coarse correlated equilibria have cost ≤ λ/(1-μ) x OPT.• essentially, bound holds if and only if the
average (rather than per-player) regret is non-positive
Proof: convex duality.
When Is a POA Bound a Smoothness Proof?
Need to show: for every pair s,s* outcomes:
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
Generally sufficient: prove POA bound for pure Nash equilibria such that:
• invoke best-response condition once/player
• hypothetical deviation by i independent of s-i
Non-example: network creation games [Fabrikant et al], [Albers et al], [Demaine et al], etc.
22
Application: POA of Non-Truthful Mechanisms
Mechanism Design: design protocol with desirable outcome despite selfish participants
• example: Vickrey (second-price) auction• focus thus far on "truthful" mechanisms
Non-truthful mechanisms: motivated by• simplicity (e.g., sponsored search
auctions)• low communication complexity• low computational complexity
23
Application: POA of Non-Truthful Mechanisms
Fact: plausible outcomes of non-truthful mechanisms = Bayes-Nash equilibria
• POA results for simple non-truthful auctions in [Christodoulou/Kovacs/Schapira ICALP 08] and [Borodin/Lucier SODA 10]
• first bound POA of Nash equilibria, then "by magic" same bound holds for Bayes-Nash
Fact: these are automatic consequences of smoothness. 24
POA of Non-Truthful Mechanisms (continued)
[Paes Leme/Tardos FOCS 10]• POA upper bounds for welfare of
Generalized Sponsored Search auction• different bounds for pure (1.618), mixed
(4), and Bayes-Nash eq (8) (without smoothness)
Open Questions: • compute best smoothness bound• prove a lower bound separating the best-
possible pure vs. Bayes-Nash POA 25
POA of Non-Truthful Mechanisms (continued)
[Bhawalkar/Roughgarden 10]• POA upper bounds for welfare of
subadditive combinatorial auctions with “item bidding”
– generalizes [Christodoulou/Kovacs/Schapira 08]
• pure Nash: non-smooth upper bound of 2• Bayes-Nash: lower bound of 2.01, upper
bound of 2 ln m [m = # goods]
Open Questions: • is Bayes-Nash POA = O(1)?
26
Local Smoothness and Splittable Congestion
GamesLocal smoothness: require smoothness
condition only for outcomes that are “close”
– assume strategy sets = convex subset of Rn
• can only decrease optimal value of λ/(1-μ) • [Harks 08]: local smoothness gives
improved POA bounds for splittable congestion games
• [Roughgarden/Schoppmann 10]: matching lower bounds => first tight bounds in this model
• [RS10]: local smoothness bounds extend to correlated eq but not to no-regret outcomes!
27
Splittable Congestion Games: Open Questions
Open Question #1: determine optimal POA bounds for no-regret sequences.
Open Question #2: is (the distribution of) every no-regret sequence a convex combination of pure Nash equilibria?
Open Question #3: prove something non-trivial about the POA of symmetric splittable congestion games. 28