Informationally Efficient Multi user communication · 2 • Motivation and existing approaches •...

1

Informationally Efficient Multi‐user communication

Yi Su

Advisor: Professor Mihaela van der Schaar

Electrical Engineering, UCLA

2

• Motivation and existing approaches

• Informationally efficient multi‐user communication– Vector cases

• Convergence conditions with decentralized information

• Improve efficiency with decentralized information

– Scalar cases• Achieve Pareto efficiency with decentralized information

• Conclusions

Outline

3

Multi‐user communication networks

Distributed routing

Power control

Peer‐to‐peer system

etc…

4

Constraints in communication networks

• Resources– Bandwidth, power,

spectrum, etc.

• Information– Real‐time

• Local observation

5


• Resources– Bandwidth, power,

spectrum, etc.



• Exchanged message

6


• Resources– Bandwidth, power,spectrum, etc.


• Local observation• Exchanged message

– Non‐real‐time• A‐priori information aboutinter‐user coupling, protocols, etc.

7


• Resources– Bandwidth, power,spectrum, etc.


• Local observation• Exchanged message

– Non‐real‐time• A‐priori information aboutinter‐user coupling, protocols, etc.

Goal: multi‐user communication without information exchange

8

A standard strategic game formulation

• Consider a tuple

– The set of players :

– The set of actions: and

– Utility function: and

– Utility region:

In communication networks, different operating

points in can be chosen based on the information

availability

9

Existing approaches


Nash equilibrium

10

Existing approaches


Nash equilibrium

• Exchanged messages

Pareto optimality

Price!

11

Existing approaches


Nash equilibrium

• Exchanged messages

Pareto optimality

Existing results usually assume some

specific action and utility structures!

Price!

12

• Results with specific action and utility structures– Pure Nash equilibrium

• Concave gamesi) : convex and compact; ii) : quasi‐concave in

• Potential games [Shapley]

• Super‐modular games [Topkis]i) is a lattice; ii)

– Pareto optimality• Network utility maximization [Kelly]

• Convexity is the watershed

Existing approaches (cont’d)

Use gradient play to find NE

13



Use best response to find NE







14











15











16

Existing approaches (cont’d)Researchers Applications Tools

Altman CDMA uplink power control S‐modular games

Berry Distributed interference compensation S‐modular games

Barbarossa Power control Potential games

Tse Spectrum sharing Repeated games

Kelly End‐to‐end congestion control Pricing

Goodman CDMA uplink power control Pricing

Low End‐to‐end flow control Pricing

Chiang Joint congestion and power control Pricing

Poor Energy efficient power and rate control Equilibrium analysis

Cioffi Power control in DSL systems Equilibrium analysis

Yates Uplink power control for cellular radio Equilibrium analysis

Wicker Selfish users in Aloha Equilibrium analysis

Lazar Non‐cooperative optimal flow control Equilibrium analysis

Special

utility

17


• Game theory– Equilibrium characterization

– Incentive design

• Optimization theory– Computational complexity

– Distributed algorithms

• Information theory– Fundamental limits

– Encoding and decoding schemes

Information is usually costless

The focus is on strategic interactions among users

Decentralization is not the focus

18


u1

u2 Pareto boundaryGlobal information

Nash equilibriumDecentralized (limited) information

General modelse.g. concave/potential/supermodular games

Specific multi‐user communication applications

But in many communication systems, information is constrained and no message passing is allowed!

19

Our goals

u1

u2


Pareto boundaryGlobal (exchanged) information

If information is constrained and

no message passing is allowed…



New classes of communication

games

20

When will it convergeto a NE ? And how fast ?

Our goals

u1

u2








games

21


How to improve an inefficient NE without message passing ?

Our goals

u1

u2








games

22



And can we still achieve Pareto optimality ?

Our goals

u1

u2








games

23






• Conclusions

Outline

24

A reformulation of multi‐user interactions


– The set of players:

– The set of actions:

– State space:

– State determination function:

and

– Utility function:

and

In standard strategic game,

It captures the structure of the coupling between action and state

25

A reformulation of multi‐user interactions


– The set of players:

– The set of actions:

– State space:

– State determination function:

and

– Utility function:

and

In standard strategic game,

Many communication

networking applications have

simple , which captures

the aggregate effects of

It captures the structure of the coupling between action and state

26

• Power control

Communication games with simple states

aggregate interference

27


• Power control

• Flow controlremaining capacity


28


• Power control

• Flow control

• Random access

remaining capacity


idle probability

29






• Conclusions

Outline

30

• Definition– A multi‐user interaction in which

A1: action set is defined to be

Additively Coupled Sum Constrained Games

31


A1: action set is defined to be


Structure of the action set:resource is constrained

32


A2: The utility function satisfies

in which is an increasing and strictly

concave function. Both and

are twice differentiable.


33







states

cost

34







states

Structure of the utility:additive coupling between

action and state

cost

35







diminishing return per invested action

states

Structure of the utility:additive coupling between

action and state

cost

36

• Power control in interference channels

Examples of ACSCG

37


Examples of ACSCG

38


Examples of ACSCG

39

• Delay minimization in Jackson networks

Examples of ACSCG (cont’d)

i

j

m

0kir

kimr

kijrk

iψ

40



i

j

m

0kir

kimr

kijrk

iψ

41



i

j

m

0kir

kimr

kijrk

iψ

42

Nash equilibrium in ACSCG

• Existence of pure NE– A subclass of concave games

• When is the NE unique? When does best response converges to such a NE?– Existing literatures are not immediately applicable

• Diagonal strict convexity condition [Rosen]• Use gradient play and stepsizes need to be carefully chosen

• Super‐modular games [Topkis]• Action space is not a lattice

• Sufficient conditions for specific and [Yu]

43

• Best response iteration

Best response dynamics

44


in which is chosen such that

• When does it converges?– By intuition, the weaker the mutual coupling is, the more likely it converges

– How to measure and quantify this coupling strength?

Best response dynamics

45





Best response dynamicssum constraint

additive coupling

state

A competition scenario in which every useraggressively uses up all his resources

46





Best response dynamicssum constraint

additive coupling

state

47

Define

represents the maximum impact that user m’s action can make over user n’s state

A measure of the mutual coupling

48

Convergence conditions

Theorem 1: If

then best response dynamics converges linearly to a unique pure NE for any set of initial conditions.

• The contraction factor is a measure of the overall coupling strength

• If is affine, the condition in Theorem 1 is not impacted by ; otherwise it may depend on .

49


Contraction mapping

Theorem 1: If




50

Theorem 1: If





Contraction mapping

is a constant for affine

51

• If have the same sign, the condition in Theorem 1 can be relaxed to

• This is true in many communication scenarios– Increasing power causes stronger interference

– Increasing input rate congests the server


52

• If have the same sign, the condition in Theorem 1 can be relaxed to

• This is true in many communication scenarios– Increasing power causes stronger interference

– Increasing input rate congests the server


Strategic complements (or strategic substitutes)

53

For , define [Walrand]

A special class of

54

For , define [Walrand]

Define

A special class of

A measure of the similarity between users’ parameters

55


Theorem 2: If


56

Theorem 1 Theorem 1

Theorem 2 Theorem 2


Theorem 2: If


Contraction mapping

57


Conclusion so far…

u1

u2

Nash equilibrium

Pareto boundary

If Information is constrained and

no message passing is available…

Concave games

Power control,Flow control

ACSCG

58


Conclusion so far…

u1

u2

Nash equilibrium

Pareto boundary



Sufficient conditions that guarantee linear convergence

Concave games

Power control,Flow control

ACSCG

59


Power control as an ACSCG

60

Performance comparison • Solutions without information exchange

– Iterative water‐filling algorithm [Yu]

• Solutions with information exchange

knP

k

knσ

k kmn m

m nH P

≠∑

user n’s spectrum

max k kkRω∑

k

knσ

k kmn m

m nH P

≠∑

61

Performance comparison • Solutions without information exchange

– Iterative water‐filling algorithm [Yu]

• Solutions with information exchange

knP

k

knσ

k kmn m

m nH P

≠∑

user n’s spectrum

max k kkRω∑

k

knσ

k kmn m

m nH P

≠∑

OSB = Optimal

Spectrum

Balancing

ASB = Autonomous

Spectrum

Balancing

62






• Conclusions

Outline

63

How to model the mutual coupling

• A reformulation of the coupling– State space

– Utility function

– State determination function

– Belief function

– Conjectural Equilibrium (CE) : a configuration of belief functions and joint action satisfying

and

n n∈=× NS S:n n nu × →S A R

:n n ns − →A S:n n ns →A S

1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

∈=

A

64





– Belief function


and



1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

∈=

A

it captures the

aggregate effect of

the other users’ actions

it models the aggregate effect

of the other users’ actions

65





– Belief function


and



1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

∈=

A

beliefs are realized each user behaves optimally

according to its expectation

it captures the

aggregate effect of

the other users’ actions

it models the aggregate effect

of the other users’ actions

66

CE in power control games [SuTSP’09]

• One leader and multiple followers

• State space– : the interference caused to user n in channel k

• Utility function

• State determination function

• Belief function (linear form)

knI

21log 1

kKn

n k kn nk

PR

Iσ=

⎛ ⎞⎟⎜ ⎟= +⎜ ⎟⎜ ⎟⎜ +⎝ ⎠∑

1,Nk k k

n in ii i nI Pα

= ≠=∑

1 1k k k kI Pβ γ= −

actual play

conceived play

67

Why Linear belief?is piece‐wise linear; , if the

number of frequency bins is sufficiently large. Linear belief is sufficient to capture the

interference coupling!

1

10,

k

jI

j kP

∂ = ≠∂

1

1

k

kIP

∂∂

68




1

10,

k

jI

j kP

∂ = ≠∂

1

1

k

kIP

∂∂

2fP

f2fσ

12 1f fH P

69




1

10,

k

jI

j kP

∂ = ≠∂

1

1

k

kIP

∂∂

2fP

f2fσ

12 1f fH P

2fP

f2fσ

12 1f fH P

70




1

10,

k

jI

j kP

∂ = ≠∂

1

1

k

kIP

∂∂

2fP

f2fσ

12 1f fH P

2fP

f2fσ

12 1f fH P

2fP

f2fσ

12 1f fH P

71




1

10,

k

jI

j kP

∂ = ≠∂

1

1

k

kIP

∂∂

2fP

f2fσ

12 1f fH P

2fP

f2fσ

12 1f fH P

2fP

f2fσ

12 1f fH P

72

Main results• Stackelberg equilibrium

– Strategy profile that satisfies

• NE and SE are special CENE:

SE:

• Infinite set of CEOpen sets of CE that contain

NE and SE may exist

12

, 0N

k k k ki i

iPβ α γ

== =∑

γ

β

1R

•

•

1NER

1SER•

•

1 11 1

1 1, .

k kk k k k

k kI I

I PP P

β γ∂ ∂= − ⋅ = −∂ ∂

( )( )* *1 1,a NE a

( )( ) ( )( )* *1 1 1 1 1 1 1 1, , ,u a NE a u a NE a a≥ ∀ ∈ A

73

Achieving the desired CE

• Conjecture‐based rate maximization (CRM)

solvable using dual method

leader followers

74

Discussion about CRM

• Essence of CRM– local approximation of the computation of SE

• Advantages– the structure of the utility function is explored

– only local information is required

– it can be applied in the cases where N>2

– if it converges, the outcome is a CE

75

Simulation results

Average rate improvements:

2‐user case: 24.4% for user 1; 33.6% for user 2

3‐user case: 26.3% for user 1; 9.7% for user 2&3

( )20.5,k

ijki jα = ≠∑

0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1/R1NE

R2/R2NE

0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1/R1NE

R2/R2NE

R3/R3NE

( )20.33,k

ijki jα = ≠∑

76

Concave games

ACSCG

Conclusions so far…

u1

u2Pareto boundary

Nash equilibrium



no message passing is allowed

Power control

77

Concave games

ACSCG

Overall efficiency may be improved!


u1

u2Pareto boundary

Nash equilibrium

Build belief, learn, and adapt



no message passing is allowed

Power control

78






• Conclusions

Outline

79

Linearly coupled games

• A non‐cooperative game model• Users’ states are linearly impacted by their competitor’s actions

• Contributions– Characterize the structures of the utility functions– Explicitly compute Nash equilibrium and Pareto boundary

– A conjectural equilibrium approach to achieve Pareto boundary without real‐time information exchange

80

A multi‐user interaction is considered a linearly coupled game if the action set is convex and the utility function satisfies

in which . In particular, the basic assumptions about include:

A1: is non‐negative;

A2: is strictly linearly decreasing in ;

is non‐increasing and linear in .

Definition

States are linearly impacted by actions

81

Denote .

A3: is an affine function,

A4:

Definition (cont’d)

Actions are linearly coupled at NE and PB

82

• For the games satisfying A1‐A4, the utility functions can take two types of form:– Type I [SuJSAC’10]

• e.g. random access

– Type II [SuTR’09]

• e.g. rate control

Two basic types

83

• For the games satisfying A1‐A4, the utility functions can take two types of form:– Type I [SuJSAC’10]

• e.g. random access

– Type II [SuTR’09]

• e.g. rate control

Two basic types

84

• Player set: – nodes in a single cell

• Action set:– transmission probability

• Payoff:– throughput

• Key issues– stability, convergence, throughput, and fairness

Type I games: wireless random access

Tx1

Rx1Tx2

TxK

Rx2

RxK

85

• Individual conjectures– state:

– linear belief:

• Two update mechanisms – Best response

– Gradient play

Conjecture‐based Random Access

actual play

conceived play

86

Main results

• Existence of CE– all operating points in action space are CE

• Stability and convergence– sufficient conditions

• Throughput performance– the entire throughput region can

be achieved with stable CE

• Fairness issue– conjecture‐based approaches

attain weighted fairness

Protocol design: how to achieve efficient outcomes?

87

How to select suitable ak?

• Adaptively alter ak when the network size changes

• Adopt aggregated throughput or “idle interval” as the indicator of the system efficiency

• Advantages– No need of a centralized solver– Throughput efficient with fairness guarantee– Stable equilibrium– Autonomously adapt to traffic fluctuation

88

Engineering interpretation• DCF vs. the best response update

– re‐design the random access protocol

89



similar different

90



similar different

CBRA makes use of 4-bit information, while DCF only uses 2 bits

91

Simulation results

• Throughput

• Stability and convergence

5 10 15 20 25 30 35 40 45 5025

26

27

28

29

30

31

32

33

34

35

36

Number of nodes

Acc

umul

ativ

e th

roug

hput

(Mbp

s) Optimal throughputP-MACConjecture-based algorithmsIEEE 802.11 DCF

0 100 200 300 400 500 60031

31.5

32

32.5

33

33.5

34

34.5

35

35.5

36

Acc

umul

ativ

e th

roug

hput

(Mbp

s)

P-MACBest responseGradient play

DCF: low throughput; P‐MAC: needs to know the number of nodes

P‐MAC: instability due to the online estimation

92

• Utility function

• Nash equilibrium

• Pareto boundary

• Efficiency loss

Conventional solutions in Type II games

93

• At stage t,

• Theorem 5: A necessary and sufficient condition for the best response dynamics to converge is

Best response dynamics in Type II games

Determine the eigenvalues of the Jacobian matrix

Observed state Linear belief

94

• Theorem 6: All the operating points on the Pareto boundary are globally convergent CE under the best response dynamics. The belief configurations lead to Pareto‐optimal operating points if and only if

– : the ratio between the immediate performance degradation and the conjectured long‐term effect

Stability of the Pareto boundary

Theorem 5 and expressions of Pareto boundary and CE

95

Pricing vs. conjectural equilibrium

• Pricing mechanism in communication networks [Kelly][Chiang]– Users repeatedly exchange coordination signals

• Conjectural equilibrium for linearly coupled games– Coordination is implicitly implemented when the participating users initialize their belief parameters

– Pareto‐optimality can be achieved solely based on local observations on the states

– No message passing is needed during the convergence process

– The key problem is how to design belief functions

96


u1

u2 Pareto boundaryGlobal (exchanged) information


Decentralized (insufficient) information

The optimal way of designing the beliefs and updating the

actions based on conjectural equilibrium is addressed

Can we still achieve Pareto optimality ?

Concave games

LCG

Random Access,Rate control



97


u1

u2 Pareto boundaryGlobal (exchanged) information


Conjectural equilibrium

Decentralized (insufficient) information

The optimal way of designing the beliefs and updating the

actions based on conjectural equilibrium is addressed

Can we still achieve Pareto optimality ?

Concave games

LCG

Random Access,Rate control

Pareto optimality can be achieved!



98

Conclusions

• We define new classes of games emerging in multi‐user communication networks and investigate the information and efficiency trade‐off– Provide sufficient convergence conditions to NE

– Suggest a conjectural equilibrium based approach to improve efficiency

– Quantify the performance improvement

99

References• J. Rosen, “Existence and uniqueness of equilibrium points for

concave n‐person games,” Econometrica, vol. 33, no. 3, pp. 520‐534, Jul. 1965.

• D. Monderer and L. S. Shapley, “Potential games,” Games Econ. Behav., vol. 14, no. 1, pp. 124‐143, May 1996.

• D. Topkis, Supermodularity and Complementarity. Princeton University Press, Princeton, 1998.

• F. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control in communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research Society, vol. 49, pp. 237‐252, 1998.

• M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering as optimization decomposition: A mathematical theory of network architectures,” Proc. of the IEEE, vol. 95, no. 1, pp. 255‐312, January 2007.

100

References (cont’d)

• W. Yu, G. Ginis, and J. Cioffi, “Distributed multiuser power control for digital subscriber lines,” IEEE J. Sel. Areas Commun., vol. 20, no. 5, pp. 1105‐1115, June 2002.

• J. Mo and J. Walrand, “Fair end‐to‐end window‐based congestion control,” IEEE Trans. on Networking, vol. 8, no. 5, pp. 556‐567, Oct. 2000.

101

References (cont’d)• Y. Su and M. van der Schaar, “Structural solutions for

additively coupled sum constrained games,” UCLA technical Report, 2010.

• Y. Su and M. van der Schaar, “Conjectural equilibrium in multiuser power control games,” IEEE Trans. Signal Processing, vol. 57, no. 9, pp. 3638‐3650, Sep. 2009.

• Y. Su and M. van der Schaar, “A new perspective on multi‐user power control games in interference channels,” IEEE Trans. Wireless Communications, vol. 8, no. 6, pp. 2910‐2919, June 2009.

• Y. Su and M. van der Schaar, “Linearly coupled communication games,” UCLA technical Report, 2009.

• Y. Su and M. van der Schaar, “Dynamic conjectures in random access networks using bio‐inspired learning,” IEEE JSAC special issue on Bio‐Inspired Networking, May 2010.

102

Linear convergence

• A sequence with limit is linearly convergent if there exists a constant such that

for k sufficiently large.

103

Solutions with information exchange

• Users aim to solve

• They can pass coordination messages

and user n behaves according to

user n’s impact over user m’s utility

104


• Gradient play

Theorem 3: If

gradient play converges for a small enough stepsize.Lipschitz continuity and gradient projection algorithm

105


• Jacobi update

Theorem 4: If

Jacobi update converges for a small enough stepsize.

Lipschitz continuity, descent lemma, and mean value theorem

106


• Convergence to an operating point that satisfies the KKT conditions is guaranteed

• Total utility is monotonically increasing

• Global optimality is guaranteed if the original problem is convex, otherwise not

• Developed for general non‐convex problem in which convex NUM solutions may not apply in general

107

Stackelberg equilibrium

• Definition– Leader (foresighted): only one

– Follower (myopic): the remaining ones

– Strategy profile that satisfies

• Existence and computation of SE in the power control games [SuTWC’09]

( )( )* *,n na NE a

( )( ) ( )( )* *, , ,n n n n n n n nu a NE a u a NE a a≥ ∀ ∈ A

108

A two‐user formulation

• Bi‐level Programming

where

upper

level

problem

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

lower

level

problem

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

1

2

1

1 2 21

1 11

22

2 1 11

2 21

max ln 1 ( )

. . , 0, ( )

argmax ln 1 ( )

. . , 0. ( )

kK

k k kkK k kk

kK

k k kk

K k kk

Pa

N P

s t P P b

Pc

N P

s t P P d

α

α

=

=

′ =

=

⎛ ⎞⎟⎜ ⎟+⎜ ⎟⎜ ⎟⎜ +⎝ ⎠

≤ ≥

⎛ ⎞′ ⎟⎜ ⎟= +⎜ ⎟⎜ ⎟⎜ +⎝ ⎠

′ ′≤ ≥

∑

∑

∑

∑

max1

max2

P

PP

P

P

2 2 2 2 2 21 1 11 1 12 22 2 2 22 2 21 11, , ,k k k k k k k k k k k kN H H H N H H Hσ α σ α= = = =

109

Problems with the SE formulation

• Computational complexity– intrinsically hard to compute

• Information required for playing SE– Global information

• Realistic assumption– Local information

– Any appropriate solutions other than SE and NE?

{ } { } { }, ,k kij iα σ max

iP

1 12,N k k k

n nnPα σ

=+∑ max

1P

110

• Priority‐based fair medium access control– Traffic classes with positive weights

• Conjecture‐based protocol

Weighted Fairness

111

Some distributed iterative algorithms

• Best response

• Jacobi update

• Gradient play stepsize

Informationally Efficient Multi user communication · 2 • Motivation and existing approaches •...

Documents

Transcript of Informationally Efficient Multi user communication · 2 • Motivation and existing approaches •...