Congestion control Lecture 6 CS 653. Why congestion control?

87
Congestion control Lecture 6 CS 653

Transcript of Congestion control Lecture 6 CS 653. Why congestion control?

Congestion control

Lecture 6CS 653

Why congestion control

Causescosts of congestion scenario 1

two senders two receivers

one router infinite buffers

no retransmission

large delays when congested

throughput staurates

unlimited shared output link buffers

Host Ain original data

Host B

out

Causescosts of congestion scenario 2

one router finite buffers sender retransmission of lost

packet

finite shared output link buffers

Host A in original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 2 always (goodput)

ldquoperfectrdquo retransmission when only loss

retransmission of delayed (not lost) packet makes

larger (than perfect case) for same

in

out

=

in

out

gt

in

out

ldquocostsrdquo of congestion more work (retransmission) for given ldquogoodputrdquo unneeded retransmissions link carries multiple copies of

pkt

R2

R2in

ou

t

b

R2

R2in

ou

t

a

R2

R2in

ou

t

c

R4

R3

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Why congestion control

Causescosts of congestion scenario 1

two senders two receivers

one router infinite buffers

no retransmission

large delays when congested

throughput staurates

unlimited shared output link buffers

Host Ain original data

Host B

out

Causescosts of congestion scenario 2

one router finite buffers sender retransmission of lost

packet

finite shared output link buffers

Host A in original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 2 always (goodput)

ldquoperfectrdquo retransmission when only loss

retransmission of delayed (not lost) packet makes

larger (than perfect case) for same

in

out

=

in

out

gt

in

out

ldquocostsrdquo of congestion more work (retransmission) for given ldquogoodputrdquo unneeded retransmissions link carries multiple copies of

pkt

R2

R2in

ou

t

b

R2

R2in

ou

t

a

R2

R2in

ou

t

c

R4

R3

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Causescosts of congestion scenario 1

two senders two receivers

one router infinite buffers

no retransmission

large delays when congested

throughput staurates

unlimited shared output link buffers

Host Ain original data

Host B

out

Causescosts of congestion scenario 2

one router finite buffers sender retransmission of lost

packet

finite shared output link buffers

Host A in original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 2 always (goodput)

ldquoperfectrdquo retransmission when only loss

retransmission of delayed (not lost) packet makes

larger (than perfect case) for same

in

out

=

in

out

gt

in

out

ldquocostsrdquo of congestion more work (retransmission) for given ldquogoodputrdquo unneeded retransmissions link carries multiple copies of

pkt

R2

R2in

ou

t

b

R2

R2in

ou

t

a

R2

R2in

ou

t

c

R4

R3

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Causescosts of congestion scenario 2

one router finite buffers sender retransmission of lost

packet

finite shared output link buffers

Host A in original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 2 always (goodput)

ldquoperfectrdquo retransmission when only loss

retransmission of delayed (not lost) packet makes

larger (than perfect case) for same

in

out

=

in

out

gt

in

out

ldquocostsrdquo of congestion more work (retransmission) for given ldquogoodputrdquo unneeded retransmissions link carries multiple copies of

pkt

R2

R2in

ou

t

b

R2

R2in

ou

t

a

R2

R2in

ou

t

c

R4

R3

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Causescosts of congestion scenario 2 always (goodput)

ldquoperfectrdquo retransmission when only loss

retransmission of delayed (not lost) packet makes

larger (than perfect case) for same

in

out

=

in

out

gt

in

out

ldquocostsrdquo of congestion more work (retransmission) for given ldquogoodputrdquo unneeded retransmissions link carries multiple copies of

pkt

R2

R2in

ou

t

b

R2

R2in

ou

t

a

R2

R2in

ou

t

c

R4

R3

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Causescosts of congestion scenario 3 four senders multihop paths timeoutretransmit

in

Q what happens as and increase

in

finite shared output link buffers

Host Ain original data

Host B

out

in original data plus retransmitted data

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Causescosts of congestion scenario 3

Another ldquocostrdquo of congestion when packet dropped any ldquoupstream

transmission capacity used for that packet was wasted

Host A

Host B

o

u

t

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Two broad approaches towards congestion control

End-end congestion control

no explicit feedback from network

congestion inferred from end-system observed loss delay

approach taken by TCP

Network-assisted congestion control

routers provide feedback to endhosts single bit indicating

congestion (SNA DECbit ATM TCPIP ECN)

explicit rate sender should send

recent proposals [XCP] [RCP] revisit ATM ideas

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Components of TCP congestion control

Slow start Multiplicatively increase (double) window

Congestion avoidance Additively increase (by 1 MSS) window

Loss Multiplicatively decrease (halve) window

Timeout Set cwnd to 1 MSS Multiplicatively increase (double) retransmission

timeout upon each further consecutive loss

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Retransmission timeout estimation

Calculate EstimatedRTT using moving average

Calculate deviation wrt moving average

Timeout = EstimatedRTT + 4DevRTT

EstimatedRTTi = (1- )EstimatedRTTi-1 + SampleRTTi

DevRTTi = (1-)DevRTTi-1 + |SampleRTTi-EstimatedRTTi-1|

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP Throughput

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP throughput A very very simple model

Whatrsquos the average throughout of TCP as a function of window size and RTT T Ignore slow startLet W be the window size when loss occurs

When window is W throughput is WT Just after loss window drops to W2

throughput to W2T Average throughput 3W4T

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP throughput A very simple model

But what is W when loss occurs

When window is w and queue has q packets TCP is

sending at rate w(T+qC) For maintaining utilization and steady state

Just before loss rate = W(T+QC) = C Just after loss rate = W2T = C For Q = CT (a common thumbrule to set router buffer

sizes) a loss occurs every frac14 (34W)Q = 3W28 packets

Q = queue capacityin number of packets

C = link capacity in packetssec

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

sum=

+=++⎟⎠

⎞⎜⎝

⎛ ++2

0

)2

(122

W

n

nW

WWW

sum=

+⎟⎠

⎞⎜⎝

⎛ +=2

021

2

W

n

nWW

2

)12(2

21

2

++⎟

⎠⎞

⎜⎝⎛ +=

WWWW

WW4

3

8

3 2 +=

packets sent per ldquoperiodrdquo =

2

8

3Wasymp

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Deriving TCP throughputloss relationship

TCPwindow

size

time (rtt)

W2

W

period

packets sent per ldquoperiodrdquo 2

8

3Wasymp

1 packet lost per ldquoperiodrdquo implies

ploss 23

8

Wasymp or

losspW

3

8=

rtt

packets

4

3utavg_thrup WB ==

rtt

packets221utavg_thrup

losspB ==

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Alternate fluid model

Rate of change of sending rate = term inversely proportional to current rate with probability (1-p) - term proportional to current rate with probability p

In steady state

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]

With many flows loss rate and delay are not affected much by a single TCP flow TCP behavior completely specified by

loss and delay pattern along path (bounded by bottleneck capacity)

Given loss rate p and delay T what is TCPrsquos throughput B packetssec taking timeouts into account

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

What is PFTK modeling

Independent loss probability p across roundsLoss acute triple duplicate acksBursty loss in a round if some packet

lost all following packets in that round also lost

Timeout if lt three duplicate acks received

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

PFTK empirical validation Low loss

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

PFTK empirical validation High loss

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Loss-based TCP

Evolution of loss-based TCPTahoe (without fast retransmit) Reno (triple duplicate acks + fast

retransmit)NewReno (Reno + handling multiple

losses better)SACK (selective acknowledgment)

common today Q what if loss not due to congestion

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Delay-based TCP Vegas

Uses delay as a signal of congestion Idea try to keep a small constant

number of packets at bottleneck queueExpected = WBaseRTTActual = WCurRTT Diff = Expected - ActualTry to keep Diff between fixed 1 and 3

More recent FAST TCP based on VegasDelay-based TCP not widely used today

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP-Friendliness

Can we try MyFavNew TCP Well is it TCP-friendly

Any alternative congestion control scheme needs to coexist with TCP in FIFO queues in the best-effort Internet or be isolated from TCP

To co-exist with TCP it must impose the same long-term load on the network No greater long-term throughput as a function

of packet loss and delay so TCP doesnt suffer Not significantly less long-term throughput or

its not too useful

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP friendly rate control (TFRC)

Use a model of TCPs throughout as a function of the loss rate and RTT directly in a congestion control algorithm

If transmission rate is higher than that given by the model reduce the transmission rate to the models rate

Otherwise increase the transmission rateEg DCCP (Datagram Congestion Control

Protocol) for unreliable congestion controlQ how to measureuse loss rate and RTT

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

High speed TCP

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP in high speed networks

Example 1500 byte segments 100ms RTT want 10 Gbps throughput

Requires window size W = 83333 in-flight segments Throughput in terms of loss rate

p = 210-10 or equivalently at most one drop every couple hours

New versions of TCP for high-speed networks needed

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCPrsquos long recovery delay

More than an hour to recover from a loss or timeout

~41000 packets

~60000 RTTs~100 minutes

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

High-speed TCP

ProposalsScalable TCP HSTCP FAST CUBICGeneral idea is to use superlinear window

increase Particularly useful in high bandwidth-

delay product regimes

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Alternate choices of response functions

Scalable TCP - S = 015p

Q Whatever happened to TCP-friendly

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

High speed TCP [Floyd]

additive increase multiplicative decrease

increments decrements depend on window size

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Scalable TCP (STCP) [T Kelly]

multiplicative increase multiplicative decrease

W W + a per ACKW W ndash b W per window with loss

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

STCP dynamics

From 1st PFLDnet Workshop Tom Kelly

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Active Queue Management

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Router Queue Management

normally packets dropped only when queue overflows ldquodrop-tailrdquo queueing

ISPISProuter

ISPISPInternet

P1P2P3P4P5P6FCFS

SchedulerFCFS

Scheduler

router

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

The case against drop-tail queue management

Large queues in routers are ldquoa bad thingrdquo Delay end-to-end latency dominated by length

of queues at switches in network Allowing queues to overflow is ldquoa bad thingrdquo

Fairness connections transmitting at high rates can starve connections transmitting at low rates

Utilization connections can synchronize their response to congestion

P1P2P3P4FCFS

Scheduler

FCFSScheduler

P5P6

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Idea early random packet drop

When queue length exceeds threshold drop packets with queue length dependent probability probabilistic packet drop flows see same

loss rate problem bursty traffic (burst arrives when

queue is near threshold) can be over penalized

P1P2P3P4P5P6FCFS

Scheduler

FCFSScheduler

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Random early detection (RED) packet drop

Use exponential average of queue length to determine when to drop avoid overly penalizing short-term bursts react to longer term trends

Tie drop prob to weighted avg queue length avoids over-reaction to mild overload

conditions

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Random early detection (RED) packet drop

Maxthreshold

Minthreshold

Average queue length

Forced drop

Probabilisticearly drop

No drop

Time

Drop probabilityMax

queue length

100

Drop probability

maxp

Weighted AverageQueue Length

min max

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

RED summary why random drop

Provide gentle transition from no-drop to all-dropProvide ldquogentlerdquo early warningAvoid synchronized loss bursts among

sources Provide same loss rate to all sessions

With tail-drop low-sending-rate sessions can be completely starved

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Random early detection (RED) today

Many (5) parameters nontrivial to tune (at least for HTTP traffic)

Gains over drop-tail FCFS not that significant

Still not widely deployed hellip

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Why randomization important

Synchronization of periodic routing updates Periodic losses observed in end-end Internet

traffic

source Floyd Jacobson 1994

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Router update operation

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to arrive at dest)

start_timer (uniform Tp +- Tr)

timeout or link fail

update

time spent in statedepends on msgs

received from others(weak coupling between routers

processing)

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Router synchronization

20 (simulated) routers broadcasting updates to each other

x-axis time until routing update sent relative to start of round

By t=100000 all router rounds are of length 120

synchronization or lack thereof depends on system parameters

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Avoiding synchronization

Choose random timer component Tr large (eg several multiples of TC)

prepareown routing

update

(time TC)

receive update from neighbor

process (time TC2)

wait

receive update from neighborprocess

ltreadygt

send update (time Td to

arrive)

start_timer (uniform Tp +- Tr)Add enough

randomizationto avoid

synchronization

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Randomization

Takeaway message randomization makes a system simple

and robust

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Background transport TCP Nice

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

What are background transfers

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples Prefetched traffic on the Web File system backup Large-scale data distribution services Background software updates Media file sharing

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Desired Properties

Utilization of spare network capacity

No interference with regular transfers Self-interference

bull applications hurt their own performance

Cross-interferencebull applications hurt other applicationsrsquo performance

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP Nice

Goal abstraction of free infinite bandwidth Applications say what they want

OS manages resources and scheduling

Self tuning transport layer Reduces risk of interference with foreground

traffic Significant utilization of spare capacity by

background traffic Simplifies application design

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Why change TCP

TCP does network resource management Need flow prioritization

Alternative router prioritization+ More responsive simple one bit priority Hard to deploy

Question Can end-to-end congestion control achieve non-

interference and utilization

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal Congestion incr queue lengths incr RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control Receiver and network unchanged TCP friendly

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

TCP Nice

Basic algorithm 1 Early Detection thresh queue length incr in RTT 2 Multiplicative decrease on early congestion 3 Allow cwnd lt 10 (despite no loss)

per-ack operation if(curRTT gt minRTT + threshold(maxRTT ndash minRTT))

numCong++ per-round operation

if(numCong gt fW) W W2 else hellip AIMD congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Nice the works

Non-interference getting out of the way in time

Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

tBAdd Mul +

Reno

Nice

Add Add Add

Mul +

Mul +

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Network Conditions

01

1

10

100

1e3

1 10 100Fo

reg

rou

nd

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference to foreground Web traffic even when there isnrsquot much spare capacity

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Scalability

01

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W lt 1 allows Nice to scale to any number of background flows

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80 of spare capacity wo stealing any bandwidth from FG

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Wide-area network experiments

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

What is TCP optimizing

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

How does TCP allocate network resources

Problem Given a network and some number of long-lived TCP connections between different source-destination routes can we model the resulting resource allocation

How to model the interaction between TCP and the network Recall PFTK like models assumed network

conditions are not affected by (a single) TCP flow

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Optimization-based approach towards congestion control

Resource allocation as optimization problem How to allocate resources (eg bandwidth) to

optimize some objective function Maybe not possible to obtain exact optimality

but optimization framework as means to

explicitly steer network towards desirable operating point

practical congestion control as distributed asynchronous implementations of optimization algorithm

systematic approach towards protocol design

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

c1 c2

Model

Network Links l each of capacity cl

Sources s (L(s) Us(xs))L(s) - links used by source sUs(xs) - utility if source rate = xs

x1

x2 x3

121 cxx le+ 231 cxx le+

Us(xs)

xs

example utilityfunction for elastic application

Q What are possible allocations with say unit capacity links

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Optimization Problem

maximize system utility (note all sources ldquoequalrdquo) constraint bandwidth used less than capacity centralized solution to optimization impractical

must know all utility functions impractical for large number of sources can we view congestion control as distributed

asynchronous algorithms to solve this problem

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0 ldquosystemrdquo problem

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

The user view

User can choose amount to pay per unit time ws Would like allocated bandwidth xs in proportion to ws

euro

max Us

w s

ps

⎝ ⎜

⎠ ⎟minus ws

subject to w s ge 0

ps could be viewed as charge per unit flow for user s

s

ss p

wx =

userrsquos utility cost

user problem

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

The network view

Suppose network knows vector ws chosen by users

Network wants to maximize logarithmic utility function

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

network problem

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Solution existence

There exist prices ps source rates xs and amount-to-pay-per-unit-time ws = psxs such that Ws solves user

problem Xs solves the

network problem Xs is the unique

solution to the system problem

sum

sum

isin

ge

leS(l)s

ls

sss

x

cx

x w s

subject to

log max0

0 wsubject to

w Umax

s

ss

ge

minus⎟⎟⎠

⎞⎜⎜⎝

⎛s

s

wp

Llcx

xU

lSsls

sss

xs

isinforalllesum

sum

isin

ge

subject to

)( max

)(

0

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Proportional Fairness

Vector of rates xs proportionally fair if feasible and for any other feasible vector xs

0

leminussum

isinSs s

ss

xxx

Result if wr=1 then Xs solves the network problem IFF it is proportionally fair

Similar result exists for the case that wr not equal 1

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Max-min Fairness

Rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Minimum potential delay fairness

Rates xr are minimum potential delay fair if Ur (xr) = -wrxr

Interpretation if wr is file size then wrxr is transfer time optimization problem is to minimize sum of transfer delays

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Max-min Fairness

rates xr max-min fair if for any other feasible rates yr if ys gt xs then p such that xp xs and yp lt xp

What is corresponding utility function

minus=

minus

infinrarr 1lim)(

1r

rr

xxU

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Solving the network problem

Results so far existence - solution exists with given properties

How to compute solution Ideally distributed solution easily

embodied in protocolShould reveal insight into existing

protocol

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

congestion ldquosignalrdquo function of aggregate rate at link l fed back to s

change inbandwidth

allocation at s

linearincrease

multiplicative decrease

⎟⎟⎠

⎞⎜⎜⎝

⎛= sum

isin

)()()(

txgtpsLl

sllwhere

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Solving the network problem

⎟⎟⎠

⎞⎜⎜⎝

⎛minus= sum

isin

)()()()(

tptxwktxdt

d

sLllsss

Results converges to solution of relaxation of

network problem xs(t)pl(t) converges to ws

Interpretation TCP-like algorithm to iteratively solves optimal rate allocation

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Source Algorithm

Source needs only its path price

kr() nonnegative nondecreasing function Above algorithm converges to unique

solution for any initial condition qr interpreted as lossmarking

probability

euro

˙ x r = kr (xr )(Ur (xr ) minus qr)

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Proportionally-Fair Controller

If utility function is

then a controller that implements it is given by

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Pricing interpretation

Can network choose pricing scheme to achieve fair resource allocation

Suppose network charges price qr ($bit) where qr= pl

Userrsquos strategy spend wr ($sec) to maximize

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Optimal User Strategy

equivalently

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Simplified TCP-Reno

suppose

then

interpretation minimize (weighted) delay

pTpT

px

2)1(2asymp

minus=

TxxU

1)( minus=

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Is AIMD special

Consider a window control as followscwnd += acwnd^n when no losscwnd -= bcwnd^m when losswhere nltm

Expected change in congestion window

Expected change in rate per unit time

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

MIMD (nm)

Consider the controller

where

Then at equilibrium

Where α = m-n For stability

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Motivation

Congestion Control

maximize user utility

Traffic Engineeringminimize network

congestionGiven routing Rli how to adapt end rate xi Given traffic xi

how to perform routing Rli

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Congestion Control Model

max sum i Ui(xi)st sumi Rlixi le cl

var x

aggregate utility

Source rate xi

UtilityUi(xi)

capacity constraints

Users are indexed by i

Congestion control provides fair rate allocation amongst users

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Traffic Engineering Model

min suml f(ul)st ul =sumi Rlixicl

var R

Link Utilization ul

Costf(ul)

aggregate cost

Links are indexed by l

Traffic engineering avoids bottlenecks in the network

ul = 1

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Model of Internet Reality

xi Rli

Congestion Control

max sumi Ui(xi) st sumi Rlixi le cl

Traffic Engineering min suml f(ul)

st ul =sumi Rlixicl

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

System Properties

Convergence Does it achieve some objective Benchmark

Utility gap between the joint system and benchmark

max sumi Ui(xi)st Rx le cVar x R

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Multipath TCP

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control

Joint routing and congestion control

Multipath TCP controller

  • Congestion control
  • Why congestion control
  • Causescosts of congestion scenario 1
  • Causescosts of congestion scenario 2
  • Slide 5
  • Causescosts of congestion scenario 3
  • Slide 7
  • Two broad approaches towards congestion control
  • TCP congestion control
  • Components of TCP congestion control
  • Retransmission timeout estimation
  • TCP Throughput
  • TCP throughput A very very simple model
  • TCP throughput A very simple model
  • Deriving TCP throughputloss relationship
  • Slide 16
  • Alternate fluid model
  • TCP throughput A better loss rate based ldquosimplerdquo model [PFTK]
  • What is PFTK modeling
  • PFTK empirical validation Low loss
  • PFTK empirical validation High loss
  • Loss-based TCP
  • Delay-based TCP Vegas
  • TCP-Friendliness
  • TCP friendly rate control (TFRC)
  • High speed TCP
  • TCP in high speed networks
  • TCPrsquos long recovery delay
  • High-speed TCP
  • Alternate choices of response functions
  • High speed TCP [Floyd]
  • Scalable TCP (STCP) [T Kelly]
  • STCP dynamics
  • Active Queue Management
  • Router Queue Management
  • The case against drop-tail queue management
  • Idea early random packet drop
  • Random early detection (RED) packet drop
  • Slide 39
  • RED summary why random drop
  • Random early detection (RED) today
  • Why randomization important
  • Router update operation
  • Router synchronization
  • Avoiding synchronization
  • Randomization
  • Background transport TCP Nice
  • What are background transfers
  • Desired Properties
  • TCP Nice
  • Why change TCP
  • Slide 52
  • Slide 53
  • Nice the works
  • Network Conditions
  • Scalability
  • Utilization
  • Wide-area network experiments
  • What is TCP optimizing
  • How does TCP allocate network resources
  • Optimization-based approach towards congestion control
  • Model
  • Optimization Problem
  • The user view
  • The network view
  • Solution existence
  • Proportional Fairness
  • Max-min Fairness
  • Minimum potential delay fairness
  • Slide 70
  • Solving the network problem
  • Slide 72
  • Slide 73
  • Source Algorithm
  • Proportionally-Fair Controller
  • Pricing interpretation
  • Optimal User Strategy
  • Simplified TCP-Reno
  • Is AIMD special
  • MIMD (nm)
  • Motivation
  • Congestion Control Model
  • Traffic Engineering Model
  • Model of Internet Reality
  • System Properties
  • Multipath TCP
  • Joint routing and congestion control