Sizing Router BuffersSizing Router Buffers

Guido Appenzeller

Thesis Defense

May 24th, 2004

2

Routers need Packet BuffersRouters need Packet Buffers

It’s well known that routers need packet buffers It’s less clear why and how much

Goal of this work is to answer the question:

How much buffering do routers need?

3

How much Buffer does a Router How much Buffer does a Router need?need?

Universally applied rule-of-thumb: A router needs a buffer size:

2T is the two-way propagation delay (or just 250ms) C is capacity of bottleneck link

Context Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines.. Usually referenced to Villamizar and Song: “High Performance

TCP in ANSNET”, CCR, 1994. Already known by inventors of TCP [Van Jacobson, 1988] Has major consequences for router design

CTB 2

CRouterSource Destination

2T

4

ExampleExample

10Gb/s linecard Requires 300Mbytes of buffering. Read and write 40 byte packet every 32ns.

Memory technologies DRAM: require 4 devices, but too slow. SRAM: require 80 devices, 1kW, $2000.

Problem gets harder at 40Gb/s Hence RLDRAM, FCRAM, etc.

5

Outline of this WorkOutline of this Work

Main Results The rule of thumb is wrong for a core routers today

Required buffer is instead of CT 2n

CT 2

Outline of this talk Where the rule of thumb comes from Why it is incorrect for a core router in the internet today Correct buffer requirements for a congested router Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

6

OutlineOutline

The Rule of Thumb Where does the rule of thumb comes from? (Answer: TCP) Interaction of TCP flows and a router buffers

The buffer requirements for a congested router Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

7

TCPTCP

CC’ > C

Only W=2 packets may be outstanding

TCP Congestion Window controls the sending rate Sender sends packets, receiver sends ACKs Sending rate is controlled by Window W, At any time, only W unacknowledged packets may be outstanding

The sending rate of TCP is

RouterSource

RTT

WR

Dest

8

Single TCP FlowSingle TCP FlowRouter without buffersRouter without buffers

CC’ > C

t

CongestionWindow

Router DestSource

W

Only W packets may be outstanding

Link not fully utilized

4

8

Rule for adjusting W If an ACK is received: W ← W+1/W If a packet is lost: W ← W/2

9

Single TCP FlowSingle TCP FlowRouter with large enough buffers for full link utilizationRouter with large enough buffers for full link utilization

B

DestCC’ > C

Source

maxW

2maxW

t

Window size RTT

For every W ACKs received, send W+1 packets

RTT

WR

10

Required buffer is height of Required buffer is height of sawtoothsawtooth

t

B

0

11

Buffer = rule of thumbBuffer = rule of thumb

12

Microscopic TCP BehaviorMicroscopic TCP BehaviorWhen sender pauses, buffer drainsWhen sender pauses, buffer drains

one RTTDrop

13

Over-buffered LinkOver-buffered Link

14

Under-buffered LinkUnder-buffered Link

15

Origin of rule-of-thumbOrigin of rule-of-thumb Before and after reducing window size, the sending rate of the

TCP sender is the same

Inserting the rate equation we get

The RTT is part transmission delay T and part queuing delay B/C . We know that after reducing the window, the queuing delay is zero.

newold RR

new

new

old

old

RTT

W

RTT

W

T

W

CBT

W oldold

2

2/

/2

CTB 2

)1( 1

noldnew WW CTBn

2)(1

1

16

Rule-of-thumbRule-of-thumb

Rule-of-thumb makes sense for one flow Typical backbone link has > 20,000 flows Does the rule-of-thumb still hold?

Answer: If flows are perfectly synchronized, then Yes. If flows are desynchronized then No.

17

OutlineOutline

The Rule of Thumb The buffer requirements for a congested router

Synchronized flows Desynchronized flows The 2T×C/sqrt(n) rule

Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

18

If flows are synchronizedIf flows are synchronized

Aggregate window has same dynamics Therefore buffer occupancy has same dynamics Rule-of-thumb still holds.

2maxW

t

max

2

W

maxW

maxW

19

When are Flows When are Flows Synchronized?Synchronized?

Small numbers of flows tend to synchronize In ns2 simulation they are synchronized In at least some cases holds for real networks as well

Large aggregates of flows are not synchronized For > 500 flows, synchronization disappears in ns2 On a Cisco GSR, 100 flows were not synchronized Measurements in the core give no indication of

synchronization C. Fraleigh, “Provisioning Internet Backbone Networks to

support Latency Sensisitve Applications”, Ph.D. Thesis, Stanford

Hohn, Veitch, Papagiannaki and Diot – “Bridging Router Performance and Queuing Theory”

20

If flows are not If flows are not synchronizedsynchronized

ProbabilityDistribution

B

0

Buffer Size

W

21

Quantitative ModelQuantitative Model Model congestion window of a flow as random variable

)(tWi model as )(][ xfxWP i iW where

For many de-synchronized flows We assume congestion windows are independent All congestion windows have the same probability distribution

2]var[][ WiWi WWE

Now central limit theorem gives us the distribution of the sum of the window sizes

)1,0()( NnntW WWn

i

22

Buffer vs. Number of FlowsBuffer vs. Number of Flowsfor a given Bandwidthfor a given Bandwidth

If for a single flow we have

Standard deviation of sum of windows decreases with n

)1,0(1

)( 11 Nn

tW nnn

i

Thus as n increases, buffer size should decrease

n

BB n 1

211 ]var[][ nn WWE

For a given C, the window W scales with 1/n and thus2

11 ]var[][

nW

nWE nn

23

Required buffer sizeRequired buffer size

2T C

n

Simulation

24

SummarySummary Flows in the core are desynchronized

Substantial experimental evidence Supported by ns2 simulations

For desynchronized flows, routers need only buffers of

n

CTB

2

25

OutlineOutline

The Rule of Thumb The buffer requirements for a congested router Buffer requirements for short flows (slow-start)

M/G/1 Model

Experimental Verification Conclusion

26

Short FlowsShort Flows

So far we were assuming a congested router with long flows in congestion avoidance mode. What about flows in slow start? Do buffer requirements differ?

Answer: Yes, however: Required buffer in such cases is independent of line

speed and RTT (same for 1Mbit/s or 40 Gbit/s) In mixes of flows, long flows drive buffer requirements

Short flow result relevant for uncongested routers

27

A single, short-lived TCP flowA single, short-lived TCP flowFlow length 62 packets, RTT ~140 msFlow length 62 packets, RTT ~140 ms

2

4

8

16

32

RTTsynfin ack

received

Flow Completion Time (FCT)

28

Modelling Short FlowsModelling Short Flows

Idea: Find buffer size by modelling queue behaviour Problem: Arrival process is hard to model

Simplify by modelling bursts as independent Buffer empties several times during one RTT

Poisson arrivals of flows

Service time is Lflow, the

flow length in packets

Poisson arrivals of bursts

Service time is the lengthof the burst: 2,4,8,16…

29

M/G/1 Model for short flowsM/G/1 Model for short flows TCP flows generate independent bursts

Service times is burst length :

Poisson arrivals of rate

To verify if this approach works, let’s compare the average queue length in the model and in simulation

]E[S

ρλ

...},,,{S

iburst

i

16842

]E[S

]E[S

-(

ρ]E[S]E[NE[Q]

i

iiQ

22

)12

30

Average Queue lengthAverage Queue length

capacity :C 40Mbit sload : 0.8

ρ)E[S]-2(1]E[Sρ 22

31

Buffer Requirements Buffer Requirements for Short Flowsfor Short Flows

P(Q = x)

Q

Buffer B

Packet Loss

Bx

Drop x)P(QP

Problem: For M/G/1 there is no closed form expression for the queue distribution

Buffers absorb fluctuations in queue, reduce packet loss Reduce retransmits, Timeouts and thereby flow completion time Utilization not a good measure of QoS as load << 1

We can find a good upper bound for loss, if we have the queue length distribution for an infinite buffer If a packet arrives and queue length is shorter than buffer,

packet will not be dropped

32

Queue DistributionQueue Distribution

We derived closed-form estimates of the queuedistribution using Effective Bandwidth

Gives very good closed form approximation

]E[S

E[S]

ρ

ρ)(κeb)P(Q bκ

2

12

33

Short Flow SummaryShort Flow Summary

Buffer requirements for short flows Can be modeled by M/G/1 model Only depends on load and burst size distribution Example - for bursts of up to size 16 at load 0.8

For 1% loss probability B = 115 Packets For 0.01% loss probability B = 230 packets etc. Bursts of size 12 is maximum for Windows XP

Independent of line speed and RTT

In mixes of flows, long flows dominate buffer requirements

34

OutlineOutline

The Rule of Thumb The buffer requirements for a congested router Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

35

Experimental Evaluation Experimental Evaluation OverviewOverview

Simulation with ns2 Over 10,000 simulations that cover range of settings

Simulation time 30s to 5 minutes Bandwidth 10 Mb/s - 1 Gb/s Latency 20ms -250 ms,

Physical router Cisco GSR with OC3 line card In collaboration with University of Wisconsin

Experimental results presented here Long Flows - Utilization Mixes of flows - Flow Completion Time (FCT) Mixes of flows - Heavy Tailed Flow Distribution Short Flows – Queue Distribution

36

Long Flows - Utilization (I)Long Flows - Utilization (I)Small Buffers are sufficient - OC3 Line, ~100ms RTTSmall Buffers are sufficient - OC3 Line, ~100ms RTT

2T C

n

99.9%

98.0%

99.5%2T C

n

2×

37

Long Flows – Utilization (II)Long Flows – Utilization (II) Model vs. ns2 vs. Physical Model vs. ns2 vs. Physical

RouterRouterGSR 12000, OC3 Line CardGSR 12000, OC3 Line Card

TCP

Flows

Router Buffer Link Utilization

Pkts RAM Model Sim Exp

100 0.5 x

1 x

2 x

3 x

64

129

258

387

1Mb

2Mb

4Mb

8Mb

96.9%

99.9%

100%

100%

94.7%

99.3%

99.9%

99.8%

94.9%

98.1%

99.8%

99.7%

400 0.5 x

1 x

2 x

3 x

32

64

128

192

512kb

1Mb

2Mb

4Mb

99.7%

100%

100%

100%

99.2%

99.8%

100%

100%

99.5%

100%

100%

99.9%

2T C

n

38

Mixes of FlowsMixes of FlowsFlow Completion TimeFlow Completion Time

FCT of 14 packet flows that share a link with long-lived flows.FCT of 14 packet flows that share a link with long-lived flows.

2T C

n

CT 2

39

Heavy-tailed flow length Heavy-tailed flow length distributiondistribution

Experiment Flow arrivals are a Poisson process Flow lengths are Pareto distributed

Results Buffers in the order of are still sufficient Number of “long-lived” flows n is now defined as

number of flows in congestion avoidance mode

2T C

n

40

Pareto Flow DistributionPareto Flow DistributionFinding the number of flowsFinding the number of flows

No.

of

flow

s in

CA

mod

e

For buffer sizing, pick n = 100

time [seconds]

41

Pareto Flow DistributionPareto Flow DistributionF

low

arr

ival

son

link

1B

ottle

neck

utili

zatio

nB

uffe

rO

ccup

ancy

42

Short Flows – Queue Short Flows – Queue DistributionDistribution

M/G/1 Model vs. GSR 12000, OC3 Line CardM/G/1 Model vs. GSR 12000, OC3 Line Card

43

OutlineOutline

The Rule of Thumb The buffer requirements for a congested router Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

44

Related WorkRelated Work

Related Publications Buffer sizing

“High Speed TCP in ANS Net” - Villamizar and Song, ACCR 1994

“TCP behaviour with many flows” – R. Morris, IEEE ICNP 1997

“Scalable TCP congestion control” – R. Morris, INFOCOM 2000

Queue Modelling “Modelling, Simulation and Measurement of Queuing

Delay” – Garetto and Towsley, SIGMETRICS 2003

45

Original ContributionsOriginal Contributions

Main original contributions of this work

Routers only require buffers of instead of

Models on TCP buffer interaction for Congestion avoidance mode Slow start

Experimental Verification

Publication “Sizing Router Buffers – Guido Appenzeller, Isaac

Keslassy and Nick McKeown, to appear at SIGCOMM 2004

CT 2n

CT 2

46

The Commercial Internet TodayThe Commercial Internet Today Today’s internet differs from our assumptions

Core is overprovisioned, almost never congested Access links are usually the bottleneck (DSL, Modem) Flows are usually limited by Maximum Window size

Maximum Window is from 6-12 (Windows) to 42 (Unix) Are the results still relevant?

Answer: Yes, we were intentionally pessimistic Routers still needs to work in case of congestion

Even if this “worst case” scenario is rare Slow access links and small TCP windows reduce buffer

requirements further Bursts are smoothed out Converges towards constant rate source, poisson packet arrivals We verified reduced buffer requirements experimentally

47

How much buffer does a router How much buffer does a router need?need?

Scenario Buffer Comments

Single flow saturates routerFew synchronized flows with congestion

Still applicable in a few select cases (e.g. Internet2 speed record)

CT 2

n

CT 2

Scenario Buffer Comments

Many flowsCongestion

Applicable for the core and edge of the internet today.

One or many flowsNot Congested, ρ < 1

M/G/1 modelOnly if there is never any congestion

The old “Rule-of-Thumb”

Our Contribution

48

Impact on Router DesignImpact on Router Design

10Gb/s linecard with 200,000 x 56kb/s flows Rule-of-thumb: Buffer = 2.5Gbits

Requires external, slow DRAM

Becomes: Buffer = 6Mbits Can use on-chip, fast SRAM Completion time halved for short-flows

40Gb/s linecard with 40,000 x 1Mb/s flows Rule-of-thumb: Buffer = 10Gbits Becomes: Buffer = 50Mbits

Questions?Questions?

BackupBackup

51

Two TCP FlowsTwo TCP FlowsTwo flows sharing a bottleneck Two flows sharing a bottleneck

synchronizesynchronize

52

Impact on Protocol DesignImpact on Protocol Design The ‘rule-of-thumb’ is due to the nature of TCP

We can easily build a TCP that requires less buffers

Latency based TCP (Vegas, FAST) suffer from smaller buffers Queuing delay too small to be used as a channel Buffer occupancy similar for different levels of congestion Unclear if these protocols work with aggregated traffic at all

For future protocol design, buffer requirements should be considered

CTBWW oldnew 2)1(2

1

CTBWWnnoldnew 2)()1(

1

11

TCP Reno: requires

General: requires

53

Smaller buffers increase lossSmaller buffers increase lossBut this in itself is not a bad thingBut this in itself is not a bad thing

Routers drop packets to throttle senders Loss rate is a function of number of flows and sum of

TCP windows

Rule-of-thumb vs. minimal buffers changes number of outstanding packets by a factor of 2 Loss will quadruple with minimal buffers

This is just the way how TCP works Quality of service better even with higher loss…

22

76.0

iW

nl