Congestion Control on High-Speed Networks Injong Rhee, Lisong Xu Computer Science Department North...

Congestion Control on High-Speed Networks

Injong Rhee, Lisong XuComputer Science Department

North Carolina State University

2

Outline

Motivation Related Work BIC-TCP Future Work

3

High-Speed Networks

NLNLSURFnet

GENEVA

UKUKSuperJANET4

ABILENEABILENE

ESNETESNET

CALRENCALREN

ItItGARR-B

GEANT

NewYork

FrFrRenater

STAR-TAP

STARLIGHT

Many high-speed networks are being developed. BW: 10Gbps, and expected to be upgraded to higher speeds in the near future.

4

ESNET: Energy Science Network

NLNLSURFnet

GENEVA

UKUKSuperJANET4

ABILENEABILENE

ESNETESNET

CALRENCALREN

ItItGARR-B

GEANT

NewYork

FrFrRenater

STAR-TAP

STARLIGHT

ESNet is funded by the Department of Energy to provide a high-speed network serving thousands of scientists worldwide

ESNETESNET

5

High-Speed Applications

Transport Protocols be able to transfer a large amount of data

over a long distance within a short amount of time

High-Speed Networks

High-Speed Applications

Weather Simulation Video Conference Telemedicine

6

TCP Performance

Ns-2: capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps100 sources, 100 ms round trip propagation delay, 1500-Byte Packet Size

155622

2500

5000

10000

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2000 4000 6000 8000 10000Link Capacity (Mbps)

Link Utilization

NS-2 Simulation (100 sec) Link Capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps, Drop-Tail Routers, 0.1BDP Buffer 5 TCP Connections, 100ms RTT, 1000-Byte Packet Size

Utilization of a link with 5 TCP connections

Cannot fully utilize the huge capacity of high-

speed networks!

7

TCP Congestion Control

The instantaneous throughput of TCP is controlled by a variable cwnd, TCP transmits approximately a cwnd number of packets per RTT

(Round-Trip Time).

Time (RTT)Slow start Congestion avoidance

Packet loss Packet loss Packet loss

cwnd

Packet loss TCP

cwnd = cwnd + 1 cwnd = cwnd * (1-1/2)

8

TCP over High-Speed Networks

Packet loss

Time (RTT)Congestion avoidance

Packet loss Packet loss

cwnd

Slow start

Packet loss

A TCP connection with 1250-Byte packet size and 100ms RTT is running over a 10Gbps link (assuming no other connections, and no buffers at routers)

100,000 10Gbps

50,000 5Gbps

1.4 hours 1.4 hours 1.4 hours

TCP

bigdecrease

slowincrease

9

Response Function of TCP

Response function of TCP is the average throughput of a TCP connection in terms of the packet loss probability, the packet size, and the round-trip time.

5.0

2.1

pRTT

MSSR =

R : Average Throughput MSS: Packet Size RTT: Round-Trip Time P : Packet Loss Probability

Response Function of TCP is :

J. Padhye, V. Firoio, D. Towsley, J. Kurose, "Modeling TCP Throughput: a Simple Model and its Empirical Validation", Proceedings of SIGCOMM 98

10

Response Function of TCP

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-10 1.0E-09 1.0E-08 1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02

Packet Loss Probability

Throughput (Mbps)

TCP

10Gbps requires a packet loss rate of 10-10, or correspondingly a link bit error rate of at most 10-14, which is an unrealistic (or at least hard) requirement for current networks

Assuming 1250-Byte packet size, and 100ms RTT

11

Outline


12

Proposed High-Speed Protocols

Window-Based Protocols AIMD (Additive Increase Multiplicative Decrease)

Jumbo Frame, GridFTP, PFTP, PSockets HSTCP (High-Speed TCP) by Sally Floyd at ICIR, Berkeley STCP (Scalable TCP) by Tom Kelly at Cambridge University FAST (Fast AQM Scalable TCP) by Steven Low at California

Institute of Technology

Rate-Based Protocols SABUL (Simple Available Bandwidth Utilization Library ) by

Robert Grossman at University of Illinois at Chicago

window-based protocols are known for safer incremental deployment. D. Bansal, H. Balakrishnan, S. Floyd, and S. Shenker, "Dynamic behavior of slowly responsive congestion controls", In Proceedings of SIGCOMM 2001, San Diego, California.

13

AIMD (Additive Increase Multiplicative Decrease)

AIMD increases cwnd by a larger number, say 32, instead of 1 per RTT. After a packet loss, AIMD decreases cwnd by 1/8, instead of 1/2

Packet loss



cwnd

Packet loss

cwnd = cwnd + 1

cwnd = cwnd + 32

cwnd = cwnd * (1-1/2)

cwnd = cwnd * (1-1/8)

TCP

14

Response Function of AIMD

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

AIMD5.0

2.1

pRTT

MSSR = TCP:

5.0

5.15

pRTT

MSSR = AIMD:

The throughput of AIMD is always about 13 times larger than that of TCP

15

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

AIMD

Properties of AIMD

Bandwidth Scalable

Bandwidth ScalabilityThe ability to achieve 10Gbps with a reasonable packet loss probability

NOT TCP Friendly

TCP-FriendlinessThe ability to share bandwidth with TCP connections on low-speed networks

16

STCP (Scalable TCP)

STCP adaptively increases cwnd, and decreases cwnd by 1/8.

Packet loss



cwnd

Packet loss

cwnd = cwnd + 1

cwnd = cwnd + 0.01*cwnd

cwnd = cwnd * (1-1/2)

cwnd = cwnd * (1-1/8)

TCP

17

HSTCP (High Speed TCP) HSTCP adaptively increases cwnd, and adaptively decreases cwnd. The larger the cwnd, the larger the increment, and the smaller the

decrement.

Packet loss



cwnd

Packet loss

cwnd = cwnd * (1-1/2)

cwnd = cwnd * (1-dec(cwnd))

cwnd = cwnd + 1

cwnd = cwnd + inc(cwnd)

TCP

18

Response Functions of HSTCP and STCP

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

AIMD

HSTCP

STCP

835.0

12.0

pRTT

MSSR = HSTCP:

pRTT

MSSR

08.0= STCP:

Bandwidth Scalable

TCP Friendly

HSTCP and STCP are both bandwidth scalable and TCP friendly

19

Outline

MotivationRelated Work BIC-TCP

Why another one? Protocol Design Parameter Setting Simulations & Experiments

Future Work

20

RTT Fairness

Different connections may have quite different round-trip times, and a good protocol should allocate bandwidth fairly among those connections

RTT fairness index = throughout ratio of two flows with different RTTs

21

RTT Fairness on Low-Speed Networks

dp

c

RTT

MSSR =

For a protocol with the following response function, where c and d are protocol-related constants.

⎟⎟⎠

⎞⎜⎜⎝

⎛

1

2

RTT

RTT

The RTT Fairness Index (or the throughput ratio of two flows) on low-speed networks is

Lisong Xu, Khaled Harfoush, and Injong Rhee, "Binary Increase Congestion Control for Fast Long-Distance Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong

On low speed networks, different protocols have the same RTT fairness

22

RTT Fairness on High-Speed Networks

dp

c

RTT

MSSR =

For a protocol with the following response function, where c and d are protocol-related constants.

d

RTT

RTT −

⎟⎟⎠

⎞⎜⎜⎝

⎛ 11

1

2

The RTT Fairness Index (or the throughput ratio of two flows) on high-speed networks is

Lisong Xu, Khaled Harfoush, and Injong Rhee, "Binary Increase Congestion Control for Fast Long-Distance Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong

On high speed networks, the RTT fairness of a protocol depends on the exponent d in the response function

23

If the response function is: then the RTT Fairness is:

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-02


Packets/RTT

TCP

AIMD

HSTCP

STCP

Slope Determines the RTT Fairness

dp

c

RTT

MSSpR =)( d

RTT

RTT −

⎟⎟⎠

⎞⎜⎜⎝

⎛ 11

1

2

The figure is shown in log-log scale, so exponent d in the response function is the slope of the line in the figure

The slope of the line determines the RTT fairness of a protocol on high-speed networks

24

Simulation Results of RTT Fairness

Inverse RTT Ratio 1 3 6

AIMD 1 6 22

HSTCP 1 29 107

STCP 1 127 389

Simulation setup: BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

Throughout ratio of two flows on a 2.5Gbps Link

25

Simulation Result of 2 STCP Flows

Throughput of two STCP flows with different RTTs on a 2.5Gbps link

Simulation setup: 2.5Gbps, BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

26

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

AIMD

HSTCP

STCPBIC

Design Goal

TCP Fairness

Scalability, RTT Fairness

27

Outline

Motivation Related Work BIC-TCP


Future Work

28

BIC (Binary Increase Congestion control)

BIC adaptively increase cwnd, and decrease cwnd by 1/8

Packet loss



cwnd

Packet loss

cwnd = cwnd + 1

cwnd = cwnd + f(cwnd, history)

cwnd = cwnd * (1-1/2)

cwnd = cwnd * (1-1/8)

TCP

29

A Search Problem

A Search Problem We consider the increase part of congestion avoidance as a

search problem, in which a connection looks for the available bandwidth by comparing its current throughput with the available bandwidth, and adjusting cwnd accordingly.

How does TCP find the available bandwidth?

Linear search

while (no packet loss){

cwnd++;

}

Q: How to compare R with A?

R = current throughput

= cwnd/RTT

A = available bandwidth A: Check for packet losses

No packet loss: R <= A Packet losses : R > A

30

Linear Search

0

32

64

96

128

160

192

224

256

0 50 100 150 200 250

Time (RTT)

cwnd

Linear Search

Available Bandwidth

31

BIC: Binary Search with Smax and Smin

BIC - Binary search

while (Wmin <= Wmax){

inc = (Wmin+Wmax)/2 - cwnd;

if (inc > Smax)

inc = Smax;

else if (inc < Smin)

inc = Smin;

cwnd = cwnd + inc;

if (no packet losses)

Wmin = cwnd;

else

break;

}

Wmax: Max Window Wmin: Min Window Smax: Max Increment Smin: Min Increment

32

Binary Search with Smax and Smin

0

32

64

96

128

160

192

224

256

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Time (RTT)

cwnd

Linear Search

Binary Search with Smax and Smin

Smin

Smax

Wmax

Wmin

Available Bandwidth

33

Binary Increase Congestion Control (BIC)

Binary search increase

Additive increase

34

Outline



Future Work

35

Setting Smax

Response Function of BIC on high-speed networks

5.0max7.2

p

S

RTT

MSSR =

Bandwidth scalability of BIC depends only on Smax

RTT Fairness of BIC on high-speed networks is the same as that of AIMD

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

Smax=16

Smax=32

Smax=64

Bandwidth scalability

36

Setting Smin

Response Function of BIC on low-speed networks

( )min,SpfRTT

MSSR =

TCP-friendliness of BIC depends only on Smin

1.0E+01

1.0E+02

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02


Throughput (Mbps)

TCP

Smin=0.1

Smin=0.01

Smin=0.001

TCP friendliness

37

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-02


Packets/RTT

TCP

AIMD

HSTCP

STCPBIC

Response Functions

Bandwidth scalability RTT Fairness

TCP-Friendliness

38

Outline


Why another one? Protocol Design Parameter Setup Simulations & Experiments

Future Work

39

Bandwidth Scalability

Ns-2: capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps100 sources, 100 ms round trip propagation delay, 1500-Byte Packet Size

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2000 4000 6000 8000 10000

Capacity (Mbps)

Link Utilization

TCP

AIMDHSTCP

STCPBIC

NS-2 Simulation (100 sec) Link Capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps, Drop-Tail Routers, 0.1BDP Buffer 5 Connections, 100ms RTT, 1000-Byte Packet Size

40

TCP - Friendliness

0%

20%

40%

60%

80%

100%

AIMDBIC

HSTCPSTCP

Long-Lived TCP Long-Lived Highspeed Background Unused

Simulation setup: 20Mbps, BDP Buffer, Drop Tail, Reverse Traffic2 TCP flows, 2 high-speed flows, and some background traffic

41

RTT Fairness

Inverse RTT Ratio 1 3 6

BIC 1 12 38

AIMD 1 6 22

HSTCP 1 29 107

STCP 1 127 389

Throughput ratio of two flows with different RTTs on a 2.5Gbps link

Simulation setup: BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

42

Buffer Size and Throughput

43

Queue Size Vs. Buffer Size

44

Packet loss vs. Buffer Size

45

Simulation Summary

AIMD HSTCP STCP BIC

Scalability TCP-Friendliness RTT Fairness

46

SLAC Experiments

Evaluation of Advanced TCP Stacks on Fast Long-Distance Production Networks by SLAC (Stanford Linear Accelerator Center) Linux TCP Parallel TCP HighSpeed TCP Scalable TCP HighSpeed TCP Low Priority Hamilton TCP BIC TCP

“BIC TCP overall performs very well in our tests” http://www-iepm.slac.stanford.edu/bw/tcp-eval/

47

Outline


48

Conclusion and Future Work

• Conclusion• Designed a congestion control protocol, which is scalable, TCP-

friendly, and has a good RTT fairness.• "Binary Increase Congestion Control for Fast Long-Distance

Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong

• Our work has been reported by some newspapers. • NBC (US)• The Washington Times (US)• The Times of India (India)• Sina (China)

• Future Work• Further improve the performance of BIC• Solve other problems in high-speed networks

49

CUBIC

0

64

128

192

256

320

384

448

512

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Time (RTT)

cwnd

( )3max KtCWCwnd −+=

50

More information:http://www.csc.ncsu.edu/faculty/rhee/export/bitcp

Thank you!

Any questions?

Congestion Control on High-Speed Networks Injong Rhee, Lisong Xu Computer Science Department North...

Documents

Transcript of Congestion Control on High-Speed Networks Injong Rhee, Lisong Xu Computer Science Department North...