FAST TCP Speaker: Ray Veune: Room 1026 Date: 25 th October, 2003 Time:10:00am.
-
date post
20-Dec-2015 -
Category
Documents
-
view
214 -
download
1
Transcript of FAST TCP Speaker: Ray Veune: Room 1026 Date: 25 th October, 2003 Time:10:00am.
FAST TCPFAST TCP
Speaker: Speaker: RayRayVeune: Veune: Room 1026Room 1026Date: Date: 2525thth October, 2003 October, 2003Time:Time: 10:00am10:00am
MotivationMotivation
Demand for ultrascale networkingDemand for ultrascale networking HENP (High Energy and Nuclear Physics)HENP (High Energy and Nuclear Physics) Data volumes of tens of Petabytes (10Data volumes of tens of Petabytes (101515) to ) to
Exabytes (10Exabytes (101818)) Require Terabit/sec (10Require Terabit/sec (1015 15 bit/sec or 1000Gbbit/sec or 1000Gb
it/sec)it/sec) Scalability problem of TCPScalability problem of TCP
Losses must be extremely rareLosses must be extremely rare TCP must induce lossTCP must induce loss Underutilization and oscillationUnderutilization and oscillation
Scalability problem of TCP Scalability problem of TCP extremely loss packet loss extremely loss packet loss
possibilitypossibility Rate = 1.3 * MTU / (RTT * sqrt(Loss))Rate = 1.3 * MTU / (RTT * sqrt(Loss)) MTU = 1500bytes, RTT = 10msMTU = 1500bytes, RTT = 10ms
BandwidthBandwidth Loss rateLoss rate
10Mbps10Mbps 2.43 x 102.43 x 10-2-2
100Mbps100Mbps 2.43 x 102.43 x 10-4-4
1Gbps1Gbps 2.43 x 102.43 x 10-6-6
10Gbps10Gbps 2.43 x 102.43 x 10-8-8
1Tbps1Tbps 2.43 x 102.43 x 10-12-12
1Tbps (RTT:100ms)1Tbps (RTT:100ms) 3.80 x 103.80 x 10-14-14
Scalability problem of TCP Scalability problem of TCP inevitable packet lossinevitable packet loss
TCP needs to create lossesTCP needs to create losses Single bit network feedback signalSingle bit network feedback signal
Scalability problem of TCP Scalability problem of TCP Underutilization and oscillationUnderutilization and oscillation
AIMD (1, 0.5)AIMD (1, 0.5) At large window size (in excess of 10,0At large window size (in excess of 10,0
00 pkts):00 pkts): Halving window on loss event is too drastiHalving window on loss event is too drasti
cc Increasing window by one packet per RTT Increasing window by one packet per RTT
is too conservativeis too conservative
FAST TCPFAST TCPAchievementAchievement
CERN (European Organization for NucCERN (European Organization for Nuclear Research) sent 1.1 Terabytes of dalear Research) sent 1.1 Terabytes of data at 5.44 Gbpsta at 5.44 Gbps
Full-length DVD film in 7 seconds !!Full-length DVD film in 7 seconds !!
FAST TCPFAST TCP
Flow based vs Packet basedFlow based vs Packet based Network delay vs Packet lossNetwork delay vs Packet loss TCP-Vegas vs TCP-RenoTCP-Vegas vs TCP-Reno Stabilized VegasStabilized Vegas
TCP VegasTCP VegasTechniquesTechniques
New Retransmission MechanismNew Retransmission Mechanism Congestion Avoidance MechanismCongestion Avoidance Mechanism Modified Slow-Start MechanismModified Slow-Start Mechanism
TCP VegasTCP VegasNew Retransmission MechanismNew Retransmission Mechanism
TimeoutTimeout nn duplicate ACKs duplicate ACKs
TCP VegasTCP VegasNew Retransmission MechanismNew Retransmission Mechanism
Check time record fCheck time record for the first duplicate or the first duplicate packetpacket
non-duplicate ACKs non-duplicate ACKs first or second after first or second after retransmissionretransmission Catch other segment Catch other segment
lost previous to retralost previous to retransmissionnsmission
TCP VegasTCP VegasCongestion Avoidance MechanismCongestion Avoidance Mechanism
Detect network delay by monitoring RDetect network delay by monitoring RTTTT
BaseRTT and ActualRTTBaseRTT and ActualRTT
Source Dest.Router
Expected = WindowSize / BaseRTTExpected = WindowSize / BaseRTT
Diff = Expected - ActualDiff = Expected - Actual Diff >> 0, Diff >> 0, decrease sending ratedecrease sending rate Diff = 0, Diff = 0, increase sending rateincrease sending rate
αα < Diff < < Diff < ββ
TCP VegasTCP VegasCongestion Avoidance MechanismCongestion Avoidance Mechanism
αα ββ DiffDiff
TCP VegasTCP VegasCongestion Avoidance MechanismCongestion Avoidance Mechanism
Extra buffers occupiedExtra buffers occupied BaseRTT: 100ms, segment size: 1KBBaseRTT: 100ms, segment size: 1KB
Expected = WindowSize / BaseRTTExpected = WindowSize / BaseRTTαα = 30KB/s, = 30KB/s, ββ=60KB/s=60KB/s
αα => 30KBps * 100ms / 1KB = 3=> 30KBps * 100ms / 1KB = 3 ββ=> 60KBps * 100ms / 1KB = 6=> 60KBps * 100ms / 1KB = 6
TCP VegasTCP VegasCongestion Avoidance MechanismCongestion Avoidance Mechanism
αα==ββ Diff <Diff <αα
increase one segment per RTTincrease one segment per RTT Diff =Diff =αα
no change in windows sizeno change in windows size Diff >Diff >αα
decrease one segment per RTTdecrease one segment per RTT
TCP VegasTCP VegasSlow-Start MechanismSlow-Start Mechanism
TCP-renoTCP-reno Send two segment for each ACK receivedSend two segment for each ACK received Exponential growth every RTTExponential growth every RTT
TCP-VegasTCP-Vegas Exponential growth every alternative RTTExponential growth every alternative RTT γγthresholdthreshold
Diff >Diff >γγ Changes from slow-start mode to linear I/D Changes from slow-start mode to linear I/D
modemode
Stability of TCP Vegas Stability of TCP Vegas Network modelNetwork model
Set of Set of LL links with finite capacities links with finite capacities cc c = c = ((ccl l , l , l L L))
NN sources indexed by sources indexed by rr Each source r uses a set of link defined Each source r uses a set of link defined
by the by the LL NN routing matrix routing matrix
RRlrlr = { = { 1 if source r uses link l0 otherwise
Stability of TCP VegasStability of TCP VegasNetwork modelNetwork model
For each link For each link ll, the congestion , the congestion measure measure ppll((tt) is call price) is call price
For each source For each source rr, it maintains a , it maintains a rate rate xxrr((tt) in packets/sec) in packets/sec
Equilibrium forward delay from Equilibrium forward delay from source source rr to link to link ll : :
Equilibrium backward delay from Equilibrium backward delay from link link ll to source to source rr : :
lr
lr
Stability of TCP VegasStability of TCP VegasNetwork modelNetwork model
Aggregate price source Aggregate price source rr observes in its pat observes in its pathh qqrr((tt) = ) = RRlr lr ppl l ((t t - - ) )
Aggregate source rate link Aggregate source rate link ll observes observes yyll ((tt) = ) = RRlrlr x xrr ((t -t - ) )
x1(t)
x2(t)
p1(t)
p2(t)
p3(t) p4(t)
llr
rlr
Stability of TCP VegasStability of TCP VegasNetwork modelNetwork model
Tr denote equilibrium RTTTr denote equilibrium RTT + + = = TrTr,, ll LL
Dynamical system of TCP VegasDynamical system of TCP Vegas
ppll ( (tt) =) =( ( yyl l ( ( t t ) – ) – ccll ) / ) / ccll if if ppll ( (tt) > 0) > 0( ( yyl l ( ( t t ) – ) – ccll ) / ) / ccl l ) )++ if if ppll ( (tt) = 0) = 0
xxrr ( (tt) = 1/) = 1/TrTr 22(t) sgn( 1 – (t) sgn( 1 – xxrr((tt))qqrr((tt) / ) / rrddrr ) )
TrTr ( (tt) = ) = ddrr + + q qrr( ( tt ) )
WhereWhere sgn(z) = 1 if z > 0, -1 if z < 0 and 0 if z = 0sgn(z) = 1 if z > 0, -1 if z < 0 and 0 if z = 0(z)(z)++ = max { 0 , z } = max { 0 , z }
lr lr
Stability of TCP Vegas Stability of TCP Vegas Approximate modelApproximate model
xxrr ( (tt) = 1/) = 1/TrTr 22(t) sgn( 1 – (t) sgn( 1 – xxrr((tt))qqrr((tt) / ) / rrddrr ) ) sgn(z) sgn(z) 2/ 2/ tan tan-1-1 ( (z)z)
xxrr ( (tt) = ) = ((2/2/TrTr 22(t)(t)))tantan-1-1 (1 – (1 – xxrr((tt))qqrr((tt) / ) / rrddrr ) )
Stability of TCP Vegas Stability of TCP Vegas Approximate modelApproximate model
In equilibrium, the source rate In equilibrium, the source rate xxrr** and and
aggregate price aggregate price qqrr** satisfy satisfy
xxrr** q qrr
** = = r r ddrr
Stability of TCP Vegas Stability of TCP Vegas Theorem 1Theorem 1
Suppose for all Suppose for all rr, , kk00TrTr max maxrr TrTr for some for some kk00.. Let M be an upper bound on the number of linkLet M be an upper bound on the number of link
s in the path of any source, M s in the path of any source, M max maxrr ll RRlrlr.. The Vegas model is locally asymptotically stable The Vegas model is locally asymptotically stable
around the equilibrium point (around the equilibrium point (xxrr** , , yyll
* * , , ppll** , , qqrr
** ) if ) if
maxmaxr r xxrr** Tr sinc Tr sinc (ň / x (ň / xrr
** Tr ) < Tr ) < / Mk / Mk0022
ň = 2ň = 2// Let Let (a) be the unique solution in ( 0, (a) be the unique solution in ( 0, /2) of/2) of tan tan = a as a strictly i = a as a strictly i
ncreasing function of ancreasing function of a ssinc inc = sin = sin / /
Stability of TCP Vegas Stability of TCP Vegas Theorem 1Theorem 1
maxmaxr r xxrr** Tr sinc Tr sinc (ň / x (ň / xrr
** Tr ) < Tr ) < / Mk / Mk0022
() is strictly increasing() is strictly increasing sincsinc() is strictly decreasing() is strictly decreasing LHS is strictly increasing in windows sLHS is strictly increasing in windows s
ize ize xxrr** Tr Tr
Theorem 1: Stability condition impose Theorem 1: Stability condition impose a limit on max windows sizea limit on max windows size
Stability of TCP Vegas Stability of TCP Vegas Corollary 2Corollary 2
maxmaxr r xxrr** Tr sinc Tr sinc (ň / x (ň / xrr
** Tr ) < Tr ) < / Mk / Mk0022
All source has the same target queue length, All source has the same target queue length, r r ddr r = = for all for all rr
Corollary 2: LHS is strictly increasing in Corollary 2: LHS is strictly increasing in qqrr** / / TT
r r , implying a lower bound on queueing delay, implying a lower bound on queueing delay
qr*qr* / /Tr Tr
sinc sinc ( ) ( )TrTrqr*qr*ňň
..
minminr r > Mk> Mk0022
Stability of TCP Vegas Stability of TCP Vegas Corollary 3Corollary 3
SinceSince () < () < / 2, sinc / 2, sinc () > 2 / () > 2 / , , kk0 0 1 1 Corollary 3: minCorollary 3: minrr qqrr
** / / Tr > 2M / Tr > 2M / If MIf M 2, then RHS bigger than 1, since 2, then RHS bigger than 1, since Tr = dTr = drr + +
qqrr**
M = 1M = 1 The stability condition cannot be satisfied if a soThe stability condition cannot be satisfied if a so
urce has more than one linkurce has more than one link Sufficient in multilink caseSufficient in multilink case Necessary and sufficient in single-link-homogenNecessary and sufficient in single-link-homogen
eous-source caseeous-source case
Stability of TCP Vegas Stability of TCP Vegas Single link with homogeneous sourceSingle link with homogeneous source
A single link of capacity A single link of capacity cc,, Shared by N homogeneous source,Shared by N homogeneous source, with round trip propagation delay with round trip propagation delay dd
Stability of TCP Vegas Stability of TCP Vegas Single link with homogeneous sourceSingle link with homogeneous source
From corollary 3: From corollary 3: qqrr** / / Tr > 2 / Tr > 2 / for all for all rr
Tr / qTr / qrr** < < /2, /2, since since Tr = d + qTr = d + qrr
* *
d / qd / qrr* * < (< (/2 – 1) => /2 – 1) => d < (d < (/2 – 1) /2 – 1) qqrr
**
Since Since qqrr* * = = / / xxrr
** => => (( N)/c N)/c cd < (cd < (/2 – 1) /2 – 1) N N Conclusion: bandwidth delay product sConclusion: bandwidth delay product s
hould be smallhould be small
Stabilized VegasStabilized Vegas
xxrr ( (tt) = ) = ((2/2/TrTr 22(t)(t)))tantan-1-1 (1 – (1 – xxrr((tt))qqrr((tt) / ) / rrddrr ) ) xxrr ( (tt) = ) = ((ww//TrTr 22(t)(t)))tantan-1-1 rr((tt))((1 – 1 – xxrr((tt))qqrr((tt)/)/rrddrr - -rr((tt) ) qqrr
((tt))))
1 – 1 – xxrr( ( t t ) ) qqrr( ( t t ) /) /r r ddrr
1 – 1 – xxrr( ( t t ) ) qqrr( ( t t ) /) /r r ddrr - -rr( ( t t ) ) qqrr( ( t )t )
rr( ( t t ) = (1 / ) = (1 / ) ) (( Tr( t ) / q Tr( t ) / qrr( ( t ) t ) )) rr( ( t t ) = ( ) = ( / w ) / w ) (( x xrr( ( t t ) ) Tr( t ) Tr( t ) ))
Stabilized VegasStabilized Vegas
The gain The gain rr( ( t t ) serves as a normalization ) serves as a normalization to to qqrr( ( t )t )
Additional differential term Additional differential term rr( ( t t ) ) qqrr( ( t t )) anticipates the future of anticipates the future of qqrr( ( t )t )
Without: xWithout: xrr( ( t t ) will increase if ) will increase if xxrr((tt))qqrr((tt)< )< rrddrr
Even Even xxrr((tt))qqrr((tt)/)/rrddrr is small, is small, xxrr( ( t t ) may de) may decrease if prices are rapidly growingcrease if prices are rapidly growing
Stabilized VegasStabilized Vegas
Stability condition for stabilized VegasStability condition for stabilized Vegas
wherewhere = tan = tan-1-1 (( (2 (2)/(1-)/(1-) ) )) Stabilized Vegas can choose a small Stabilized Vegas can choose a small
( a>0, ( a>0, (0,1) (0,1) )) such that RHS can be larger f such that RHS can be larger for better stability of the original Vegas or better stability of the original Vegas cd < cd < ((/2 – 1) /2 – 1) N N
22 + + 22aa22
22 + a + a22cd < ( - 1 ) N
Simulation ResultsSimulation Results
Reno/Reno/RenoReno
Reno/Reno/VegasVegas
Regas/ReRegas/Renono
Vegas/Vegas/VegasVegas
Throughput Throughput (KB/s)(KB/s)
60/10960/109 61/12361/123 66/11966/119 74/13174/131
Throughput Throughput RatioRatio
1.00/1.01.00/1.000
1.02/1.131.02/1.13 1.10/1.091.10/1.09 1.23/1.201.23/1.20
Retranmissions Retranmissions (KB)(KB)
30/2230/22 43/1.843/1.8 1.5/1.81.5/1.8 0.3/0.10.3/0.1
Retransmit Retransmit RatiosRatios
1.00/1.01.00/1.000
1.43/0.081.43/0.08 0.05/0.820.05/0.82 0.01/0.010.01/0.01One-on-One (300KB and 1MB) TransfersOne-on-One (300KB and 1MB) Transfers
cc = 200KB/s = 200KB/s50ms delay50ms delay
Simulation ResultsSimulation Results
= 20= 20 N = 100 flowsN = 100 flows Fixed packet size of 1KBFixed packet size of 1KB FIFO /w Droptail, queue capacity = 200FIFO /w Droptail, queue capacity = 200
0000 ( a , ( a , ) = ( 0.5 , 0.015 ) ) = ( 0.5 , 0.015 )
Simulation ResultsSimulation Resultscc = 100 pkts/s and = 100 pkts/s and dd = 10ms = 10ms
Simulation ResultsSimulation Resultscc = 1000 pkts/s and = 1000 pkts/s and dd = 10ms = 10ms
Simulation ResultsSimulation Resultscc = 100 pkts/s and = 100 pkts/s and dd = 100ms = 100ms
Simulation ResultsSimulation Results
Experimental ResultsExperimental Results
FAST TCP was first demonstrated publicly in during the SuperComputing Conference (SC2002) in Baltimore, MD, in November 16–22 2002
Caltech-SLAC research team CERN DataTAG StarLight TeraGrid Cisco Level(3).
Experimental ResultsExperimental Results
Experimental ResultsExperimental ResultsThroughput and utilizationThroughput and utilization
#flow#flow ThroughThroughputput
MbpsMbps
utilizatiutilizationon
DelaDelayy
msms
DistanDistancece
kmkm
DuratioDurationn
ss
Bmps Bmps 10101515
TransfTransfer GBer GB
11 925 925 (266)(266)
95% 95% (27%)(27%)
180180 10,03710,037 3,6003,600 9.28 9.28 (2.67(2.67
))
387 387 (111)(111)
22 1,797 1,797 (931)(931)
92% 92% (48%)(48%)
180180 10,03710,037 3,6003,600 18.03 18.03 (9.35(9.35
))
753 753 (390)(390)
77 6,1236,123 90%90% 8585 3,9483,948 21,60021,600 24.1724.17 15,39615,396
99 7,9407,940 90%90% 8585 3,9483,948 4,0304,030 31.3531.35 3,7253,725
1010 8,6098,609 88%88% 8585 3,9483,948 21,60021,600 33.9933.99 21,64721,647 SC2002 FAST experimental result Current TCP implementation in Linux v2.4.18
on Jan 27-28, 2003
FAST TCPFAST TCPConclusionConclusion
Problem of current TCP Problem of current TCP EquilibriumEquilibrium Dynamic problemDynamic problem
FAST TCPFAST TCP Equation-based control with queuing Equation-based control with queuing
delaydelay TCP VegasTCP Vegas Stabilized VegasStabilized Vegas
Q&AQ&A
Thank youThank you