Computer Communications 31 (2008) 3883–3893

Contents lists available at ScienceDirect

Computer Communications

journal homepage: www.elsevier .com/locate /comcom

PBProbe: A capacity estimation tool for high speed networks q

Ling-Jyh Chen a,*, Tony Sun b, Bo-Chun Wang a, M.Y. Sanadidi c, Mario Gerla c

a Institute of Information Science, Academia Sinica, 128, Sec. 2, Academia Road, Taipei 11529, Taiwanb PacketMotion Inc., 110 Baytech Drive, 2nd Floor San Jose, CA 95134, USAc Department of Computer Science, University of California at Los Angeles 4732 Boelter Hall, Los Angeles, CA 90095, USA

a r t i c l e i n f o a b s t r a c t

Article history:Received 17 August 2007Received in revised form 19 May 2008Accepted 19 May 2008Available online 14 August 2008

Keywords:AnalysisHigh speed networksLink capacity estimationSimulationExperiments

0140-3664/$ - see front matter � 2008 Elsevier B.V. Adoi:10.1016/j.comcom.2008.05.047

q A preliminary version of this paper was published iIFIP Networking Conference [10]. This version extendwith additional traffic models. Furthermore, it contaiof testbed experiments than those in the previouPBProbe’s compatibility with general networks. Hencemuch more thorough and authoritative presentation

* Corresponding author. Tel.: +886 2 2788 3799x17E-mail address: cclljj@iis.sinica.edu.tw (L.-J. Chen)

Knowledge about the bottleneck capacity of an Internet path is critical for efficient network design, man-agement, and usage. In this paper, we propose a new technique, called PBProbe, for estimating high speedlinks rapidly and accurately. Although it is based on CapProbe, instead of relying solely on packet pairs,PBProbe employs the concept of ‘‘Packet Bulk” and adapts the bulk length to compensate for the wellknown problem with packet pair-based approaches, namely the lack of accurate timer granularity. Asa result, PBProbe not only preserves the simplicity and speed of CapProbe, but also correctly estimateslink capacities over a much larger range. Using analysis, we evaluate PBProbe with various bulk lengths,network configurations, and traffic models. We then perform a set of experiments to evaluate the accu-racy of PBProbe on the Internet over wired and wireless links. Finally, we perform emulation and Internetexperiments to verify the accuracy and speed of PBProbe on high speed links (e.g., the Gigabit Ethernetconnection). The results show that PBProbe is consistently fast and accurate in the majority of test cases.

� 2008 Elsevier B.V. All rights reserved.

1. Introduction

Estimating the bottleneck capacity of an Internet path is a fun-damental research problem in computer networking, since knowl-edge of the capacity is critical for efficient network design,management and usage. In recent years, with the growing popular-ity of emerging technologies, such as overlay, peer-to-peer (P2P),sensor, grid, and mobile networks, it has become increasinglyimportant to have a simple, fast and accurate tool for capacity esti-mation and monitoring. To accommodate the diversity of networkarrangements, a capacity estimation tool should also be scalableand applicable to a variety of network configurations.

A number of techniques have been proposed for capacity esti-mation of generic Internet paths [3,7,14,16,18,20,29]. Among them,Pathrate [14] and CapProbe [18] are widely recognized as fast andaccurate tools for generic network scenarios. However, CapProbe isa round-trip estimation scheme that only works well on paths witha symmetric bottleneck link, while Pathrate is based on histogramsand may converge slowly if the initial dispersion measurementsare not unimodal. As a result, CapProbe has difficulty estimating

ll rights reserved.

n the Proceedings of the 2006s the analysis section in [10]ns a more comprehensive sets paper, and demonstrates, this journal submission is a

of PBProbe.02; fax: +886 2 2782 4814..

the capacity of asymmetric links [12], and Pathrate performspoorly on wireless links [18].

To address the above problems, specialized capacity estimationtools have been proposed for specific and emerging network sce-narios. For instance, ALBP [24] and AsymProbe [12] are intendedfor capacity estimation on asymmetric links, and AdHoc Probe[11] tries to estimate the end-to-end path capacity of wireless net-works. However, recent studies have shown that, for emerginghigh speed network links (i.e., gigabit links), estimating high speedlink capacity remains a major challenge [17,19]; hence, a simple,fast, and accurate technique is desirable.

In this paper, we propose a capacity estimation tool, called PBP-robe, for high speed network links. Although PBProbe is inspired byCapProbe, instead of relying solely on one pair of packets, it em-ploys the concept of ‘‘Packet Bulk” to adapt the number of probepackets in each sample in accordance with the dispersion measure-ment. More specifically, when the bottleneck link capacity is ex-pected to be low, PBProbe uses one pair of packets as usual (i.e.,the bulk length is 1). For paths with high bottleneck capacities,PBProbe increases the bulk length and sends several packets to-gether. This enlarges the dispersion between the first and lastpackets, and resolves the well known timer granularity problem.Timer granularity is the main challenge in estimating the capacityof high capacity links [17,19]. This issue is discussed in detail laterin the paper.

We study the performance of PBProbe under different Poissoncross traffic loads via analytical models; and evaluate the tech-nique’s accuracy and speed through testbed experiments, a labora-tory testbed, and on high speed Internet paths. We also compare it

3884 L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893

with Pathrate. The results show that PBProbe can accurately andrapidly estimate the link capacities in all tested scenarios, therebyoutperforming Pathrate in most experiments.

The remainder of this paper is organized as follows. In Section 2,we summarize related work on capacity estimation. In Section 3,we describe PBProbe. Section 4 contains an analysis of PBProbeand an evaluation of its speed and accuracy with Poisson cross traf-fic, Deterministic cross traffic, and Pareto ON/OFF cross Traffic. InSection 5, we describe PBProbe experiments for general networkscenarios, and demonstrate the approach’s compatibility with gen-eral Internet links. In Section 6, we evaluate PBProbe on high speedlinks (e.g., Gigabit Ethernet) in our emulator testbed as well as onthe Internet. Then, in Section 7, we summarize our conclusions.

2. Background and overview

2.1. Related work

Previous research on capacity estimation relied on delay varia-tions among probe packets, as illustrated by Pathchar [16], or ondispersion among probe packets, as in Nettimer [20] and Pathrate[14]. Pathchar-like tools (such as pchar [7] and Clink [3]) are lim-ited in terms of accuracy and speed, as reported in [18,21]. More-over, they evaluate the capacity of a link based on the estimatesof previous links along the path; thus, estimation errors accumu-late and amplify with each link measured [29].

Dispersion-based techniques suffer from other problems. In par-ticular, Dovrolis’ analysis in [14] shows that the dispersion distribu-tion can be multi-modal due to cross traffic, and that the strongestmode of such a distribution may correspond to (1) the capacity ofthe path; (2) a ‘‘compressed” dispersion, resulting in capacity over-estimation; or (3) the Average Dispersion Rate (ADR), which is al-ways lower than the capacity. Another dispersion-based tool, SProbe[29], exploits the SYN and RST packets of the TCP protocol to estimatethe downstream link capacity, and employs two heuristics to filterout samples affected by cross traffic. However, SProbe does not workefficiently when the network is heavily utilized [23].

Unlike the above approaches, CapProbe [18] uses both dispersionmeasurements and end-to-end delay measurements to filter outpacket pair samples distorted by cross traffic. The method has beenshown to be both fast and accurate in a variety of scenarios. The ori-ginal implementation of CapProbe uses ICMP packets as probe pack-ets, and measures the bottleneck capacity on a round-trip basis. As aresult, the capacity estimate does not reflect the higher capacity linkwhen the path is asymmetric. Other difficulties are encounteredwhen intermediate nodes block ICMP packets [25], or employ prior-ity schemes to delay ICMP packet forwarding (e.g., the Solaris oper-ating system limits the rate of ICMP responses, and is thus likely toperturb CapProbe measurements) [30].

Fig. 1. (a) Over-estimation caused by ‘‘compression”, (b) und

Recent capacity estimation studies have extended the targetnetwork scenarios to more diverse environments. For instance,Lakshminarayanan et al. evaluated tools for estimating the capac-ity and available bandwidth of emerging broadband access net-works [22], while Chen et al. extended CapProbe to estimate theeffective path capacity in ad hoc wireless networks [11]. In addition,ABLP [24] and AsymProbe [12] have been developed for capacityestimation of the increasingly popular asymmetric links, such asDSL and satellite links.

Nonetheless, capacity estimation on high speed links remains achallenge. Though recent studies verified the accuracy of Pathratein estimating gigabit links [28], the evaluations were conducted onan emulator-based testbed, which cannot represent realistic Inter-net dynamics [15]. Thus, an experimental evaluation of capacityestimation on high speed links is still required.

In this paper, we propose a novel packet bulk technique calledPBProbe for estimating the capacity of high speed links. PBProbeis based on CapProbe, but it probes the bottleneck link capacityusing UDP packets (instead of the ICMP packets used by CapProbe).We summarize the CapProbe algorithm in the next subsection.

2.2. CapProbe: an overview

CapProbe is a packet pair-based capacity estimation techniquethat has proven both fast and accurate in several scenarios [18].Conceptually, when a back-to-back packet pair is transmitted overa network, and assuming both packets arrive at the bottleneck linkunperturbed (i.e., back-to-back) by cross traffic on previous links,they are always dispersed at that link according to the bottleneckcapacity. Although the dispersion will accurately reflect the bottle-neck capacity (as shown in Fig. 1c), if either packet in a pair hasbeen queued due to cross traffic, the dispersion of the samplemight be expanded or compressed; ‘‘expansion” leads to under-estimation and ‘‘compression” leads to over-estimation of thecapacity (as shown in Fig. 1a and b).

CapProbe combines dispersion measurements and end-to-enddelay measurements to filter out packet pair samples distortedby cross traffic. The basic idea is that if a sample has not beenqueued by cross traffic, it can be used to estimate the bottleneckcapacity correctly. For such ‘‘good” samples, the sum of the delaysof the packet pairs, called the delay sum, does not include cross-traffic induced queuing delay, and is indeed smaller than the delaysum of the samples distorted by cross traffic. Thus, these ‘‘good”samples can be identified easily because the delay sum will bethe minimum of the delay sums of all packet pair samples. We callthis delay sum the minimum delay sum. In this way, the capacitycan be estimated by the equation:

C ¼ PT; ð1Þ

er-estimation caused by ‘‘expansion”, (c) the ideal case.

Fig. 2. Illustration of PBProbe. (a) Phase I: measuring forward direction linkcapacity; (b) phase II: measuring backward direction link capacity.

L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893 3885

where P is the sample packet size, and T is the dispersion of thesample packet pair with the minimum delay sum.

However, since CapProbe relies on dispersion measurements foraccurate capacity estimation, it is inaccurate when estimating highspeed links or when operating on slow machines [19]. More specifi-cally, according to Eq. (1), when C is extremely large, T must becomesmall or P must become very large. Since the accuracy of the T mea-surement is limited by the system timer’s granularity, the only fea-sible solution for CapProbe estimation on high speed links is toenlarge the packet size. However, if the packet size becomes largerthan the Maximum Transmission Unit (MTU), the ‘‘big” packet willbe segmented into several fragments before entering the network.The fragments will then be reassembled as the original packet sizeat the receiver [27]. The latency caused by segmentation (at the sen-der) and reassembly (at the receiver) expands the dispersion mea-surement and therefore results in under-estimation. To preventsuch additional latency, we propose sending multiple MTU packetsback-to-back (i.e., virtually, the probe packets are input to the net-work as one ‘‘big” packet). As a result, the dispersion measurementcan be used to accurately estimate the capacity using the CapProbealgorithm because the segmentation and reassembly latency canbe avoided. We discuss the design, analysis, and evaluation of PBP-robe in the following sections.

3. The proposed approach: PBProbe

Like CapProbe, PBProbe estimates the link capacity by activelysending a number of probes over the network and using the minimumdelay sum filter to identify a ‘‘good” sample.1 However, instead ofemploying a packet pair, PBProbe uses a packet bulk of length k in eachprobing and measures the capacity of each direction separately. Specif-ically, PBProbe is comprised of two phases. In the first phase, it estimatesthe capacity of the forward link; and in the second, it estimates thecapacity of the backward link. Fig. 2 details the PBProbe algorithm.

In the first phase (shown in Fig. 2a), host A sends a START packetto host B to initiate the estimation process, after which B sends aRequest To Send (RTS) packet to A every G time units. On receiptof the RTS packet, A immediately sends B a packet bulk of lengthk (note: bulk length = k means that k + 1 packets are sent back-to-back). For the i-th probing sample, suppose B sends the RTSpacket at time tsend(i) and receives the j-th packet (in the i-th sam-ple) at time trcv(i, j). The delay sum, Si, and the dispersion, Di, of thei-th packet bulk sample are given by

Si ¼ ðtrcvði;1Þ � tsendðiÞÞ þ ðtrcvði; kþ 1Þ � tsendðiÞÞ; ð2ÞDi ¼ trcvði; kþ 1Þ � trcvði;1Þ: ð3Þ

If none of the k + 1 probe packets experience cross-traffic in-duced queueing, the sample will reflect the correct capacity. Thus,the ‘‘good” sample (say, the m-th sample) is identified by applyingthe minimum delay sum filter to all probing samples (say, n sam-ples) as follows:

m ¼ arg mini¼1...n

Si: ð4Þ

Therefore, the capacity is estimated by using the dispersion ofthe m-th sample with the minimum delay sum:

C ¼ kPDm

; ð5Þ

where P denotes the size of each probe packet. Since the packet bulksamples are only delivered in the forward direction, the estimatedcapacity corresponds to the bottleneck in that direction.

1 Note that, like previous studies [14,16,18], for simplicity, we do not consider theissues of traffic shaping and multi-path diversity in this study.

At the end of the first phase, B sends an END packet to A, andPBProbe enters the second phase, as shown in Fig. 2b. In this phase,A first sends an RTS packet (every G time units) to B, which replieswith a packet bulk of length k immediately on receipt of each RTSpacket. Similar to the first phase, PBProbe measures the delay sumand dispersion for each sample, and estimates the capacity in thebackward direction by using the minimum delay sum filter.

3.1. The inter-sample period: G

PBProbe assesses the link capacity by sending a packet bulkevery G time units. The value of G is critical to the convergencetime of PBProbe because the larger the value of G, the slower PBP-robe’s estimation becomes. However, G should not be too small;otherwise, PBProbe may create congestion in the network and re-quire longer to converge. Therefore, for PBProbe, we set G as:

G ¼ 2Dm0

U; ð6Þ

where Dm0 is the dispersion of the good sample (i.e., Dm0 ¼ kPC ) that

has the minimum delay sum among all probing samples seen sofar; and U is the maximum network utilization allowed for PBProbeestimation. We provide a short proof below to show that, if G ¼ 2Dm0

U ,the network utilization constraint U can be guaranteed.

Proof. Since k P 1, we know that

G ¼ 2Dm0

U

PDm0

Uþ Dm0

kU

¼ kPCUþ P

CU¼ ðkþ 1ÞP

CUð7Þ

) ðkþ 1ÞPG

6 CU: ð8Þ

Let R denote the data rate of the introduced packet bulk probes, i.e.,R ¼ ðkþ1ÞP

G ; then, we can conclude that R 6 CU. In other words, theprobing data rate is never larger than the load constraint U. h

3.2. The packet bulk length: k

The major difference between CapProbe and PBProbe is that thelatter sends packet bulks. The purpose of using packet bulks is toovercome the system timer’s granularity, and avoid the additionallatency caused by segmentation and reassembly when the packetsize is larger than the MTU.2 PBProbe applies Algorithm 1 to

2 The ‘‘path MTU”, i.e., the largest packet size that can traverse this path withoutbeing fragmented, can be found by the Path MTU Discovery technique [26].

3886 L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893

automatically determine the proper bulk length, k, for capacityestimation.

Algorithm 1. The algorithm for determining the appropriate bulklength, k, in PBProbe

k 1count 0D 1repeat

Fig. 3. The expected number of samples (N) with different link utilization (q) andpacket bulk length (k) under Poisson cross traffic.

t1 time()Send START packetReceive a packet bulk (length k) and measure D0

if D0 < Dthresh thenk k � 10count 0

elseD min(D0 ,D)G 2D/Ucount count + 1t2 time( )Sleep(G � (t2 � t1))

end ifuntil count == n

In the beginning, k is initialized to 1, i.e., PBProbe behaves likeCapProbe by using a packet pair to probe the link capacity. How-ever, whenever the measured dispersion is smaller than a certainthreshold, say Dthresh, the algorithm increases the bulk length (k)tenfold and restarts the estimation process. Clearly, the decisionabout the Dthresh value depends on the system timer’s granularity.In this work, we set Dthresh = 1 ms for all experiments.

3.3. The number of samples: n

CapProbe applies a sophisticated convergence test to determinewhether a good sample has been obtained. To simplify the imple-mentation, PBProbe simply estimates the link capacity using afixed number of samples, n. Obviously, the larger the value of n,the more accurate the PBProbe estimates will be. The time requiredfor one PBProbe capacity estimate is linearly proportional to thevalue of n. More specifically, the larger the value of n, the longerPBProbe will require to estimate the capacity. Based on the exper-iment results reported in previous CapProbe studies [13,18], we setn = 200 throughout this paper.

3 The utilization of the Abilene backbone network, which is the gigabit backbone ofInternet2, is seldom higher than 30% [1].

4. Analysis

We now present a queueing model that can predict the proba-bility of a ‘‘good” sample for a single link with three types of crosstraffic distribution: Poisson, Deterministic, and Pareto ON/OFF dis-tribution. For simplicity, we assume the probing samples of PBP-robe arrive according to a Poisson distribution so that they takewhat amounts to ‘‘a random look” at the link. We also assume thatthe probing samples do not constitute a significant load on the net-work because they are sent infrequently. Finally, we assume thebuffers are large enough so that probe packets will not be droppeddue to buffer overflow. The results of our analysis are presented inthe following subsections.

4.1. Poisson cross traffic

First, we analyze PBProbe with Poisson cross traffic on the bot-tleneck link. Suppose the arrival rate and the service rate of thePoisson cross traffic are k and l, respectively, the service time ofone single probing packet is s, and the bottleneck link utilizationis q. The probability that the first probe packet will find the systemempty, i.e., p, is given by

p ¼ 1� kl¼ 1� q: ð9Þ

Since none of the probe packets of a ‘‘good” sample experiencequeueing delay, the probability of no queueing delay for theremaining k probe packets (i.e., no cross traffic packets arrive inthe ks period) is e�k(ks). Therefore, the probability of obtaining a‘‘good” sample, i.e., p0, is given by

p0 ¼ p ekð�ksÞ ¼ ð1� qÞe�kks: ð10Þ

The expected number of samples, N, required to obtain a goodsample is then derived by

N ¼X1n¼1

np0ð1� p0Þn�1 ¼ 1

p0¼ ekks

1� q: ð11Þ

Suppose the probe packets and cross traffic packets are of equalsize, then s ¼ 1

l ¼qk. Therefore, N can be rewritten as

N ¼ ekq

1� q: ð12Þ

The relationship between the expected number of requiredsamples for one good sample (N) and link utilization (q) for differ-ent packet bulk lengths (k) is shown in Fig. 3.

From Fig. 3, we observe that when k = 1 (i.e., packet pair basedCapProbe), N is around 25 when the utilization (q) is as high as 0.9.However, as k increases, N increases exponentially. For instance,when q = 0.3,3 N is around 30 when k = 10, but reaches approxi-mately 1.5 � 1013 when k = 100. We find that the estimation speedof PBProbe (i.e., the number of samples required to obtain a goodsample) is closely related to the packet bulk length employed.Though a large packet bulk can improve the accuracy of dispersionmeasurements, a much larger number of attempts will be neededto derive a good sample, which will slow down the estimation pro-cess considerably.

4.2. Deterministic cross traffic

In this subsection, we analyze the probability of obtaining agood sample by using the packet bulk technique under determin-istic cross traffic. Suppose the deterministic cross traffic deploysthe same packet size with a rate equal to k packets per time unit(i.e., the interval between any two contiguous cross traffic packetsis 1/k time units), and the transmission time of each cross traffic

Fig. 4. Illustration of the time interval between the arrival of two deterministiccross traffic packets.

Fig. 5. The expected number of samples (N) with different link utilization (q) andpacket bulk length (k) under deterministic cross traffic.

Fig. 6. Illustration of the time interval between the arrival of two Pareto crosstraffic samples (Case 3).

L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893 3887

packet is tx. To obtain a good sample of length k, the system mustfinish servicing the previous cross traffic and k probe packets with-in 1/k time units, as shown in Fig. 4. Therefore, the probability ofobtaining a good sample is given by

p0 ¼max 0;1� tx þ ks1k

!¼maxð0;1� kq� ktxÞ: ð13Þ

When the cross traffic packets and the probe packets are thesame size (i.e., tx = s), p0 can be rewritten as

p0 ¼maxð0;1� kq� ksÞ ¼maxð0;1� kq� qÞ: ð14Þ

To ensure that one can always obtain a good sample for accu-rate capacity estimation, p0 must be larger than 0. Thus,

1� kq� q > 0) q <1

kþ 1: ð15Þ

Then, the expected number of samples required to obtain agood sample is given by

N ¼X1n¼1

np0ð1� p0Þn�1 ¼ 1

p0¼ 1

1� ðkþ 1Þq : ð16Þ

Fig. 5 depicts the relationship between N and q with packetbulks of different length, k. The results show that when the utiliza-tion (q) is smaller than 1

kþ1, a good sample can be obtained veryrapidly (i.e., N is small); when q is very close to 1

kþ1, N suddenly in-creases to a very large number; and when q is larger than 1

kþ1, N isno longer very meaningful, since p0 = 0 in this case and it is impos-sible to get any good samples. As a result, the larger k becomes, thesmaller the effective range of q will be.

4.3. Pareto ON/OFF cross traffic

Pareto ON/OFF cross traffic has two states: ON and OFF. In theON state, the cross traffic is actually deterministic; whereas inthe OFF state, the cross traffic source does not transmit any pack-ets. The distribution of ON and OFF periods follows the Pareto dis-tribution. We assume that such periods are independently andidentically distributed (i.i.d.). The probability density function(PDF), f(t), of the ON/OFF periods and its mean, t, are given by

f ðtÞ ¼ aba

taþ1 0 < b 6 t; 1 < a 6 2: ð17Þ

and

t ¼ aba� 1

; ð18Þ

where a and b are, respectively, the shape and scale parameters ofthe Pareto distribution. Since the mean length of the ON periods isequal to that of the OFF periods, we assume the mean arrival rate ofthe cross traffic is k; hence, the mean arrival rate during ON periodswill be 2k.

Suppose tx is the transmission time of a single cross traffic pack-et, and s is the transmission time for a single probe packet. Theanalysis of the probability of obtaining a good sample can be di-vided into three cases as follows:

1. tx <12k < tx þ ks.

In this case, the inter-arrival time of the cross traffic (during anON state) is longer than the service time (tx); thus, there will beidle time during an ON period. However, since the idle time isnot long enough to serve k probe packets (i.e., ks time units),the good sample can only arrive during an OFF period. Accord-ing to the analysis presented in [18], the probability of obtain-ing a good sample can be derived from the probability that theresidual life time Y will be larger than ks:

p0 ¼12

P½Y P ks� ¼1

2ab

ks

� �a�1if ks P b

12 1� ða�1Þks

ab

h iif ks < b:

8<: ð19Þ

2. 12k > tx þ ks.In this case, there is idle time during an ON period that is longenough to serve k probe packets. Therefore, the good samplecan arrive during an ON or an OFF period. Since the cross trafficduring an ON period is deterministic, the probability of obtain-ing a good sample during an ON period can be derived by themethod presented in Section 4.2. Therefore, the probability ofobtaining a good sample in this case will be the summation ofthe probabilities of obtaining a good sample during an ON stateand an OFF state, given by

p0 ¼Pon þ 1

2ab

ks

� �a�1if ks P b

Pon þ 12 1� ða�1Þks

ab

h iif ks < b;

8<: ð20Þ

where Pon ¼ 12 ½1� 2kðtx þ ksÞ�.

3. tx >12k.

In the third case, the inter-arrival time of the cross traffic pack-ets (during an ON period) is shorter than the service time of asingle cross traffic packet; thus, the system will be busy servingcross traffic packets during the entire ON periods, as well as inthe first few time units of the OFF periods. Consequently, thegood sample can only arrive during an OFF period when thereis enough idle time to serve k probe packets. As shown inFig. 6, the probability of obtaining a good sample will be 1minus the proportion of the busy period (i.e., k

l), minus the timerequired to service k probe packets (i.e., ks

2t). Since the idle time

may not be long enough to serve k probe packets either, werequire that the probability can not be smaller than 0 by

p0 ¼max 0;1� kl� ks

2t

� �: ð21Þ

Table 1

munications 31 (2008) 3883–3893

In summary, we have analyzed the packet bulk technique using

Path properties of the employed symmetric Internet connections

Path Bulk length k Time spent (s)

UCLA M NTNU 10 10UCLA M CalTech 10 10UCLA M UCLA 10 10

Poisson, Deterministic, and Pareto ON/OFF cross traffic. The resultsshow that the technique can rapidly obtain a good sample (a prob-ing packet bulk with no queueing delay) when the utilization of thebottleneck link is low. For a heavily loaded link, the packet bulkneeds a larger number of samples to accurately estimate the linkcapacity. In the following sections, we evaluate PBProbe furthervia a detailed set of experiments on an emulator-based testbedand on the Internet.

5. Evaluation on general Internet links

In this section, we present the results of evaluating the accuracyof PBProbe on general Internet connections. We evaluate PBProbeon symmetric Internet links in Section 5.1, and then present theevaluation results of PBProbe for emerging first/last-mile asym-metric access links and wireless links in Sections 5.2 and 5.3,respectively.

5.1. Symmetric links

In the Internet experiments on the general symmetric fastEthernet links, one of the selected paths is a local connection with-in the UCLA campus network (UCLA M UCLA; capacity: 100 Mbps;round-trip time: 0.2 ms). The other two paths are from UCLA to theCalifornia Institute of Technology (UCLA M CalTech; capacity:100 Mbps; round-trip time: 8 ms) and from UCLA to National Tai-wan Normal University (UCLA M NTNU; capacity: 100 Mbps;round-trip time: 180 ms).

For each network path, PBProbe was run 20 times to determinethe average capacity estimate and standard deviation. The normal-ized capacity estimates (i.e., the ratio of estimated capacity to realcapacity) and the 95% confidence intervals (i.e., ±2 standard devia-tions) of the experiment results are illustrated in Fig. 7. The em-ployed bulk length k and the average time required to make oneestimate (in one direction) of each path are shown in Table 1.

From the table, we observe that PBProbe adapted its bulklength, k, to 10 in all experiments. The reason is that when k = 1,PBProbe sends packet pairs as probing samples, which is the sameas CapProbe. Hence, the dispersion measurements are smaller thanthe threshold value (i.e., Dthresh = 1 ms). As a result, the bulk lengthadaptation algorithm increases k to ensure that the measured dis-persion is larger than Dthresh, thereby overcoming the system’sinadequate timer granularity.

Compared with the experiment results reported in [18], PBP-robe achieves the same estimation accuracy as CapProbe and Path-

3888 L.-J. Chen et al. / Computer Com

Fig. 7. PBProbe capacity estimation (mean and 95% confidence intervals) on100 Mbps Internet links.

rate on 100 Mbps links. As shown in Fig. 7, PBProbe estimates thebottleneck capacity accurately in most cases. Note that the capac-ity estimate for the path from NTNU to UCLA fluctuates slightlymore than the estimates for the other paths. This is because heavycross traffic on the path reduces the probability of obtaining a goodpacket bulk sample.

In terms of estimation speed, PBProbe’s performance is betweenthat of CapProbe and Pathrate. It requires slightly more time thanCapProbe because of packet bulk length adaptation and the fixednumber of samples. Nevertheless, it is worth noting that, byemploying packet bulks instead of packet pairs, PBProbe canaccommodate systems with a lower timer granularity as well ashigher capacity links; therefore, it is more flexible than CapProbe.

5.2. Asymmetric links

The emerging first/last-mile Internet access links, such as DSLand Cable modem links, are usually asymmetric. Existing asym-metric estimation techniques are either very complicated (e.g.,ALBP [24]) or they require fine-scale system timer granularity sup-port (e.g., AsymProbe [12]). In the following, we describe a set ofexperiments that evaluate the applicability of PBProbe to asym-metric access links.

We selected two asymmetric links from our local host (con-nected to the Internet via a 100 Mbps ethernet link) to twoDSL hosts. One was a Verizon DSL link with 1.5 M/384 Kbps linkcapacity on the DOWN/UP links and approximately 30 ms round-trip delay time; the other was a HiNet DSL link with 3 M/640 Kbps link capacity on the DOWN/UP links and approxi-mately 50 ms round-trip delay time.4 The experiments were per-formed 20 times for each destination host, and k was observed tobe 1 in all runs (i.e., when k = 1, the measured dispersion was al-ready larger than the threshold value). In Fig. 8, we plot the nor-malized average capacity estimates with 95% confidence intervalsfor each link direction.

The results clearly demonstrate PBProbe’s accuracy in estimat-ing all asymmetric link capacities. Specifically, the average capacityestimates are all within 90% accuracy compared to the real physi-cal link capacity. Given a 95% confidence level, all the capacity esti-mates remain within 80% accuracy.

5.3. Wireless experiments

Next, we evaluate PBProbe on a one-hop wireless link becausesuch links are becoming more prevalent in first/last mile scenarios.The wireless link used is based on IEEE 802.11b, and the transmis-sion rate is configured as 1, 2, 5.5, and 11 Mbps, respectively. Foreach transmission rate, we repeated the PBProbe experiment 20times to collect the average and standard deviation results. Fig. 9shows the experiment results.5 From the figure, we observe thatPBProbe also yields accurate capacity estimates for wireless links.

4 The DSL connections were provided by Verizon (http://www.verizon.net) andHiNet (http://www.hinet.net).

5 The capacity estimates were normalized to the effective capacity as reported athttp://www.uninett.no/wlan/throughput.html.

Fig. 10. NIST Net emulation testbed.

Fig. 9. PBProbe experiment results (mean and 95% confidence intervals level, i.e., ±2standard deviation, of 20 runs) on last mile IEEE 802.11b links with differenttransmission modes.

Fig. 8. PBProbe experiment results (mean and 95% confidence intervals level, i.e., ±2standard deviations, of 20 runs) on Up and Down links of DSL connections.

L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893 3889

The normalized capacity estimates are very close to 1 in all transmis-sion modes, and the standard deviations are also small.

6. Evaluation on high speed links

We now evaluate the performance of PBProbe on high speedlinks (i.e., over 100 Mbps). First, we perform emulation experi-ments to validate the speed and accuracy of PBProbe in a con-trolled environment using different bulk lengths, and thenconduct Internet experiments to study its performance in a morerealistic environment.

6.1. Emulation experiments

The emulator-based experiments were run on our laboratorytestbed with the configurations illustrated in Fig. 10. The testbedcomprised three Linux-based desktops (Fedora Core 3 with kernelversion 2.6.9-1.667), each of which had an Intel Pentium 4 2 GHzCPU, 2 GB RAM, and two Intel PRO/1000 GT 32-bit PCI GigabitEthernet adapters. In the first scenario (Fig. 10a), three test ma-chines were connected in series with 1 Gbps links. The NISTNetemulator [6] was installed on the middle machine to create a bot-tleneck link on the gigabits path. No cross traffic was added to thispath. In the second scenario (Fig. 10b), three test machines were

connected to a one gigabit switch with a 1 Gbps link. A Poissontraffic generator [8] was installed on one of the three machines,and PBProbe was installed on the other two machines. The Poissontraffic generator was configured to generate different levels ofcross traffic in order to validate the accuracy and speed of PBProbeunder different link utilization levels.

6.1.1. No cross trafficFirst, we evaluated the accuracy of PBProbe with different k set-

tings on high speed links by disabling the bulk length (k) adapta-tion algorithm. Fig. 10a illustrates the testbed configuration inwhich no cross traffic was present during the experiments. We var-ied the bottleneck link capacity, and ran the experiment 20 timesfor each capacity setting. The results of k = 10 and 100 are shownin Fig. 11.

Since there was no cross traffic in these experiments, packets ina packet bulk were expected to traverse the path back-to-backwithout being disturbed. Therefore, ideally, the measured disper-sion of each packet bulk should represent the undistorted disper-sion corresponding to bottleneck link capacity. Moreover, basedon these perfect dispersions, the capacity estimate should be accu-rate and consistent for every probing sample.

However, since PBProbe requires accurate dispersion measure-ments, the capacity estimates tend to become inaccurate when thetimer granularity is inadequate. Table 2 shows the required timergranularity of PBProbe on links with different capacities and differ-ent bulk lengths. Obviously, when the bottleneck link capacity ishigh or the bulk length is small, a high timer granularity is re-quired. For instance, with link capacity = 100 Mbps and bulklength = 1 (i.e., packet pairs), only a powerful processor can satisfythe required resolution of 0.12 ms. As the capacity increases to1 Gbps, it becomes very difficult to achieve the required 0.012 msresolution, and the measurement accuracy will be degraded as aconsequence. Indeed, this trend can be observed in Fig. 11 a andb, where, for small k, the capacity estimates become increasinglyinaccurate and inconsistent when the required timer granularityincreases (or equivalently, when the bottleneck capacity isenlarged).

However, the results also reveal that, when measuring highcapacity links, a large k can successfully minimize the inaccuracyand inconsistency of capacity estimates caused by poor timer gran-ularity. For instance, when the bottleneck link capacity is600 Mbps, PBProbe with bulk length k = 100 can accurately esti-mate the capacity (as illustrated by the very small coefficient of

Fig. 12. Capacity estimates of PBProbe (k = 100) under different Poisson crosstraffic.

Fig. 11. The results of PBProbe estimation using the NISTNet emulator: (a)normalized capacity estimates; and (b) coefficient of variation of capacity estimateswith various bottleneck capacity settings.

Table 2Required system timer granularity for accurate PBProbe estimation (assuming theprobe packet size is 1500 bytes)

k Bottleneck link capacity

1 Gbps (ms) 100 Mbps (ms) 10 Mbps (ms)

1 0.012 0.12 1.210 0.12 1.2 12

100 1.2 12 120

6 More information about Internet utilization can be found at http://netmon.grnet.gr/weathermap/andhttp://people.ee.ethz.ch/oetiker/webtools/mrtg/users.html.

3890 L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893

variation in capacity estimates). In contrast, when k = 10, PBProbecan only measure around 60% of the link capacity, and the coeffi-cient of variation is much larger. Based on the experiment results,and the analysis in Table 2, we conclude that our testbed hosts canonly provide a timer granularity of approximately 1 ms. Therefore,in the following experiments, we fix Tthresh = 1 ms in PBProbe’s kadaptation algorithm.

6.1.2. Poisson cross trafficThe objective of the second set of experiments was to investi-

gate the impact of different levels of cross traffic on the accuracyand speed of PBProbe. Fig. 10b illustrates a testbed topology inwhich a Poisson traffic generator inputs traffic to the probing pathof PBProbe. We varied the generator’s traffic rate from 0 to500 Mbps, and performed 20 trials of PBProbe for each cross trafficrate with the bulk length k fixed at 100. The relationship betweenthe number of employed samples and the average capacity esti-mates is plotted in Fig. 12.

The results show that PBProbe estimated 950 Mbps capacityvery rapidly (in less than 50 samples) when the Poisson cross traf-fic was very light (i.e., the Poisson cross traffic varied from 0 to100 Mbps). As the cross traffic increased, the accuracy of PBProbeestimates decreased. Specifically, PBProbe achieved around 90%accuracy when the link utilization was 0.2, and 85% accuracy whenit was 0.3. However, when the link utilization was greater than40%, PBProbe only achieved around 80% accuracy after 1000 sam-ples. The results are consistent with the analytical results shownin Fig. 3, where the required number of samples increases expo-nentially with a link’s utilization.

We find that, if the bulk length is large, PBProbe can esti-mate a link’s capacity very rapidly when the link utilization islow, but it requires a large number of probing samples whenthe link is heavily utilized. Fortunately, most current Internetgigabit backbones are moderately loaded. As reported in[1,2,4,5,9].6 the utilization of Internet backbones is mostly below15%. Therefore, in practice, PBProbe is still adequate for capacityestimation on high speed Internet links. To validate the feasibilityand capability of PBProbe in more realistic scenarios, we describeexperiments performed on high speed Internet paths in the nextsubsection.

6.2. Internet experiments

We conducted Internet experiments to evaluate PBProbe inrealistic environments. Five Internet hosts and five Internet gigabitpaths (within California: UCLA and CalTech; cross country: UCLA–PSC, PSC–GaTech, GaTech–UCLA; and international: NTNU–UCLA)were selected for the experiments. The selected Internet hostsare listed in Table 3, and the topology and path properties of theselected paths are shown in Fig. 13.

Figs. 14 and 15 illustrate the experiment results (i.e., the nor-malized mean and the coefficient of variation of capacity estimatesin 20 runs). From Fig. 14, it is clear that the normalized capacityestimates are mostly within 90% accuracy range, except for thethree outgoing links from UCLA to CalTech, GaTech, and PSC. Thiswas because the utilization of the outgoing links of the UCLA back-bone was around 30%, which was much higher than the utilizationof the incoming link (around 15%) [2]; therefore, in this case, PBP-robe required a large number of samples to correctly estimate the

Fig. 13. Topology and path properties (bottleneck capacity and round trip delay) ofselected gigabit Internet paths.

Table 3Description of the participating hosts in the gigabit Internet experiments

Abbrev. Full name Location

CalTech California Institute of Technology CA, USAGaTech Georgia Institute of Technology GA, USANTNU National Taiwan Normal University Taipei, TaiwanPSC Pittsburgh Supercomputing Center PA, USAUCLA University of California at Los Angeles CA, USA

L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893 3891

capacity. Since we set n = 200 in all experiments, PBProbe couldonly estimate around 80% of the link capacity.

Fig. 15 also shows that the coefficient of variation of PBProbeestimates is below 0.15 for all tested links, i.e., the capacity esti-mates (20 runs) of each high-speed link are very stable. Compari-son with the results shown in Fig. 11 shows that PBProbe canadapt its bulk length to the most appropriate value (i.e., k = 100)so that it can consistently and accurately estimate the capacity ofhigh speed links.

6.3. Comparison of PBProbe and Pathrate

We have evaluated PBProbe in a variety of network scenarios.Here, we compare the performance of PBProbe and Pathrate interms of accuracy, speed, and bandwidth consumption (i.e., over-head). The experiments were performed on the set of high speedInternet links illustrated in Fig. 13, and the results of the capacityestimates and time required are detailed in Table 4.

The results in Table 4 show that PBProbe can measure at least85% of the bottleneck capacity of all tested links, but Pathrate ismore accurate and measures at least 90% of the bottleneck capacityof all links. However, for links with long delay and/or high utiliza-

Fig. 14. PBProbe experiment results (me

tion, Pathrate required more than 1000 s to estimate the capacity.The reason is that if the distribution of the measured dispersion isnot unimodal after the first phase, Pathrate will start the secondphase to probe the network using different packet train lengthsand packet sizes. Once Pathrate enters the second phase, it takesa long time to determine the correct link capacity. As a result,the technique converges rapidly if the dispersion distribution isunimodal in the first phase; otherwise, it is much slower thanPBProbe.

We also compared the packet overhead incurred by PBProbeand Pathrate on high speed Internet links. Table 5 shows the com-parison on one of the high speed links, the UCLA–GaTech link. Fromthe experiment results, we observe that PBProbe is more expensivein terms of computation time than Pathrate, if the latter only usesone phase to estimate the capacity of high speed links. In the casewhere Pathrate is required to enter the second phase, PBProbe andPathrate produce comparable amounts of packet overhead. How-ever, since Pathrate slows down substantially after entering thesecond phase, the bandwidth consumption (i.e., bits per second)is much smaller than that of PBProbe. It is worth noting that, eventhough the bandwidth consumption of PBProbe (approximately2 Mbps) seems relatively high, in reality, it is only 0.2% of the bot-tleneck capacity (i.e., 1 Gbps); thus, it does not intrude on othertraffic flows in the network.

The experiment results suggest that there are trade-offs be-tween PBProbe and Pathrate for high-speed path capacity estima-tion. On the one hand, PBProbe yields very good estimationresults rapidly (less than 20 s in most cases). Given more time, itwill progressively improve the estimates, since better samplescan be obtained. On the other hand, Pathrate tends to produceaccurate results, but the required time may vary from approxi-mately 20 s to 20 min. Therefore, it may not be ideal for scenarioswhere the bottleneck capacity must be estimated in a very shorttime.

It should also be noted that the packet overhead of PBProbe isproportional to the employed bulk length k. While measuring ahigh speed link, PBProbe increases its bulk length and in turn in-creases the packet overhead to compensate for the limited supportof the system’s timer granularity. Nonetheless, thanks to the Uparameter employed, the bandwidth consumption of PBProbe isrestricted by the upper bound of the utilization. Hence, PBProbecan carefully control the trade-off between the bandwidth con-sumption and the time needed to satisfy the requirements of dif-ferent applications.

7. Conclusion

We have studied a classic problem of link capacity estimation,and proposed a technique, called PBProbe, for estimating the

an of 20 runs) on high speed links.

Table 4Comparison of PBProbe and Pathrate on Internet links

PBProbe Pathrate

Capacity (Mbps) Time (s) Capacity (Mbps) Time (s)

CalTech ? UCLA (1 Gbps) 919.6 14 933.2 17UCLA ? CalTech (1 Gbps) 839.4 14 945.3 1146GaTech ? UCLA (1 Gbps) 928.1 14 932.9 18UCLA ? GaTech (1 Gbps) 840.3 14 968.1 1223PSC ? GaTech (1 Gbps) 959.9 13 995.4 1122GaTech ? PSC (1 Gbps) 905.9 13 947.0 17PSC ? UCLA (1 Gbps) 889.6 14 935.6 20UCLA ? PSC (1 Gbps) 845.2 15 905.6 20NTNU ? UCLA (600 Mbps) 580.6 20 575.6 1641UCLA ? NTNU (600 Mbps) 588.4 21 573.4 1641

Table 5Comparison of PBProbe and Pathrate overhead on Internet gigabit links

GaTech ? UCLA UCLA ? GaTech

PBProbe Pathrate PBProbe Pathrate

Spent time 14 s 18 s 14 s 1223 sTotal packets 20,213 2,414 20,213 27,630Total bytes 30,319,500 3,543,752 30,319,500 39,707,740BW consumption 2.166 Mbps 1.575 Mbps 2.166 Mbps 0.260 Mbps

Fig. 15. PBProbe experiment results (coefficient of variation of 20 runs) on high speed links.

3892 L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893

bottleneck capacity of emerging high speed links. PBProbe is basedon the CapProbe algorithm, but it uses a ‘‘packet bulk” to adaptthe number of packets in each probing according to different net-work characteristics. As a result, it preserves the simplicity, speed,and accuracy of CapProbe, and compensates for the poor systemtimer granularity problem on high speed links. Using extensiveanalysis, emulation, and Internet experiments, we evaluated theaccuracy, speed, and overhead of PBProbe in various network con-figurations. The results show that PBProbe can correctly and rap-idly estimate the bottleneck capacity in almost all test cases.Compared to other capacity estimation techniques, PBProbe isideal for real deployments that require online and timely capacityestimation. The proposed technique can improve various applica-tions, such as peer-to-peer streaming and file sharing, overlaynetwork structuring, pricing and QoS enhancements, as well asnetwork monitoring.

Acknowledgments

We thank the following researchers for their help in carryingout the various PBProbe measurements: Sanjay Hegde (CaliforniaInstitute of Technology), Che-Chih Liu (National Taiwan NormalUniversity), Cesar A.C. Marcondes (University of California, LosAngeles), and Anders Persson (University of California, Los Ange-

les). We also thank the anonymous reviewers for their valuablecomments and suggestions.

This work was co-sponsored by the National Science Counciland the National Science Foundation under Grant Nos. NSC-94-2218-E-001-002 and CNS-0435515, respectively.

References

[1] Abilene Network Traffic. Available from: <http://loadrunner.uits.iu.edu/weathermaps/abilene/>.

[2] Cenic Network Statistics. Available from: <http://cricket.cenic.org/grapher.cgi>.

[3] Clink: A Tool for Estimating Internet Link Characteristics. Available from:<http://allendowney.com/research/clink/>.

[4] Garr-b: Network Weather Map. Available from: <http://www.noc.garr.it/mappe/backbone.shtml>.

[5] Grnet: Network Weathermap. Available from: <http://netmon.grnet.gr/map.shtml>.

[6] Nistnet: Network Emulation Package. Available from: <http://www.antd.nist.gov/itg/nistnet/>.

[7] pchar: A Tool for Measuring Internet Path Characteristics. Available from:<http://www.kitchenlab.org/www/bmah/Software/pchar/>.

[8] Poisson Traffic Generator. Available from: <http://www.spin.rice.edu/Software/poisson_gen/>.

[9] Twaren Weather Map. Available from: <http://mrtg.twaren.net/mrtg/wmap/>.[10] L.-J. Chen, T. Sun, L. Lao, G. Yang, M.Y. Sanadidi, M. Gerla, Estimating link

capacity in high speed networks, in: IFIP Networking, 2006, pp. 98–109.[11] L.-J. Chen, T. Sun, G. Yang, M.Y. Sanadidi, M. Gerla, Adhoc probe: path capacity

probing in ad hoc networks, in: International Conference on Wireless Internet,2005, pp. 156–163.

[12] L.-J. Chen, T. Sun, G. Yang, M.Y. Sanadidi, M. Gerla, End-to-end asymmetric linkcapacity estimation, in: IFIP Networking, 2005, pp. 780–791.

[13] L.-J. Chen, T. Sun, G. Yang, M.Y. Sanadidi, M. Gerla, Monitoring access linkcapacity using tfrc probe, Computer Communications 29 (10) (2006) 605–1613.

[14] C. Dovrolis, P. Ramanathan, D. Moore, What do packet dispersion techniquesmeasure? in: IEEE Infocom, 2001, pp. 905–914.

[15] S. Floyd, V. Paxson, Difficulties in simulating the internet, IEEE/ACMTransactions on Networking 9 (4) (2001) 392–403.

[16] V. Jacobson, Pathchar: A Tool to Infer Characteristics of Internet Paths.Available from: <ftp://ftp.ee.lbl.gov/pathchar/>.

[17] G. Jin, B. Tierney, System capability effect on algorithms for networkbandwidth measurement, in: ACM SIGCOMM Conference on InternetMeasurement, 2003, pp. 27–38.

[18] R. Kapoor, L.-J. Chen, L. Lao, M. Gerla, M.Y. Sanadidi, Capprobe: a simple andaccurate capacity estimation technique, in: ACM SIGCOMM, 2004, pp. 67–78.

[19] R. Kapoor, L.-J. Chen, M.Y. Sanadidi, M. Gerla, Accuracy of link capacityestimates using passive and active approaches with capprobe, in: IEEESymposium on Computers and Communications, vol. 2, 2004, pp. 1085–1090.

[20] K. Lai, M. Baker, Measuring bandwidth, in: IEEE Infocom, 1999, pp. 235–245.[21] K. Lai, M. Baker, Measuring link bandwidths using a deterministic model of

packet delay, in: ACM SIGCOMM, 2000, pp. 283–294.[22] K. Lakshminarayanan, V.N. Padmanabhan, J. Padhye, Bandwidth estimation in

broadband access networks, in: ACM SIGCOMM Conference on InternetMeasurement, 2004, pp. 314–321.

[23] S.-J. Lee, P. Sharma, S. Banerjee, S. Basu, Rodrigo Fonseca, Measuringbandwidth between planetlab nodes, in: International Workshop on Passiveand Active Network Measurement, 2005, pp. 292–305.

[24] Y. Lin, H. Wu, S. Cheng, W. Wang, C. Wang, Measuring asymmetric linkbandwidths in internet using a multi-packet delay model, in: IEEEInternational Conference on Communications, vol. 3, 2003, pp. 1601–1605.

[25] M.J. Luckie, A.J. McGregor, H.-W. Braun, Towards improving packet probingtechniques, in: ACM SIGCOMM Workshop on Internet Measurement, 2001, pp.145–150.

L.-J. Chen et al. / Computer Communications 31 (2008) 3883–3893 3893

[26] J. Mogul, S. Deering, Path mtu Discovery. IETF RFC 1191, November1990.

[27] J. Postel, Internet Protocol. IETF RFC 791, September 1981.[28] R. Prasad, M. Jain, C. Dovrolis, Evaluating pathrate and pathload with realistic

cross-traffic, in: 2003 Bandwidth Estimation Workshop.

[29] S. Saroiu, P.K. Gummadi, S.D. Gribble, Sprobe: A Fast Technique for MeasuringBottleneck Bandwidth in Uncooperative Environments. Available from:<http://sprobe.cs.washington.edu/>.

[30] S. Savage, Sting: A tcp-based network measurement tool, in: USENIXSymposium on Internet Technologies and Systems 1999.