Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer...

Internet Performance Measurements and

Measurement Techniques

Jim KuroseDepartment of Computer Science

University of Massachusetts/Amhersthttp://www.cs.umass.edu/~kurose

Overview

• Introduction– why and what to measure

• Measuring per-hop performance– tricks, successes, “failures”

• End-to-end measurements– correlation in end-end loss, delay– “confidence” in measurements

• What lies ahead?

What “performance” to measure?

• packet delay• packet loss• link or path capacity/availability• where?

• over what time scale?– sub-second, minute, hours?

end-to-end

per-hop

Why measure?end-end measurements:

• benchmarking, monitoring (e.g., Imeter)

• fault identification (e.g., routing instabilities)

• understanding end-end perf– misordering, loss (e.g., tcp

studies by Paxson)– correlation-time scale for

end-end loss, delayuse in adaptive applications

per-hop measurements:

• network operations (proprietary?)

• understanding where in end-end path performance impairments occur – use in reliable

multicast protocols, active services, network modeling

Measuring Per-hop performance• Question: what is loss, delay, capacity at given hop?• Question: what per-hop delays does a packet see?

• Complication:– routers do not report performance stats to end users

• need to infer performance statistics– cleverly use little “machinery” that we have– develop an inferencing methodology

per-hop

Clever use of existing protocols

• traceroute, pathchar: – use ICMP packets and time-to-live (TTL) field– each router decrements TTL on forwarding– TTL = 0 results in ICMP error msg back to sender

• Used to discover all routers on path to destination

ttl=3ttl=2

ttl=1

ICMP errrouter = x

x

Clever use of exiting protocols (cont.)

• ICMP/TTL-field trick also gives link bandwidth:– find min roundtrip delay to x-

1 (use many probe pkts)– find min rt delay to hop x– difference gives prop. delay

plus transmission delay– vary pkt size to get link

bandwidth

• gives variable queueing delay, loss of path to x– isolating hop x behavior

difficult

hop x-1 hop x

data packet (d bits)

ICMP reply (r bits)

d/bw

2*prop

r/bw

time

Can we measure per-hop delays?

• motivation - a typical modeling paper: “We model the network as a single link ….”

• is this a valid assumption?– does a packet generally experience “most”

of its delay at one link?

Measuring per-hop delays:• send unicast probes along path• use “IP options” on probes to gather timestamps

– packet passing through specified router timestamped

• problem: only 4 timestamps in each packet• solution: send multiple probes at one time:

x

y

x y

y

xts(x)

y

xts(x)

ts(y)

probe 1

probe 2

x x x xx x x x

Measuring per-hop delays:• problem: IP options packets treated differently

– data packets forwarded on fast path– IP options packet detoured (hopefully briefly)

• solution: send non-option packet with probes– only analyze probes when non-option packet delay

close to probe delay (hope: negligible options processing delays)

probe 1 x x x xprobe 2 x x x x

options

forwarding

non-option pkt

Analyzing the per-hop data

• Consider only probes with e-e queueing delays > 100ms

• filter cases where probe and option pkt delays “close” (20 ms)

• Hypothesis: e-e delays of filtered probes from same distribution as all probes– hypothesis rejected with

negligible probability of being wrong :-(

Title:

Creator:gnuplotPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

Can we measure per-hop packet delay/loss?

• timestamping approach not statistically valid

• inspiration (!) from another on-going effort: multicast loss

• question: where in a multicast tree does loss occur?– backbone? edges? – implications for design of

reliable multicast protocols

Title: /tmp/xfig-export003787Creator: fig2devCreationDate: Sat May 11 23:53:21 1996

Using multicast to infer per-hop performance

• correlation of received mcast pkts provides glimpse inside

• simple loss model– independent loss probabilitiesk on link k

• method– multicast n packets from source– data: list of packets received at each receiver– check consistency of data with independent loss model– analysis: Maximum Likelihood Estimator

• find which maximizes Prob[data | ]

R1 R2

R1 R2

1

32

Multicast inference: evaluation

• through ns simulations– 2-8 receivers– different topologies– TCP, on/off background sources

• approach tracks probe loss well• good estimate of background traffic loss

Multicast Inference: to-do listObservations:• multicast-based inference promising for loss• applicable to delaysResearch questions:• what if topology partially unknown?• can we identify bottleneck links?Potential Applications:• Internet weather map• use in adaptive applications

UMass collaboration with AT&T, LBNL

End-End Loss Delay Characteristics

• Question: time correlation of e-e loss, delay ?

• Application:– adjustment of FEC for audio, video, data– playout delay adjustment for audio– analytic models: how many “states” needed in

Markovian models?

• Approach: collect/analyze point-point, multicast traces of periodically generated UDP probes

Analysis Issues:

• stationarity of traces:– look for increasing trends in avg, variance

over trace– non-stationary traces not considered

• removal of clock skew– algorithm for removing constant clock drift

• how “confident” are we in the measured value?• 150 hours of measurement data

– there’s an exception to every “typical” result

Analysis Metrics:

• delay autocorrelation: dj: measured delay of pkt j

• loss autocorrelation: xj = 0 if pkt j received

= 0 if pkt j lost

• conditional average delay given loss:

dad d d d

d dii

j i j

j

( )( )

( ), ,...

21

E d ld

ljj j i

k i

k[ | ] , ,...

1 1

lax x x x

x xii

j i j

j

( )( )

( ), ,...

21

Delay Autocorrelation

• Note: typically autocorrelation dies down quickly

Title:


Conditional Delay Given Loss:

• Interesting behavior!• Loss appears to be predictor of near term

higher-than average delays

Title:


Loss Autocorrelation:

• generally: loss correlation timescale < 500 ms

• modeling: length of consecutive losses, successful reception can be modeled accurately by 2 or 3 state Markov process

Title:/usr/mcast/yajnik/lossdata/traces3-loss-autoc.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

How many states needed in analytic model?

• For n-state Markov model, determine transition probabilities from observed data

Title:/usr/mcast/yajnik/lossdata/20Dec97.uni/baddist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

Title:/usr/mcast/yajnik/lossdata/20Dec97.uni/gooddist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

• needed: rigorous hypothesis testing of agreement between model and observed distributions

“Confidence” in loss probability estimates

• suppose: we send 10 packets and see 3 lost– view loss as a random process– is loss rate “really” 30%?– could be true loss rate is 20% or 50% !– if we sample more, we’d have more “confidence” in

the estimate

• goal: interval estimator for loss rate– e.g.: 95% confident that true loss in range [p1,p2]

• use: adaptive applications (e.g., using RTCP)

Example:

Bernoulli loss process: each pkt lost independently

with probability p

95% confidence interval around MLE: find [ p1, p2 ] such that Pr{loss[k, n] | p = p1} = Pr{loss[0, k] | p = p2} = 0.025

n = 10 k = 3 MLE = k/n = 0.3

Loss probability confidence: model

p1 0.07 p2 0.65

0 1 2 3 4 5 6 7 8 9 10

0.1

0.2

0.3

0.4

0.5

0 1 2 3 4 5 6 7 8 9 10

0.1

0.2

0.3

0.4

0.5

Loss probability estimation: intervals

0 1000 2000 3000 4000 5000

0.2

0.4

0.6

0.8

1

MLE = 0.50

0.10

0.05

0.02

0.01

number of packets sent n

95%confidenceintervalwidthrelativeto MLE

What’s ahead?

• need for statistically rigorous, empirically verified, end-user oriented performance measurement tools and technique– research just beginning

• middleware: network-to-user performance feedback?– when and in what form?

• informed use of performance measurements in:– adaptive applications– active services

For More Information …..

• This talk: ftp://gaia.cs.umass.edu/pub/kurose/intel98.ps

• Group publications: http://gaia.cs.umass.edu/papers• WWW sites:

– Cooperative Association for Internet Data Analysis www.caida.org– National Laboratory for Applied Network Research www.nlanr.net

Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer...

Documents

Transcript of Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer...