Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri...
-
Upload
jack-bentley -
Category
Documents
-
view
215 -
download
1
Transcript of Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri...
![Page 1: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/1.jpg)
collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham,Rolf Reidi
Network Bandwidth Estimation and Tomography
Rob Nowak & Rich Baraniuk UW-Madison Rice University
spin.rice.edu
![Page 2: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/2.jpg)
• too complex to measure everywhere, all the time• traffic measurements expensive (hardware, bandwidth)
1969 1993
Internet Boom
![Page 3: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/3.jpg)
companies do not share data or performance information
Proprietary Concerns
![Page 4: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/4.jpg)
• bits are bundled into packets• packets go through routers• queues absorb bursts of packets• congestion: queues fill up, large delays, packet drops
Networking 101
![Page 5: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/5.jpg)
Network Measurement & Inference
Internet equivalent model
Path Modeling and Bandwidth Estimation
Network Tomography
![Page 6: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/6.jpg)
Brain Tomography
unknown object
statistical model
measurements
Maximumlikelihood estimate
maximizelikelihood
physics
data
prior knowledge MRF model
counting &projection
Poisson
![Page 7: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/7.jpg)
unknown object
statistical model
measurements
Maximumlikelihood estimate
maximizelikelihood
physics
data
prior knowledge
Network Tomography
queuing behavior
routing &counting
binomial /multinomial
![Page 8: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/8.jpg)
Network Tomography
From link-level traffic measurements, infer end-to-end traffic flow rates
Vardi ’96, Tebaldi & West ’98
Cao, Davis, Vander Wiel, Yu ’00
![Page 9: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/9.jpg)
y = packet losses or delays measured at the edge
A = routing matrix (graph)
= packet loss probabilities or queuing delays for each link
= randomness inherent traffic measurements
),|(),( AypAl likelihood function
Ay
Network Tomography (MINC Project, Towsley-Duffield)
![Page 10: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/10.jpg)
Probe packets experience similar queuing effects and may interact with each other
Probing the Network
probe =packet stripe
cross-traffic(2)packet (1)packet
delay
![Page 11: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/11.jpg)
Network Tomography: The Basic Ideasender
receivers
![Page 12: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/12.jpg)
Network Tomography: The Basic Ideasender
receivers
![Page 13: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/13.jpg)
Maximum Likelihood Estimation via EM
)()1( ny
)()2( ny
Suppose we were able to measure losses/delays on each link
)(1 nx
)(2 nx)(3 nx
)(3 nx
Expectation-Maximization (EM) alternates between computing expectation of unobserved internal measurements and the desired estimates of link-by-link loss/delay distributions
Problem: How to computemaximum likelihood estimates of link-by-link loss/delay distributions from end-to-end measurements ?
![Page 14: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/14.jpg)
0
1
2
53 4
Topology ID via Probe Interactions
35
d
![Page 15: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/15.jpg)
Topology ID via Probe Interactions
0
1
2
53 4
34
d
35 we can infer that receivers 3 & 4 have a longer shared path than 3 & 5
![Page 16: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/16.jpg)
Finding the Maximum Likelihood Tree
Stochastic search through “forest” via Metropolis-Hastings
![Page 17: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/17.jpg)
True topology
estimated topology
Internet measurement experiments
UNO
![Page 18: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/18.jpg)
What have we learned?
• Clever probing and sampling schemes reveal “hidden” network structure and behavior
• Simple inference algorithms are effective, intuitive, easy to implement, scale nicely
• MLE criteria are easily modified to include prior information: Bayesian or regularized MLE methods are straightforward
Complex interplay between measurement/probing techniques, statistical modeling, and computational methods for optimization
![Page 19: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/19.jpg)
Open Problems: Placement/Coverage
How should measurement devices be deployed ?
Logical graph coverage of physical topology ?
Can random graph models shed some light ?
![Page 20: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/20.jpg)
Open Problems: Spatio-temporal Correlation
competing traffic
Can we detect correlations? Can we exploit them in measurement and mapping applications?
Fuse tomography and bandwidth estimation
Long-range dependence of network traffic
Correlations due to competing traffic flows
![Page 21: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/21.jpg)
Open Problems: Detection and Localization
Detecting and locating anomalous behavior rather than estimating everything
Estimation Hypothesis Testing
How can we capitalize on conventional wisdom: most links are ‘good’ and only a few are ‘bad’ ?
![Page 22: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/22.jpg)
Open Problems: Timing and Synchronization
• Hardware solutions (expensive)• Software solutions (more practical)
- sophisticated software clocks (Veitch ’02)
- crude software clocks (ICMP timestamping) and statistical averaging
sender network
sendermonitor
receivermonitor
receiver
How to accurately measure time ?
![Page 23: Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.](https://reader035.fdocuments.us/reader035/viewer/2022070305/55151cbd55034673228b4ece/html5/thumbnails/23.jpg)
Open Problems: Network Security
How can measurement and monitoring across the Internet help detect and prevent malicious activities ?