Hierarchical Clustering and Network Hierarchical Clustering and Network Topology IdentificationTopology Identification
Mark Coates Rob NowakDepartment of Electrical and Computer Engineering
Rui Castro
Copyright © 2004 - Rui Castro
Topology IdentificationTopology Identification
Pairwise delay measurements reveal topology
Ratnasamy & McCanne (1999)Duffield, et al (2000,01,02)
Bestravos, et al (2001)Coates, et al (2001)Shih & Hero (2002)
Copyright © 2004 - Rui Castro
Topology IdentificationTopology Identification
Challenges: • 12 % never respond,15 % multiple interfaces - Barford et al (2000)
• detect level-2 topology “invisible” to IP layer (e.g., switches)
Copyright © 2004 - Rui Castro
Relationship between Topology ID Relationship between Topology ID and Hierarchical Clusteringand Hierarchical Clustering
Copyright © 2004 - Rui Castro
0
1
2
53 4
Do not need clock
synchronization!!
Sandwich ProbingSandwich Probing
Copyright © 2004 - Rui Castro
0
1
2
53 4
more shared queues larger
Topology imposesconstraints
we can infer that receivers 3 & 4 have a longer shared path than 3 & 5
Sandwich ProbingSandwich Probing
Copyright © 2004 - Rui Castro
0
1
2
53 4
Delay CovarianceDelay Covariance
more shared queues larger covarianceCopyright © 2004 - Rui Castro
individual measurement
Multiple measurements
CLT
0
1
2
53 4
Measurement FrameworkMeasurement Framework
Key Assumptions:
• stationarity
• fixed (but unknown) routes
• temporal independence
• spatial independence
Copyright © 2004 - Rui Castro
The maximum likelihood tree (MLT) is defined as
where
Maximum Likelihood Tree - MLTMaximum Likelihood Tree - MLT
Two Approaches:
• Binary tree construction based on bottom-up, recursive selection and pair-merging process
• Markov Chain Monte Carlo (MCMC) tree search
measurementsunknown similarity metric values,measurement likelihoodforest of possible trees,monotonicity constrain set, for tree
product of Gaussian densities
Copyright © 2004 - Rui Castro
Traceroute topology
ALT topology
UNO
Internet Experiments – Internet Experiments – Sandwich ProbingSandwich Probing
MCMC topology
Copyright © 2004 - Rui Castro
Traceroute topology
Estimated topology
Internet Experiments – Internet Experiments – RTT Delay CovarianceRTT Delay Covariance
Thanks toYolanda Tsang & Mehmet Yildiz
Copyright © 2004 - Rui Castro
• Clever probing and sampling schemes reveal “hidden” network structure and behavior
• Likelihood based methods are a natural choice to account for uncertainty in the data
• Sampling methods relying solely on RTT can be devised
Complex interplay between measurement/probing techniques, statistical modeling, and computational methods for optimization
Final Remarks and CommentsFinal Remarks and Comments
R. Castro, M. Coates and R. Nowak, "Likelihood Based Hierarchical Clustering", IEEE Transactions in Signal Processing, August 2004.
R. Castro, M. Coates, G. Liang, R. Nowak and B. Yu, "Network Tomography: Recent Developments", Statistical Science, 2004 (invited paper, to appear).
Copyright © 2004 - Rui Castro
Top Related