Post on 10-Feb-2016
description
1
A General Introduction to Tomography & Link Delay
Inference with EM Algorithm
Presented by Joe, Wenjie Jiang21/02/2004
2
Outline of Talk Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link
delay using EM algorithm Conclusion
3
Terminology “Tomography”
Brain Tomography
Access is difficult!
Network Tomography
Access is difficult!Vardi 1996
4
Why tomography?What is the: Bandwidth? Loss rate? Link Delay? Traffic demands? Connectivity of links
in the network? (Topology Inference)
Path: a connection between two end nodes, each consisting of several links.
Link: a direct connection with no intermediate routes/hosts.
5
Motivation Identify congestion points and
performance bottlenecks Dynamic routing Optimized service providing Security: detection of
anomalous/malicious behavior Capacity planning
6
Why tomography - Difficulty Decentralized, heterogeneous and unregulated
nature of the internal network. No incentive for individuals to collect and
distribute these info freely. Collecting all statistics impose an impracticable
overhead expense ISP regards the statistics highly confidential Relaying measurements to decision-making
point consumes bandwidth.
7
Why tomography - Solution Widespread internal
network monitoring is expensive and infeasible
Edge-based measurement and statistical analysis is practical and scalable
8
Brain Tomography
9
Network Tomography
10
Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link
delay using EM algorithm Conclusion
11
Introduction to tomography Use a limited number of measurements to
infer network (link) performance parameters, using:-- Maximum Likelihood Estimator -- Estimation Maximization-- Bayesian Inference
and assuming a prior model. Categories of problems:
-- Link level parameter estimation-- Sender-Receiver traffic intensity.-- Topology Inference
12
Introduction to tomography (2) Two forms of network tomography:
-- link-level metric estimation based on end-to-end, traffic measurements (counts of sent/received packets, time delays between sent/received packets)-- path level (sender-receiver path) traffic intensity estimation based on link-level measurements (counts of packets through nodes)
Passive or Active measurements? Multicast or Unicast?
13
Problem Description To solve the linear system:
A, ө and εhave special structures. Goal: to maximize the likelihood function
14
Problem Description (2)
A = routing matrix (graph) ө = packet queuing delays
for each link y = packet delays measu
red at the edge ε= noise, inherent rando
mness in traffic measurements
Statistical likelihood function
15
Problem Description (3)
An virtual multicast tree with four receivers
l1
l2 l3
l4 l5 l6 l7
l1 l2 l3 l4 l5 l6 l7
Y1 Y2 Y3 Y4
Y1=X1+X2+X4
16
Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link del
ay using EM algorithm Conclusion
17
Physical Topology
Measure end-to-end (from sender to receiver) delays
18
Logical Topology
Logical topology is formed by considering only the branching points in the physical topology
Infer the logical link-level queuing delay distributions!
19
The basic idea of internal link delay tomography
Send a back-to-back packet pair from a sender, each packet heading to a different receiver
Use the fact that delays are highly correlated on shared links
Queuing delay difference between these two end can be attributed to the unshared links
20
Delay Estimation Measure end-to-end delay of packet
pairsPackets experience the same delay on link1
d2=dmin=0 d3>0 Extra delay on link 3!
21
Packet-pair measurements
)()2( nx
)()2( ny )()1( ny
Key Assumptions• Fixed known routes
• Temporal independence
• Spatial independence
• Packet-pair delays are identical on share links.
N delay measurements in all
22
Parameters
α1
α2α3
α4 α5
α6 α7 α8α9
αi = parameter of delay pmf on link i
23
Link delay model• αi = delay pmf on link i• Link delay model could be m
ultinomial• quantized delay model: dela
y= {0, 1, 2, 3,…,L,∞}
• αi= {αi0,αi1,αi2,...,αiL,αi ∞ }
• αij=P{ delay(link i) = j }
• αi0+αi1+αi2,...,αiL+αi ∞=10
0. 020. 040. 060. 080. 10. 120. 140. 160. 180. 2
0 1 2 3 4 … … L
probabi l i ty
24
Goal
);( YL
N
n
nypYL1
)( );();(
);( )( nyp is the probability of the event of n-th measurement
is the probability of the event of all measurements
Our goal: find );(maxarg
YL
25
Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link
delay using EM algorithm Conclusion
26
Review of MLE (Maximum Likelihood Estimation)
27
Review of MLE (Maximum Likelihood Estimation) The basic idea of MLE: God always let the event
with the biggest probability happen the most likely -- The MLE of ө is to make the sample occur the most likely
Note we assume X={x1,…xN} to be i.i.d The solution could be easy or hard depending on
the form of p(ө|X) e.g. p(ө|X) is a single Gaussian ө=(μ, σ2), we can
set the derivative of logL(ө|X) to zero and solve it directly.
28
Complete Data The sample X={x1,…xN} together with the
missing (or latent) data Y is called complete data.
The complete likelihood is
where p(x, y|ө) is the joint density of X and Y given the parameter ө.
The complete log-likelihood is
29
Complete MLE By the definition of conditional density,
where p(y|x,ө) is the conditional density of Y given X=x and ө
The complete MLE
30
Basic idea of EM Given X=x and ө= өt-1, where өt-1 is the current estim
ates the unknown parameters log p(x,Y| ө) is a function of Y whose unique best Me
an Squared Error (MSE) predicator is
31
EM steps
32
The magic of EM the direct MLE of
is relatively hard to solve But the MLE of complete log-likelihood is
relatively easier to obtain since is a function of x and y, (y is
hidden), we use the expectation of y under x and
So E-step
M-step
33
Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link del
ay using EM algorithm Conclusion
34
EM in link delay inference
α1
α2α3
α4 α5
α6 α7 α8α9
x1
x2 x3
x4 x5x6 x7
x8
x9
Note that here notation x and y have opposite meaning of x, y stated in previous EM algorithm
35
EM in link delay inference (2) Complete data Z=(X,Y) the complete data log-likelihood:
Pα[Y|X] has nothing to do with α
mi,j is the total number of packets experience a delay j on link i over N measurements.
][log]|[log][]|[log],[log);,( XPXYPXPXYPYXPYXL
Liii mi
mi
mii
M
ii
LXPXPXPXP
XPXPYXL
,1,0, ][]1[]0[][
][log][log);,(1
36
EM in link delay inference (3)
Liii
Liii
mLi
mi
mi
mi
mi
mii LXPXPXPXP
,1,0,
,1,0,
,1,0,
][]1[]0[][
The MLE of αwould be
L
jji
jiji
m
m
1,
,,
37
EM in link delay inference (4)nm
nmmmp 210
210
n
ii mmm
m
21
MLE
which is the frequency of event mi1i
i
A simple example is that we toss a die, P( the result i)=αi
(i=1,2…6) mi= how many times we see result i
38
EM in link delay inference (5) We notice that is similar to
only different that should be replaced by
So the MLE
);,( YXL
jim ,
],|[ )1(,,
ijiji YmEm
L
jji
jiji
m
m
1,
,,
39
EM in link delay inference (6)
],|)([
],|1[],|1[
1
],|[
)1(1
)1()(11
)1()(,
1 )(,
)1(,,
iNn
ijidelay
Nn
Nn
ijidelayji
Nn jidelayji
ijiji
YjidelayP
YEYEm
m
YmEm
Probability Propagation
40
A simple example
delay on each link fall into {0,1,2,3}
y1 y2
x1
x2 x3
0
1
2 3
}41,
41,
41,
41{},,,{
}41,
41,
41,
41{},,,{
}41,
41,
41,
41{},,,{
333231303
232221202
131211101
αij=P{ delay (link i) = j }
41
A simple example (2)Suppose there are 5 measurements:{ (3,2), (4,2), (6,5), (0,0), (4,1)}
y1 y2
x1
x2 x3
0
1
2 3
)](),(|0[
],|)([
2115
10,1
)1(1,
)0( nynyxPm
YjidelayPm
n
iNnji
42
A simple example (3)
y1 y2
x1
x2 x3
0
1
2 3
41
41
41
]0[]2[]3[]0[]2,3[
]0[]0|2,3[
][]|2,3[
]0[]0|2,3[
]2,3|0[)]1(),1(|0[
132
132
1121
3
01121
1121
211211 )0()0(
xPxPxPxPxxP
xPxyyP
jxPjxyyP
xPxyyP
yyxPyyxP
j
Bayes Formula
43
A simple example (4)
y1 y2
x1
x2 x3
0
1
2 3
31
64/164/164/164/1]2,3|0[
0410
41
]3[]3|2,3[41
41
41
]2[]2|2,3[41
41
41
]1[]1|2,3[
211
1121
1121
1121
yyxP
xPxyyP
xPxyyP
xPxyyP
44
A simple example (5)
y1 y2
x1
x2 x3
0
1
2 3
340100
31
0]1,4|0[1]0,0|0[0]5,6|0[0]2,4|0[31]2,3|0[
0,1
211
211
211
211
211
m
yyxPyyxPyyxPyyxP
yyxPsimilarly:
45
A simple example (6) ji
0 1 2 3
1 4/3 11/6 5/6 1
2 1 1/3 5/6 17/6
3 17/6 5/6 4/3 0
jim ,
mi,j computed in the first iteration.
46
A simple example (7)
154
16/56/113/43/4
3,12,11,10,1
0,10,1
1,
,,
mmmmm
m
mL
jji
jiji
the physical meaning of α1,0 is that: the number of packets that experience delay 0 on link i divided by the total number of packets that travel through link i
47
A simple example (8) ji
0 1 2 3
1 4/15 11/30 1/6 1/5
2 1/5 1/15 1/6 17/30
3 17/30 1/6 4/15 0
ji ,
αi,j computed in the first iteration
48
A simple example (9)
j i
0 1 2 3
1 0.4 0.4 0 0.22 0.2 0 0 0.83 0.4 0.2 0.4 0
ji ,
Iteration: iterate E-step and M-step, until some termination criteria is satisfied!
After 6 iterations, αi,j converges to a fixed value.
49
A simple example (9){ (3,2), (4,2), (6,5), (0,0), (4,1)}
y1 y2
x1
x2 x3
0
1
2 3
00. 10. 20. 30. 40. 50. 60. 70. 8
0 1 2 3
l i nk1l i nk2l i nk3
50
Complexity
51
Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link
delay using EM algorithm Conclusion
52
Conclusion+
The field is just emerging. Deploying measurement/probing schemes and inference
algorithms in larger networks is the next key step.
53
Problems The spatial-temporally stationary and
independent traffic model has limitations, especially in heavily loaded networks.
A trend for highly uncooperative environment for active probing – passive traffic monitoring techniques, for example based on sampling TCP traffic streams
54
Thank you!
The End