Correctness of Gossip-Based Membership under Message Loss Maxim GurevichIdit Keidar Technion.
-
Upload
arturo-woolf -
Category
Documents
-
view
214 -
download
1
Transcript of Correctness of Gossip-Based Membership under Message Loss Maxim GurevichIdit Keidar Technion.
Correctness of Gossip-Based Membership under Message Loss
Maxim Gurevich Idit Keidar
Technion
The Setting
•Many nodes – n▫10,000s, 100,000s, 1,000,000s, …
•Come and go▫Churn
•Fully connected network▫Like the Internet
•Every joining node knows some others▫(Initial) Connectivity
2
Membership: Each node needs to know some live nodes
•Each node has a view ▫Set of node ids▫Supplied to the application▫Constantly refreshed
•Typical size – log n
3
Applications
•Applications▫Gossip-based algorithm▫Unstructured overlay networks▫Gathering statistics
•Work best with random node sample▫Gossip algorithms converge fast▫Overlay networks are robust, good expanders▫Statistics are accurate
4
Modeling Membership Views
•Modeled as a directed graph
u v
w
v y w …
y
5
Modeling Protocols: Graph Transformations
•View is used for maintenance•Example: push protocol
… … w …… … z …u v
w
v … w …
w
z
6
Desirable Properties?
•Randomness▫View should include random samples
•Holy grail for samples: IID▫Each sample uniformly distributed▫Each sample independent of other samples
Avoid spatial dependencies among view entries Avoid correlations between nodes
▫Good load balance among nodes
7
What About Churn?
•Views should constantly evolve▫Remove failed nodes, add joining ones
•Views should evolve to IID from any state•Minimize temporal dependencies▫Dependence on the past should decay quickly ▫Useful for application requiring fresh samples
8
Global Markov Chain
•A global state – all n views in the system•A protocol action – transition between global
states•Global Markov Chain G
u v u v
9
Defining Properties Formally
•Small views▫Bounded dout(u)
•Load balance▫ Low variance of din(u)
•From any starting state, eventually(In the stationary distribution of MC on G)▫Uniformity
Pr(v u.view) = Pr(w u.view) ▫Spatial independence
Pr(v u. view| y w. view) = Pr(v u. view) ▫Perfect uniformity + spatial independence load balance
10
Temporal Independence
•Time to obtain views independent of the past•From an expected state▫Refresh rate in the steady state
•Would have been much longer had we considered starting from arbitrary state▫O(n14) [Cooper09]
11
Existing Work: Practical Protocols
•Tolerates asynchrony, message loss•Studied only empirically ▫Good load balance [Lpbcast, Jelasity et al 07] ▫Fast decay of temporal dependencies [Jelasity et al 07] ▫ Induce spatial dependence
Push protocol
u v
w
u v
w
w
z z
12
v … z …
Existing Work: Analysis
•Analyzed theoretically [Allavena et al 05, Mahlmann et al 06]
▫ Uniformity, load balance, spatial independence ▫Weak bounds (worst case) on temporal independence
•Unrealistic assumptions – hard to implement ▫ Atomic actions with bi-directional communication▫ No message loss
… … z …… … w …u v
w
v … w …
w
zShuffle protocol
z
*
13
Our Contribution : Bridge This Gap
•A practical protocol▫Tolerates message loss, churn, failures▫No complex bookkeeping for atomic actions
•Formally prove the desirable properties▫Including under message loss
14
… …
Send & Forget Membership•The best of push and shuffle•Some view entries may be empty
u v
w
v … w … u w
u w
15
S&F: Message Loss
•Message loss▫Or no empty entries in v’s view
u v
w
u v
w
16
S&F: Compensating for Loss
• Edges (view entries) disappear due to loss• Need to prevent views from emptying out• Keep the sent ids when too little ids in view▫ Push-like when views are too small
u v
w
u v
w
17
S&F: Advantages over Other Protocols
•No bi-directional communication▫No complex bookkeeping▫Tolerates message loss
•Simple▫Without unrealistic assumptions▫Amenable to formal analysis
Easy to implement
18
•Degree distribution▫Closed-form approximation without loss▫Degree Markov Chain with loss
•Stationary distribution of MC on the global graph G▫Uniformity▫Spatial Independence▫Temporal Independence
•Hold even under (reasonable) message loss!
Key Contribution: Analysis
19
Degree Distribution without loss•In all reachable graphs:▫dout(u) + 2din(u) = const▫Better than in a random graph – indegree bounded
•Uniform stationary distribution on reachable states in G
•Combinatorial approximation of degree distribution▫The fraction of reachable graphs with specified node
degree▫Ignoring dependencies among nodes
20
Degree Distribution without Loss: Results
•Similar (better) to that of a random graph•Validated by a more accurate Markov model
0
0.05
0.1
0.15
0.2
0 10 20 30 40Node indegree
Binomial
S&F Analytical
S&F Markov
21
0
0.05
0.1
0.15
0.2
0 20 40 60 80Node outdegree
Binomial
S&F Analytical
S&F Markov
Setting Degree Thresholds to Compensate for Loss
•Note: dout(u) + 2din(u) = const invariant no longer holds – indegree not bounded
22
Key Contribution: Analysis
23
•Degree distribution▫Closed-form approximation without loss▫Degree Markov Chain with loss
•Stationary distribution of MC on the global graph G▫Uniformity▫Spatial Independence▫Temporal Independence
…
Degree Markov Chain
•Given loss rate, degree thresholds, and degree distributions
• Iteratively compute the stationary distribution
Transitions without loss
Transitions due to loss
State corresponding to isolated node
outdegree0 2 4 6
inde
gree
0
1
2
3
…
…
…
…
…
…
…
24
Results• Outdegree is bounded by the
protocol• Decreases with increasing loss
• Indegree is not bounded by the protocol
• Still, its variance is low, even under loss
• Typical overload at most 2x
0
0.05
0.1
0.15
0.2
0.25
0 20 40 60 80Node outdegree
loss=0loss=0.01loss=0.05loss=0.1
0
0.05
0.1
0.15
0.2
0.25
0 10 20 30 40Node indegree
loss=0loss=0.01loss=0.05loss=0.1
25
•Degree distribution▫Closed-form approximation without loss▫Degree Markov Chain with loss
•Stationary distribution of MC on the global graph G▫Uniformity▫Spatial Independence▫Temporal Independence
Key Contribution: Analysis
26
Uniformity
•Simple!•Nodes are identical•Graphs where uv isomorphic to graphs
where uw•Same probability in stationary distribution
27
•Degree distribution▫Closed-form approximation without loss▫Degree Markov Chain with loss
•Stationary distribution of MC on the global graph G▫Uniformity▫Spatial Independence▫Temporal Independence
Key Contribution: Analysis
28
Decay of Spatial Dependencies
•Assume initially > 2/3 independent good expander
•For uniform loss < 15%, dependencies decay faster than they are created
u v
w
uv
w
u does not delete the sent ids
…
…
u w
29
Decay of Spatial Dependencies: Results
•1 – 2loss rate fraction of view entries are independent▫E.g., for loss rate of 3% more than 90% of entries
are independent
30
•Degree distribution▫Closed-form approximation without loss▫Degree Markov Chain with loss
•Stationary distribution of MC on the global graph G▫Uniformity▫Spatial Independence▫Temporal Independence
Key Contribution: Analysis
31
Temporal Independence
•Start from expected state▫Uniform and spatially independent views
•High “expected conductance” of G•Short mixing time▫While staying in the “good” component
32
Temporal Independence: Results•Ids travel fast enough▫Reach random nodes in O(log n) hops▫Due to “sufficiently many” independent ids in views
•Dependence on past views decays within O(log n view size) time
33
Conclusions
•Formalized the desired properties of a membership protocol
•Send & Forget protocol▫Simple for both implementation and analysis
•Analysis under message loss▫Load balance▫Uniformity▫Spatial Independence▫Temporal Independence
34
Thank You