Symphony: Distributed Hashing in a Small World Gurmeet Singh Manku Mayank Bawa Prabhakar Raghavan...
-
Upload
lindsey-harrington -
Category
Documents
-
view
221 -
download
1
Transcript of Symphony: Distributed Hashing in a Small World Gurmeet Singh Manku Mayank Bawa Prabhakar Raghavan...
Symphony: Distributed Hashing in a Small World
Gurmeet Singh Manku Mayank Bawa
Prabhakar Raghavan
Presented by Satpreet Singh
MotivationGOAL: To maintain a large DHT over a WAN
DESIRED CHARACTERISTICS:• Scalability (work for a range of network sizes)• Stability (handle churn)• Performance (provide low-latency lookups & low maintenance costs with churn)• Flexibility (provide design knobs, preferably run-time)• Simplicity (easy to understand, code, deploy…)
SOLUTION: Symphony (fuses ideas from Chord & Klienberg’s Small World greedy routing algorithm/result)
Features/Advantages of Symphony1. Low state maintenance (conseq. of low degree)• Fewer pings/keep-alives, less (ambient) control traffic• Distributed locking and coordination overhead over smaller
sets of nodes• Smaller bootstrapping time when a node joins• Smaller recovery time when a node leaves 2. Smooth out-degree vs latency tradeoff • Only protocol that offers this tuning knob even at run time!• Out-degree not fixed at runtime, or as function of network
size. 3. Flexibility and support for heterogeneity• Different nodes can have different number of links
4. Fault tolerance• Only short links are bolstered. No backups for long links
Architecture: Overview
node long link short link
A typical Symphony network
• Establish keyspace as [0, 1) (wrap around a ring like in Chord)
• Every node ‘manages’ subrange from own-id. to next-clockwise-node’s-id. (~ equi-sized)
• Objects hash to m-bit hash-key K, managed by node that manages real number containing K/2m
•2 Short-links: one with each immediate neighbor
Architecture: Overview
node long link short link
A typical Symphony network
• k (≥ 1) Long Links (uni-/bi-direct.)- draws a rand. number (x) from a Probability Distribution Function- contacts manager of (x) using a Routing Protocol- Establishes a link (if incoming links at manager ≤ 2k, if not resample PDF)
PDF is a type of harmonic distribution (so, Symphony)Pn(x) = 1/(x ln [n]) in x Є [1/n, 1]
= 0 otherwise
PDF estimates [n] using an Estimation Protocol
Network Size Estimation ProtocolGoal: to estimate ‘n’ - the current total number of nodes in the DHT
So if,- S is any set of s distinct nodes, -Xs is the sum of segment-lengths managed by them,Estimated n = s/Xs
•All s nodes update their estimate,• Experimentally s = 3 found good enough,• So simply use node and it’s two immediate neighbors Fact: Impact of increasing s on avg. latency is insignificant
x = Length of arc1/x = Estimate of n (Idea from Viceroy)
Routing Protocol(s)Klienberg’s Small World result: A message can be routed to any node by greedy routing in O(log2n) hops, in a construction where each node has one link to each of it’s 4 directional neighbors and a single long-distance link to a node chosen from a suitable PDF.
To lookup hash key x Є [0,1), contact the manager of x :
Unidirectional Routing Protocol: Node forwards a lookup for x along (short or long) link that minimizes the clockwise distance to x
Bidirectional Routing Protocol: Node forwards a lookup for x along (short or long) link that minimizes the absolute distance to x
In both cases, expected path length in an n-node network with k = O(1) links is O(1/k log2n) hops.
Bidirectional & 1-Lookahead reduce latency by 40% and 30% each
Join/Leave ProtocolsJOIN: • The new node chooses its id x from [0, 1) uniformly at random• Using the routing protocol it identifies the current manager of x• It then runs the estimation protocol using s = 3• X then uses Pnx
to establish its long distance linksCost = k links * O(1/k log2n) msgs. = O(log2n) messages
LEAVE:• All out- and in- links to x’s long distance neighbors are snapped• Other nodes whose outgoing links to x are just broken, reinstate those links with other nodes• The immediate neighbors of x establish short-links between themselves• Successor of x initiates estimation protocol over s = 3 neighborsCost = O(log2n) messages
Re-linking Protocols etc…RE-LINKING: • nx = x’s current estimate of n• nx
link = x’s estimate when long distance links were last established• When nx and nx
link differ → stale estimate• Re-link only when nx / nx
link is not in the range [0.5, 2]• Re-linking gains are marginal, cost high: O(log2n) messages
LOOKAHEAD:• Node can maintain list of neighbor’s neighbors • Improves choice of neighbor for routing queries• No extra messages – piggyback on keep-alives of TCP link• Cost = O(k2) space. Number of long-links remains unchanged
FAULT TOLERANCE:• Deletion of short links more detrimental as leads to node isolation• Make f copies of node’s content in f next clockwise nodes
Experimental Data:SETUP: • Large DHT: 25 to 215 nodes simulated in network• Four kinds of test networks: Static, Expanding, Expanding-Relink & Dynamic
Estimate Protocol Performance: estimate improves for log(n) neighbors, but impact on avg. latency is minimal (later)
Experimental Data:
Routing Protocol Performance: Increasing links beyond 2 has marginal benefits. Bidirectional routing is good (30% reduction in latency)
Experimental Data:
Lookahead Performance: • 1-Lookahead reduces avg. latency by 40% for small value of k.• Also, it does not entail an increase in the no. of long-links per node. • Neighbor-lists are exchanged periodically piggy-backed on normal routing traffic or keep-alives
Experimental Data:
Fault-tolerance motivation: • [Left] On deleting a random set of links (short + long), successful lookups drops quite quickly [deletion of short links causes node isolation quickly]• [Right] Impact of removing only long links not as severe (and only avg. latency goes up)• Thus, only fortify short links. Make f copies of content in clockwise direction.
Comparisons & Conclusions:
Conclusions: • For large DHTs, (25 to 215 nodes) Symphony outperforms others• Avg. TCP links = 10; Latency is about 8 hops; • Lower costs of Join/Leave compared to Chord etc.• Number of neighbors not fixed at outset; No backup links
Wrapping up…Symphony...• is a simple protocol for managing large DHTs• supports a dynamic network of hosts with relatively short lifetimes• scales well• has low lookup latency • has low maintenance cost• requires few neighbors per node• supports heterogenity in nodes (run-time knobs) • provides flexibility in design
Questions… ?