Distributed Hash Tables - Chord - ERNET€¦ · Distributed Hash Tables Chord Smruti R. Sarangi...
Transcript of Distributed Hash Tables - Chord - ERNET€¦ · Distributed Hash Tables Chord Smruti R. Sarangi...
OverviewDesign of Chord
Results
Distributed Hash TablesChord
Smruti R. Sarangi
Department of Computer ScienceIndian Institute of Technology
New Delhi, India
Smruti R. Sarangi Chord 1/29
OverviewDesign of Chord
Results
Outline
1 Overview
2 Design of ChordBasic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
3 Results
Smruti R. Sarangi Chord 2/29
OverviewDesign of Chord
Results
Comparison with Pastry
Chord vs PastryEach node and each key’s id is hashed to a unique value.The process of lookup tries to find the immediate successorto a key’s id.The routing table at each node contains O(log(n)) entries.Inserting and deleting nodes requires O(log(n)2) mes-sages.Sarangi View , : More robust than Pastry, and more ele-gant.
Smruti R. Sarangi Chord 3/29
OverviewDesign of Chord
Results
Comparison with other Systems
The Globe system assigns objects to locations, and ishieararchial. Chord is completely distributed and decentral-ized.CAN
Uses a d-dimensional co-ordinate space.Each node maintains O(d) state, and the lookup cost isO(dN1/d ).Maintains a lesser amount of state than Chord, but has ahigher lookup cost.
Smruti R. Sarangi Chord 4/29
OverviewDesign of Chord
Results
Features of Chord
Automatic load balancingFully distributedScalable in terms of state per node, bandwidth, and lookuptime.Always availableProvably correct.
Smruti R. Sarangi Chord 5/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Outline
1 Overview
2 Design of ChordBasic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
3 Results
Smruti R. Sarangi Chord 6/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Consistent Hashing
DefinitionConsistent Hashing: It is a hashing technique that adapts verywell to resizing of the hash table. Typically k/n elements need tobe reshuffled across buckets. k is the number of keys and n is thenumber of slots in a hash table.
1
2
3
4
5bucket 1
bucket 2
buck
et 3
Smruti R. Sarangi Chord 7/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Structure of Chord
Each node and key is assigned a m bit identifier.The hash for the node and key is generated by using theSHA-1 algorithm.The nodes are arranged in a circle (recall Pastry).Each key is assigned to the smallest node id that is largerthan it. This node is known as the successor .
Objective
For a given key, efficiently locate its successor.
Efficiently manage addition and deletion of nodes.
Smruti R. Sarangi Chord 8/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Properties of Chord’s Hashing Algorithm
For n nodes, and k keys, with high probability1 Each node stores at most (1 + ε)k/n keys2 Addition and deletion of nodes leads to a reshuffling of O(k/n)
keys
Previous papers prove that ε = O(log(n))There are techniques to reduce ε using virtual nodes.
Each node contains log(n) virtual nodes.Not scalable ( Not necessarily required )
Smruti R. Sarangi Chord 9/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Chord’s Routing(Finger) Table
Let m be the number of bits in an idNode n contains m entries in its finger table.
successor→ next node on the identifier circlepredecessor→ node on the identifier circle
The i th finger contains:finger[i].start = (n + 2i−1) mod 2m, (1 ≤ i ≤ m)finger[i].end = (n + 2i − 1) mod 2m
finger[i].node = successor(finger[i].start)
Basic OperationfindSuccessor(keyId)→ nodeId
Smruti R. Sarangi Chord 10/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Outline
1 Overview
2 Design of ChordBasic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
3 Results
Smruti R. Sarangi Chord 11/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Finger Table- II
1
3
8
1115
22
40
3+1, 3+2, 3+4
3+8
3+16
3+32
14key
Smruti R. Sarangi Chord 12/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Algorithms
Algorithm 1: findSuccessor in Chord
1 n.findSuccessor(id) begin2 n’← findPredecessor(id)
return n’.successor(id)
3 end4 n.findPredecessor(id) begin5 n′ ← n
while id /∈ (n′,n′.successor()) do6 n’← n’.closestPrecedingFinger(id)7 end8 end
Smruti R. Sarangi Chord 13/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
closestPrecedingFinger(id)
1 n.closestPrecedingFinger(id) begin2 for i← m to 1 do3 if finger[i].node ∈ (n, id) then4 return finger [i].node5 end6 end7 return n8 end
Smruti R. Sarangi Chord 14/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
O(log(n)) Routing Complexity
1
3
8
1115
22
35
3+2
3+2i
i-1
17node
predecessor
2i-1
2i-1
<
Smruti R. Sarangi Chord 15/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Outline
1 Overview
2 Design of ChordBasic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
3 Results
Smruti R. Sarangi Chord 16/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Node Arrival
Each node maintains a precessor pointerInitialize the predecessor and the fingers of the new node.Update the predecessor and fingers of other nodesNotify software that the node is ready
Smruti R. Sarangi Chord 17/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Node Arrival - II
n initially contacts n′
1 n.join(n′) begin2 n.initFingerTable(n′)
updateOthers()
3 end
Smruti R. Sarangi Chord 18/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Algorithm 2: initFingerTable in Chord
1 n.initFingerTable(n′) begin2 finger[1].node← n′.findSuccessor(finger[1].start)
successor← finger[1].nodepredecessor← successor.predecessorsuccessor.predecessor← npredecessor.successor← nfor i← 1 to m-1 do
3 if finger[i+1].start ∈ (n, finger[i].node) then4 finger[i+1].node← finger[i].node
5 end6 else7 finger[i+1].node← n’.findSuccessor(finger[i+1].start)
8 end9 end
10 end Smruti R. Sarangi Chord 19/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
updateOthers()
1 n.updateOthers() begin2 for i ← 1 to m do3 pred ← findPredecessor (n - 2i−1)
pred .updateFingerTable(n, i)
4 end5 end6 pred.updateFingerTable(n, i) begin7 if n ∈ (pred , finger [i].node) then8 finger[i].node← n
p ← predecessorp.updateFingerTable(n,i)
9 end10 end
Smruti R. Sarangi Chord 20/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Stabilization of the Network (run periodically)
1 n.stabilize() begin2 x ← successor.predecessor
if x ∈ (n, sucessor) then3 successor← x
4 end5 successor.notify(n)
6 end7 n.notify(n’) begin8 if (predecessor is null) OR (n′ ∈ (predecessor, n)) then9 predecessor← n′
10 end11 end
Smruti R. Sarangi Chord 21/29
OverviewDesign of Chord
Results
Basic StructureAlgorithm to find the SuccessorNode Arrival and Stabilization
Stabilization-II
1 n.fix_fingers() begin2 i← random() finger[i].node← find_successor (finger[i].start)3 end
Smruti R. Sarangi Chord 22/29
OverviewDesign of Chord
Results
Results
Evaluation Setup
Network consists 104 nodesNumber of keys : 105 to 106
Each experiment is repeated 20 timesThe major results are on a Chord protocol simulator
Smruti R. Sarangi Chord 23/29
OverviewDesign of Chord
Results
Effect of Virtual Nodes
The number of keys per node decreases with the number ofvirtual nodes .For 1 virtual node, we can have up to 500 keys per node(mean 100).For 10 virtual nodes, we can have roughly 50 to 200 keysper node (mean 100).
source [1]
Smruti R. Sarangi Chord 24/29
OverviewDesign of Chord
Results
Average Path Length
The path length in Chord grows with the number of nodes.It is roughly normally distributed about the mean. For amean of 6 nodes (path length), the ±3σ range varies from1 to 11.
Number of nodes Path length (approx.)10 2100 3
1000 4.310000 6.2
source [1]
Smruti R. Sarangi Chord 25/29
OverviewDesign of Chord
Results
Other DHT Systems: Tapestry
Tapestry160 block id, Octal digitsRouting table like pastry (digit based hypercube)Does not have a leaf set or neighborhood table.
Smruti R. Sarangi Chord 26/29
OverviewDesign of Chord
Results
Other DHT Systems: Kademlia
KademliaBasis of bit-torrentEach node has a 128 bit idEach digit contains only 1 bitFind the closest node to a keyValues are stored at several nodesNodes can cache the values of popular keys.
Smruti R. Sarangi Chord 27/29
OverviewDesign of Chord
Results
Other DHT Systems: CAN
CAN – Content Addressable NetworkIt uses a d-dimensional multi-torus as its overlay network.Node uses standard routing algorithms for tori. It uses O(d)space. (Note: This is independent of n)Each node contains a virtual co-ordinate zone.Node Arrival: Split a zoneNode Departure: Merge a zone
Smruti R. Sarangi Chord 28/29
OverviewDesign of Chord
Results
Chord: A Peer-to-Peer Lookup Service for Internet Applica-tions, by I. Stoica, R. Morris, D. Karger, F. Kaashoek, H. Bal-akrishnan, Proc. ACM SIGCOMM, San Diego, CA, Septem-ber 2001.
Smruti R. Sarangi Chord 28/29