LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems
-
Upload
judith-williamson -
Category
Documents
-
view
26 -
download
2
description
Transcript of LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems
![Page 1: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/1.jpg)
LessLogA Logless File Replication Algorithm for Peer-to-Peer
Distributed Systems
Kuang-Li HuangTai-Yi Huang
Jerry C. Y. Chou
Embedded Operating Systems (EOS) Labhttp://eos.cs.nthu.edu.tw/
Department of Computer ScienceNational Tsing Hua University, Taiwan
![Page 2: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/2.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Outlines
IntroductionSystem model
Basic model Advanced model Fault-tolerant model
Experimental resultsConclusions and future work
![Page 3: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/3.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Related Work Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet
Applications. IEEE/ACM Transactions on Networking, 2003
Pastry: Scalable, Distributed Object Location and Routing for Large-Scale Peer-to-Peer Systems. IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), 2001
Tapestry: A Resilient Global-Scale Overlay for Service Deployment. IEEE Journal on Selected Areas in Communications, January 2004
SCAN: A Dynamic, Scalable, and Efficient Content Distribution Network. International Conference on Pervasive Computing, 2002
![Page 4: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/4.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
LessLog Contributions
An efficient bitwise replication algorithm Independent with underlying lookup protocols Make no use of client-access logs
A complete set of file-access APIFault-tolerant featuresSelf-organized for dynamic system change
![Page 5: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/5.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Basic System Model
There are totally N live nodes, N = 2m.Each node is assigned a unique ID in [0, N-1]
Randomly assigned or any user-specified way Such ID is called the PID of the node Denoted by P(i)
A complete N-node binomial tree is built for each node Totally N different physical trees
![Page 6: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/6.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Virtual Tree We use one unique virtual tree to construct each of
N different physical trees
111111
111100
101011
010111
101000
000011
000000
010100
VID
![Page 7: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/7.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Physical Tree for Node 3
111
110 101 011
100 001
000
010
3
2 1 7
0 6
4
3=011
key=100
complement
110⊕100=010
101⊕100=001
5
![Page 8: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/8.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Physical Tree for Node 6
111111
110110 101101 011011
100100 001001
000000
010010
66
77 44 22
55 33 00
11
![Page 9: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/9.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Properties of Virtual Tree
A node has i children nodes if its leftmost i bits in VID are all 1’s
The node of VID i has more or the same offspring nodes than the node of VID j if i > j
Given the PID of the root node in a lookup tree, we can do PID VID mapping for each node in the lookup tree
![Page 10: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/10.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
LessLog Tree Traversal
111111
110110 101101 011011
100100 001001
000000
010010
33
22 11 77
00 66 55
44
![Page 11: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/11.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
LessLog INSERT / GET API
111111
110110 101101 011011
100100 001001
000000
010010
33
22 11 77
00 66 55
44ask Hashing(file)=3
connect to P(6)
![Page 12: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/12.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
LessLog REPLICATE API
111111
110110 101101 011011
100100 001001
000000
010010
33
Each replication reduces 50% load!
![Page 13: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/13.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Advanced System ModelThere are totally N live nodes, N ≦ 2m.Each node is assigned a unique ID in [0, 2m-
1] Randomly assigned or any user-specified way Denoted by P(i) Live nodes and dead nodes
A complete 2m-node binomial tree is built for each node Totally 2m different physical trees
![Page 14: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/14.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
The 1st live node with largest VID
Advanced LessLog INSERT / GET API
111111
110110 101101 011011
100100 001001
000000
010010
33
22 11 77
00 66 55
44 ask Hashing(file)=3
![Page 15: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/15.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Overloaded Node with Largest VID
111
110 101 011
100 001
000
010
3
2 1 7
0 6 5
4
100
011
010
001Sorted by VID
![Page 16: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/16.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Self-organized LessLog
Joining (Leaving) a node Check whether it is the largest VID Ex. P(6) join, P(1), P(4) are dead nodes
1
111
110 101 011
100 001010
6
7 4 2
5 3 0
100
7111
110 011
010
6 3
2
1015
1004
0011
0000
![Page 17: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/17.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Fault-Tolerant LessLogFiles can always be accessedLessLog stores each file in 2b nodes, b ≦ m2b identical subtrees
111
110
101
011
100
001
000
010
3
2 1 7
0 6 5
4
1
1
1
1
0
0 0
0
![Page 18: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/18.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
22 identical subtrees
1110
1100
1010
0110
1000
0010
0000
0100
2
0 6 10
4 8 14
12
1111
1101
1011
0111
1001
0011
0001
0101
3
1 7 11
5 9 15
13
![Page 19: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/19.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
0
100
200
300
400
500
600
700
1 3 5 7 9 11 13 15 17 19 21
log-based
LessLog
random
Experimental Results Compared with log-based and random algorithms
at an evenly-distributed model
incoming requests/1000
rep
lica
s
![Page 20: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/20.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
0
100
200
300
400
500
600
700
1 3 5 7 9 11 13 15 17 19 21
log-basedLessLograndom
Experimental Results Compared with log-based and random algorithms
at an locality model
incoming requests/1000
rep
lica
s
![Page 21: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/21.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Conclusions and Future Work
An efficient bitwise replication algorithm Built on top of any existing lookup protocol
Make no use of client-access logs Significant performance improvement
Fault-tolerant & self-organized futures Concurrent joins / leaves / failures
Under implementation at PlanetNet systems Real performance numbers
![Page 22: LessLog A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems](https://reader035.fdocuments.us/reader035/viewer/2022062422/5681355a550346895d9cbfea/html5/thumbnails/22.jpg)
EOS Lab, National Tsing Hua University IPDPS 2004
Thank You