Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.
![Page 1: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/1.jpg)
Finding Nearby Objects in Peer-to-Peer Networks
Kirsten Hildrum
![Page 2: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/2.jpg)
Object Location
![Page 3: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/3.jpg)
The Big Picture
![Page 4: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/4.jpg)
Behind the Cloud
?
![Page 5: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/5.jpg)
Two solutions
• One directory– One participant stores
a lot, but average is low
– Stretch is high
• Everyone has a directory– Low stretch!– Everyone has lots of
data– Changes are very
expensive
![Page 6: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/6.jpg)
Distribute directoryEach peer is responsible for part of directory
Q: Who is responsible for my object?A: Hash peers into same namespace as
objects.
Q: How do I get there?A: Peer-to-peer networks route to names.
Must efficiently self-organize into network
![Page 7: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/7.jpg)
Efficiency is…
With respect to network• Few neighbors• Short routes (few hops
& low stretch)• Easy routing (No
routing tables!)• Dynamic
With respect to directory• Not too many entries
(4,4)
hard
easy
![Page 8: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/8.jpg)
• Low stretch dynamic peer-to-peer network
• Tolerate failures in network
• Adapting to network variation
• Related results
• Future work
Outline
![Page 9: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/9.jpg)
System Neighbors
Motivating Structure
Hops
CAN, 2001 O(r) grid O(rn1/r)
Chord, 2001 O(log n) hypercube O(log n)
Pastry, 2001 O(log n) hypercube O(log n)
Tapestry, 2001 O(log n) hypercube O(log n)
Distributed Hash Tables
• These systems give– Guaranteed location – Join and leave algorithms– Load-balanced storage
• No stretch guarantees
![Page 10: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/10.jpg)
Low Stretch Approaches
System Stretch Space Balanced Metric
Awerbuch Peleg, 1991
polylog polylog no General
PRR, 1997 O(1) O(log n) yes Special
Thorup-Zwick O(k2) O(kn1/k) yes General
RRVV, 2001 polylog polylog yes General
• Not dynamic
Result #1: Make a dynamic low-stretch scheme
![Page 11: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/11.jpg)
PRR/Tapestry
City
State
Country
![Page 12: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/12.jpg)
PRR/Tapestry
Two object types: red and blue, so two trees
Level 1
Level 2
Level 3
![Page 13: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/13.jpg)
Neighbor TableFor “5471” (Octal)
Routing Levels1234
1xxx
2xxx
0xxx
3xxx
4xxx
5xxx
6xxx
7xxx
50xx
51xx
52xx
53xx
54xx
55xx
56xx
57xx
540x
541x
542x
543x
544x
545x
546x
547x
5470
5471
5472
5473
Ø
5475
Ø
5477
Balancing Load
1
NodeID5123
3
3
2
22
4
3
NodeID5471
NodeID5416
NodeID5061
NodeID5432
NodeID5455
NodeID5470
![Page 14: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/14.jpg)
Building It• Every node is a root!• Can extend to more than two colors• Pefix routing in base b promoting at rate 1/b• b=16 match one hex digit or four bits at a time
• Space: O(b logb n) – Every node at every level stores b parents
We want the links to be of geometrically increasing length
![Page 15: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/15.jpg)
Node Joins
• Correctness O(log n) insert & delete – Concurrent inserts in a lock-free fashion
• Keeping low stretch– Required a nearest-neighbor search algorithm
as subroutine– Keep O(1) stretch in special metrics
• Algorithms implemented in version of Tapestry used in OceanStore
![Page 16: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/16.jpg)
Concurrent Insert
![Page 17: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/17.jpg)
Solving the Problem
Theorem 1 When peer A is finished inserting, it knows about all relevant peers that have finished insertion.
Hildrum, Kubiatowicz, Rao & Zhao, “Distributed Object Location in a Dynamic Network” SPAA2002
Sequential Consistency(without locks)
BA
![Page 18: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/18.jpg)
0
20
40
60
80
100
120
0 5 10 15 20 25 30 35
Time (minutes)
Co
nsi
stan
cyNode Join in Practice
largegroup join
~210
killnodes
RouteToID operations on 40 node clusterChurn: nodes join/leave every 10 seconds, average lifetime = 2mins
150 128 203
Constant churn
![Page 19: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/19.jpg)
Dynamic challenge: find parent
• Need to find closest at level
• Idea Use routing structure backwards
• Other approaches• Clarkson ‘97
• Karger & Ruhl ‘02
• Krauthgamer Lee ‘03
![Page 20: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/20.jpg)
Growth Restriction
• Growth Restriction:– Ball of radius 3r
contains only a factor of c more nodes than ball of radius r.
– Stronger than “polynomial growth”
• Base b at least c– Promote less than 1/c
r3r
![Page 21: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/21.jpg)
Algorithm Idea, in pictures
new
![Page 22: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/22.jpg)
The Trimming Lemma
• Circle of radius r around new with at least one level i node.
• To find level (i-1) must find its parent.
• Level (i-1) node in little circle must point to a level i node in 3r of new
<2r
r
3r
![Page 23: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/23.jpg)
How Many?
• Keep fixed k= O(log n) • If k red nodes in little
circle, then about ck in big circle
• Expect only about ck/b blue nodes
• If c/b < 1, expect less than k
r
3r
log n per level & log n levels, so total is O(log2 n)
![Page 24: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/24.jpg)
Summary
In growth-restricted networks (with no additional space!):
Theorem 2 [HKRZ02] Can find nearest neighbor with high probability with O(log2 n) messages
Theorem 3 [HKMR04] Can find nearest neighbor, and messages is O(log n) with high probability
+ more dynamic, low-stretch network
![Page 25: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/25.jpg)
• Low stretch dynamic peer-to-peer network
• Tolerate failures in network
• Adapting to network variation
• Related results
• Future work
Outline
![Page 26: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/26.jpg)
Behind the Cloud Again
![Page 27: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/27.jpg)
Dealing with faults
• Multiple paths– Castro et. al– One failure along path,
path breaks
• Wide path– Paths faulty at the same
place to break
• Exponential difference in width effect
• “retrofit” Tapestry to do latter in slightly malicious networks
Failed!
Still good…
![Page 28: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/28.jpg)
Effective even for small overhead
Theorem 4 In growth restricted spaces, can make probability of failed route less than 1/nc for width O(clog n)Hildrum & Kubiatowicz, DISC02
0
10
20
30
40
50
60
70
80
90
100
0.1 0.2 0.3 0.4 0.5
Fraction of Bad Nodes
% f
aile
d r
ou
tes 1
2
3
4
5
6
![Page 29: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/29.jpg)
Wide path vs. multiple paths
0
10
20
30
40
50
60
70
80
90
0 0.1 0.2 0.3 0.4 0.5 0.6
Fraction of Bad nodes
Fa
ile
d P
ath
s
4
4 Single
![Page 30: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/30.jpg)
• Low stretch dynamic peer-to-peer network
• Tolerate failures in network
• Adapting to Network Variation
• Related results
• Future work
Outline
![Page 31: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/31.jpg)
Network not homogeneous
Previous schemes assumed global growth rate. But what if there isn’t one?
San Francisco
Nebraska
Paris
![Page 32: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/32.jpg)
Using the local c
• Idea: Determine local c and use it
• Challenge: Paths still meet up, stretch still low, and space reasonable
• Technique: “Fix” or match bits– In variable size chunks– PRR97, LAND, Tapestry,
Pastry matched bits in fixed size chunks
r3r
![Page 33: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/33.jpg)
Summary
• Previous approaches assumed a global c• Had to know c in advance of construction Hildrum, Krauthgamer & Kubiatowicz
[SPAA04]:
Dynamic, locally optimal low-stretch network
![Page 34: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/34.jpg)
• Low stretch dynamic peer-to-peer network
• Tolerate failures in network
• Adapting to network variation
• Related results
• Future work
Outline
![Page 35: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/35.jpg)
Congestion
• Latency might not be right measure
• Reduce bandwidth usage
• When demands are known, can use linear program to find a set of paths
• But what if demands are not known…
![Page 36: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/36.jpg)
Summary
• Räcke02 showed scheme exists O(log3 n)• Azar, Cohen, Fiat, Kaplan, & Räcke in STOC03
find optimal routing, but not hierarchical
Theorem 6 Hierarchical scheme can be constructed in polynomial time. (Within O(log2 n log log n) of optimal)– Harrelson, Hildrum, & Rao, “A Polynomial-time Tree
Decomposition to Minimize Congestion” SPAA03
![Page 37: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/37.jpg)
Online Auctions for Digital Goods
Online Answer bidder i before seeing the next bid– Bob offers you 50¢, yes or no?– Don’t get to see Alice’s bid before answering
Digital Each additional copy is free– Want to sell to everyone at their bid– Can’t sell to everyone at their bid, or else all bids will be low
Bar-Yossef, Hildrum, Wu, “Online Auctions for Digital Goods” SODA 2002:
Theorem 7 Constant competitive ratio when opt makes O(maxBid log log maxBid)
Theorem 8 When best fixed price sells few, can get:
)maxBid(loglogexp O
![Page 38: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/38.jpg)
• Low stretch dynamic peer-to-peer network
• Tolerate failures in network
• Adapting to network variation
• Related results
• Future work
Outline
![Page 39: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/39.jpg)
Future Work• Object location questions
– Tradeoff between degree, stretch, load balance as function of growth rate
– Smooth transition between general-metric systems and growth-restricted systems
• Distributed low-congestion network
• Peer-to-peer networks with selfish players
• Sensor networks
![Page 40: Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum.](https://reader031.fdocuments.us/reader031/viewer/2022032703/56649d2a5503460f949febd5/html5/thumbnails/40.jpg)
Thank You!