Towards a Scalable and Towards a Scalable and Robust DHTRobust DHT

Baruch AwerbuchBaruch AwerbuchJohns Hopkins UniversityJohns Hopkins University

Christian ScheidelerChristian ScheidelerTechnical University of MunichTechnical University of Munich

22 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Holy Grail of Distributed SystemsHoly Grail of Distributed Systems

Scalability and Robustness

Adversarial behavior increasingly pressing issue!

33 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Why is this difficult???Why is this difficult???Scalability: Scalability: minimizeminimize resources needed for operations resources needed for operationsRobustness:Robustness:maximizemaximize resources needed for attack resources needed for attack

Scalable solutions seem to be easy to attack!Scalable solutions seem to be easy to attack!

44 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Distributed Hash TableDistributed Hash TableProblem:Problem: maintain a hash table of data maintain a hash table of data items across multiple sitesitems across multiple sites

Basic operations:Basic operations: Join(s):Join(s): site s joins the system site s joins the system Leave(s):Leave(s): site s leaves the system site s leaves the system Insert(d):Insert(d): insert data item d into hash table insert data item d into hash table Lookup(name):Lookup(name): lookup data item with given lookup data item with given

namename

data

sites

55 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Distributed Hash TableDistributed Hash TableScalable DHT:Scalable DHT:

Bounded-degreeBounded-degree overlay network overlay networkFewFew copies per data item copies per data item

data

sites

66 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Distributed Hash TableDistributed Hash TableRobust DHT:Robust DHT:

Network can sustain Network can sustain constant fractionconstant fraction of adversarial sites of adversarial sitesData layer can handle Data layer can handle arbitraryarbitrary collection of insert and collection of insert and lookup requestslookup requests

data

sites

77 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

DHT by Karger et al.DHT by Karger et al.

0 1

data

sites

88 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Overlay NetworkRobust Overlay NetworkTwo central conditionsTwo central conditions (given n sites): (given n sites):

Balancing condition:Balancing condition: only only O(log n)O(log n) sites in sites in intervals of size intervals of size (c log n)/n(c log n)/nMajority condition:Majority condition: adv. sites in minority in adv. sites in minority in all intervals of size all intervals of size (c log n)/n(c log n)/n

Balancing condition: Balancing condition: scalabilityscalabilityMajority condition: Majority condition: robustnessrobustness via majority via majority decisiondecision

0 1

99 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to satisfy conditions?How to satisfy conditions?Chord:Chord: uses cryptographic hash function to map uses cryptographic hash function to map

sites to points in [0,1)sites to points in [0,1)randomlyrandomly distributes honest sites distributes honest sitesdoes notdoes not randomly distribute adversarial sites randomly distribute adversarial sites

1010 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to mix adversarial sites?How to mix adversarial sites?

CAN:CAN: map sites to map sites to randomrandom points in [0,1) points in [0,1)

1111 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to mix adversarial sites?How to mix adversarial sites?Group spreading Group spreading [AS04][AS04]::

Map sites to random points in [0,1)Map sites to random points in [0,1)Limit lifetime of pointsLimit lifetime of points

Too expensive!

1212 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to mix adversarial sites?How to mix adversarial sites?Our approach:Our approach:

nn honest nodes honest nodes (not under control of (not under control of adversary) andadversary) and nn adversarial nodesadversarial nodesAdversary canAdversary can adaptively adaptively join and leave join and leave with its nodeswith its nodesObliviousOblivious join operation that can maintain join operation that can maintain conditions under conditions under anyany adversary adversary

Naive solution:Naive solution: perturb perturb everythingeverything after each after each join operationjoin operation

1313 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to mix adversarial sites?How to mix adversarial sites?

Card shuffling [Diaconis & Shahshahani 81]:

random transposition

(n log n) transpositions: random permutation

(log n) transpositions per join operation??

1414 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Random transpositionsRandom transpositions

Cannot preserve balancing condition!!

1515 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

How to mix adversarial sites?How to mix adversarial sites?Rule that works:Rule that works: k-cuckoo rule k-cuckoo rule

evict k/n-region

n honest n adversarial

< 1-1/k

1616 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Are we done?Are we done?

Dilemma:Dilemma: we we cannot cannot randomly distribute randomly distribute the datathe dataReason:Reason: data data unsearchableunsearchable!!So we need hash function, but then So we need hash function, but then open open to adversarial attacksto adversarial attacks on insert, lookup on insert, lookup operationsoperations

1717 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

What is the problem?What is the problem?Adversary can select requests to Adversary can select requests to differentdifferent data items at data items at samesame region region

Any medicine against that???Any medicine against that???

0 1

1818 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Yes!!Yes!!

1919 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Data ManagementRobust Data ManagementLong, long ago…Long, long ago…Deterministic PRAM simulation:Deterministic PRAM simulation:

Pioneered by Mehlhorn and Vishkin 84Pioneered by Mehlhorn and Vishkin 84Problem:Problem: simulate PRAM of simulate PRAM of nn processors and processors and mm memory cells on complete network of memory cells on complete network of nn processors with memoryprocessors with memory

Central ideas:Central ideas:Choose Choose 2c-12c-1 fixed hash functions with fixed hash functions with expansion expansion properties, properties, c=c=(log m)(log m) Majority trick: Majority trick: update and lookup any update and lookup any cc copies copies

2020 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Data ManagementRobust Data ManagementWhy Why deterministic deterministic strategies?strategies?Randomness Randomness expensiveexpensive in dynamic systems in dynamic systems

with adversarial presence!with adversarial presence!

More complex for dynamic networks:More complex for dynamic networks:CongestionCongestion during routing during routingContentionContention at destination regions at destination regions

2121 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Data ManagementRobust Data ManagementExpansion properties (probabilistic proof): Expansion properties (probabilistic proof):

simple simple threshold rulethreshold rule sufficient sufficient

sources

2222 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Data ManagementRobust Data ManagementStrategy:Strategy:

Run several attemptsRun several attemptsIn each attempt:In each attempt: For each remaining request, route 2c-1 For each remaining request, route 2c-1

packets, one for each copypackets, one for each copy Discard packets at congested nodesDiscard packets at congested nodes If c packets of a request successful, then If c packets of a request successful, then

request successfulrequest successful

How many attempts???How many attempts???

2323 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Robust Data ManagementRobust Data ManagementAnswer:Answer: O(log n)O(log n) many attempts with many attempts with

congestion threshold congestion threshold O(polylog).O(polylog).

Theorem 1:Theorem 1: For For anyany set of set of nn lookup lookup requests, one request per node, the requests, one request per node, the lookup protocol can serve all requests in lookup protocol can serve all requests in O(polylog)O(polylog) communication rounds. communication rounds.

Theorem 2:Theorem 2: Same for insert requests Same for insert requests

2424 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

ConclusionConclusionWe presented high-level solution for a We presented high-level solution for a scalable and robust DHT.scalable and robust DHT.

Open problems:Open problems:Low-levelLow-level attacks (DoS!) attacks (DoS!)Adversary controls join-leave behavior of Adversary controls join-leave behavior of adversarial adversarial andand honest nodes honest nodesElementaryElementary random number generator random number generator

2525 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Questions?

2626 Towards a Scalable and Robust DHTTowards a Scalable and Robust DHT

Hash tableHash table

data

sites

0 1