Vitus Masters Defense
-
date post
13-Sep-2014 -
Category
Technology
-
view
1.038 -
download
0
description
Transcript of Vitus Masters Defense
PlanetenWachHundNetz
Instrumentation Infrastructure for PlanetLab
Vitus Lorenz-Meyer
Peer-to-Peer
• Distributed on open internet• All participants both receive & provide
services to/from others• Not centrally administered• Membership changes over time (churn)• Example: file sharing (napster, gnutella,…)
Any node can publish a named file Any node can obtain file from another node who
has it. Range of strategies to find nodes containing
desired contentVitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso2
Problem
Rel. Work
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso3
The Problem
• P2P systems hard to tune, requires understanding of complex behavior Requires instrumentation & analysis
• Many P2P systems constructed without scalable instrumentation infrastructure Frequently done in ad-hoc manner
Data transmitted to single collection & analysis node Inadequate for understanding behavior of large systems
of many (hundreds to MILLIONS) of nodes• My work: development of a flexible tool to
enable scalable instrumentation algorithms, data structures
P2P
Related work (1 of 2: cousins)
• Distributed Database Mngmt. Systems Select data at sources Optimize joins (run near to sources…) Commercially used in non-p2p configurations P2P (research): PIER, Sophia
• Sensor Networks Unmanaged radio-connected nodes
provide “network” of surveilance SQL; Compiled into a 3-step process Software communicates through same mechanism IrisNet, TAG
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso4
Related work (2of2: Siblings)
• Aggregation Overlays Information collection subsystem Nodes provide information tuples
Internal aggregation language Computed using parallel prefix of pre-defined
assoc/comm ops Astrolabe, SDIMS, SOMO
• Google’s MapReduce Data selection & aggregation in distributed
system User provides “map” and “reduce” program
Not fully p2p (resource mgmt. overlay)
High-level Approach
• User specifiable programs like MapReduce • Split data collection into 3 ‘phases’
1. Generate values on all nodes2. Pairwise aggregation throughout system3. Evaluate results
• emit measured vals (val,num=1)• Aggregate: (val1+val2,num1+num2)• Evaluate (avg) (val / num)
Easy to use: user provides 3 programs (scripts)
Impl.Rel. Work
Illustration of Binary Aggregation Impl.Rel. Work
Why is this hard in P2P?
• Problem: membership churn• Nodes continuously enter & leave
system• Nobody in charge (p2p)
• Nobody knows membership list!
Exposes following challenges1. Finding all participating nodes2. Constructing an (appx) balanced tree
Impl.Rel. Work
Building Structure Upon Anarchy: Key Based
Routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso9
GoalAppr.
02160
2158
2159
2159 + 2158
“Chord” Routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso10
GoalAppr.
Chord lookup
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso11
GoalAppr.
Building a tree upon KBR
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso12
GoalAppr.
a aa b
b
d ea b d
e
g if hd ea b
f
g
h
i
Building a tree: FTT & KBT
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso13
GoalAppr.
FTT: finger-based tree• Operation associated with a “target” node• Systems send data to finger closest to target• Ambiguous
Depends on all nodes’ fingertables• Tree useful only for aggregation
• KBT: Maps tree on key-space Operation associated w/ target node System/tree-node mapping:
Node assigned to node w/ nearest key
Non-ambiguous Tree useful for both dissemination &
aggregation Single, global tree
Our Structure
101…
111…101…
111…101… 110…100…
001…
011…001…
011…001… 010…000…
101…
011…
001…
110…
As 001..!
GoalAppr.
• KMR: Subset of KBT, rooted at specific node• One tree / root
Better load-balancing• Tree fully determined by set of active nodes and root
Implementation details
• PWHN-Server layered on FreePastry• PWHN-Client connects to PWHN-Server and
makes query• Callee builds tree making itself root
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso15
Goal
SS S
SS
S
S
S
C
Appr.
Our Goal
• Develop toolkit for data collection/aggregation in P2P networks Useful for PlanetLab-community
• Extend MR’s model to P2P K.I.S.S.
Users provide programs for gen/agg/eval• Use techniques from P2P
Construct aggregation tree upon key-based-routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso16
ExampleImpl. details
Example (1)
• First implementation: Script version, flat, to test approach
• Example 1: Overall average system load Gen emits (1,<1load>,<5load>,<15load>) for each server Agg adds all numbers Eval divides last 3 numbers by first to get average
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso17
EvaluationGoal
Example (2)
• PWHN client (Java) Can start and stop
server Used for specifying
all programs and parameters (Servers, username for flat, method)
Front-end for connecting to servers and making query
Allows saving and graphically representing result
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso18
EvaluationGoal
Example (3)
• Graphing of queried results
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso19
EvaluationGoal
Bar Chart Color bubbles on world map
Example (4)
Graphing of tree
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso20
EvaluationGoal
Graphing of paths of the query
Evaluation
• Minimize disruption Minimize incoming
bytes to client
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso21
SynopsisExamples
•More efficientLower average fan-in of aggregation tree
Evaluation: Fern
• Global Update latency histogram
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso22
SynopsisExamples
10 clients 701 clients
Summary
• PWHN - Instrumentation toolkit Extends MR’s model to P2P Uses P2P techniques (DHTs) Combines FTT and KBT to be more efficient Conclusion: Useful tool that is more efficient
than to build infrastructure into software What did I do?
Survey of systems that provide aggregation in dynamic networks
Classification and naming of aggregation trees upon DHTs
Design and implementation of my own tool (KMR/PWHN)
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso23
Examples
Questions
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso24
Synopsis