Post on 29-Dec-2015
Università degli Studi dell’AquilaAcademic Year 2012/2013
Course: Algorithms for Distributed SystemsInstructor: Prof. Guido Proietti
Schedule: Monday: 14.15 – 16.00 – Room 2.4 Wednesday: 11.15 – 13.00 – Room 0.4Questions?: Wednesday 14.00 - 16.00
Slides plus other infos:http://www.di.univaq.it/~proietti/didattica.html
Two main ingredients in the course:
Distributed Systems + AlgorithmsDistributed system (DS): Broadly speaking, we refer to a set of autonomous computational devices (say, processors) performing multiple operations/tasks simultaneously, and which influence reciprocally either by taking actions or by exchanging messages (using an underlying communication infrastructure)
We will be concerned with the computational aspects of a DS. We will analyze a DS depending on the cooperativeness level of its processors:
– Obedient: always cooperate honestly with the system Classic field of distributed computing
– Adversarial: may operate against the system Fields of fault-tolerance and security in DS
– Selfish: choose strategically how to behave Emerging field of game-theoretic aspects of DS
Two main ingredients in the course:
Distributed Systems + AlgorithmsAlgorithm (informal definition): effective method, expressed as a finite list of well-defined instructions, for solving a given problem (e.g., calculating a function, implementing a goal, reaching a benefit, etc.)
Regardless of their nature, processors in a DS follow a strategy which is dictated by an algorithm!
We will analyze these algorithms in (almost) every respect: existence, correctness, finiteness, efficiency (computational complexity), effectiveness, robustness, stability (w.r.t. to a given solution concept), etc.
Modern Distributed ApplicationsIn the old days: a number of workstations over a LAN
TodayCollaborative computing
Military command and control Online strategy games Massive computation
Distributed Real-time Systems
Process Control Navigation systems, Airline Traffic Monitoring (ATM)
Mobile Ad hoc Networks Rescue Operations, emergency operations, robotics
Wireless Sensor Networks Habitat monitoring, intelligent farming
Grid and Cloud computing
Stock market…
Some Issues in Building Distributed Applications
Reliability (connectivity)
Security (cryptography)
Consistency (mutual exclusion)
Cooperativeness (game theory)
Fault-tolerance (failures, recoveries…)
Performance: What is the efficiency of the designed algorithm?
Scalability: How is the performance affected as the number of processors increase ?
Course structureFIRST PART: Algorithms for COOPERATIVE DS
1. Leader Election2. Minimum spanning tree3. Maximal independent set
SECOND PART: Algorithms for UNRELIABLE DS1. Benign failures: consensus problem2. Byzantine failures: consensus problem
THIRD PART: Algorithms for CONCURRENT DS1. Mutual exclusion
Mid-term Written Examination: Last week of November
FOURTH PART: DS SECURITY (to be confirmed; alternatively, WIRELESS DS)1. Elements of cryptography
FIFTH PART: Algorithms for STRATEGIC DS• Strategic equilibria theory
SIXTH PART: Implementation theory for STRATEGIC DS• Algorithmic mechanism design (AMD)• AMD for Graph optimization problems
Final Oral Examination: this will be concerned with either the whole program or just the second part of it, depending on the outcome of the mid-term exam
Cooperative distributed Cooperative distributed algorithms: algorithms:
Message Passing SystemMessage Passing System
Cooperative distributed Cooperative distributed algorithms: algorithms:
Message Passing SystemMessage Passing System
A Formal ModelA Formal Model
The System Topology: a network (connected undirected graph)
Processors (nodes) Communication channels (edges)
Degree of synchrony: asynchronous versus synchronous (universal clock)
Degree of symmetry: anonymous (processors are indistinguishable) versus non-anonymous
Degree of Uniformity: uniform (number of processors is unknown) versus non-uniform
Local algorithm: the algorithm associated with each single processor
Distributed algorithm: the “composition” of local algorithms
Notation n processors: p0, p1, … , pn-1.
Each processor knows nothing about the network topology, except for its neighbors, numbered from from 1 to r
Communication takes place only through message exchanges, using buffers associated with each neighbor, namely outbufi[k], inbufi[k], i=1,…,r.
qi: the state set for pi, containing a distinguished initial state; each state describes the internal status of the processor and the status of the buffers
Configuration and events
System configuration: A vector [q0,q1,…,qn-1] where qi is the state of pi
Events: Computation events (internal computations plus sending of messages), and message delivering (receipt of messages) events
Asynchronous Systems
No upper bound on delivering times
Admissible execution: each message sent is eventually delivered
Synchronous Systems Each processor has a (universal)
clock, and computation takes place in rounds.
At each round each processor: 1. Reads the incoming messages buffer2. Makes some internal computations3. Sends messages which will be read in
the next round.
Message Complexity The total number of messages
sent during any admissible execution of the algorithm (in other words, the number of message delivery events).
However, the size of a message will count as well…
Time Complexity Synchronous: The number of
rounds until termination.Asynchronous: not really
meaningful
• Example: Distributed Depth-First Search visit of a graph– Visiting a (connected) graph G=(V,E) means to
explore all the nodes and edges of the graph– General overview of a sequential algorithm:
– Begin at some source vertex, r0
– when reaching any vertex v» if v has an unvisited neighbor, then visit it and
proceed further from it» otherwise, return to parent(v)
– when we reach the parent of some vertex v such that parent(v) = NULL, then we terminate since v = r0
– DFS defines a tree, with r0 as the root, which reaches all vertices in the graph
– sequential time complexity = Θ(|E|+|V|) (we use Θ notation because every execution of the algorithm costs exactly |E|+|V|, in an asymptotic sense)
• Distributed DFS: an asynchronous algorithm– Distributed version (token-based): the token
traverses the graph in a depth-first manner using the algorithm described above
1. Start exploration (visit) at a waking-up node (root) r.2. When v is visited for the first time:
2.1 Inform all of its neighbors that it has been visited.2.2 Wait for acknowledgment from all neighbors.2.3 Resume the DFS process.2.4 If no unvisited neighbor node exists, then pass token
back to the parent node
– Message complexity is Θ(|E|) (optimal, because of the trivial lower bound of (|E|) induced by the fact that every node must know the status of each of its neighbors – this requires an atomic message for each graph edge)
• Distributed DFS (cont’d.)•Time complexity analysis (sync. DS)
We ensure that vertices visited for the first time know which of their neighbors have been visited; thus we make no unnecessary vertex explorations:
»algorithm: freeze the DFS process; inform all neighbors of v that v has been visited; get Ack messages from those neighbors; restart DFS process constant number of rounds (i.e., 2) for each new discovered node
»|V| nodes are discovered time complexity = Θ(|V|)