Sergio Rajsbaum 2006 Lecture 1 Introduction to Principles of Distributed Computing Sergio Rajsbaum...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Sergio Rajsbaum 2006 Lecture 1 Introduction to Principles of Distributed Computing Sergio Rajsbaum...
Sergio Rajsbaum 2006
Lecture 1Introduction to Principles of
Distributed Computing
Sergio RajsbaumMath Institute
UNAM, Mexico
Sergio Rajsbaum 2006
Lecture 1
• Part I: Two-phase commit. An example of a distributed protocol
• Part II: What is a distributed system and its parameters. Problems solved in such a system. The need for a theoretical foundation
Sergio Rajsbaum 2006
Part I: Two-phase commit
An example of an important distributed DBMS protocol
Sergio Rajsbaum 2006
Principles of Distributed Computing
• With the development of networking infrastructures and software, it is more and more rare to find a system that does not interact with other systems
• Distributed computing studies systems where components interact and collaborate
• Principles of distributed computing tries to understand the fundamental possibilities and limitations of such systems, with a precise, scientific approach
• Goal: to design efficient and reliable systems, and techniques to design them, analyze them and prove them correct, or to prove impossibility results when no protocol exists
Sergio Rajsbaum 2006
Distributed Commit
An example from databases[ Garcia-Molina, Ullman, Widom, Database Systems, 2001 ]
Sergio Rajsbaum 2006
Distributed Commit
• A distributed transaction with components at several sites should execute atomically
• Example: A manager of a chain of stores wants to query all the stores, find the inventory of toothbrushes at each, and issue instructions to move toothbrushes from store to store in order to balance the inventory.
• The operation is done by a single global transaction T that has component Ti at the i-th store and a component T0 at the office where the manages is located.
Sergio Rajsbaum 2006
Sequence of activities performed by T
1. Component T0 is created at the site of the manager2. T0 sends messages to all the stores instructing them to
create components Ti3. Each Ti executes a query at store i to discover the number
of toothbrushes in inventory and reports this number to T04. T0 takes these numbers and determines, by some algorithm
we shall not discuss, what shipments of toothbrushes are desired. T0 then sends messages such as “store 10 should ship 500 toothbrushes to store 7” to the appropriate stores
5. Stores receiving instructions update their inventory and perform the shipments
Sergio Rajsbaum 2006
Atomicity
• Make sure it does not happen: some of the actions of T get executed, but others do not
• We do assume atomicity of each Ti, through mechanisms such as logging and recovery
• Failures make difficult the achievement of atomicity of T– A site fails or is disconnected from the network
– A bug in the algorithm to redistribute toothbrushes instructs store 10 to ship more than it has
Sergio Rajsbaum 2006
Example of failures
• Suppose T10 replies to T0’s 1st message with its inventory.
• The machine at 10 then crashes, the instructions form T0 are never received by T10
• However, T7 sees no problem, and receives the instructions from T0
• Can distributed transaction T ever commit?
Sergio Rajsbaum 2006
Two-phase commit
• To guarantee atomicity distributed DBMS use a protocol for deciding whether or not to commit a distributed transaction
• Each component of the transaction will commit, or non will
• The protocol is coordinator based. It could be the site at which the transaction originates, such as T0. Call it C
Sergio Rajsbaum 2006
Phase 1
1. C sends to each site “prepare T”2. Each site receiving it, decides whether to commit or abort
its component of T. It must eventually send this response to C
3. At this point the site performs all actions associated with the local T, to be sure the local component does not abort later due to a local failure, and sends “ready T” to C. Only C could instruct it to abort later on
4. If the site send “don’t commit” is can locally abort T, since T will surely abort even if another wants to commit
Sergio Rajsbaum 2006
Phase 2
1. It begins when C receives the responses “ready” or “don’t commit” by all. Or after a timeout, in case of a site fails or gets disconnected
2. If C receives “ready” from all then it decides to commit T. Sends “commit T” to all
3. If C receives a “don’t commit” it aborts T, and sends “abort” to all
4. If a site receives “commit” it commits its local component of T; if it receives “abort” it aborts it
Sergio Rajsbaum 2006
Recovery in case of failures
Two cases: when C fails or when another fails
Sergio Rajsbaum 2006
Recovery of a non-C site
• Suppose a site S fails during a two-phase commit. Need to make sure when it recovers is does not introduce inconsistencies
• The hard case: its last log is it sent a “ready” to C• Then it must communicate to at least one other site
to find out the global decision for T. In the worst case, no other site can be contacted and the local component of T must be kept active
Sergio Rajsbaum 2006
What if C fails?
• The surviving participant sites must wait for C to recover, or after a waiting time, elect a new coordinator
• Leader election problem is in itself a complex problem. The simple solution of interchanging IP addresses and electing the smallest works often, but may fail
• Once a new leader C’ exists, it polls the sites about the status of all transactions
• If some site had already received a “commit” from C then C’ sends “commit” to all
• If some site had already received a “ abort” it sends it to all• If no site received such messages, and at least one was not
ready to commit, it is safe to abort T
Sergio Rajsbaum 2006
The hard case
• No site had received “commit” or “abort” and every site is ready to commit
• C’ cannot be sure if C found some reason to abort T (perhaps for a local reason, or some delays), or to commit it and it had already committed it locally.
• Must wait until the original C recovers• In real systems the administrator has the ability to
intervene manually and force the waiting transaction components to finish
• The result is a possible loss of atomicity, and the person executing T will be notified
Sergio Rajsbaum 2006
So, is the protocol correct ?
Is it efficient?
Is manual intervention unavoidable ??
Sergio Rajsbaum 2006
How to analyze the protocol ?
• What exactly does it mean to be correct? – Not just “either all commit or all abort”– What is the termination correctness condition ?
• Under what circumstances it is correct? – E.g. Trivial if no failures are possible. – What if there are malicious failures?– How to choose the time-outs?
• What is its performance? – Depends on the timeouts, delays. – On the type and number of failures
• Is it efficient? Are there more efficient protocols?
Sergio Rajsbaum 2006
Part II:
What is a distributed system and its parameters. Problems solved in such a
system. The need for a theoretical foundation
Sergio Rajsbaum 2006
What is distributed computing?
• Any system where several independent computing components interact
• This broad definition encompasses– VLSI chips, and any modern PC
– tightly-coupled shared memory multiprocessor
– local area cluster of workstations
– internet, WEB, Web services
– wireless networks, sensor networks, ad-hoc networks
– cooperating robots, mobile agents, P2P systems
Sergio Rajsbaum 2006
Computing components
• Referred to processors or processes in the literature
• Can represent a– microprocessor – process in a multiprocessing operating system– Java thread– mobile agent, mobile node (e.g. laptop), robot– computing element in a VLSI chip
Sergio Rajsbaum 2006
Interaction – message passing vs. shared memory
• Processors need to communicate with each other to collaborate, via
• Message passing– Point-to-point channels, defining an interconnection
graph– All-to-all using an underlying infrastructure (e.g.
TCP/IP)– Broadcast; wireless, satellite
• Shared memory– Shared-objects: read/write, test&set, compare&swap, etc– Usually harder to implement, easier to program
Sergio Rajsbaum 2006
A distributed system
processors
Communicationmedia
collaborate
Sergio Rajsbaum 2006
Failures
• Any system that includes many components running over a long period of time must consider the possibility of failures
• of processors and communication media
• of different severity– from processor crashes or message loses, to– malicious Byzantine behavior
Sergio Rajsbaum 2006
A distributed system is one in which the failure of a
computer you didn't even know existed can render
your own computer unusable.
Leslie Lamport
Sergio Rajsbaum 2006
A note on parallel vs.
distributed computing
Sergio Rajsbaum 2006
Parallel vs. distributed computing
• Distributed computing – focuses on the more loosely coupled end of the spectrum– Each processor has its own semi-independent agenda, but needs to
coordinate with others for fault-tolerance, sharing resources, availability, etc.
– Processors tend to be somewhat independent: sometimes physically separated, running at independent speeds, communicating through a media with unpredictable delays
– Multi-layered, many different issues need to be addressed: communication, synchronization, fault-tolerance…
• Parallel processing – studies the more tightly-coupled end of the spectrum– usually all processors are dedicated to perform one large task– as fast as possible. Parallelizing a problem is a main goal.
Sergio Rajsbaum 2006
Many kinds of problems
• Clock synchronization• Routing• Broadcasting• Naming• P2P, how to share and find resources• sharing resources, mutual exclusion• Increasing fault-tolerance, failure detection• Security, authentication, cryptography• Database transactions, atomic commitment• Backups, reliable storage, file systems• Applications, airline reservation, banking, electronic
commerce, publish/subscribe systems, web search, web caching, …
Sergio Rajsbaum 2006
Multi-layered, complex interactionsAn example
• A fault-tolerant broadcast service is useful to build a higher level database transaction module
• Naming, authentication is required• And may work more efficiently if clocks are tightly
synchronized• And good routing schemes should exist• If the clock synchronization is attacked, the whole
system may be compromised
Sergio Rajsbaum 2006
Chaos
We need a good foundation,
principles of distributed computing
Sergio Rajsbaum 2006
Chaos
• Too many models, problems and orthogonal, interacting issues
• Very hard to get things right, to reproduce operating scenarios
• Sometimes it is easy to adapt a solution to a different model, sometimes a small change in the model makes a problem unsolvable
Sergio Rajsbaum 2006
Distributed computing theory• Models
– Good models [Schneider Ch.2 in Distributed Systems, Mullender (Ed.)]
– Relation between models: solve a problem only once; solve it in the strongest possible model
• Problems– Search of paradigms that represent fundamental distributed
computing issues– Relations between problems: hierarchies of solvable and unsolvable
problems; reductions• Solutions
– Design algorithms, verification techniques, programming abstractions
– Impossibility results and lower bounds• Efficiency measures
– Time, communication, failures, recovery time, bottlenecks, congestion
Sergio Rajsbaum 2006
BibliographyTheory of distributed computing textbooks
• Attiya, Welch, Distributed Computing, Wiley-Interscience, 2 ed., 2004
• Garg, Elements of Distributed Computing, Wiley-IEEE, 2002
• Lynch, Distributed Algorithms, Morgan Kaufmann,1997
• Tel, Introduction to Distributed Algorithms, Cambridge U., 2 ed. 2001
Sergio Rajsbaum 2006
Bibliographyothers
• Distributed Algorithms and Systems http://www.md.chalmers.se/~tsigas/DISAS/index.html
• Conferences: DISC, PODC,…
• Journals: Distributed Computing,…– Special issue PODC 20th anniversary, Sept. 2003
• ACM SIGACT News Distributed Computing Column. Also one in EATCS Bulletin
Sergio Rajsbaum 2006