Forestry in India A Story of Management Failure Presented by Smruti(42) Nirbhay(29)
Advanced Distributed Systems - ERNETsrsarangi/csl860/docs/intro.pdf · Message-Oriented...
Transcript of Advanced Distributed Systems - ERNETsrsarangi/csl860/docs/intro.pdf · Message-Oriented...
PlanIntroduction
Building a Distributed System
Advanced Distributed SystemsCourse Plan and Overview
Smruti R. Sarangi
Department of Computer ScienceIndian Institute of Technology
New Delhi, India
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
Lectures - I
Lecture Topic1 Introduction and Structure2 Communication – Client Server, RPC
Message Queue, Stream3 Multicast Communication – Epidemic, Gossip4 Naming : DNS, Chord5 Naming : Pastry, Tapestry, LDAP6 Physical Clock , Logical Clock, Totally ordered multicast7 Mutual Exclusion Algorithms8 Leader Election Algorithms9 Consistency - I10 Consistency - II11 Fault Tolerance – Introduction, Log based recovery12 Byzantine Fault Tolerance
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
Lectures - II
Lecture Topic13 Paxos14 Distributed Commit15 Security – Channels, Concepts16 Security Applications – Kerberos
Diffie Helmann Key Exchange17 Corba, EJB18 AFS and NFS19 Akamai and Corona20 Dynamo and Voldemort21 Coda, Fawn, and Google FS22 Web Services23 Dryad-LinQ
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
Grading Scheme
Component WeightageAttendance 15%Midterm 15%End term (take home) 20%Programming Assignment 1 15%Programming Assignment 2 15%Programming Assignment 3 20%
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Distributed System: Definition
A distributed system is a piece of software that ensures that:
a collection of independent computers appears to itsusers as a single coherent system
Two aspects:1 independent computers2 single coherent system ⇒ middleware .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Goals of Distributed Systems
GoalsMaking resources availableDistribution transparencyOpennessScalability
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Distribution Transparency
Transp.Description
Access Hides differences in data representation and invocationmechanisms
Location Hides where an object residesMigration Hides from an object the ability of a system to change that
object’s locationRelocation Hides from a client the ability of a system to change the
location of an object to which the client is boundReplication Hides the fact that an object or its state may be replicated
and that replicas reside at different locationsConcurrency Hides the coordination of activities between objects to
achieve consistency at a higher levelFailure Hides failure and possible recovery of objects
NoteDistribution transparency is a nice a goal, but achieving it is a different story.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Distribution Transparency
Transp.Description
Access Hides differences in data representation and invocationmechanisms
Location Hides where an object residesMigration Hides from an object the ability of a system to change that
object’s locationRelocation Hides from a client the ability of a system to change the
location of an object to which the client is boundReplication Hides the fact that an object or its state may be replicated
and that replicas reside at different locationsConcurrency Hides the coordination of activities between objects to
achieve consistency at a higher levelFailure Hides failure and possible recovery of objects
NoteDistribution transparency is a nice a goal, but achieving it is a different story.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Degree of Transparency
Aiming at full distribution transparency may be too much
Users may be located in different continents
Completely hiding failures of networks and nodes is (theoreticallyand practically) impossible
You cannot distinguish a slow computer from a failing oneYou can never be sure that a server actually performed anoperation before a crash
Full transparency will cost performance , exposing distribution ofthe system
Keeping Web caches exactly up-to-date with the masterImmediately flushing write operations to disk for fault toler-ance
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Degree of Transparency
Aiming at full distribution transparency may be too much
Users may be located in different continents
Completely hiding failures of networks and nodes is (theoreticallyand practically) impossible
You cannot distinguish a slow computer from a failing oneYou can never be sure that a server actually performed anoperation before a crash
Full transparency will cost performance , exposing distribution ofthe system
Keeping Web caches exactly up-to-date with the masterImmediately flushing write operations to disk for fault toler-ance
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Degree of Transparency
Aiming at full distribution transparency may be too much
Users may be located in different continents
Completely hiding failures of networks and nodes is (theoreticallyand practically) impossible
You cannot distinguish a slow computer from a failing oneYou can never be sure that a server actually performed anoperation before a crash
Full transparency will cost performance , exposing distribution ofthe system
Keeping Web caches exactly up-to-date with the masterImmediately flushing write operations to disk for fault toler-ance
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Degree of Transparency
Aiming at full distribution transparency may be too much
Users may be located in different continents
Completely hiding failures of networks and nodes is (theoreticallyand practically) impossible
You cannot distinguish a slow computer from a failing oneYou can never be sure that a server actually performed anoperation before a crash
Full transparency will cost performance , exposing distribution ofthe system
Keeping Web caches exactly up-to-date with the masterImmediately flushing write operations to disk for fault toler-ance
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Openness of Distributed Systems
Open distributed system ⇒ Be able to interact with servicesfrom other open systems, irrespective of the underlying en-vironment:
Systems should conform to well-defined interfacesSystems should support portability of applicationsSystems should easily interoperate
Achieving openness ⇒ At least make the distributed systemindependent from heterogeneity of the underlying environ-ment:
HardwarePlatformsLanguages
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Openness of Distributed Systems
Open distributed system ⇒ Be able to interact with servicesfrom other open systems, irrespective of the underlying en-vironment:
Systems should conform to well-defined interfacesSystems should support portability of applicationsSystems should easily interoperate
Achieving openness ⇒ At least make the distributed systemindependent from heterogeneity of the underlying environ-ment:
HardwarePlatformsLanguages
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scale in Distributed Systems
ObservationMany developers of modern distributed system easily use the adjective“scalable” without making clear why their system actually scales.
Scalability – At least three components:
Number of users and/or processes (size scalability)
Maximum distance between nodes (geographical scalability)
Number of administrative domains (administrative scalability)
ObservationMost systems account only, to a certain extent, for size scalability. The(non)solution: powerful servers. Today, the challenge lies in geograph-ical and administrative scalability.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scale in Distributed Systems
ObservationMany developers of modern distributed system easily use the adjective“scalable” without making clear why their system actually scales.
Scalability – At least three components:
Number of users and/or processes (size scalability)
Maximum distance between nodes (geographical scalability)
Number of administrative domains (administrative scalability)
ObservationMost systems account only, to a certain extent, for size scalability. The(non)solution: powerful servers. Today, the challenge lies in geograph-ical and administrative scalability.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scale in Distributed Systems
ObservationMany developers of modern distributed system easily use the adjective“scalable” without making clear why their system actually scales.
Scalability – At least three components:
Number of users and/or processes (size scalability)
Maximum distance between nodes (geographical scalability)
Number of administrative domains (administrative scalability)
ObservationMost systems account only, to a certain extent, for size scalability. The(non)solution: powerful servers. Today, the challenge lies in geograph-ical and administrative scalability.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Techniques for Scaling
Hide communication latencies – Avoid waiting for responses;do something else:
Make use of asynchronous communicationHave separate handler for incoming responseProblem: not every application fits this model
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Techniques for Scaling
DistributionPartition data and computations across multiple machines:
Move computations to clients (Java applets)Decentralized naming services (DNS)Decentralized information systems (WWW)
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Techniques for Scaling
Replication/cachingMake copies of data available at different machines:
Replicated file servers and databasesMirrored Web sitesWeb caches (in browsers and proxies)File caching (at server and client)
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scaling – The Problem
ObservationApplying scaling techniques is easy, except for one thing:
Having multiple copies (cached or replicated), leads to in-consistencies : modifying one copy makes that copy differ-ent from the rest.Always keeping copies consistent and in a general way re-quires global synchronization on each modification.Global synchronization precludes large-scale solutions.
ObservationIf we can tolerate inconsistencies, we may reduce the need forglobal synchronization, but tolerating inconsistencies is applica-tion dependent .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scaling – The Problem
ObservationApplying scaling techniques is easy, except for one thing:
Having multiple copies (cached or replicated), leads to in-consistencies : modifying one copy makes that copy differ-ent from the rest.
Always keeping copies consistent and in a general way re-quires global synchronization on each modification.Global synchronization precludes large-scale solutions.
ObservationIf we can tolerate inconsistencies, we may reduce the need forglobal synchronization, but tolerating inconsistencies is applica-tion dependent .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scaling – The Problem
ObservationApplying scaling techniques is easy, except for one thing:
Having multiple copies (cached or replicated), leads to in-consistencies : modifying one copy makes that copy differ-ent from the rest.Always keeping copies consistent and in a general way re-quires global synchronization on each modification.
Global synchronization precludes large-scale solutions.
ObservationIf we can tolerate inconsistencies, we may reduce the need forglobal synchronization, but tolerating inconsistencies is applica-tion dependent .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scaling – The Problem
ObservationApplying scaling techniques is easy, except for one thing:
Having multiple copies (cached or replicated), leads to in-consistencies : modifying one copy makes that copy differ-ent from the rest.Always keeping copies consistent and in a general way re-quires global synchronization on each modification.Global synchronization precludes large-scale solutions.
ObservationIf we can tolerate inconsistencies, we may reduce the need forglobal synchronization, but tolerating inconsistencies is applica-tion dependent .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Scaling – The Problem
ObservationApplying scaling techniques is easy, except for one thing:
Having multiple copies (cached or replicated), leads to in-consistencies : modifying one copy makes that copy differ-ent from the rest.Always keeping copies consistent and in a general way re-quires global synchronization on each modification.Global synchronization precludes large-scale solutions.
ObservationIf we can tolerate inconsistencies, we may reduce the need forglobal synchronization, but tolerating inconsistencies is applica-tion dependent .
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Types of Distributed Systems : Structure
Centralized Layered
Unstructured Peer to Peer Structured Peer to Peer
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Types of Distributed Systems : Synchronous vsAsynchronous
DefinitionSynchronous System The requester waits for the response be-fore placing other requests.
DefinitionAsynchronous System The requester does not wait for the re-sponse before placing other requests. One example of suchsystems are publish/subscribe systems. A given set of nodessubscribe for a service. When a node is ready to publish somedata, it looks up the list of subscribers and sends them mes-sages.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliable
The network is secureThe network is homogeneousThe topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secure
The network is homogeneousThe topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneous
The topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not change
Latency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not changeLatency is zero
Bandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not changeLatency is zeroBandwidth is infinite
Transport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zero
There is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
OverviewDeveloping Distributed Systems: Pitfalls
Developing Distributed Systems: Pitfalls
ObservationMany distributed systems are needlessly complex caused bymistakes that required patching later on. There are many falseassumptions :
The network is reliableThe network is secureThe network is homogeneousThe topology does not changeLatency is zeroBandwidth is infiniteTransport cost is zeroThere is one administrator
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Computation and Communication
Components of a distributed SystemNodes that run the distributed software. ( Computation )Communication network. ( Communication )
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Computing Nodes
A compute node can either be a process or a thread
ThreadA thread is a light weight process that shares the address spacewith other threads.
ProcessA process is the runtime image of a program. It does not shareits address space.
We can use regular memory reads and writes to commu-nicate across threads. However, inter-process communica-tion requires OS intervention .In practice, there are thousands of threads and processesdistributed across hundreds of sites.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Computing Nodes - II
Generic solution: Message passing across threads/processes.Minimize shared data.Fetch shared data through messages from a database.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
How do Nodes Talk to Each Other?
Distributed systems predominantly use application layer pro-tocols.This protocol layer is known as middleware .They use standard sockets to send messages to other nodes.Paradigms for sending messages
1 Synchronous vs Asynchronous2 Transient vs Persistent
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Middleware Layer
ObservationMiddleware is invented to provide common services and proto-cols that can be used by many different applications
A rich set of communication protocols(Un)marshaling of data, necessary for integrated systemsNaming protocols , to allow easy sharing of resourcesSecurity protocols for secure communicationScaling mechanisms , such as for replication and caching
NoteWhat remains are truly application-specific protocols...such as?
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Basic RPC operation
Application developers are familiar with a simple procedure modelWell-engineered procedures operate in isolation (black box)There is no fundamental reason not to execute procedures on sep-arate machines
Call local procedureand return results
Call remoteprocedure
Returnfrom call
Client
Request Reply
ServerTime
Wait for result
ConclusionCommunication between caller & callee can be hidden by usingprocedure-call mechanism.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Basic RPC operation
Implementationof add
Client OS Server OS
Client machine Server machine
Client stub
Client process Server process1. Client call to
procedure
2. Stub buildsmessage
5. Stub unpacksmessage
6. Stub makeslocal call to "add"
3. Message is sentacross the network
4. Server OShands messageto server stub
Server stubk = add(i,j) k = add(i,j)
proc: "add"int: val(i)int: val(j)
proc: "add"int: val(i)int: val(j)
proc: "add"int: val(i)int: val(j)
1 Client procedure calls client stub.2 Stub builds message; calls local OS.3 OS sends message to remote OS.4 Remote OS gives message to stub.5 Stub unpacks parameters and calls
server.
6 Server returns result to stub.7 Stub builds message; calls OS.8 OS sends message to client’s OS.9 Client’s OS gives message to stub.
10 Client stub unpacks result and returnsto the client.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
RPC: Parameter passing
Parameter marshalingThere’s more than just wrapping parameters into a message:
Client and server machines may have different data repre-sentations (think of byte ordering)Wrapping a parameter means transforming a value into asequence of bytesClient and server have to agree on the same encoding :
How are basic data values represented (integers, floats,characters)How are complex data values represented (arrays, unions)
Client and server need to properly interpret messages ,transforming them into machine-dependent representations.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
RPC: Parameter passing
RPC parameter passing: some assumptions
Copy in/copy out semantics: while procedure is executed, noth-ing can be assumed about parameter values.
All data that is to be operated on is passed by parameters. Ex-cludes passing references to (global) data.
ConclusionFull access transparency cannot be realized.
A remote reference mechanism enhances access transparency:
Remote reference offers unified access to remote data
Remote references can be passed as parameter in RPCs
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Asynchronous RPCs
EssenceTry to get rid of the strict request-reply behavior, but let the clientcontinue without waiting for an answer from the server.
Call local procedure
Call remoteprocedure
Returnfrom call
Request Accept request
Wait for acceptance
Call local procedureand return results
Call remoteprocedure
Returnfrom call
Client Client
Request Reply
Server ServerTime Time
Wait for result
(a) (b)
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Deferred synchronous RPCs
Call local procedure
Call remoteprocedure
Returnfrom call
Client
RequestAcceptrequest
ServerTime
Wait foracceptance
Interrupt client
Returnresults Acknowledge
Call client withone-way RPC
VariationClient can also do a (non)blocking poll at the server to seewhether results are available.
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
RPC in practice
C compiler
Uuidgen
IDL compiler
C compiler C compiler
Linker Linker
C compiler
Server stubobject file
Serverobject file
Runtimelibrary
Serverbinary
Clientbinary
Runtimelibrary
Client stubobject file
Clientobject file
Client stubClient code Header Server stub
Interfacedefinition file
Server code
#include#include
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Client-to-server binding (DCE)
Issues(1) Client must locate server machine, and (2) locate the server.
Endpointtable
Server
DCEdaemon
Client1. Register endpoint
2. Register service3. Look up server
4. Ask for endpoint
5. Do RPC
Directoryserver
Server machineClient machine
Directory machine
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Outline
1 Plan
2 IntroductionOverviewDeveloping Distributed Systems: Pitfalls
3 Building a Distributed SystemComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Message-Oriented Communication
Transient MessagingMessage-Queuing SystemMessage BrokersExample: IBM WebSphere
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Transient messaging: sockets
Berkeley socket interface
SOCKET Create a new communication endpointBIND Attach a local address to a socketLISTEN Announce willingness to accept N connectionsACCEPT Block until request to establish a connectionCONNECT Attempt to establish a connectionSEND Send data over a connectionRECEIVE Receive data over a connectionCLOSE Release the connection
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Transient messaging: sockets
connect
socket
socket
bind listen read
read
write
write
accept close
close
Server
Client
Synchronization point Communication
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Message-oriented middleware
EssenceAsynchronous persistent communication through support ofmiddleware-level queues. Queues correspond to buffers at com-munication servers.
PUT Append a message to a specified queueGET Block until the specified queue is nonempty, and re-
move the first messagePOLL Check a specified queue for messages, and remove
the first. Never blockNOTIFY Install a handler to be called when a message is put
into the specified queue
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Message broker
ObservationMessage queuing systems assume a common messaging pro-tocol : all applications agree on message format (i.e., structureand data representation)
Message broker: Centralized component that takes care ofapplication heterogeneity in an MQ system
Transforms incoming messages to target formatVery often acts as an application gatewayMay provide subject-based routing capabilities ⇒ Enter-prise Application Integration
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
Message broker
Queuinglayer
Brokerprogram
Repository with conversion rules and programsSource client Destination client
OS OSOS
Message broker
Network
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
IBM’s WebSphere MQ
Application-specific messages are put into, and removedfrom queuesQueues reside under the regime of a queue managerProcesses can put messages only in local queues, or throughan RPC mechanism
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
IBM’s WebSphere MQ
Message TransferMessages are transferred between queuesMessage transfer between queues at different processes,requires a channelAt each endpoint of channel is a message channel agentMessage channel agents are responsible for:
Setting up channels using lower-level network communica-tion facilities (e.g., TCP/IP)(Un)wrapping messages from/in transport-level packetsSending/receiving packets
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
IBM’s WebSphere MQ
MCA MCAMCA MCA
MQ Interface
Stub StubServerstub
Serverstub
Send queue
Program ProgramQueuemanager
Queuemanager
Routing table
Enterprise networkRPC(synchronous)
Local network
Message passing(asynchronous)
To other remotequeue managers
Client's receivequeueSending client Receiving client
Channels are inherently unidirectionalAutomatically start MCAs when messages arriveAny network of queue managers can be createdRoutes are set up manually (system administration)
Smruti R. Sarangi Advanced Distributed Systems
PlanIntroduction
Building a Distributed System
ComputationCommunicationRemote Procedure CallsMessage-Oriented Communication
IBM’s WebSphere MQ
Routing
By using logical names, in combination with name resolution to localqueues, it is possible to put a message in a remote queue
SQ1SQ2
SQ1
SQ1SQ1SQ2
QMBQMCQMD
SQ1
SQ1SQ1
SQ1
SQ2SQ1
SQ1
SQ1SQ1
QMA
QMA QMA
QMC
QMCQMB
QMD
QMBQMD
Routing tableRouting table
Routing table Routing table
QMB
QMC
QMA
LA1LA1
LA1
LA2LA2
LA2
QMCQMA
QMA
QMDQMD
QMC
Alias tableAlias table
Alias table
QMD
SQ1
SQ2
SQ1
Smruti R. Sarangi Advanced Distributed Systems