Post on 05-Jan-2016
3. Communication in distributed system
3.1 Introduction
Interprocess communication is at the heart of distributed systems
Distributed Application
Communication middleware
Low-level message passing underlying network
IP addressport, ...
Name, ...
Middleware Protocols
An adapted reference model for networked communication.
2-5
P69
Distributed Middleware
• Communication middleware: RPC, RMI, MPI– name DCE, CORBA
– message-oriented– various communication forms
• Authentication, authorization protocols• Distributed commit protocols
Client-Server TCP
Normal operation of TCP. Transactional TCP.
2-4
P65
Two issues for communication mechanism:• How indicate destination?• How many methods (forms)?
S
TQ
P
Indicate sending and receiving processes
1) <IP address, port no.> ex, <202.119.32.6, 80>
2) Unique name
ex, www.nju.edu.cn; procedure name( )
S
TQ
P
1. Addressing
…...
Domain name IP address Type
www.nju.edu.cn 202.119.32.6 Acs.nju.edu.cn 202.119.36.5 A …… …...
DNS
FTPd webserver
ftp://nju.edu.cnwww.nju.edu.cn
202.119.32.6
80 21
<202.119.32.6, 21><202.119.32.6, 80>
3) Mailbox (queue, indirect)
multi-in, multi-out
2. Methods 1) Synchronous/asynchronous
sender
sender
receiver
receivermailbox
P1 P2
send
receivewaiting
receivewaiting
send
Synchronous: sender will be blocked if receiver is not ready
send( ), recv( ); RPC
P1 P2
send
send
receive
receive
Internet?
Asynchronous UDP, Asynchronous RPC
2) Transient/persistent• Persistent communication:
A message is stored by communication system as long as it takes to deliver it. Neither the sender nor the receiver need to be up.
Ex, email
Synchronous wait, no broadcast safe, simple
Asynchronous buffer full,not safe parallelism, broadcast
Defects merits
Comparison
P100-1
Persistence in Communication P100
• Transient communication:
A message is stored by communication system as long as the sending and receiving applications are executing
Ex, send, recv in Socket
Question: difference (relation) between 1) and 2)
transient synchronous
persistent asynchronous
As to: 1) sender/receiver; 2) message
send, recv; RPC
UDP; asyn RPC
Two successive messages have a temporal
relationship.
Ex, movie, audio stream
3) Stream-oriented communication
P1 P2 Proc G(u,v)
Call P2.G(x,y)
node 1 node 2
……
…..
3.2 RPC P68
2003.7 Worm.msBlast
1. Conventional procedure call
Node 1
process Procedure Q(u,box,v)
{ … { …
Q(i,buf,j) …
…
} }
return addr.i
bufj
sp
stack
result
2. How does RPC work?
node 1 node 2
process Procedure Q(u,box,v)
{ … { …
Q(i,buf,j) …
… }
}
?
stack stack
return
sp
return addr 2i
bufjsp
result
return addr.1
bufj
i
proc Qint i ? bufint j resultresult
Original procedure call: Now
push return addr.1
push i
Q(i,buf,j) push buf
push j
goto Q goto c_stub
take result
C_stub
• pack parameters & name• send them to S_stub• receive result• unpack result• return
S_stub
• receive message• unpack message• push return addr.• goto Q( )• take & pack result• send it to C_stub
Q( ){ }
transform
Steps of a Remote Procedure Call
• Client procedure calls client stub in normal way• Client stub builds message, calls local OS• Client's OS sends message to remote OS• Remote OS gives message to server stub• Server stub unpacks parameters, calls server• Server does work, returns result to the stub• Server stub packs it in message, calls local OS• Server's OS sends message to client's OS• Client's OS gives message to client stub• Stub unpacks result, returns to client
P72
Passing Value Parameters
Steps involved in doing remote computation through RPC
2-8
P74
Issues to be handledMachine may be not identicalExecute in different spacesMachine may crash
3. Parameter passingImplementation should consider:
Different types of computers Pointer or call by reference
1) Deal with different types of computers
5 Jill
(a) Original message on the Pentium(b) The message after receipt on the SPARC(c) The message after being inverted. The little numbers in boxes indicate the address of each byte
P75
How represent parameters? – In intermediate form– Indicating in type i
typei typej
Message should include parameters and their type
2) Deal with call by reference
proc Qint i
char bufint j
Node1 Node2Main( )
{ int B[100] proc write(name,buf,len)
… { int len, buf[ ];
write(f1,B,100) …
… }
proc read(name, buf, len)
{ int len, buf[ ];
read(f2,B,100) …
}Write
f1B=5200
100 message
B5200
5200
Writefile f1
Array 100B[1]B[2]……
B[100]int 100
message
Readfile f2
Array 100int 100
message
read
write
proc write(in char name, in int buf[], in int len);
proc read(in char name, out int buf[], in int len);
– Input parameter deliver the array– Output parameter needn’t
4. Where stubs come from?Write server specification (interfaces) including:
Server name, version, list of procs
Stub compiler generates c_stub & s_stub
Specification of file server
Specification of file_server, version 3.1:
long read(in char name[MAX_PATH], out char buf[BUF_SIZE], in long bytes, in lon
g position);
long write(in char name[MAX_PATH], in char buf[BUF_SIZE], in long bytes, in lon
g position);
int create(in char name[MAX_PATH], in int mode);
int delete(in char name[MAX_PATH]);
end; returnIDL
5. Steps for using RPC• write specification• compile• write server code and link with s_stub• write client code and link with c_stub
specificationC_stub
S_stub Stubcompiler
The steps in writing a client and a server in DCE RPC.
P82
user.c file_server.c
proc read( )
read( ) proc write( )
c_stub s_stub
user.exe file_server.exe
read( )
{…… }
{…… }
6. Dynamic binding --- How client find server
1) Register: name, version, handle(IP,port), unique id
2) Look up: name, version (first time)
3) Reply: handle, unique id
4) Request: unique id, parameters
5) Deregister: name, version, unique id
binder
client server
12
3
4
• Advantage: flexible– Server move won’t affect client
– Binder can even load if multiple servers support the same interface
• Disadvantage– Overhead at running time
– Binder is bottleneck
Addressing: name
Method: synchronous, transient
RPC
NFS
XDR
Socket/TLITCP|UDP
IPARP
Driver
ICMP
FTPTelnet
AddressingMethod
7. Extended RPC --- asynchronous RPC
(a) The interconnection between client and server in a traditional RPC
(b) The interaction using asynchronous RPC
A B100 P79
A client and server interacting through two asynchronous RPCs
2-13
A client wants to prefetch network addresses of a set of hosts
P80
3.3 RMI (or Remote Object Invocation)
Common organization of a remote object with client-side proxy.
P86
1. Distributed object
RMI Compares to RPC:• Same:
– the steps of call is similar– separating interface from implementation
• Difference– procedure is a segment of program.– object is a variable of a data type, method is in
object.
Object reference compare to pointer. System provides
2. Object references Object references are similar to pointer, but not the same.
object
object reference
remote
point
local object
distributed obj
…
employee
promote() {…}dismiss() {… }
hire() {… }
Corp server
Sale server
salesman
bonus() {… }
advertise
cost() {… }
personnel
1. IP address
2.3.
?
Object servers
P152
personnel.hire( )
Information object reference contains
• network address of the machine • server id or endpoint (i.e.port number)• object id
P89
• May use location server
locationserver
client server
12
3
4
Server can move
• Server id instead of server’s endpoint. A daemon listens to a well-known endpoint and knows every server.
3. Implementation of object references
server1
server2keep a table
daemon
4. Several remote object invocation
1) CORBA
Open, heterogeneous environment
2) DCOM
Microsoft world
Object Object
C++ JavaCfunc P( )
How introduce object?
chapter 9
3) Java RMI
• Java environment • Distributed objects have been integrated into the
language
Summary
Addressing: name
Method: synchronous, transient
3.4 Message-oriented communication
1. Message-oriented transient communication 1) RPC and RMI
advantages: hide communication; message-orienteddisadvantage: synchronous; both need to be up
2) Recall socket P105
addressing: IP address, portforms: synchronous/asynchronous, transientshortage: low-level, byte-oriented/TCP-IP
P99
Process0
Process1
Process5Process4
Process3
Process2
Process1
Process0
Process2
3) MPI P107
JupiterWorld
designed for parallel applications; offer efficient
communication primitives; for high-speed nets.
communicator
• Addressing (groupID, processID)
• Forms support most transient communication forms
? A B; email
Need for various communication forms• Synchronous/asynchronous• Transient/persistent
100
Various combinations: P103
Why list so many forms? Where to use?
For efficiency.
(a) email
(c) A B
(e) send recv
(f) RPC
100
(persistent, asynchronous)
(transient, asynchronous)
(transient, synchronous)
(transient, synchronous/asyn)
2. Message-oriented persistent communication
1) Message-queuing model
Addressing: name of destination queue
Method: persistent, asynchronous
2) Basic idea• Application inserts messages in specific queue • the messages are forwarded over a series of
communication servers• eventually the messages are delivered to destination
P109
Message-Queuing System
The general organization of a message-queuing system with routers.
2-29
P112
Four combinations for loosely-coupled communications using queues.
2-26
P110
3) Basic interface to a queue
Install a handler to be called when a message is put into the specified queue.
Notify
Check a specified queue for messages, and remove the first. Never block.Poll
Block until the specified queue is nonempty, and remove the first messageGet
Append a message to a specified queuePut
MeaningPrimitive
callback
P110
4) Implementation
All machines in message-queuing system should have a queue manager which manages:
• queues: source queue, destination queue (name)• transmission of messages from S to D• mapping queues to network locations
Two kinds of implementations
a) Create its own forwarding servers (routers), like multicast gateway + Mbone
b) with the aid of Internet routers, like email
a) Architecture of IBM MQSeries
For IBM mainframes, access large scale database
Addressing: (destQM, destQ),
Example: IBM MQSeries
General organization of IBM's MQSeries message-queuing system.
2-31
P116
Message Transfer
The general organization of an MQSeries queuing network using routing tables and aliases.
(QA,SQ1)
Why do message-queue systems send messages to
queue, instead of directly to receivers?
Some applications need this kind of communication
form. For example, tasks are sent to a set of processes
who perform these tasks.
Two ways: • a dispatcher + workers• workers
P143
3.5 Stream-oriented communication
1. Requirements• Real life
– audio stream (simple stream)– movie (complex stream)
• Features– time-dependent information.
– data stream
Stream media?
P119
Addressing: <IP address, port> or
destination name
Method: continuous data stream
• Requirement for transmission:– timing for continuous data stream– synchronization for complex stream
Ex, – Samples in audio stream are played at intervals
of 1/44100 sec.– Video and audio sub-streams need synchronize.
Data Stream
Setting up a stream between two processes across a network.
P122
MP3
An example of multicasting a stream to several receivers.
Differentiated services
4. Stream Synchronization
• Meaning
Maintain temporal relation between streams which may be continuous or discrete data stream.
• Essence
Synchronize at the level of data units.
What is a data unit?
a single audio stream a series of samples
a video stream a series of frames
Data unit
Ex1, synchronization for stereo audio stream takesplace every 1/44100 sec
a data unit = a sample
Ex2, what about synchronization between an audio stream and a video stream?
video frame 30Hz
audio sample 44100Hz
sample/frame 44100/30=1470
an audio data unit = 1470 samples
1/30 sec
• Synchronization mechanism
A process executes read (write) operations on several simple streams, ensuring timing and synchronization.
Ex, alternating between reading an image and reading a block of audio samples every 33ms
sender receiver
audio streamvideo stream
• Does sender send two separate streams or a complex stream?
A complex stream. • Where synchronization should take place, sending
or receiving side?
Sender: multiplex different streams into a single stream containing all data units, i
ncluding those for synchronization(timestamp)
Receiver: demultiplexes the stream, using the timestamps of each packet
Synchronization Mechanisms (2)
The principle of synchronization as supported by high-level interfaces.
2-41
Offer interface to
specify the rates
Summary
1. Key issues
for designing communication mechanism• Addressing: how indicate sender and receiver,
LAN, WAN• Methods : synchronous/asynchronous
persistent/transient
stream-oriented communication• Middleware
Examples?
• How does it work?
• Issues to be handle: type, reference (object?)
• IDL: is used to write server’ specification C_stub
• Steps for using RPC S_stub
• Binder
3. RMI• Difference between RPC and RMI
• Object reference and its features
2. RPC
– address, serverID, objectID• CORBA etc
4. Message-oriented communication• Why need it?• Transient communication: MPI• Persistent communication: Message-queuing system
• Requirement for transmission• Features• Synchronization
5. Stream-oriented communication