Post on 28-Feb-2021
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Sistemas Operativos Distribuidos
Distributed SystemCommunications
Distributed SystemCommunications
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support2
Distributed Communication
1. Distributed Communication:• Interaction Models• Communication Primitives • Addressing• Failure Management
2. Distributed Development:• Message Passing• RPCs Approaches• Distributed Objects
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support3
Distributed System Communication
Native communication model depends on hardware architecture:– SMPs use shared memory.– Clusters or distributed systems use message passing.
High-level (programming level) communication mechanisms are:– Shared memory areas.– Point-to-point multicast messages– Remote procedure calls or distributed object frameworks.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support4
Shared Memory Communication
Requires:– Uniform or Non-uniform memory access (Hardware).– Operating system functionalities (Software):
• Process management and shared memory allocation.• Thread programming.
– Atomic operations (Hardware/Software):• Mutex and semaphore instructions.
– No special failure management control.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support5
Communication Layers
TCP/UDP
Protocol andRepresentation
RMI/RPCApp./Services
3) Remote procedure callsand distributedobjects.
ATM/Ethernet
Interfaceand
CommunicationLogic
App./Services
2) Low-levelcommunicationfacilities. DistributedOperating Systems.
TCP/UDPAPI (sockets)
Applications&
Services
1) Pure messagepassing. Networkapplications.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support6
Message Passing Communication
Requires:– Network devices (Hardware).– Operating system funcionalities (Software):
• Specific I/O control (network messages).– Network Protocols(Hardware/Software):
• Media access, transport, ...– Special failure management control:
• Partial failures.• Security issues.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support7
Communication Primitives
Elemental communication functions.– send(target,message).– receive(source,message).– connect(target).– close().– ...
Implementation:–An API function. –Network messages.–Status information.
Characteristics:– Blocking vs. Non-blocking. – Synchronous vs. Asynchronous.– Reliable vs. No-reliable.– State vs. stateless
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support8
Blocking vs. Non-blocking
The blocking characteristics are:– Blocking primitives: The operation interrupts caller execution until
the operation is “completed”. – Non-blocking primitives: The operation does not stop the
execution. No finalization status information returned.
Their meaning depends on the primitive function:– Non-blocking send: The sender copies the message into kernel
space and continues its execution.– Non-blocking receive: If there is any data available the receiver
reads it, otherwise the execution continues.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support9
Synchronous vs. Asynchronous
These characteristics represents the interaction of two primitives (send/receive) in a communication:
– Synchronous communication: Send and receive must be executed at the same instant..
– Asynchronous communication: Send primitive does not require the receiver to be waiting.
Synchronous communication uses an intermedia buffer either at kernel level or user level..This function can be used to force independent processes to synchronise.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support10
Reliability
A reliable communication guaranties that a message sent is always received. Otherwise the sender is informed.Reliability uses special acknowledgement messages (ACKs).
Reliability is provided either by:– The communication protocol (TCP yes but UDP no).– The communication logic of the sender and receiver.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Sistemas Operativos Distribuidos11
State and Stateless Services
• State vs. Stateless serves:– Indicates whether the server maintains state information about the clients.
• Sate service benefits:– Shortert request messages.– Better performance (information is retained in memory).– Allows optimization strategies:
• Predictive strategies: Analyzing client’s access patterns. • Stateless service benefits:
– Fault tolerance (server restarting).• Selfcontained requests.
– Less number of messages: There is no session start/finish.– Less resource-demanding on server-side (no memory usage)
• Inherently state-based services (distributed locks).• State management using stateless services (HTTP+cookies).
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support12
Communication Primitives
SENDERSENDER
SenderKernel
REDRECEIVERRECEIVER
ReceiverKernel
12
7
3
64
8 5
msgACK
• Non-blocking send: [1:8] Sender copies the message into the kernel.
• Blocking send: [1:2:7:8] Sender waits for the kernel to send the message through the network.
• Reliable blocking send: [1:2:3:6:7:8]: Sender waits for the opposite kernel to receive the message.
• Explicit blocking send: [1:2:3:4:5:6:7:8]: Idem, but the application (not the kernel) send the acknowledgement.
• Request-reply: [1:2:3:4:<service>:5:6:7:8]: Sender waits for the receiver to complete the operation requested.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support13
Addressing
Information necessary to identify communication elements. How a message receptor is described.
Mechanisms:– Location-dependent address:
• E.g.: IP address + port number.• Provides no transparency.
– Location-independent address (logical address):• Allows transparency.• Requires a logical address translation mechanism:
– Using broadcast messages.– Using a localisation server (which records all element addresses)..
• Address caches improves performance.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support14
Multicast Communication
Multiple message receivers for a single message.– Requires:
• Group membership control.• Efficient message propagation (Multicast routing).
– Applications:• Fault tolerance due to replicated servers.• Discovery of services in a distributed environment.• Better performance in replicated systems.• Software actualisation.• Collective operations in parallel processing.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support15
Message Passing
A prototype of message passing communication is:
CLIENTmsg
Send(msg)Send(msg)msg
SERVER msg
Receive(msg)
Message msg,reply;msg=<data to be sent>send(msg);receive(reply);if(isOK(reply))<operation is OK>
else<operation error>
...
Message op,ack;receive(op);if(validOp(op))ack=OK
elseack=ERROR
send(ack);...
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support16
Representation Formats
Binary-type communication in heterogeneous domain should tackle the following problems:
– Numeric data size and format.– Byte ordering.– Text characters: ASCII vs. EBCDIC.
little-endianArchitecture
Original data: 5
0 0 0 5Value: 0x224+0x216+0x28+5
big-endianArchitecture
Data received: 83.886.080
0 0 0 5Value: 5x224+0x216+0x28+0
0 0 0 5
0123
3210
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support17
Remote Procedure Call
...send(msg)...receive(rpy)
...receive(msg)...send(rpy)
Message passing (low-level interaction)
...
...x=do_it(756)...
int do_it(int cod){......return val;
}
Remote procedure calls (higher level) Programming oriented
Clie
ntServer
Clie
ntServer
¿?
msg
rpy
¿?
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support18
Remote Procedure Call
RPCs evolution:– Proposed by Birrel & Nelson in 1985.– Sun RPC is the foundation of many common nowaday’s services (NFS
or NIS).– A complete RPC development framework was designed in 1990: DCE
(Distributed Computing Environment) by OSF.– Object oriented perspective since CORBA and JavaRMI. Remore
method invocation.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support19
Overall RPC Execution
Client:– The program calls the procedure.– This call packs the arguments of the invocation in a message and
sends them to the other process.– The process waits the response
Server:– A message with packed argument is received.– These arguments are used to call an internal procedure in the
server.– The result of the procedure is packed and sent back to the client.
Objective: To make remote procedure invocation look like conventional procedure invocation.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support20
Necessary Elements
• Client code.• Server code.• Representation format (XDR).• Interface definition.• Server location.• Failure semantics.
...
...x=do_it(756)...
int do_it(int cod){......return val;
}
Clie
nt
Server
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support21
Client Code/Server Code
The abstraction layer elements in a RPC are called stubs.
CLIENT SYSTEM
APPLICATION CODE
PREPAREINPUT
ADAPTOUTPUT
CALLSTART
CALLEND
RPCEXECUTION
LIBRARY
SENDINPUT
RECEIVEOUTPUT
CLIENTSTUB 1
28
9
5
3
PROCEDURES
REMOTEPROCEDUREEXECUTION
SERVER SYSTEM
RECEIVE
SERVERSTUB
PREPAREOUTPUT
SEND BACKOUTPUT
ADAPTINPUT
RPC.EXECUTION
LIBRARY
4
6
7
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support22
Stubs
It is automaticly generated by the RPC software tools:– They are independent of client and server implementation.Only
depend on the interface.Tasks performed by the stubs:
– Server look-up.– Parameter packing and message construction.– Send messages.– Wait for message reception and unpacking the results.
The common issues are implemented in a RPC library.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support23
Representation Format
One of the functions performed by the stubs is packing the arguments: marshalling.
Representation problems:– Client and server might be executed on different hardware
architectures..– XDR (external data representation) an estándar to represent different
data types.– Types of parameters (in/out):
• Pointer parameters: An address has a meaning only inside a specific address space.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support24
Interface Definition
IDL (Interface Definition Language) is alangage to represent anddesribe remote interfaces:
– Many IDL variants:• Integrated with the program language (Cedar, Argus).• Specific for interface defintion (Sun’s RPC and RPC of DCE).
– Define procedures and argument types (Not implementation).– It is the foundation of automatic stub generation.
Server TicketServer{
procedure void reset();procedure int getTicket(in string ident);procedure bool isValid(in int ticket);
}
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support25
Communication Errors
• The server could be down.• The client and server uses different service versions.• How a client is notified of an error:
– Returning an error code (-1)• It is not transparent
– E.g.: add(a,b)– Throwing an exception
• Requires a language that handles exceptions.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support26
Error Semantics
Message from client to server lost
Time-out: If there is no response then replay.
Reliable protocol solves this.
Message from server to client lost
Time-out (from client): How long is this time out? DANGEROUS
Some operations are re-done.
Request number and reply bit.
Client fails
Request submitted but the client dies before reply.
Computation time wasted (server).
Client re-staring could be hazarous.
Server fails
Server dies before or after execute the operations (CLIENT DOESN’T KNOW).
Semantics: At leas once (1+), At most once (0-1), Exactly once (IDEAL).
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support27
Middleware Environments: CORBA
Part I
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support28
Middleware Environments
1. From RPCs to RMIs:• Remote methods• Distributed objects
2. Distributed Objects Frameworks:• OMG’s CORBA• Microsoft’s COM+ / .NET• Java Distributed Computing
3. CORBA:• CORBA architecture/elements• Objects adaptor• Communication protocols
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support29
From RPCs to RMIs
The extension of remote procedure calls to an object-oriented framework develops what is called remote method invocation (RMI).
RMIs provide:– Remote methods are assigned to the concept of distributed objects.– More appropriate to object oriented development.– Allow event-driven development.
New issues:– Remote object reference.– Volatile and persistent objects.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support30
Distributed Objects
Characteristics:– Middleware: Abstraction layer. It provides communication for
distributed objects. Abstraction form:• Object location.• Communication protocols.• Hardware.• Operating system.
– Distributed object model: Represents the elements of the object-oriented paradigm accepted by the technology: Inheritance, Interfaces, Exceptions, Polymorphism, ...
– Garbage Collection: Selects unused objects (following object references) and releases their resources.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support31
Distributed Objects Technologies
– ANSA (1989-1991) first project to propose an object-oriented formalisms to represent distributed systems.
– Microsoft’s DCOM, COM+ and .NET.– OMG’s CORBA.– Java Technologies from Sun Microsytems:
• Remote Method Invocation (RMI).• Enterprise Java Beans (EJB).• Jini.
– Other proprietary technologies.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support32
CORBA
Standard developed by the Object Management Group (OMG)
OMG: A group of companies that work together on the definition of standards for heterogeneous environment interoperability.
– OMG was funded in 1989– At the present moment there are 800+ companies as members of
OMG.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support33
OMA
(Object Management Architecture)
Abstract architecture used as reference to define distributed applications on a heterogeneous domain. CORBA is the technology related to this generic architecture..
CORBA specification is divided in several formal models:– Object Model– Interaction Model– ...
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support34
ORB
Services Common Facilities
Domain Interfaces
Applications
OMA
An application in OMA is a set of distributed collaborative objects. These objects are divided into different groups:
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support35
Servervoid withdraw(long){
....
.... }
Clientx->withdraw(30)
ORB
ORB
The communication between any pair of objects is based on the ORB operations. An elemental operation scenario is:
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support36
CORBA IDL
(Interface Definition Language)This language is used to describe the methods a distributed object offers to other elements of the system.
interface Account{
void deposit(in long amount);void withdraw(in long amount);long balance();
};
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support37
CORBA IDL
Language Mappings:– Translate an IDL description into another programming language:
• C++, • Ada, • COBOL, • SmallTalk, • Java,• ...
– This translation generates two code pieces called Stub and Skeleton. These code modules represent client and server communicationissues.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support38
CORBA IDL
The client code (stub) inludesmarshalling operations.
Marshalling: Translation of the methodparameters into a commonrepresentation forORBcommunication.
interface Account
{
void withdraw(in long amount);
}
Account.idl
IDL Compiler
Account_Skel.c++
class Account_Stub : ...
{
void withdraw(CORBA::Long &amount)
{ <MARSHALLING> }
}
Account_Stub.c++
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support39
CORBA IDL
The IDL-generated server code(skeleton) performs the oppositeoperation (de-marshalling).
De-marshalling: Unpaking callerparameters from the message andbuilding back the procedure call.
IDL Compiler
Account_Stub.c++
class Account_Skel : ...
{
virtual
void withdraw(CORBA::Long &account)=0;
void __withdraw()
{
<DE-MARSHALLING>
withdraw(c);
}
}
Account_Skel.c++
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support40
CORBA IDL
Object Implementation:– The actual object implementation is summoned by the Skeleton
methods. It is used to be a derived class of this skeleton.
class Account_Impl: public Account_Skel{
void withdraw(CORBA::Long &amount){
money += amount;}
};
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support41
ORB ORB
DII Stub ORBInterface
ORBInterface
Skel. DSI
Object Adapter (OA)
Client Server Object
ORB Components
The complete communication architecture in CORBA has the following elements:
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support42
ORB Components
Stub:– Client code that represents the remote object to be called. It emulates
the remote code methods invoked by the client. Stubs perform communication functions instead of the actual method calls.
Skeleton:– Server code of the object. It has the opposite role of the stub. It
emulates client actions for remote method calls. It performs local method calls to the actual object implementation.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support43
ORB Components
DII:– (Dynamic Invocation Interface)– It is an alternative to static stub usage that allows clients to request
server operations that were unknown during compilation phase.
DSI:– (Dynamic Skeleton Interface)– It is a dynamic alternative to static skeletons when they are defined
during compilation phase. This feature is useful for server thatincludes new services during execution phase.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support44
ORB Components
ORB and ORB Interface:– This component deals with interconnection issues between client and
server spaces (& many other functions). This element hides different PS and hardware differences.
– There are the same features provided for both client and server sides.The two main ORB tasks are:
– Object location: The client does not know where remote object computer is.
– Communication between client and server: Independently of protocol and implementation issues (language, operating system, ...)
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support45
ORB Component
Object Adapter:– Remote object use this element to be registered in, This element
keep all the object references inside the same service space (samenode). The object adapter is able to redirect remote methodinvocations from clients to the appropriate skeletons.
There are two object adapters specified by OMG:– BOA: (Basic Object Adapter).– POA: (Portable Object Adapter).
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support46
ORB Components
The main object adapter tasks are:– Two-fold multiplexing (per object and per method). – To keep remote object implementation information (stored into an
Implementation Repository). This element is in charge of the object activation (if it isn’t active).
– To provide different activation modes:• Persistent: Object status is kept between two executions.• Shared: All the clients share the same object.• Un-shared: Each client has its own object implementation.• Per-method: Each method call activates a new object.
– To generate object references inside the environment. This object reference is unique.
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support47
ORB ORB
DII Stub ORBInterface
ORBInterface
Skel. DSI
Object Adapter (OA)
Client Server Object
CORBA Communication
Communication mechanism:1- Client calls stub method. Client stub marshals the parameters. 2- Client stub uses ORB sending to addressed the request to the remote object.3- Server ORB receives the request and forwards it to assigned Object Adapter (it
could be more than one).
1
2 3
4
5
6
7
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support48
CORBA Communication4- The Object Adapter solves which object is referred to and which of its method is
requested.5- Server skeleton de-marshals request arguments and calls the actual object.6- Object implementation performs the operation and the result (and any out/in-out
parameter) are sent back to the skeleton.7- Result reply is analogous.
ORB ORB
DII Stub ORBInterface
ORBInterface
Skel. DSI
Object Adapter (OA)
Client Server Object
1
2 3
4
5
6
7
Fernando PérezJosé María Peña
Víctor RoblesFrancisco Rosales
Operating System Support49
ORB Implementation
ORB is a logical bus that glues objects and clients. The implementation options for this element are:
– Inside client/server: Additional code that both client and server link.– System daemon: A system service that centralised the operations.– Inside OS: As a part of the OS kernel.– Library: When client and server shares the same memory space
(same computer and process).