Topics in Distributed Systems CS350, CS550, CS580F.
-
Upload
anna-willis -
Category
Documents
-
view
241 -
download
0
description
Transcript of Topics in Distributed Systems CS350, CS550, CS580F.
Topics in Distributed Systems
CS350, CS550, CS580F
©2010 D. J. Foreman2
Topics
1. Types of Distributed systems2. Communications3. Naming4. Synchronization5. Consistency & Replication6. Fault Tolerance7. Security
©2010 D. J. Foreman3
Introduction
©2010 D. J. Foreman4
Definition of a Distributed System
A collection of functions implemented across a set of independent devices that appears to its users as a single coherent system.
Horizontal - all devices are peers Vertical - there is a control hierarchy
Note: Devices may be computers or ….
Types of Distributed systems
Client-Server Peer-peer Processor Pool Network Operating Systems
©2010 D. J. Foreman5
©2010 D. J. Foreman6
Examples of Distributed Systems
An "ATM machine" A remote file access mechanism A database A chat room A computing "grid" (like SETI)
©2010 D. J. Foreman7
Design
All “usually” have: Multi-threaded/Multi-process operation User interface (need not be graphical) Point(s) of control (any system shuts-down all)
©2010 D. J. Foreman8
Some Problems (topics)
System Character codes (ASCII/EBCDIC) Number representation (big- vs. little-Endian)
Size of an "int" (16/32/64 - bit) Float
Shared Memory Operational Compatibility
Thread control System services Process handling
Pointers & References Objects
©2010 D. J. Foreman9
Hardware Systems
Multiprocessors (modularized shared memory) Bus traffic cache incoherence Limited scalability (network cost/speed) NUMA – Non-Uniform Memory Access
(some local, some shared via network)
Homogeneous Multicomputers Private memory + switching circuits Bus-based connection: ethernet routing Switch-based: routing via h/w net (mesh, n-cube)
e.g., Clusters of Workstations
Heterogeneous Multicomputers No commonality, no global view
©2010 D. J. Foreman10
Software
Tightly coupled (true "Distributed System") Maintains global view
Multiprocessor (one O/S)Multicomputer (homogeneous vs. heterogeneous)
Loosely coupled (NOS) Each heterogeneous computer runs its own O/S Not much transparency as seen by applications BUT: Local services are shared on the network
©2010 D. J. Foreman11
Some Classical Distributed Systems
Locus – distributed UNIX Lightweight kernel systems w/UNIX emulation
Amoeba – groups of threads (processor pool) Mach – synch msg-based IPC, threads
Argus ('79) – "Guardians" for transactions Clouds – object-based (like methods) V-system (1984)– IPC via kernel (msgs only),
true multi-threading (unavailable in UNIX '84) Cedar - µcoded VM, IDE, concurrent execution
(see list of papers)
©2010 D. J. Foreman12
Classical Systems, continued
XDFS - ca. 1977 - servers Cambridge File Server – ca.1982 - indexes on
servers used by OS to build its directory Sun NFS – networked UNIX FS
©2010 D. J. Foreman13
Open vs Closed Systems
Open Extensible Services external to kernel
Closed Non-extensible Services internal to kernel
Services are: File system Process mgmt Network communications Names Time Security/authentication
©2010 D. J. Foreman14
Atomicity
Semaphores Unstructured – requires app to test/signal
Mutexes Monitors
A programming constructShared variablesProcedures
lock and unlock functions (may be kernel-provided) Threads (Pthreads, POSIX threads)
Butenhof, "Programming with POSIX threads", AW Lewis & Berg, "Multithreaded Programming with
Pthreads", PTR-PH
©2010 D. J. Foreman15
Number Representations
What is an "int" 16 bits??? 32 bits???
FP number storage Many mechanisms, depending on h/w Less portable than integers
©2010 D. J. Foreman16
An example using int's
32-bit compiler, treats "int" as 32 bitsint x; x=1; // x is 00000000 00000000 00000000 00000001
Y(x); // all 32 bits ---> 16-bit target system Target of call compiled on 16-bit system
function Y (int val) // "int" is 16 bits{print val;} // prints 0 ---- WHY???? ---
'Y' only gets the 1st 16 bits of x because it doesn't KNOW there are more to get !!! ---
WHY??? -- Same problem for 32-bit 64-bit calls and
whatever comes next. Remember Y2K!
©2010 D. J. Foreman17
Operational Compatibility
Thread controls Pthreads on Windows (compatible w/POSIX) POSIX threads
seen on Linux & Solaris MS threads Solaris threads others (OS/2, MVS)
System services (commands, signals, file manipulation)
Process handling (creation, deletion) Communication (IPC: msgs vs pipes)
©2010 D. J. Foreman18
Memory
©2010 D. J. Foreman19
Memory Problems
a. Little Endian (Intel): lowest # byte = highest valueb. Big Endian (Sun): highest # byte = highest value
Receiver must know data types
Original(Sun)
as received(Windows)
improper correction
5*2245
©2010 D. J. Foreman20
Shared Memory
Inhibits distribution (coherency lost, cost to build) When NOT shared, problems arise:
Consistency Access (speed, mechanisms) Compatibility (endian) etc.
Atomicity Program level
Compare & Swap Test & OP (add, subtract, set-bit)
Kernel/library supported Semaphores, Mutexes, Condition variables
©2010 D. J. Foreman21
Distributed Memory
Partial solutions for heterogeneous systems Small local (non-shared) memory Bus and caches to a main store Bus and caches to each processor
Full solutions for multi-computer Interconnection (h/w switching) networks
Limited scalability VERY expensive, but VERY fast
©2010 D. J. Foreman22
Communications
2
Primitives - 1
Reliable – e.g.; TCP Ack Ordered Guaranteed delivery
Unreliable – e.g.; UDP No ack No guarantee No ordering (bottles in the ocean)
©2010 D. J. Foreman23
Primitives - 2
Blocking/non-blocking Buffered/unbuffered Distinguished messages
Send_get (sender is blocked) Get_request Send_reply (original sender is unblocked)
©2010 D. J. Foreman24
©2010 D. J. Foreman25
Methods
Shared variables - coherence?? Message passing
RPC – Remote Procedure Call Very good for Client-server Hides message passing
RMI – Remote Method Invocation (also Remote Object Invocation)
Like RPC, but with system-wide objects Java server implements a Security Manager & registry
Very much like DCE MOM – Message Oriented Middleware
Streams (TCP/IP) Needed for continuous information flow (video, audio)
RPC Protocols
Requestor Receiver (server)
©2010 D. J. Foreman26
Request
Ack ReplyAck
Request
Ack Reply
Request
Reply
T0
Tn
Marshaling
Heterogeneous systems No guarantee of data conformity On guarantee of directionality (endian-ness)
Data must be converted Data format must be standardized Communication types
Virtual circuits –reliable, flow controlled Datagrams unreliable, no TCP “connect”
©2010 D. J. Foreman27
©2010 D. J. Foreman28
RPC Steps
1. Client procedure calls client stub in normal way2. Client stub builds message, calls local OS3. Client's OS sends message to remote OS4. Remote OS gives message to server stub5. Server stub unpacks parameters, calls server6. Server does work, returns result to the stub7. Server stub packs it in message, calls local OS8. Server's OS sends message to client's OS9. Client's OS gives message to client stub10.Stub unpacks result, returns to client
2
©2010 D. J. Foreman29
Passing Value Parameters
a) Original message on the PCb) The message after receipt on a SPARC WSc) The message after being inverted. Numbers in
dotted-boxes indicate the address of each byte
2
©2010 D. J. Foreman30
Berkeley Sockets (1)
Delete the connectionClose
Input dataReceive
Output dataSend
Allow a specific connection request to an ethereal port, selected by the O/S, freeing the "listen" port
Accept
Signal availability for connection requestsListen
Assign a "handle" to the socketBind
Create an endpoint from a connectionSocket
Basic TCP/IP 'commands'
2
©2010 D. J. Foreman31
Connection-oriented communication pattern using sockets. Note the ordering of Reads & Writes vs time.
Berkeley Sockets: Client-Server
S o c ket B ind Lis ten A c c ep t R ead W rite C lo s e
S e rve r
C lien tS o c ket C o nnec t W rite R ead C lo s e
2
t0
©2010 D. J. Foreman32
Berkeley Sockets: Peer-Peer
S o c ket, B ind ,L is ten , A c c ep t R ead W rite
P e e r 1
P eer 2
C o nnec t W riteR eadS o c ket, B ind ,L is ten , A c c ep t
C o nnec t
Read & Write are independent threads
2
©2010 D. J. Foreman33
Berkeley Sockets: Producer-Consumer
S o c ket B ind L is ten A c c ep t R ead C lo s e
C onsum e r
P ro d u cerS o c ket, B ind C o nnec t W rite C lo s e
Note the one-way nature of communication
2
©2010 D. J. Foreman34
Naming
4
©2010 D. J. Foreman35
What is a name?
Basic objects File Value Program
Data structures across a network How to reference How to modify
Types Human readable Machine readable
©2010 D. J. Foreman36
Synchronization
5
©2010 D. J. Foreman37
Mechanisms
Explicit signals Semaphores, conditions, events
Test and Set shared variables Consider chip cycle vs. bus cycle
©2010 D. J. Foreman38
Methods
Semaphores Conditional critical sections Messages Event counts and sequencers Monitors Modules, common procedures and entries Path expressions Managers I/O commands
©2010 D. J. Foreman39
Problems
Race conditions - cause loss of data
Action sequences - hierarchy to be maintained
Data protection
©2010 D. J. Foreman40
Consistency and Replication
6
©2010 D. J. Foreman41
World view
Users must see same data at all times Updates must be controlled Rollbacks must maintain consistency
©2010 D. J. Foreman42
Fault Tolerance
7
©2010 D. J. Foreman43
Partitioning
Problem: when is the system partitioned? Response time? Keep-alive? Other?
Solutions Rollback? Re-start? Cut-off? Other?
©2010 D. J. Foreman44
Security
8
©2010 D. J. Foreman45
Whose data is it?
Must protect data and programs Viruses Worms Errors Theft
©2010 D. J. Foreman46
Virtualization
8
Problems
Hardware differences on ends of real systems Time delays cannot be simulated
©2010 D. J. Foreman47
©2010 D. J. Foreman48
Papers
8
©2010 D. J. Foreman49
Introductory Papers
The papers listed in the following section, while dated in the early 1970s and 80's, form some of the foundations of contemporary implementations and research.
©2010 D. J. Foreman50
Basic Concepts
Synchronizing Resources, G. Andrews, 1981 Specification of Synchronizing Processes, K.
Ramanathan, 1983 Concepts and Notations for Concurrent
Programming, G. Andrews, 1983 Distributed Operating Systems, A.
Tanenbaum, 1985
©2010 D. J. Foreman51
Programming & Concept Papers
The Temporal Semantics of Concurrent Programs, A. Pneuli, 1981
Time, Clocks, and the Ordering of Events in a D.S., L. Lamport, 1978
Determining the Last Process to Fail, D. Skeen, 1985
Correctness Proofs of D. Termination Algorithms, K. Apt, 1986
Monitors: An OS Structuring Concept, C.A.R. Hoare, 1974
©2010 D. J. Foreman52
Actual System Papers
An Introduction to the V System, E. Berglund, 1986
The Design of the Saguaro D.O.S., G. Andrews,1987
XMS: A Rendezvous-based D. S. Software Architecture, N. Gammage, 1985