Distributed Computing Systems

CSCI 4780/6780

Scalability

Concept Example

Centralized services A single server for all users

Centralized data A single on-line telephone book

Centralized algorithmsDoing routing based on complete information

• Scalability with respect to size

• Scalability with respect to geographical location

• Scalability with respect to management

Scalability Problems

Example• Problem: Design a Highly Scalable Map

(Direction) Service

• Data – Route information

• Task – Computing shortest (least cost) routes

• Option 1: Store all data at a single server, perform all computations there

• Option 2: Store data centrally, but perform operations on multiple nodes

• Option 3: Store data at multiple locations and perform operations in a distributed fashion

Characteristics of Distributed Algorithms

• Single machine does not have complete information

• Decisions are made based on local information

• Failure of a node does not cause the algorithm to fail

• No global clock

• Distributed algorithms are much harder than centralized algorithms

Geographical Scalability Challenges

• Synchronous communication– Waiting for a reply does not scale well!!

• Unreliable networks– Broadcasting does not work

• Centralized components– Will eventually become bottlenecks

• Multiple administrative domains– Security mechanisms do not come for free

Scalability Techniques

• Hiding communication latencies– Asynchronous communication– Reduce amount of data transmitted

• Distribution– Spreading work across system

• Caching and replication– Make copies of data and services

• Balance load– Avoid hot spots

Scaling Techniques

1.4

The difference between letting:

a) a server or

b) a client check forms as they are being filled

Scaling Techniques (2)

1.5

An example of dividing the DNS name space into zones.

Pitfalls

• Network is reliable

• Network is secure

• Network is homogeneous

• Topology does not change

• Latency is zero

• Bandwidth is infinite

• Transport cost is zero

• Single administrator

Types of Distributed Systems

• High-Performance Computing Systems• Cluster computing• Grid computing

• Distributed Information Systems• Transaction processing systems• Enterprise application integration• P2P systems

• Distributed Pervasive Systems• Mobile networks• Sensor networks

Cluster Computing

• Super computing using PCs/workstations

• Used for parallel programming

• Homogeneity among nodes

• Master allocates nodes to tasks

• Compute nodes run standard operating systems (and may be a very thin middleware)

• Middleware consists of parallel programming libraries

Cluster Computing Systems

An example of a cluster computing system.