18 June 2001

Optimizing Distributed System Performance via Adaptive

Middleware Load BalancingOssama Othmanossama@uci.edu

Douglas C. Schmidtschmidt@uci.edu

Department of Electrical and Computer Engineering

University of California, Irvine

Irvine, California 92697 USA

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Introduction

Network

Clients

Req

uest

s

Rep

lies

IDC

Servers • Motivation– Given a resource

intensive distributed application

• Clients typically greatly out number servers

• Some servers can be more loaded than others

• Requests generated by clients are often “bursty” and unpredictable

• Solution– Adaptive load

balancing

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Basic Scenario and Concepts

LoadBalancer

Clients

Req

uest

s

Rep

lies

ReplicasReplica Groups

• Load balancing goals– Use load balancing to

distribute client requests equitably among several replicas, within a replica group

– Ensure differences in replica loads are kept to a minimum

• Common Problem– Load balancing

algorithms in use may be very good but underlying mechanism is often inefficient

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Load Balancing Strategies• Client binding granularity

– Per-session• Client permanently

forwarded to a replica– Per-request

• Requests forward on client’s behalf

– On-demand• Client can be rebound

to another replica whenever necessary

• Balancing policy– Non-adaptive

• No load feedback used when binding clients

– Adaptive• Load feedback taken in

to account

: Client : Replica

: Load Balancer

1. send_re

quest() 2. send_request()

: Client : Replica

: Load Balancer

1. send_re

quest()

2. L

OCATION_FORW

ARD()

3. send_request()

Per-session / On-Demand

`

Per-request

: Load Balancer

load_advisory()report_load()

: Replica

Adaptive

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Load Balancing Architectures• Load balancing

architecture comprised of a combination of client binding granularity and balancing policy

• Given the strategies just described, there are six possible architectures

• Three common architectures– Non-adaptive per-

session– Adaptive per-request– Adaptive on-demand

: Client : Server Replica

: Load Balancer

1. send_request()

2. LOCATIO

N_FORWARD()

3. send_request()

: Client : Server Replica

: Load Balancer

1. send_request()

2. send_request()

load_advisory()

report_load()

The front-endserver proxy.

: Client : Server Replica

: Load Balancer

1. send_request()

2. LOCATIO

N_FORWARD() load_advisory()

report_load()

3. send_request()

Non-adaptive Per-Session

Adaptive Per-request

Adaptive On-demand

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Load Balancing Experiment Testbed

Dual CPUReplica Host

Dual CPULoad Balancer

HostQuad CPUClient Host

Dual CPUReplica Host

100 MBpsNetwork Switch

• Testbed hardware– Client host (1)

• Quad CPU 400 MHz Pentium II Xeon, 1GB RAM

– Replica hosts (2), and load balancer host (1)

• Dual CPU 733 MHz Pentium III, 512MB RAM

• Testbed software– Debian GNU/Linux 2.1

“potato” (glibc 2.1, kernel 2.2.16)

– TAO “Latency” performance test

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Latency OverheadLatency Comparison

0

100

200

300

400

500

600

700

Classic LatencyPerformance Test

Latency Test w/Per-Session Load

Balancer

Latency Test w/Per-Request Load

Balancer

Latency Test w/TAOOn-Demand Load

Balancer

Lat

ency

(u

secs

)

Latency - Jitter

Average Latency

Latency + Jitter

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Throughput Overhead

Throughput Comparison

0500

10001500200025003000350040004500

Classic LatencyPerformance

Test

Latency Testw/Per-SessionLoad Balancer

Latency Testw/Per-RequestLoad Balancer

Latency Testw/TAO On-

Demand LoadBalancer

Th

rou

gh

pu

t (e

ven

ts p

er s

eco

nd

)

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Non-adaptive Per-session Effectiveness

Loads Under Non-Adaptive Per-Session Strategy

0

5

10

15

20

251 40 79 118

157

196

235

274

313

352

391

430

Elapsed Time (seconds)

Lo

ad (

req

ues

ts/s

eco

nd

)

Replica 1Replica 2Replica 3Replica 4

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Adaptive On-demand Effectiveness

Loads Under Adaptive On-Demand Strategy

020406080

100120140160180200

1 52 103

154

205

256

307

358

409

460

511

562

613

664

715

766

Elapsed Time (seconds)

Lo

ad (

req

ues

ts/s

eco

nd

)

Replica 1

Replica 2

Replica 3

Replica 4

University of California, Irvine

O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing

Conclusion• Load balancing can be performed at several levels

– The network level– The operating system level– The middleware level

• Network-based and OS-based suffer from several limitations– Inability to support application-defined load metrics

at run-time– Lack of adaptability due to absence of load feedback,

and lack of control over replicas• Middleware-based load balancing has a clear advantage

since it suffers from neither of these limitations