Injecting Realistic Burstiness to a Traditional Client-Server Benchmark

© 2006 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Injecting Realistic Burstiness to a Traditional Client-Server Benchmark

Ningfang Mi College of William and Mary

Giuliano Casale SAP Research

Ludmila Cherkasova Hewlett-Packard Labs

Evgenia Smirni College of William and Mary

Presenter: Lucy Cherkasova

2 International Conference on Autonomic Computing and Communications (ICAC) 2009

Origin of Burstiness

• Enterprise and Internet applications:

Clients DB Server

Front Server

Web + Application

Server

HTTP request

HTTP reply

SQL query

SQL reply

Burstiness

??

Highly Correlated Arrivals

?


Client-Server Benchmark

• E.g., TPC-W (On-line bookstore Web site)

• Exponentially distributed user think timesExponentially distributed user think times

Clients DB Server

Front Server

Web + Application

Server

HTTP request

HTTP reply

SQL query

SQL reply

Burstiness

??

Highly Correlated Arrivals

?


• Accounts for randomness and variability … • … but not for burstinessbut not for burstiness

Can we ignore burstiness in the arrival process?

Typical Client-Server Benchmark

BurstinessBurstinessVariabilityVariability

Serv

ice t

ime

Serv

ice t

ime

Request number Request number


Why Need to Inject Burstiness?

• Burstiness impacts the performance of resource allocation mechanisms.

• Example: Session-based admission control (SBAC)−User session: sequence of transaction requests−Session is a unit of work−Typically, long sessions are “sales”.−Useful system throughput is the number of

completed sessions−Admission controller admits/rejects sessions

based on observed CPU utilization of the server (a combination of last measurement and some history).

L. Cherkasova, P. Phaal. Session Based Admission Control: a Mechanism for

Peak Load Management of Commercial Web Sites. IEEE J. TOC, June 2002.


SBAC

Reject a new session when utilization is above the threshold

Abort an accepted session when the server queue is full

highly undesirable

Front ServerWeb +

ApplicationServer

DB Server

New Client Arrival

Requests from already accepted clients

limited server queue


Impact of Burstiness

• We performed experiments for the same workload with different arrival patterns: non-bursty vs bursty

• Aborted ratio = aborted sessions/accepted sessions

highly undesirable

Queue Size Non-bursty Bursty

250 0.04% 11.37%

512 0.00% 6.28%

800 0.00% 2.50%


Why Need to Inject Burstiness? (2)• Service level agreement (SLA)

−support given response time guarantees for accepted sessions

• SLA of 1.2s can be supported for 98% of requests with queue size =250 for non-bursty traffic

• Only 90% of requests meet SLA=1.2s bursty traffic.

0

0.2

0.4

0.6

0.8

1

1.2

250 512 800

90th

95th

98th

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

250 512 800

90th

95th

98th

Queue Size Queue Size

Resp

onse

Tim

e (

s)

Resp

onse

Tim

e (

s)

Non-Bursty Bursty


Limitations of Standard TPC-W

• Think times are drawn randomly from the exponential distribution identical for all clients

• Exponential think times are incompatibleincompatible with the notion of burstiness.

Need to inject burstiness into user think times.


Our Methodology

•Basic Idea: modify the distribution of client think time to create bursty arrivals−Regulate the arrivals by using a 2-phase

Markovian Arrival ProcessMarkovian Arrival Process (MAP).• MAPs are variations of popular On/OFF traffic

models that can be easily shaped to create correlated inter-arrival times

• All clients share a MAP(2) to draw think times

• A new module for client-server benchmarks

−Regulate the intensity of traffic surges by using the index of dispersionindex of dispersion. • A simple tunable knob of burstiness


Index of Dispersion (I)• Popular burstiness index in networking• Definition

− SCV – the squared coefficient of variation (variance/mean2)− ρk – autocorrelation coefficients

• i.e., correlation of service times− Exponential: I = SCV = 1

)21(1

k

kSCVI variabilityburstines

s

BurstinessBurstinessVariabilityVariability

Serv

ice

tim

e

Serv

ice

tim

e

Request number Request number


Markovian Arrival Process (MAP)

• MAPs have ability to provide variabilityvariability and temporal localitytemporal locality.

• We use a class of MAPs with two states only

Normal

Traffic

λlong

Traffic Surge

λshort

2 states: λshort > λlong

pl,s

ps,l

ps,spl,l

time

Num

. of

arr

ivals

pl,s, ps,l, ps,s, pl,s shape correlation


MAP Fitting

• Input − Estimated mean service demands at servers: E[Di]

− Mean user think time E[Z]

− The pre-defined index of dispersion I

• Output− A MAP(2) to draw user think times


MAP Fitting (2)

Key: determine (Key: determine (λλshortshort,, λλlonglong, , ppl,sl,s,, p ps,ls,l))• Condition for traffic surge

• Condition for normal traffic

• Mean think time

• We use non-linear optimizer to search for such f and ps,l and find a MAP(2) to best match the predefined I

fDEi ishort /)(1

])[),(max(1 ZEDENfi ilong

)][

][(

1

1

,,

short

longlssl ZE

ZEpp

Departure > Arrival

Arrival > Departure the arrival rate is f times higher than the throughput of the system

the arrival rate is f times slower for balanced system throughput

Balancing the height and the width of the burst


Realistic values for Burstiness

−What is the range of realistic values for defining burstiness via index of dispersion I ? • Exponential: I = SCV = 1

• Bursty: values of thousands,

−e.g., FIFA World Cup 1998, one of the servers over 10 days, I = 6300


TPC-W Testbed

• On-line bookstore Web site • Testbed: clients + front server + DB server

−Constant number of emulated browsers (EBs)

• User session−sequence of transaction requests

−think time (mean=7 sec) between two transaction requests

• 14 transactions types grouped in three mixes:−Browsing mix

−Shopping mix

−Ordering mix


Validation – Arrival Process

• Arrival clients to the system (front server)

Think times drawn by a MAP(2) with I create the bursty conditions.

Shopping Mix

Non-bursty (I=1)

Time (s)

Num

ber

of

act

ive c

lients Bursty (I=4000)

Time (s)

Num

ber

of

act

ive c

lients


Validation – Utilization DistributionShopping Mix

Non-bursty (I=1) Bursty (I=4000)

pd

fpd

f

pd

fpd

f

Utilization (%)Utilization (%)

Utilization (%) Utilization (%)

Front

DB


Validation - Average Latency

0

500

1000

1500

2000

2500

3000

3500

200 400 600 800 1000 1200Number of EBs

non- bursty

I=4000

Browsing Mix

Resp

onse

tim

e (

ms)

0

200

400

600

800

1000

1200

1400

1600

200 400 600 800 1000 1200Number of EBs

non- bursty

I=4000

Shopping Mix

Resp

onse

tim

e (

ms)


Validation – Latency Distributions

0%

20%

40%

60%

80%

100%

0 2000 4000 6000 8000 10000

non-bursty

I=4000

0%

20%

40%

60%

80%

100%

0 1000 2000 3000 4000 5000

non-bursty

I=4000

Browsing Mix

CD

F

Shopping Mix

Response time (ms) Response time (ms)

CD

F

0.83

2.98

0.04

1.25


Conclusion• Burstiness critical for autonomic system design

− need representative benchmarks for system evaluation− need reproducible and controllable bursty workloads

• Traditional client-server benchmarks ignore burstiness in arrival flows− e.g., TPC-W with exponential think times

• Explicitly inject burstiness − a simple and tunable parameter: index of dispersion− can introduce different intensity of traffic surges

• http://www.cs.wm.edu/~ningfang/tpcw_codes/

• Supported by NSF grants CNS-0720699 and CCF-08114171 and HPLabs gift.

Injecting Realistic Burstiness to a Traditional Client-Server Benchmark

Documents

Transcript of Injecting Realistic Burstiness to a Traditional Client-Server Benchmark