Lecture 9: Deterministic Fluid Models and Many-Server Heavy-Traffic Limits

IEOR 4615: Service Engineering

Professor Whitt

February 19, 2015

Outline

• Deterministic Fluid Models

– Directly From Data: Cumulative Arrivals and Departures

– Directly from Mt/M/st+M BD Model (deterministic view)

• Many-Server Heavy-Traffic Limits for QueueingModels

– Fluid Model Obtained in the Limit as Scale Increases

– Ultimately, limits explained by LLN and CLT

From Data to Fluid Models• To analyze data, we plot cumulative arrival and departure

functions:

• For large systems (bird’s eye view), the functions look smoother.

• Directly from event-based (call-by-call) measurements.

• For example, an isolated service-station:– A(t) = cumulative # arrivals from time 0 to time t;

– D(t) = cumulative # departures from system during [0, t];

– L(t) = A(T) − D(t) = # customers in system at t.

Arrivals and Departures from a Bank Branch Face-to-Face Service

From Data to Fluid Models

Deterministic Fluid Model

• Describe impact of Predictable Variability

– Time-Varying Arrival Rate

– Ignoring Stochastic Variability

• Idealistic Smooth Model

– Ordinary Differential Equation (ODE)

Phases of CongestionHall, textbook:

Four points of view

• Cumulative Arrivals and Departures

• Rates (⇒ Peak Load)

• Queues (⇒ Congestion)

• Outflows (⇒ end of rush-hour)

Phases of Congestion via Rates

• Time lag in congestion after peak arrival rate

• Changing Departure Rate

arrival rateα(t)

(maximum possible service rate)

Queueing System as a Tub (Hall, p.188)

• A(t) – cumulative arrivals function.• D(t) – cumulative departures function.• – arrival rate (dot = derivative d/dt)• – processing (departure) rate.• c(t) – maximal potential processing rate.• q(t) – total amount in the system at time t.

Mathematical Fluid Models: General Setup

)()( tDt

)()( tAt

Mathematical Fluid Model

Differential equation:

• λ(t) – arrival rate at time t ∈ [0,T].

• c(t) – maximal potential processing rate.

• δ(t) – effective processing (departure) rate.

• q(t) – total amount in the system at time t.

Then q(t) is a solution of

].,0[,)0();()()( 0 Ttqqtttq

Mathematical Fluid Model:Multi-Server Queue

• s(t) statistically-identical servers, each with service rate μ.

• c(t) = μs(t): maximal potential processing rate.

• : processing rate.

i.e.,

How to actually solve? Discrete-time approximation: Start with Then, for

].,0[,)0());(),(min()()( 0 Ttqqtqtsttq

.))(),(min()()()( 1111 ttqtstttqtq nnnnn

))(),(min()( tqtst

.)(,0 000 qtqt :1 ttt nn

.))(),(min()()0()(00 tt

duuqusduuqtq

Mathematical Fluid Model:Multi-Server Queue with Abandonment

• θ – Abandonment rate of customers in queue

• Processing rate:

• The fluid model:

• Deterministic View of Mt/M/st+M BD Model– Parameters: λ(t), μ, s(t),θ

].,0[,)0(

;)]()([))(),(min()()(

0 Ttqq

tstqtqtsttq

)]()([))(),(min()( tstqtqtst

Many-Server Heavy-Traffic Limit

Sequence of Mt/M/st+M Models Indexed by nLet n →∞.

Parameters:

• λn(t) = nλ(t) arrival rate at time t [heavy-traffic]

• sn(t) = ns(t) number of servers at time t [large scale]

• μn(t) = μ individual service rate (constant)

• θn(t) = θ individual abandonment rate (constant)

Stochastic Processes:

• An(t) number of arrivals in system n in [0,t]

• Dn(t) number of departures in system n in [0,t]

• Qn(t) number of customers in system n at time t

• Wn(t) potential waiting for arrival at time t (with infinite patience)

Fluid Approximation from Many-Server Heavy-Traffic Limit

Fluid Approximation from Many-Server Heavy-Traffic Limit

Fluid Approximation from Many-Server Heavy-Traffic Limit

Many-Server Heavy-Traffic LimitsSequence of Mt/M/st+M Models Indexed by n

Parameters:• λn(t) = nλ(t) arrival rate at time t [heavy-traffic]• sn(t) = ns(t) number of servers at time t [large scale]• μn(t) = μ individual service rate (constant)• θn(t) = θ individual abandonment rate (constant)Stochastic Processes:• An(t) number of arrivals in system n in [0,t]• Dn(t) number of departures in system n in [0,t]• Qn(t) number of customers in system n at time t• Wn(t) potential waiting for arrival (with infinite patience) Limits (Fluid Limit = Law of Large Numbers): As n →∞,• An(t)/n → Λ(t) = ∫

t λ(s) ds (integral of fluid arrival rate)

• Dn(t)/n → D(t) = ∫t δ(s) ds (integral of fluid departure rate)

• Qn(t)/n → q(t) fluid content at time t• Wn(t) → w(t) potential waiting time for atom of fluid

Refined Many-Server Heavy-Traffic LimitSequence of Queueing Models Indexed by n

Mt/M/st+M

Fluid Limits (Fluid Limit = Law of Large Numbers): As n →∞,

• An(t)/n → Λ(t) = ∫t λ(s) ds (integral of fluid arrival rate)

• Dn(t)/n → D(t) = ∫t δ(s) ds (integral of fluid departure rate)

• Qn(t)/n → q(t) fluid content at time t

• Wn(t)→ w(t) waiting time

Stochastic Limits (Gaussian Limit = Central limit Theorem):

As n →∞,

• √n[(An(t)/n) - Λ(t)] → XA(t)

• √n[(Dn(t)/n) - D(t)] → XD(t) Gaussian limit processes

• √n[(Qn(t)/n) - q(t)] → XQ(t)

• √n[Wn(t)- w(t)] → XW(t)

Three Many-Server Heavy-Traffic Limiting RegimesSequence of Stationary M/M/s+M Models

Parameters:• λn = nλ - c √n arrival rate at time t [No time-varying parameters]• sn = ns number of servers at time t [large scale]• μn = μ individual service rate (constant)• θn = θ individual abandonment rate (constant)Limiting Regimes• λ>s μ overloaded or Efficiency-Driven (ED)• λ<s μ underloaded or Quality-Driven (QD)• λ=s μ critically loaded – need to look more closelyExpanding the Critically Loaded Regime: λ=sμ• More general arrival rate scaling: • Quality-and-Efficiency-Driven (QED) regime = Halfin-Whitt (1981) regime

– λn = ns μ[1 – (β/√n)] (λ = s μ and c = -s μ β)– (1-ρn)√n =β where ρn = λn/snμ = λn/nsμ

Delay Probability Approximationin the M/M/s/∞ Model in the QED Regime

• Quality-and-Efficiency-Driven (QED) regime = Halfin-Whitt (1981) regime

– λn = ns μ[1 – (β/√n)]

– (1-ρn)√n =β where ρn = λn/snμ = λn/nsμ

– P(Wn > 0) → α with 0 < α < 1. (Wn steady state wait before starting service in model n)

P(Wn > 0) ≈ α(β) = HW(β) = 1/[βΦ(β)/φ(β)]

Where Φ(x) = P(N(0,1) < x) standard normal cdf and φ(β) is the associated density function

Implications for Staffingin the M/M/s/∞ Model

P(Wn > 0) ≈ Target = α = α (β)= HW(β) = 1/[βΦ(β)/φ(β)]

Where Φ(x) = P(N(0,1) < x) standard normal cdfand φ(β) is the associated density function

Use Square-Root Staffing Formula: Set

S = S(λ/μ) = (λ/μ) + β (λ/μ)1/2

For β = HW-1(α) (inverse function)

λ/μ = offered load (infinite-server model)

Implications for Staffingin the M/M/s+M Model

P(Wn > 0) ≈ Target = α = α(β, γ) = G(β, γ) = 1/[1 + γh(β/γ)h(-β)]

Garnett function from Garnett et al. (2002)

Where γ = (θ/μ)1/2, h(x) = φ(x)/[1- Φ(x)], Φ(x) = P(N(0,1) < x) standard normal cdf and φ(x) is the associated density function.

Use Square-Root Staffing Formula: Set

S = S(λ/μ) = (λ/μ) + β (λ/μ)1/2

For β = G-1(Target, γ)

ReferencesFluid Models

1. Chapter 6, especially Section 6.4, of Hall (1991) QueueingMethods for Services and Manufacturing, Prentice Hall.

2. Chapters 1 and 2 of Newell (1982) Applications of QueueingTheory, second edition, Chapman and Hall.

Many-Server heavy-Traffic Limits3. Halfin, S., W. Whitt. (1981) Heavy-traffic limits for queues with

many exponential servers. Operations Research, 29, 567-588.

4. Whitt, W. (1992) Understanding the Efficiency of Multi-Server Service Systems. Management Science, 38, 708-723.

More Advanced ReferencesFluid Models

1. Hampshire, R. C., W. A. Massey. (2010). A tutorial on dynamic optimization with application to dynamic rate queues. TutORials in Operations Research, presented at the INFORMS National Meeting in Austin Texas. (www.princeton.edu~wmassey)

Many-Server Heavy-Traffic Limits2. Garnett, O., A. Mandelbaum, M. I. Reiman. (2002) Designing a call

center with impatient customers. Manufacturing Service Oper. Management 4, 208–227. (http://iew3.technion.ac.il/serveng)

3. Pang, G., R. Talreja, W. Whitt. (2007) Martingale proofs of many-server heavy-traffic limits for Markovian queues. Probability Surveys, 4, 193-267.