Queuing Theory

Queuing Theory deals with systems of the following type:

Typically we are interested in how much queuing occurs or in the delays at the servers.

Queuing Theory

InputProcess

Server Process(es)

Output

Queuing Theory Notation• A standard notation is used in queuing theory to

denote the type of system we are dealing with.• Typical examples are:– M/M/1 Poisson Input/Poisson Server/1 Server– M/G/1 Poisson Input/General Server/1 Server– D/G/n Deterministic Input/General Server/n Servers– E/G/ Erlangian Input/General Server/Inf. Servers

• The first letter indicates the input process, the second letter is the server process and the number is the number of servers.

• (M = Memoryless = Poisson)

4

Definition of a queueing system

Customer arrivals

Departure of impatient customers

Departure of served customers

• A queueing system can be described as follows:"customers arrive for a given service, wait if the service cannot start immediately and leave after being served"

• The term "customer" can be men, products, machines, ...

5

History of queueing theory

• The theory of queueing systems was developed to provide models for forecasting behaviors of systems subject to random demand.

• The first problems addressed concerned congestion of telephone traffic (Erlang, "the theory of probabilities and telephone conversations ", 1909)

• Erlang observed that a telephone system can be modeled by Poisson customer arrivals and exponentially distributed service times

• Molina, Pollaczek, Kolmogorov, Khintchine, Palm, Crommelin followed the track

6

Interests of queueing systems

Queuing theory found numerous applications in:

– Trafic control (communication networks, air traffic, …)

– Planing (manufacturing systems, computer programmes, …)

– Facility dimensioning (factories, ...)

7

Classification of queueing systems

8

Characteristics of simple queueing systems

Queueing systems can be characterized with several criteria:

• Customer arrival processes• Service time• Service discipline• Service capacity• Number of service stages

9

Notation of Kendall

The following is a standard notation system of queueing systems

T/X/C/K/P/Z with

– T: probability distribution of inter-arrival times– X: probability distribution of service times– C: Number of servers– K: Queue capacity– P: Size of the population– Z: service discipline

10

Customer arrival processT/X/C/K/P/Z

• T can take the following values:– M : markovian (i.e. exponential)– G : general distribution– D : deterministic– Ek : Erlang distribution– …

• If the arrivals are grouped in lots, we use the notation T[X] where X is the random variable indicating the number of customers at each arrival epoch– P{X=k} = P{k customers arrive at the same time}

• Some arriving customers can leave if the queue is too long

11

Service timesT/X/C/K/P/Z

• X can take the following values:– M : markovian (i.e. exponential)– G : general distribution– D : deterministic– Ek : Erlang distribution– …

Erlang distribution Ek with parameter m

k exponential servers with parameter m

12

Number of serversT/X/C/K/P/Z

In simple Queuing Systems, servers are identical

13

Queue capacityT/X/C/K/P/Z

Loss of customers if the queue is full

Capacity K

14

Size of the populationT/X/C/K/P/Z

The size of the population can be either finite or infinite

For a finite population, the customer arrival rate is a function of the number of customers in the system: l(n).

15

Service disciplineT/X/C/K/P/Z

Z can take the following values:

• FCFS or FIFO : First Come First Served

• LCFS or LIFO : Last Come First Served

• RANDOM : service in random order

• HL (Hold On Line) : when an important customer arrives, it takes the head of the queue

• PR ( Preemption) : when an important customer arrives, it is served immediately and the customer under service returns to the queue

• PS (Processor Sharing) : All customers are served simultaneously with service rate inversely proportional to the number of customers

• GD (General Discipline)

Chapter 30 Introduction to Queueing Theory16

• = interarrival time• l = mean arrival rate

= 1/E[]• s = service time per job• m = mean service rate per

server = 1/E[s] (total service rate

for m servers is mm)

Time

Previous arrival Arrival Begin

ServiceEnd

Service

w s

• nq = number of jobs waiting to receive service.• ns = number of jobs receiving service• n = number of jobs in system

n = nq + ns• r = response time = w + s• w = waiting time

r

Note that all of these variables are random variablesexcept for l and m .

Chapter 30 Introduction to Queueing Theory17

Rules for All Queues

• Stability Condition– If the number of jobs becomes infinite, the system is

unstable. For stability, the mean arrival rate less than the mean service rate.

l < mm

– Does not apply to finite buffer system or the finite population systems They are always stable. • Finite population: queue length is always finite. • Finite buffer system: arrivals are lost when the

number of jobs in the system exceed the system capacity.

Chapter 30 Introduction to Queueing Theory18

Rules for All Queues

• Number in System vs. Number in Queue–n = nq + ns

– E[n] = E[nq]+E[ns]–Also, if the service rate of each server is

independent of the number in queue Cov(nq,ns) = 0 Var[n] = Var[nq]+Var[ns]

Chapter 30 Introduction to Queueing Theory19

Rules for All Queues

• Number vs. Time (Little’s law)– If jobs are not lost due to buffer overflow, the mean

jobs is related to its mean response time as follows: mean number of jobs in system

= arrival rate x mean response time– Similarly

mean jobs in queue = arrival rate x mean waiting time– For finite buffers can use effective arrival rate, that is,

the rate of jobs actually admitted to the system.

Chapter 30 Introduction to Queueing Theory20

Rules for All Queues

• Time in System vs. Time in Queue– Time spent in system, response time, is the

sum of waiting time and service timer = w + s

– In particular:E[r] = E[w] + E[s]

– If the service rate is independent of jobs in queue Cov(w,s) = 0 Var[r] = Var[w] + Var[s]

Chapter 30 Introduction to Queueing Theory21

Proof of Little’s LawMean jobs in system = arrival rate x mean response time

Arrival time Departure timejob1 0 4job2 3 6job3 5 7

Chapter 30 Introduction to Queueing Theory22

Proof of Little’s Law

J: total time spent inside the system by all three jobs (the hatched area)

30.4 (a): Mean time spent in the system = J/N30.4 (b): Mean number in the system = J/T = J/N * N/T = Mean time spent in the system * Mean arrival rate

Chapter 30 Introduction to Queueing Theory23

Applying Little’s Law

• Example: – A disk server satisfies an I/O request in average

of 100 msec. I/O rate is about 100 requests/sec. What is the mean number of requests at the disk server?

– Mean number at server = arrival rate x response time

( =100 requests/sec )x (0.1 sec) =10 requests

Chapter 30 Introduction to Queueing Theory24

Types of Stochastic Processes

• A stochastic process is a family of random variables indexed by the parameter t, such as time.

e.g.) Number of jobs at CPU of computer system at time t is a random variable n(t)

• Time and state can be discrete or continuous

Chapter 30 Introduction to Queueing Theory25

Types of Stochastic Processes

• Markov Process– If future states depend only on the present and

are independent of the past (memoryless) then called markov process

– A discrete state Markov Process is a Markov chain

– M/M/m queues can be modeled using Markov process• time spent by a job – Markov process• the number of jobs - Markov chain

Chapter 30 Introduction to Queueing Theory26

Types of Stochastic Processes

• Birth-Death Process– The Markov chain in which transitions

restricted to neighboring states.– Can represent states by integers, s.t. process in

state n can only go to state n+1 or n-1– Arrival (birth) causes state to change by +1 and

departure after service (death) causes state to change by –1

– Only if arrive individually, not in bulk

Chapter 30 Introduction to Queueing Theory27

Types of Stochastic Processes

• Poisson Processes– If inter arrival times are IID and

exponentially distributed, then number of arrivals over interval [t,t+x] has a Poisson distribution Poisson Process–Popular in queuing theory because

arrivals are then memory less

Chapter 30 Introduction to Queueing Theory28

Properties of Poisson Process

(a) Merging k Poisson streams with mean rate li gives another Poisson stream with mean rate l = li

(b) If Poisson stream split into k substreams with probability pi, each substream is Poisson with mean rate lpi

Chapter 30 Introduction to Queueing Theory29

Properties of Poisson Process

(c ) If arrivals to single server with exponential service times are Poisson

with mean , d epartures are also Poisson with mean , if (<)

(d )Same relationship holds for m servers as long as total arrival rate less than total service rate

Chapter 30 Introduction to Queueing Theory30

Types of Stochastic Processes

PoissonProcesses

Birth-deathProcesses

MarkovProcesses