Renewal theory and its applications - win.tue.nlresing/isp/ISP4.pdfRenewal theory and its...

Renewal theory and its applications


Stella Kapodistria and Jacques Resing

September 16th, 2014

ISP


Definition of a Renewal process


If we substitute the Exponentially distributed inter-arrival times of the Poissonprocess by any arbitrary sequence of iid r.v. {X1,X2, . . .} we can generalizethe definition of the counting process.

DefinitionLet the sequence of nonnegative random variables {X1,X2, . . .} beindependent and identically distributed. For any sequence of iid r.v. we candefine a counting process as

N(t) = max{n : Sn ≤ t} with Sn =n∑

j=1

Xj

Then the counting process {N(t), t ≥ 0} is said to be a renewal process.

The definition implies:

(i) N(t) ≥ 0

(ii) N(t) is integer valued

(iii) If s < t , then N(s) ≤ N(t)

(iv) For s < t , N(t)− N(s) equals the number of events in (s, t ].



Poisson process

DefinitionThe counting process {N(t), t ≥ 0} is called a Poisson process with rate λ, if{X1,X2, . . .} are iid having the exponential distribution at rate λ.

In this case we know that:

(i) N(t) ∼ Poisson(λt) and m(t) = E[N(t)] = λt

(ii)∑n

j=1 Xj ∼ Erlang(λ, n) and E[Sn] = n/λ

(iii) By the Strong Law of Large Numbers (SLLN) it follows that,

Sn

n→ 1

λ, n→∞ (w.p.1)



Poisson process

DefinitionThe counting process {N(t), t ≥ 0} is called a Poisson process with rate λ, if{X1,X2, . . .} are iid having the exponential distribution at rate λ.

In this case we know that:

(i) N(t) ∼ Poisson(λt) and m(t) = E[N(t)] = λt

(ii)∑n

j=1 Xj ∼ Erlang(λ, n) and E[Sn] = n/λ

(iii) By the Strong Law of Large Numbers (SLLN) it follows that,

Sn

n→ 1

λ, n→∞ (w.p.1)

The strong law of large numbers states that the sample averageof iid rvs converges almost surely≡ (w.p.1) to the expected value

X na.s.−−→ E[X ], as n→∞.

That is,Pr

(lim

n→∞X n = E[X ]

)= 1.

SLLN


Distribution of N(t)


The distribution of N(t) can be obtained, at least in theory, by noting that

N(t) ≥ n⇐⇒ Sn ≤ t

Then,

P[N(t) = n] = P[N(t) ≥ n]− P[N(t) ≥ n + 1]

= P[Sn ≤ t ]− P[Sn+1 ≤ t ]

Now since the random variables Xi , i ≥ 1, are iid having a distribution F , itfollows that

Sn =n∑

j=1

Xj ∼ Fn

where Fn denotes the n-fold convolution of F with itself (Section 2.5).Therefore, we obtain

P[N(t) = n] = Fn(t)− Fn+1(t)



The mean-value function

The mean-value function: approach 1

We define the mean-value or the renewal function as

m(t) = E[N(t)]

The mean-value function can be obtained by noting that

N(t) ≥ n⇐⇒ Sn ≤ t

Then,

m(t) = E[N(t)] =∞∑

n=1

P[N(t) ≥ n]

=∞∑

n=1

P[Sn ≤ t ]

=∞∑

n=1

Fn(t)

ExampleSuppose m(t) = 2t . What is the distribution P[N(10) = n] =?





We define the mean-value or the renewal function as

m(t) = E[N(t)]

The mean-value function can be obtained by noting that

N(t) ≥ n⇐⇒ Sn ≤ t

Then,

m(t) = E[N(t)] =∞∑

n=1

P[N(t) ≥ n]

=∞∑

n=1

P[Sn ≤ t ]

=∞∑

n=1

Fn(t)

ExampleSuppose m(t) = 2t . What is the distribution P[N(10) = n] =?

Remember:

E[X ] =∞∑

x=1

P[X ≥ x ]





There is one-to-one correspondence between the renewal process and itsmean-value function!

We define m̃(s) to be the Laplace-Stieltjes transform of m(t)

m̃(s) =∫ ∞

0e−st dm(t)

Then we can prove that

m̃(s) =ψ(s)

1− ψ(s)

with ψ(s) = E[e−sX ] we have denoted the Laplace-Stieltjes transform of theinter-arrival times {X1,X2, . . .}.







m̃(s) =∫ ∞

0e−st dm(t)


m̃(s) =ψ(s)

1− ψ(s)


ψ(s) =m̃(s)

1 + m̃(s)

Equivalently







m̃(s) =∫ ∞

0e−st dm(t)


m̃(s) =ψ(s)

1− ψ(s)


how do we prove it?





There is one-to-one correspondence between the renewal process and itsmean-value function!We want to determine m(t) for t ≥ 1. We will prove a basic renewal equationas follows

m(t) = E[N(t)] =∫ ∞

0E[N(t)|X1 = x ]f (x)dx

=

∫ t

0E[N(t)|X1 = x ]f (x)dx +

∫ ∞t

E[N(t)|X1 = x ]f (x)dx

=

∫ t

0[1 + m(t − x)]f (x)dx +

∫ ∞t

0f (x)dx

= F (t) +∫ t

0m(t − x)f (x)dx

This last equation is called the renewal equation and can sometimes besolved to obtain the mean-value function.




Proposition 7.2 E[SN(t)+1] = (m(t) + 1)µ

Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ

timetS_8NHtL< S_8NHtL+1<

NHtimeL

NHtL





Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ

Remark (Wald’s Equation)For any sequence {X1,X2, . . .} of iid r.v. with mean E[X ] = µ and a r.v. Nindependent from the sequence {X1,X2, . . .} we can easily prove that

E[N∑

i=1

Xi ] = E[N]E[X ] = E[N]µ

timetS_8NHtL< S_8NHtL+1<

NHtimeL

NHtL





Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ

Proof.We define E[SN(t)+1] = g(t), then

g(t) =∫ ∞

0E[SN(t)+1|X1 = x ]f (x)dx

=

∫ t

0E[SN(t)+1)|X1 = x ]f (x)dx +

∫ ∞t

E[SN(t)+1|X1 = x ]f (x)dx

=

∫ t

0[x + g(t − x)]f (x)dx +

∫ ∞t

xf (x)dx

= µ+

∫ t

0g(t − x)f (x)dx





Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ


g(t) = µ+

∫ t

0g(t − x)f (x)dx

If we substitute g(t) = (m(t) + 1)µ yields

m(t) = F (t) +∫ t

0m(t − x)f (x)dx

which completes the proof.





Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ


g(t) = µ+

∫ t

0g(t − x)f (x)dx

If we substitute g(t) = (m(t) + 1)µ yields

m(t) = F (t) +∫ t

0m(t − x)f (x)dx

which completes the proof.

What if we didn’t know the answer?





Proposition (7.2)

E[SN(t)+1] = (m(t) + 1)µ


g(t) = µ︸︷︷︸k(t)

+

∫ t

0g(t − x)f (x)dx

For any known function k(t) the renewal type equation has a unique solution:

g(t)= k(t) +∫ t

0k(t − x)dm(x)

(m(t) = F (t) +∫ t

0m(t − x)f (x)dx)

Then by setting k(t) = µ immediately yields the result.


Limit Theorems

Limit Theorems

Let {X1,X2, . . .} be a sequence of iid r.v., with E[Xj ] = µ, and we define therenewal process {N(t), t ≥ 0} as

N(t) = max{n : Sn ≤ t} with Sn =n∑

j=1

Xj

By the SLLN it follows that,Sn

n→ µ, n→∞ (w.p.1)

However, sinceSN(t)

N(t)=

∑N(t)j=1 Xj

N(t)is the average of N(t) iid distributed random variables, it follows by SLLN

SN(t)

N(t)→ µ, N(t)→∞ (w.p.1)

But since N(t)→∞ when t →∞, we obtain

SN(t)

N(t)→ µ, t →∞ (w.p.1)


Limit Theorems

Limit Theorems

Proposition ( 7.1)With probability 1,

N(t)t→ 1

µas t →∞

Proof.Recall that by SLLN

SN(t)/N(t)→ E[X ] = µ as t →∞ (w.p.1)

Secondly,

SN(t) ≤ t < SN(t)+1

SN(t)

N(t)≤ t

N(t)<

SN(t)+1

N(t)SN(t)

N(t)≤ t

N(t)<

SN(t)+1

N(t) + 1N(t) + 1

N(t)


Limit Theorems

Limit Theorems

Proposition ( 7.1)With probability 1,

N(t)t→ 1

µas t →∞

Proof.Recall that by SLLN

SN(t)/N(t)→ E[X ] = µ as t →∞ (w.p.1)

Secondly,

SN(t) ≤ t < SN(t)+1

SN(t)

N(t)≤ t

N(t)<

SN(t)+1

N(t)

��µ

SN(t)

N(t)≤ t

N(t)<��

��*µ

SN(t)+1

N(t) + 1��*1

N(t) + 1N(t)

as t →∞


Limit Theorems

Limit Theorems

Theorem (Elementary Renewal Theorem)

m(t)t→ 1

µas t →∞

RemarkLet E[X ] = µ and Var [X ] = σ2, then

limt→∞

E[N(t)]t

=1µ

and limt→∞

Var [N(t)]t

=σ2

µ3

Theorem (Central Limit Theorem for Renewal Process)Let E[X ] = µ and Var [X ] = σ2, then

N(t)− t/µ√tσ2/µ3

d→ N(0, 1)


Limit Theorems

Example 7.7

Example 7.7

Suppose that potential customers arrive at a single-server bank inaccordance with a Poisson process having rate λ. Furthermore, suppose thatthe potential customer will enter the bank only if the server is free when hearrives; if upon arrival the customer sees the bank teller occupied he willimmediately leave. If we assume that the amount of time spent in the bank byan entering customer is a random variable having distribution G, then

(a) what is the rate at which customers enter the bank?

(b) what proportion of potential customers actually enter the bank?

In answering these questions, suppose that at time 0 a customer has justentered the bank.


Limit Theorems

Example 7.7

Example 7.7

Suppose that potential customers arrive at a single-server bank inaccordance with a Poisson process having rate λ. Furthermore, suppose thatthe potential customer will enter the bank only if the server is free when hearrives; if upon arrival the customer sees the bank teller occupied he willimmediately leave. If we assume that the amount of time spent in the bank byan entering customer is a random variable having distribution G, then

(a) what is the rate at which customers enter the bank?

(b) what proportion of potential customers actually enter the bank?

In answering these questions, suppose that at time 0 a customer has justentered the bank.

how do we solve it?


Limit Theorems

Example 7.9

Example 7.9

Consider the renewal process whose inter-arrival distribution is theconvolution of two exponentials; that is, F = F1 ∗ F2, where Fi(t) = 1− e−µi t ,i = 1, 2. Imagine that each renewal corresponds to a new machine being putin use, and suppose that each machine has two componentsinitiallycomponent 1 is employed and this lasts an exponential time with rate µ1, andthen component 2, which functions for an exponential time with rate µ2, isemployed. When component 2 fails, a new machine is put in use (that is, arenewal occurs). Now consider the process {X (t), t ≥ 0} where X (t) is i if atype i component is in use at time t . Calculate

(a) the probability that the machine in use at time t is using its firstcomponent.

(b) the expected excess time E[Y (t)] := E[SN(t)+1 − t ].

(c) the mean-value function.

In answering these questions, suppose that at time 0 a component 1 has justbeen employed.


Limit Theorems

Example 7.9

Example 7.9

Consider the renewal process whose inter-arrival distribution is theconvolution of two exponentials; that is, F = F1 ∗ F2, where Fi(t) = 1− e−µi t ,i = 1, 2. Imagine that each renewal corresponds to a new machine being putin use, and suppose that each machine has two componentsinitiallycomponent 1 is employed and this lasts an exponential time with rate µ1, andthen component 2, which functions for an exponential time with rate µ2, isemployed. When component 2 fails, a new machine is put in use (that is, arenewal occurs). Now consider the process {X (t), t ≥ 0} where X (t) is i if atype i component is in use at time t . Calculate

(a) the probability that the machine in use at time t is using its firstcomponent.

(b) the expected excess time E[Y (t)] := E[SN(t)+1 − t ].

(c) the mean-value function.

In answering these questions, suppose that at time 0 a component 1 has justbeen employed.

how do we solve it?


Limit Theorems

Example 7.10

Example 7.10

Two machines continually process an unending number of jobs. The time thatit takes to process a job on machine 1 is a Gamma random variable withparameters n = 4, λ = 2, whereas the time that it takes to process a job onmachine 2 is Uniformly distributed between 0 and 4. Approximate theprobability that together the two machines can process at least 90 jobs bytime t = 100.


Limit Theorems

Example 7.10

Example 7.10

Two machines continually process an unending number of jobs. The time thatit takes to process a job on machine 1 is a Gamma random variable withparameters n = 4, λ = 2, whereas the time that it takes to process a job onmachine 2 is Uniformly distributed between 0 and 4. Approximate theprobability that together the two machines can process at least 90 jobs bytime t = 100.

how do we solve it?


Renewal Reward Processes


Consider a renewal process {N(t), t ≥ 0} having inter-arrival times{X1,X2, . . .} and suppose that each time a renewal occurs we receive areward. We denote by Rn, the reward earned at the time of the n-th renewal.We assume that the Rn, n ≥ 1, are iid r.v. If we let

R(t) =N(t)∑n=1

Rn

then R(t) represents the total reward earned by time t . Let

E[R] = E[Rn], E[X ] = E[Xn]

Proposition (7.3)If E[R] <∞ and E[X ] <∞, then

a) limt→∞

R(t)t = E[R]

E[X ](w.p.1)

b) limt→∞

E[R(t)]t = E[R]

E[X ]



Example 7.11

Example 7.11

Suppose that potential customers arrive at a single-server bank inaccordance with a Poisson process having rate λ. However, suppose that thepotential customer will enter the bank only if the server is free when hearrives. That is, if there is already a customer in the bank, then our arriver,rather than entering the bank, will go home. If we assume that the time spentin the bank by an entering customer is a random variable having distributionG, and

that each customer that enters makes a deposit and that the amounts thatthe successive customers deposit in the bank are iid r.v. having a commondistribution H, then the rate at which deposits accumulate — that is,limt→∞(total deposits by the time t)/t — is given by

E[deposits during a cycle]E[time of cycle]

=µH

µG + 1/λ

where µG + 1/λ is the mean time of a cycle, and µH is the mean of thedistribution H.



Example 7.16

Example 7.16 (The Average Age of a Renewal Process)

Consider a renewal process having inter-arrival distribution F and defineA(t) := t − SN(t) at time t . We are interested in

lims→∞

∫ s0 A(t)dt

s= average value of age



Example 7.16

Example 7.16 (The Average Age of a Renewal Process)

Consider a renewal process having inter-arrival distribution F and defineA(t) := t − SN(t) at time t . We are interested in

lims→∞

∫ s0 A(t)dt

s= average value of age

SolutionAssume that

∫ s0 A(t)dt represents our total earnings by time s:

lims→∞

∫ s0 A(t)dt

s→ E[reward during a renewal cycle]

E[time of a renewal cycle]

Now since the age of the renewal process a time t into a renewal cycle is justt, we have

reward during a renewal cycle =

∫ X

0tdt

where X ∼ F is the time of the renewal cycle. Hence, we have that

average value of age = lims→∞

∫ s0 A(t)dt

s=

E[X 2]/2E[X ]


Regenerative Processes


Consider a stochastic process {X (t), t ≥ 0} with state space {0, 1, 2, . . .},having the property that there exist time points at which the process(probabilistically) restarts itself. That is, suppose that with probability one,there exists a time T1, such that the continuation of the process beyond T1 isa probabilistic replica of the whole process starting at 0.

Note that this property implies the existence of further times T2,T3, . . .,having the same property as T1. Such a stochastic process is known as aregenerative process.

From the preceding, it follows that T1,T2, . . ., constitute the arrival times of arenewal process, and we shall say that a cycle is completed every time arenewal occurs.

Example

(1) A renewal process is regenerative, and T1 represents the time of the firstrenewal.

(2) A recurrent Markov chain is regenerative, and T1 represents the time ofthe first transition into the initial state.




We are interested in determining the long-run proportion of time that aregenerative process spends in state j .

To obtain this quantity, let us imagine that we earn a reward at a rate 1 perunit time when the process is in state j and at rate 0 otherwise. That is, if I(s)represents the rate at which we earn at time s, then

I(s) =

{1, if X (s) = j0, if X (s) 6= j

and

total reward earned by t =∫ t

0I(s)ds

Proposition (7.4)For a regenerative process, the long-run proportion of time in state j

=E[amount of time in j during a cycle]

E[time of a cycle]



Example 7.18

Example 7.18

Consider a positive recurrent continuous time Markov chain that is initially instate i . By the Markovian property, each time the process reenters state i itstarts over again. Thus returns to state i are renewals and constitute thebeginnings of new cycles. By Proposition 7.4, it follows that

the long-run proportion of time in state j

=E[time in j during an i–i cycle]

E[Ti,i ]

where E[Ti,i ] represents the mean time to return to state i . If we take j toequal i , then we obtain

proportion of time in state i =1/vi

E[Ti,i ]



Example 7.19

Example 7.19 (A Queueing System with Renewal Arrivals)

Consider a waiting time system in which customers arrive in accordance withan arbitrary renewal process and are served one at a time by a single serverhaving an arbitrary service distribution. If we suppose that at time 0 the initialcustomer has just arrived, then {X (t), t ≥ 0} is a regenerative process,where X (t) denotes the number of customers in the system at time t . Theprocess regenerates each time a customer arrives and finds the server free.



Alternating Renewal Processes

Alternating Renewal Processes

Consider a system that can be in one of two states: on or off. Initially it is on,and it remains on for a time Z1; it then goes off and remains off for a time Y1.It then goes on for a time Z2; then off for a time Y2; then on, and so on.

We suppose that the random vectors (Zn,Yn), n ≥ 1 are iid; but we allow Zn

and Yn to be dependent. In other words, each time the process goes on,everything starts over again, but when it then goes off, we allow the length ofthe off time to depend on the previous on time.

We are concerned with Pon, the long-run proportion of time that the system ison

Pon =E[Z ]

E[Y ] + E[Z ]=

E[on]E[off] + E[on]



Example 7.23

Example 7.23 (The Age of a Renewal Process)

Suppose we are interested in determining the proportion of time that the ageof a renewal process is less than some constant c. To do so, let a cyclecorrespond to a renewal, and say that the system is “on” at time t if the age att is less than or equal to c, and say it is “off” if the age at t is greater than c.In other words, the system is “on” the first c time units of a renewal interval,and “off” the remaining time.



Example 7.23

Example 7.23 (The Age of a Renewal Process)

Suppose we are interested in determining the proportion of time that the ageof a renewal process is less than some constant c. To do so, let a cyclecorrespond to a renewal, and say that the system is “on” at time t if the age att is less than or equal to c, and say it is “off” if the age at t is greater than c.In other words, the system is “on” the first c time units of a renewal interval,and “off” the remaining time.

SolutionLet X denote a renewal interval, we have

proportion of time age is less than c =E[min(X , c)]

E[X ]

=

∫∞0 P[min{X , c} > x ]dx

E[X ]

=

∫ c0 P[X > x ]dx

E[X ]

=

∫ c0 (1− F (x))dx

E[X ]



Example 7.23

Summary Renewal theory

1 Definition of a Renewal process

2 Distribution of N(t)The mean-value function

3 Limit TheoremsExample 7.7Example 7.9Example 7.10

4 Renewal Reward ProcessesExample 7.11Example 7.16

5 Regenerative ProcessesExample 7.18Example 7.19Alternating Renewal ProcessesExample 7.23



Example 7.23

Exercises

Introduction to Probability ModelsHarcourt/Academic Press, San Diego, 9th ed., 2007Sheldon M. Ross

Chapter 7

Sections 7.1, 7.2, 7.3, 7.4, 7.5

Exercises: 2, 4, 5, 10, 11, 12, 15, 19, 26, 32, 37, 44

Renewal theory and its applications - win.tue.nlresing/isp/ISP4.pdfRenewal theory and its...

Documents

Transcript of Renewal theory and its applications - win.tue.nlresing/isp/ISP4.pdfRenewal theory and its...