Shock model in Markovian environment

Shock Model in Markovian Environment*

Gang Li,1 Jiaowan Luo2

1 School of Mathematics and Statistics, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario K1S 5B6, Canada

2 School of Mathematics, Central South University, Changsha, Hunan 410075, People’s Republic of China

Received 31 March 2004; accepted 19 December 2004DOI 10.1002/nav.20068

Published online 3 February 2005 in Wiley InterScience (www.interscience.wiley.com).

Abstract: A Markov modulated shock models is studied in this paper. In this model, both the interarrival time and the magnitudeof the shock are determined by a Markov process. The system fails whenever a shock magnitude exceeds a pre-specified level �.Nonexponential bounds of the reliability are given when the interarrival time has heavy-tailed distribution. The exponential decayof the reliability function and the asymptotic failure rate are also considered for the light-tailed case. © 2005 Wiley Periodicals, Inc.Naval Research Logistics 52: 253–260, 2005.

Keywords: shock model; semi-Markov; reliability; martingale; bounds; asymptotic

1. INTRODUCTION

A common assumption in the classical shock models isthat the shock arrival is a renewal process, the magnitude ofshocks are i.i.d. and independent of the shock arrival pro-cess. However, in many practical problems, the interarrivaltime and the magnitude are correlated to each other and notnecessarily have the same distribution. To provide morerealistic models for this kind of problems, recently, someshock models with correlated element are introduced [5, 6,8, 12, 15]. In these papers, for shock models with somecorrelation structure, the optimal replacement policies (seeFeldman [5, 6] and Zucherman [15]) and system life distri-bution (see Igaki, Sumita, and Kowada [8], and Gut andJurg [7]) are studied. Here, we are going to study a shockmodel in Markovian environment, which incorporate theinfluence of external system states on the shock magnitudesand shock interarrival times by introducing in a Markovchain. In this model, both the interarrival time and themagnitude of the shock are governed by a Markov process.The shock arrives according to a Markov renewal process,the magnitude is determined by the state of a Markovprocess, the system fails whenever the magnitude of one

shock exceeds a prespecified level. Of interest is the lifedistribution of the system. A similar model has been studiedby Igaki, Sumita, and Kowada [8], where the Laplace trans-form of the system life distribution is given under theassumption that the joint distribution of interarrival time andshock magnitude is absolutely continuous and has partialderivative. In this paper, without those conditions, we con-sider the bounds and asymptotic property of the system lifedistribution by another approach.

In studying the life distribution of an extreme shocksystem, the interarrival time distribution plays a critical role.To some extent, it dominates the property of the extremeshock system’s life distribution. While there are two typesof interarrival distributions: light-tail and heavy-tail distri-bution, most existing works merely assume light tailedinterarrival distribution, such as exponential, normal distri-bution. That doesn’t mean that heavy-tail interarrival distri-bution is not necessary in reliability. In fact, many shockarrival processes exhibit heavy-tail behavior. For example, arepairable system may have a Pareto, log-normal, or heavy-tailed Weibull interfailure distribution [3, 10, 11], andheavy-tail distribution has properties that are qualitativelydifferent from light-tailed distribution. Conclusions reachedunder light-tail assumptions may be misleading or incorrectin the presence of heavy-tailed distributions. Consequently,it is of particular significance to analysis a shock model withheavy-tailed interarrival distribution. In present paper, wewill study both light-tail and heavy-tail cases. For the light-

* This research is supported in part by NSFC under Grant No.10301036.

Correspondence to: G. Li ([email protected]); J. Luo([email protected])

© 2005 Wiley Periodicals, Inc.

tail case, some asymptotic results of the reliability functionis proved. For the heavy-tail case, some bounds of thereliability function are provided.

As application, this model can be applied to the reliabilityevaluation of repairable system, networking traffic, powersystem, dam and so on.

The rest of this paper is organized as follows. In Section2, we will describe the shock model in detail. Some upperbounds of the reliability function are derived in Section 3.Section 4 is devoted to study the exponential decay of thereliability function. The asymptotic failure rate of the sys-tem is yielded in Section 5.

2. MODEL

Consider a shock model {( Jn, Zn, Yn), n � 0, 1, . . . },where { Jn} denotes the state of the environment, Zn � 0denotes the interarrival time between the (n � 1)th and nthshock, Yn � 0 is the magnitude of the nth shock, especially,Y0 � Z0 � 0.

Denote

Sn � �k�0

n

Zn.

Assume {( Jn, Sn), n � 0, 1, . . .} is a Markov renewalprocess, where { Jn} is an irreducible Markov chain withstate space � � {1, 2, . . . , M} and transition matrix

P � �pij�M�M;

the semi-Markov kernel is

Fij�x� � Pr�Sn�1 � Sn � x, Jn�1 � j�Jn � i�.

Here, we assume Fij( x) is nonlattice. Let

pij � �Fij� – Fij��.

and

F� i�t� � 1 � Fi�t� � �j�1

M

Fij�t�, i � 1, 2, . . . , M.

The nth shock comes at time Sn, which causes a damageYn � 0. Yn depends on Jn. Given Jn � i, Yn has adistribution

PrYn � x�Jn � i � Gi�x�,

and Yn is conditionally independent of Zn given Jn.The system fails whenever the magnitude of a shock

exceeds a prespecified deterministic level � � 0.Let

p�i� � 1 � q�i� � Pr� � Y1�J1 � i,

q � min1�i�M

q�i�, q � max1�i�M

q�i�, (1)

and

T � minSn�� Yn.

Given the initial state i, the reliability function is denoted as

Ri�t� � PrT � t�J0 � i.

Let � � (�1, �2, . . . , �M) be the steady state distributionof the semi-Markov process which is associated with {( Jn,Sn), n � 0, 1, . . . }; denote

R�t� � PrT � t � �i�1

M

�iRi�t�

as the reliability function under stationary environment. Inthe following, let Pr� denote the conditional probabilitygiven the initial distribution �, Pri denote the conditionalprobability given the initial state i. Ei and E� are thecorresponding conditional expectations.

The number of the shocks which arrive before time t isdenoted as

N�t� � maxn�Sn � t.

3. SOME BOUNDS FOR THE RELIABILITYFUNCTION

In this section, By constructing a super-Martingale, wewill give some bounds for the life distribution of the shockmodel.

First, we give a lemma about the reliability function. Inthe following, we let �k�1

f p(k) � 1, if f 1.

LEMMA 1: For the model described in previous section,the reliability function given J0 � i is

Ri�t� � Ei��k�1

N�t�

q�Jk�� (2)

254 Naval Research Logistics, Vol. 52 (2005)

and the reliability function is

R�t� � E��k�1

N�t�

q�Jk��. (3)

PROOF: Since

Ri�t� � PrT � t�J0 � i

� �n�0

� �lk�1

k�1,2. . .n

M

PrT � t, N�t� � n, Jk � lk, k

� 1, 2, . . . , n � J0 � i}

� �n�0

� �lk�1

k�1,2,. . .,n

M �k�1

n

q�lk�PrN�t� � n, Jk � lk, k

� 1, 2, . . . , n�J0 � i}

� Ei��k�1

N�t�

q�Jk��,

by the same way, (2) is proved. �

To prove the main result in this section, we need thefollowing lemma. It is shown that under some conditions,N(t), t � 0 is finite with probability 1.

LEMMA 2: Suppose Fij(0) pij, i, j, � 1, 2, . . . , M,then Pr(N(t) �) � 1.

PROOF: Denote

Gij�x� – PrZ1 � x�J0 � i, J1 � j,

then

Gij�x� �Fij�x�

pij,

so, Fij(0) pij, i, j, � 1, 2, . . . , M, this just meansGij(0) 1. Choose bij � 0 and 1 � �ij � 0 such that

1 � Gij�bij� � �ij.

Let

� � min1�i,j�M

�ij, b � min1�i,j�M

bij;

thus

1 � Gij�b� � �.

Construct an independent renewal process N(t) with inter-arrival distribution

G�x� � � 1 � �, x � b,1, x � b.

Obviously

G�x� � Gij�x�, i, j, � 1, 2, . . . , M.

Hence, there exist a version of N(t) and N(t) [13] such that

PrN�t� � N�t� � 1, t � 0.

But

PrN�t� � n � � 0, n � �t�,��t��1�1 � ��n��t�, n � �t�,

where

�t� � t

b.

Since EN(t) � (see [13]), it is easy to see that

PrN�t� � � � PrN�t� � � � 1.

This completes the proof. �

REMARK 3: Similar to the proof in last lemma, we canconstruct a renewal process to bound the Markov renewalprocess when the probability measure is changed to Pri orPr�; therefore, we also have Pri(N(t) �) � 1, i � 1,2, . . . , M and Pr�(N(t) �) � 1.

To construct a super-Martingale, we need introduce aclass of function first. This kind of function is widelyapplied in reliability study, many distribution functions be-long to this class.

DEFINITION 4: A function L : [0, �) 3 [0, �) iscalled an NBU (NWU) function if L is nonincreasing,positive, with L(0) � 1, L(�) � 0, and

255Li and Luo: Shock Model in Markovian Environment

B�x y� � ��B�x�B�y�.

The following theorem is one of our main results in thepaper. It gives the upper bound of the reliability function.

THEOREM 5: Assume that there exists an NWU func-tion L such that

�0

�

L�1�x� dFi�x� �1

q�i�

for any 1 � i � M and Fij(0) pij, i, j, � 1, 2, . . . , M,then

R�t� �L�t�

q. (4)

PROOF: First, we construct a super-martingale

U0 � 1, Un � �k�1

n

�L�1�Zk�q�Jk�1��, n � 1, 2, . . . .

Since

E�Un�1�J0, Z0, J1, Z1, . . . , Jn, Zn

� �k�1

n

�L�1�Zk�q�Jk�1��E��L�1�Zn�1�q�Jn� � Jn�

� �k�1

n

�L�1�Zk�q�Jk�1�� Un, (5)

Un is a super-martingale with respect to {( Ji, Zi), i � 0, 1,2, . . . }.

Let

��t� � N�t� 1, t � 0.

Note that for any n � �

� ��t, � � n � � Sn�1� � � t, Sn� � � t,

�(t) is a stop time with respect to {( Ji, Zi), i � 0, 1,2, . . . ,}.

Thus by the optimal stop theory (see [4, 9], Lemma 2, and(2)),

1 � E�U0 � E�U� � E� �k�1

��t�

�L�1�Zk�q�Jk�1��

� qE��L�1�S��t�� k�2

��t�

q�Jk�1�� qR�t�L�1�t�,

where the first inequality follows from the NWU property offunction L and (1), the second inequality follows fromLemma (2). Then (4) follows. �

Correspondingly, we have the bounds for conditionalreliability given the initial state of the Markov chain.

THEOREM 6: Under the condition of Theorem 5, wehave

Ri�t� �q�i�Ei�L

�1�Z1��

qL�t�.

PROOF: The same notations as in the proof of Theorem5 are used here. Similarly, we can prove that (5) also holdwhen E� is changed to Ei. So Un is a super-martingale withrespect to {( Ji, Zi), i � 0, 1, 2, . . . }. By the optimal stoptheory, we have

q�i�Ei�L�1�Z1�� EiU1 � J0 � i � EU�

� Ei��k�1

��t�

�L�1�Zk�q�Jk�1�� qEi�L�1�S��t��

k�2

��t�

q�Jk�1�� qRi�t�L�1�t�.

This complete the proof. �

By condition on Z�(t), we can refine the bounds in The-orems 5 and 6.

THEOREM 7: Under the condition of Theorem 5, wehave

Ri�t� �q�i�Ei�L

�1�Z1��

q��t� (6)

and

R�t� ��t�

q, (7)


where

��1�t� � inf0�w�t1�i�M

�L�t � w��1 �w� �L�y��1dFi�y�

F� i�w�.

PROOF: Similar to the proof of Theorem 6, by theoptimal stop theory, we have

q�i�Ei�L�1�Z1�� EiU1 � EiU�

� Ei��k�1

��t�

�L�1�Zk�q�Jk�1�� qEi��L�SN�t��L�Z��t��

�1 �k�2

��t�

q�Jk�1�� qRi�t��

�1�t�; (8)

denote W(t) � t � SN(t) and

� � J0 � i, Jm � jm, ��t� � k, W�t� � w,

m � 1, 2, . . . , k � 1.

The last inequality in (8) holds because

Ei��L�SN�t��L�Z��t��1 �

k�2

��t�

q�Jk�1� �� Ei��L�t � w�L�Zk��

�1 �m�2

k

q�jm�1� �� L�1�t � w� �

m�2

k

q�jm�1�Ei�L�Zk��1 � �

� ��1�t� �m�2

k

q�jm�1�.

This completes the proof of (6). (7) can be proved in thesame way. �

REMARK 8: For all of the three bounds in this section,the bound in Theorem 7 is the tightest; and the bound inTheorem 6 is better than that in Theorem 5. That meansmore information about the initial state or the interarrivaldistribution leads to a much tighter bound.

EXAMPLE 9: Consider the lifetime of a dam, which isaffected by earthquakes. Suppose the magnitude of an earth-

quake can be classed into two types, denoted as {1, 2}.Suppose the magnitude of the earthquake and its interarrivaltime constitute a Markov renewal process [1] with Markovkernel:

Fij�x� � pij�1 � �1 x

��1��i� , � � 0, �i � 0.

[Here, assuming the intershock time has Pareto distributionmeans that a earthquake may not happen for a long timewith a higher probability (comparing with light-tailed dis-tribution).] And, assume the dam will break down withpossibility 0 1 � qi 1 when an earthquake withmagnitude i � {1, 2} happen. Thus, this is a Markov-modulated shock model.

Suppose pi� �i; let

L1�x� � �1 x

��1��

,

where

� � sup�� 0 �1 �i

�i � ��

1

qi, i � 1, 2,

then

�0

� �1 x

��1��

dFi�x� �1 �i

�i � ��

1

qi.

It is easy to prove that L1(t) is an NWU function; then byTheorem 5

R�t� �L1�t�

minq1, q2,

by Theorem 6

Ri�t� �qi�1 �i�

��i � �� minq1, q2L1�t�, i � 1, 2,

and by Theorem 7

Ri�t� �qi�1 �i�

��i � �� minq1, q2

� �

1 �1 t

��1��

,

i � 1, 2,

where


� max�1, �2.

REMARK 10: Since the negative exponential function isan NWU function, all of the bounds can be a negativeexponential function as long as the moment generatingfunction exist.

3. ASYMPTOTIC PROPERTY OF THERELIABILITY

In this section, we will study the exponential decay of thereliability Ri(t). First, we construct a Markov renewal equa-tion.

LEMMA 11: The conditional reliability of the systemRi(t), i � 1, 2, . . . , M, satisfies the following equation:

Ri�t� � F� i�t� �j�1

M �0

t

Rj�t � x�q�j� dFij�x�. (9)

PROOF: Since

Pr�Z1 � t, T � t, J1 � j � J0 � i�

� �0

t

Pr�Z1 � dx, T � t, J1 � j � J0 � i�

� �0

t

Pr�T � t � J0 � i, J1 � j, Z1 � dx,�Pr�Z1 � dx,

J1 � j � J0 � i�

� �0

t

Pr�T � t � x � J0 � j�q�j�Pr�Z1 � dx, J1 � j � J0 � i�,

then

Ri�t� � Pr�Z1 � t � J0 � i� Pr�Z1 � t, T � t � J0 � i�

� �j�1

M

Pr�Z1 � t, J1 � j � J0 � i� �j�1

M

Pr�Z1 � t, T � t, J1 � j � J0 � i�

� F� i�t� �j�1

M �0

t

Rj�t � x�q�j� dFij�x�.


Based on the renewal equation, we have following theo-rem.

THEOREM 12: Suppose there exist a � 0 such thatA � (aij)M�M has a spectral radius 1, where

aij � �0

�

exq�j� dFij�x�; (10)

then, there exist vectors v � (v1, v2, . . . , vM) and h �(h1, h2, . . . , hM)� such that

vA � v, Ah � h

and vi � 0, hi � 0, i � 1, 2, . . . , M; thus

Ri�t� hi ¥k�1

M vk �0� exF� k�x� dx

¥k,j�1M vkhj �0

� xexq�j� dFkj�x�e�t, t3 �.

PROOF: Obviously, F� i is direct Riemann integrable.Based on the renewal equation (9), this theorem is a directconclusion of the renewal theory in [2], the result is easy toget. �

EXAMPLE 13: Consider the reliability of a power sys-tem. Suppose the system has two states {1, 2} and theevolution of the states can be modeled as a semi-Markovprocess with a semi-Markov kernel [14]:

� 0 1 � �1 0.12t�e�0.12t

1 � e�0.2t 0 � .

When the state changes, a power surge will happen. Withprobability 0.22 (0.08), this may result in a failure of thesystem if the state following the change is 1 (2). Apparently,this is a Markov modulated shock model. Therefore, byTheorem 12, � 0.01446, and

A � � 0 1.18940.8408 0 � .

Choose

h � �0.7654 0.6435��, v � �0.6435 0.7654�

such that Ah � h and vA � v; then

R1�t� 1.0953e�0.01446t,


R2�t� 0.9209e�0.01446t.

4. ASYMPTOTIC FAILURE RATE OF THESYSTEM

In this section, by constructing a Markov renewal equa-tion, the asymptotic failure rate of the system is given undercertain conditions.

LEMMA 14: Suppose Fij, i, j � 1, 2, . . . , M, isabsolutely continuous and its density function is fij(t), thenthe density function ri(t) of the system lifetime T given theinitial state i is the solution of the following equation:

ri�t� � �j�1

M

p�j�fij�t� �0

t �j�1

M

rj�t � x�q�j� dFij�x�. (11)

PROOF: First we have

ri�t� � lim�30

Prt � T � t � � J0 � i

�. (12)

Since

Prt � Z1 � t �, Y1 � � � J0 � i

� �j�1

M

Prt � Z1 � t �, Y1 � �, J1 � j � J0 � i

� �j�1

M

Prt � Z1 � t � � Y1 � �, J1 � j, J0 � iPrY1

� �, J1 � j � J0 � i

� �j�1

M

Prt � Z1 � t � � J1 � j,

J0 � i}PrY1 � � � J1 � jpij

� �j�1

M Prt � Z1 � t �, J1 � j � J0 � i

pijp�j�pij

� �j�1

M

Prt � Z1 � t �, J1 � j � J0 � ip�j�

we have

Prt � T � t � � J0 � i

� Prt � Z1 � t �, Y1 � � � J0 � i �0

t

Prt � T � t �, Z1 � dx � J0 � i

� �j�1

M

Prt � Z1 � t �, J1 � j � J0 � ip�j� �0

t �j�1

M

Prt � T � t �, Z1 � dx, J1 � j � J0 � i

� �j�1

M

Prt � Z1 � t �, J1 � j � J0 � ip�j� �0

t �j�1

M

Prt � T � t � � J0 � i, J1 � j, Z1 � dxPrZ1 � dx, J1 � j � J0 � i

� �j�1

M

Prt � Z1 � t �, J1 � j � J0 � ip�j� �0

t �j�1

M

Prt � x � T � t � x � � J0 � jq�j�PrZ1 � dx, J1 � j � J0 � i

� �j�1

M

Prt � Z1 � t �, J1 � j � J0 � ip�j� �0

t �j�1

M

Prt � x � T � t � x � � J0 � jq�j� dFij�x�. (13)


Combining (12) and (13) completes the proof. �

The Markov renewal equation is proved in the lastlemma, then the limit property of the density function is adirect result of the renewal theory.

THEOREM 15: Suppose there exists a � 0 such thatA � (aij)M�M has a spectral radius 1, where

aij � �0

�

exq�j� dFij�x�; (14)

then there exist vectors v � (v1, v2, . . . , vM) and h � (h1,h2, . . . , hM)� such that

vA � v, Ah � h

and vi � 0, hi � 0, i � 1, 2, . . . , M. Furthermore,suppose exfij( x), i, j � 1, 2, . . . , M, is directly Riemannintegrable; then

ri�t� hi ¥k�1

M vk �0� exfk�x� dx

¥k,j�1M vkhj �0

� xexq�j� dFkj�x�e�t, t 3 �,

where

fi�x� � �j�1

M

p�j�fij�x�.

PROOF: Obviously, fi is directly Riemann integrable.Based on the renewal equation (11) and the renewal theory[2], we get the result. �

Combining Theorem 11 and 15, we have the followingcorollary:

COROLLARY 16: Under the condition of Theorem 15,we have

limt3�

ri�t�

Rt�t��

¥k�1M vk ¥j�1

M �p�j�lkj��

¥k�1M vk�¥j�1

M lkj�� 1�, (15)

where

lkj�� 0

�

exfkj�x� dx.

PROOF: Since

�0

�

exF� k�x� dx �¥j�1

M lkj�� 1

,

from Theorems 11 and 15,

limt3�

ri�t�

Ri�t��

hi ¥k�1M vk �0

� exfk�x� dx

hi ¥k�1M vk �0

� exF� k�x� dx

� ¥k�1

M vk ¥j�1M �p�j�lkj��

¥k�1M vk �¥j�1

M lkj�� 1�.


EXAMPLE 17: Consider Example 13. By Corollary 16,we can get the asymptotic failure rate 0.0145.

REFERENCES

[1] Y. Altinok and D. Kolcak, An application of the semi-Markov model for earthquake occurrences in North Anatolia,Turkey, J Balkan Geophys Soc 2(4) (1999), 90–99.

[2] S. Asmussen, Applied probability and queues, Springer, NewYork, 2003.

[3] W.R. Blischke and D.N.P. Murthy, Reliability: Modeling,prediction, and optimization, Wiley, New York, 2000.

[4] R.J. Elliott, Stochastic calculus and applications, Springer,New York, 1982.

[5] R.M. Feldman, Optimal replacement with semi-Markovshock models, J Appl Probab 13 (1976), 108–117.

[6] R.M. Feldman, Optimal replacement with semi-Markovshock models using discounted cost, Math Oper Res 2(1977), 78–90.

[7] A. Gut and H. Jurg, Extreme shock models, Extremes 2(3)(1999), 295–307.

[8] N. Igaki, U. Sumita, and M. Kowada, Analysis of Markovrenewal shock models, J Appl Probab 32 (1995), 821–831.

[9] S. Karlin and H.M. Taylor, A first course in stochasticprocesses, Academic, San Diego, 1975.

[10] G. Last and R. Szekli, Asymptotic and monotonicity proper-ties of some repairable systems, Adv Appl Probab 30 (1998),1089–1110.

[11] N.R. Mann, R.E. Schafer, and N.D. Singpurwalla, Methodsfor statistical analysis of reliability and life data, Wiley,Toronto, 1974.

[12] M.J.M. Posner and D. Zuckerman, Semi-Markov shock mod-els with additive damage, Adv Appl Probab 18 (1986), 772–790.

[13] S. Ross, Stochastic processes, Wiley, New York, 1983.[14] J.F.L. van Casteren, M.H.J. Bollen, and M.E. Schmieg, Re-

liability assessment in electrical power systems: TheWeibull-Markov stochastic model, IEEE Trans Ind Appl 36(2000), 911–915.

[15] D. Zucherman, Optimal stopping in a semi-Markov shockmodels, J Appl Probab 15 (1978), 629–634.


Shock model in Markovian environment

Documents

Transcript of Shock model in Markovian environment