A multivariate mixture of Weibull distributions in reliability modeling

Statistics & Probability Letters 45 (1999) 225–235www.elsevier.nl/locate/stapro

A multivariate mixture of Weibull distributions inreliability modeling

Kaushik Patra, Dipak K. Dey ∗

Department of Statistics, University of Connecticut, 196 Auditorium Road, Storrs, CT 06269-3120, USA

Received July 1998; received in revised form January 1999

Abstract

A new class of multivariate distribution is derived where each component is a mixture of Weibull distribution. Theapproach in this paper is based on the introduction of an exponentially distributed latent random variable. The new classincludes several multivariate and bivariate models including Marshall and Olkin type. The moment generating functionand, hence, the correlation structure is obtained for the bivariate situation. The distribution of the minimum in a competingrisk framework is discussed and various other properties including correlation structures are investigated. c© 1999 ElsevierScience B.V. All rights reserved

MSC: 62N05; 62H12

Keywords: Competing risk; Maximum likelihood estimation

1. Introduction

Multivariate lifetime data arise quite frequently in many types of industrial settings. For example, a pro-duction analyst might be interested in time-to-failure measurements for machinery components. The Weibulldistribution is a versatile family of life distributions to model such life data concerning many types of manu-factured items. However, common handling or a similar environment often induce dependence among thecomponents of a system. In such situations, a bivariate or a multivariate Weibull distribution is used to modeldependent components such that individual lifetime is also Weibull. There is an extensive literature now forthe bivariate and multivariate Weibull distribution; for example, Gumbel (1960); Freund (1961); Marshall andOlkin (1967, 1988); Block and Basu (1974); Clayton (1978); Sarkar (1987) and a host of others.In the context of modeling-dependent lifetimes, the bivariate exponential model of Marshall and Olkin

(1967) is by far most popular and well justi�ed on physical bases. The dependence of the component life-times from this fatal shock model, arises from simultaneous failures of both components. Lu (1989) extendedideas of Marshall–Olkin model to bivariate Weibull models. Further, Lu and Bhattacharya (1990) constructed

∗ Corresponding author.E-mail address: [email protected] (D.K. Dey)

0167-7152/99/$ - see front matter c© 1999 Elsevier Science B.V. All rights reservedPII: S0167 -7152(99)00062 -0

226 K. Patra, D.K. Dey / Statistics & Probability Letters 45 (1999) 225–235

a new family of bivariate Weibull family with Weibull marginals. However, in all the above-mentionedwork, the marginals have monotonic hazard function, e.g., in Weibull and in exponential. Often in prac-tice, the hazard function of the individual component distribution is nonmonotonic, e.g., bathtub shape. Thishappens in particular, when the individual component life can be a mixture of few subcomponent distribu-tions. To construct such nonmonotonic hazard, a natural approach is to consider the individual componentlife distribution as a mixture of Weibull distributions. Such a class of mixture distribution is quite rich,since it can incorporate wide varieties of hazard functions. There are several ways to construct mixturesof distributions, viz., using physical mixing or subjective mixing. For details, see Lynn and Singpurwalla(1997). The approach we adopt is to introduce a latent variable and de�ne the observable lifetimes asfunctions of this variable. Observed life distribution of components sharing common latent variable will bedependent.The format of the paper is as follows. In Section 2, we develop the multivariate mixture of Weibull

model using the latent variable approach. Bivariate exponential mixture model is developed as a special case.Section 3 focuses on competing risk models under Weibull and exponential mixture set up. We conclude inSection 4 with possibilities of inclusion of covariates.

2. The models

In this section we work with the mixture of Weibull distributions and derive a multivariate distribution wherethe dependence among the components is characterized by a latent random variable independently distributedof the individual component. We also develop a bivariate distribution with more details as a special case.

2.1. A multivariate mixture model

Consider an r-component system where the lifetime of ith component, say, Xi is a mixture of Weibulldistributions, i = 1; : : : ; r; i.e.,

[Xi]D=

k∑j=1

aijXik |Xij ∼ W (�ij; �ij) j = 1; : : : ; k ;

where [X ] denotes the distribution of X and the density of Xij ∼ W (�ij; �ij) is given as

fXij (x) = �ij�ijx�ij−1 exp(−�ijx�ij); x¿ 0; �ij ¿ 0; �ij ¿ 0 for all i; j (2.1)

with (ai1; : : : ; aik) is the vector of mixing probabilities corresponding to ith component, i.e., aij¿0 for all i; jand

∑kj=1 aij = 1:

Consider also an exponentially distributed random variable, Z , with scale parameter �0 which is independentof Xij for all i; j: Z will be used as a latent variable to introduce dependence among Xi’s. The density of Zis given by

fZ(z) = �0 exp(−�0z); z ¿ 0; �0¿ 0: (2.2)

Note that by independence assumption in the above model, we can say that Z is also independent of Xi forall i = 1; : : : ; r:De�ne Si=Min(Xi; Z) for i=1; 2; : : : ; r: Then the vector S=(S1; S2; : : : ; Sr) follows a multivariate distribution

and obviously they are dependent as they commonly share the in uence of the latent random variable Z . Thus,

K. Patra, D.K. Dey / Statistics & Probability Letters 45 (1999) 225–235 227

the joint multivariate survival function of S1; : : : ; Sr is given by

S(s1; : : : ; sr) = Pr(S1¿s1; : : : ; Sr ¿ sr) =r∏i=1

[Pr(Xi ¿ si){Pr(Z ¿s0)}1=r]

=r∏i=1

k∑j=1

aij exp{−(�ijs

�iji +

�0s0r

)}; (2.3)

where s0=max(s1; : : : ; sr)¿ 0: Note that the presence of s0 makes it very di�cult to calculate the multivariatedensity of (S1; S2; : : : ; Sr) as we have to take mixed derivatives over all possible partitions of the sample space.Now for simplicity, we consider the case when r = 2 and k = 2; that is under the scenario of the bivariatetwo component mixture Weibull distributions. Then, from (2.3) it follows that the joint survival function willhave the following form:

S(s1; s2) = Pr(S1¿s1; S2¿s2)

=p11e−(�11s�111 +�12s

�122 +�0s0) + p12e−(�11s

�111 +�22s

�222 +�0s0)

+p21e−(�21s�211 +�12s

�122 +�0s0) + p22e−(�21s

�211 +�22s�222 +�0s0); (2.4)

where X1 and X2 have mixture Weibull distribution denoted by

X1 ∼ [a1W (�11; �11) + (1− a1)W (�12; �12)] ;X2 ∼ [a2W (�21; �21) + (1− a2)W (�22; �22)] ;

where W denotes the Weibull distribution, and

p11 = a1a2; p12 = a1(1− a2); p21 = (1− a1)a2 and p22 = (1− a1) (1− a2):Note that p11 + p12 + p21 + p22 = 1, i.e., the resultant distribution is also a mixture of some well-de�neddistributions. Let us now calculate the bivariate density of (S1; S2), denoted by fs1 ; s2 (s1; s2): The form of thedensity given as

f(s1; s2) = f1(s1; s2) if s1¿s2

= f2(s1; s2) if s1¡s2

= f0(s0; s0) if s1 = s2 = s0;

where

f1(s1; s2) =p11(�11�11s�11−11 + �0)�21�21s

�21−12 e−(�11s

�111 +�21s

�212 +�0s1)

+p12(�11�11s�11−11 + �0)�22�22s

�22−12 e−(�11s

�111 +�22s

�222 +�0s1)

+p21(�12�12s�12−11 + �0)�21�21s

�21−12 e−(�12s

�121 +�21s

�212 +�0s1)

+p22(�12�12s�12−11 + �0)�22�22s

�22−12 e−(�12s

�121 +�22s

�222 +�0s1)

and

f2(s1; s2) =p11(�21�21s�21−12 + �0)�11�11s

�11−11 e−(�11s

�111 +�21s

�212 +�0s2)

+p12(�22�22s�22−12 + �0)�11�11s

�11−11 e−(�11s

�111 +�22s

�222 +�0s2)


+p21(�21�21s�21−12 + �0)�12�12s

�12−11 e−(�12s

�121 +�21s

�212 +�0s2)

+p22(�22�22s�22−12 + �0)�12�12s

�12−11 e−(�12s

�121 +�22s

�222 +�0s2): (2.5)

Note that f0(s0; s0) cannot be obtained by using the derivative approach, instead we obtain it using thefollowing identity:∫ ∞

0

∫ s1

0f1(s1; s2) ds2 ds1 +

∫ ∞

0

∫ s2

0f2(s1; s2) ds1 ds2 +

∫ ∞

0f0(s0; s0) ds0 = 1: (2.6)

The following theorem provides an approach of obtaining the joint bivariate density when the componentrandom variables can be equal with positive probability.

Theorem 2.1. If the bivariate cumulative density function has the following form:

S(x; y) = e−(�1x�1+�2y�2+�0z); where z =max(x; y)

then the joint density function is given by

fX;Y (x; y) = f1(x; y) if x¿y

= f2(x; y) if x¡y

= f0(x; y) if x = y;

where

f1(x; y) = (�1�1x�1−1 + �0)�2�2y�2−1e−(�1x�1+�2y�2+�0x);

f2(x; y) = (�2�2y�2−1 + �0)�1�1x�1−1e−(�1x�1+�2y�2+�0y);

f0(x; x) = �0e−(�1x�1+�2x�2+�0x):

Proof. The form of f1(x; y) and f2(x; y) can be obtained by di�erentiating S(x; y) with respect to x and y,and f0(x; x) will be obtained from the identity given in (2.6). Now,∫ ∞

0

∫ x

0f1(x; y) dy dx = 1−

∫ ∞

0(�1�1x�1−1 + �0)e−(�1x

�1+�2x�2+�0x) dx

and ∫ ∞

0

∫ y

0f2(x; y) dx dy = 1−

∫ ∞

0(�2�2x�2−1 + �0)e−(�1x

�1+�2x�2+�0x) dx:

However,

1−[∫ ∞

0

∫ x

0f1(x; y) dy dx +

∫ ∞

0

∫ y

0f2(x; y) dx dy

]

=∫ ∞

0(�1�1x�1−1 + �2�2x�2−1 + �0)e−(�1x

�1+�2x�2+�0x) dx

+∫ ∞

0�0e−(�1x

�1+�2x�2+�0x) dx − 1

=∫ ∞

0�0e−(�1x

�1+�2x�2+�0x) dx =∫ ∞

0f0(x; x) dx

implies f0(x; x) = �0e−(�1x�1+�2x�2+�0x) x¿ 0; which completes the proof.


Following Theorem 2.1 we see that in our original set-up

f0(s0; s0) =p11�0e−(�11s�110 +�21s

�210 +�0s0) + p12�0e−(�11s

�110 +�22s

�220 +�0s0)

+p21�0e−(�12s�120 +�21s

�210 +�0s0) + p22�0e−(�12s

�120 +�22s

�220 +�0s0): (2.7)

Next we consider as a special case the model for bivariate mixture of exponential distributions.

2.2. Bivariate mixture of exponential distributions

In this section besides deriving the form of the bivariate mixture exponential distributions, we also derivethe joint moment generating function to study dependence structure. The model for exponential distributionis obtained from (2.5) and (2.6) by simply putting �ij = 1 for all i and j: Thus, the density is given by

f1(s1; s2) =p11(�11 + �0)�21e−(�11s1+�21s2+�0s1) + p12(�11 + �0)�22e−(�11s1+�22s2+�0s1)

+p21(�12 + �0)�21e−(�12s1+�21s2+�0s1) + p22(�12 + �0)�22e−(�12s1+�22s2+�0s1);

f2(s1; s2) =p11(�21 + �0)�11e−(�11s1+�21s2+�0s2) + p12(�22 + �0)�11e−(�11s1+�22s2+�0s2)

+p21(�21 + �0)�12e−(�12s1+�21s2+�0s2) + p22(�22 + �0)�12e−(�12s1+�22s2+�0s2)

and

f0(s0; s0) =p11�0e−(�11+�21+�0)s0 + p12�0e−(�11+�22+�0)s0

+p21�0e−(�12+�21+�0)s0 + p22�0e−(�12+�22+�0)s0 :

Now, the moment generating function, denoted by M (t1; t2) is given as

M (t1; t2) = E[exp(t1S1 + t2S2)]

=∫ ∞

0

∫ s1

0exp(t1s1 + t2s2)f1(s1; s2) ds2 ds1

+∫ ∞

0

∫ s2

0exp(t1s1 + t2s2)f2(s1; s2) ds1 ds2

+∫ ∞

0exp{(t1 + t2)s0}f0(s0; s0) ds0:

After lengthy algebraic manipulation, it can be shown that

M (t1; t2) =p11

(�11 + �21 + �0 − t1 − t2){�21(�11 + �0)�21 − t2 +

�11(�21 + �0)�11 − t1 + �0

}

+p12

(�11 + �22 + �0 − t1 − t2){�22(�11 + �0)�22 − t2 +

�11(�22 + �0)�11 − t1 + �0

}

+p21

(�12 + �21 + �0 − t1 − t2){�21(�12 + �0)�21 − t2 +

�12(�21 + �0)�12 − t1 + �0

}

+p22

(�12 + �22 + �0 − t1 − t2){�22(�12 + �0)�22 − t2 +

�12(�22 + �0)�12 − t1 + �0

}: (2.8)


Note that the univariate moment generating functions can be obtained by considering the marginal distributionsof S1 and S2 which are given, respectively, as

fS1 (s1) = a1(�11 + �0)e−(�11+�0)s1 + (1− a1) (�12 + �0)e−(�12+�0)s1 ; s1¿ 0

and

fS2 (s2) = a2(�21 + �0)e−(�21+�0)s2 + (1− a2) (�22 + �0)e−(�22+�0)s2 ; s2¿ 0:

Thus the moment generating function of S1 is

M1(t1) =a1(�11 + �0)(�11 + �0 − t1) +

(1− a1) (�12 + �0)(�12 + �0 − t1) :

Similar expression can be obtained for the moment generating function of S2: Thus it follows after somealgebraic manipulation that

E(S1) =a1

�11 + �0+1− a1�12 + �0

;

Var(S1) =1

(�11 + �0)2+

1(�12 + �0)2

−(

a1�12 + �0

+1− a1�11 + �0

)2;

E(S2) =a2

�21 + �0+1− a2�22 + �0

;

Var(S2) =1

(�21 + �0)2+

1(�22 + �0)2

−(

a2�22 + �0

+1− a2�21 + �0

)2and

Cov(S1; S2) =p11�0

(�11 + �21 + �0) (�11 + �0) (�21 + �0)+

p12�0(�11 + �22 + �0) (�11 + �0) (�22 + �0)

+p21�0

(�12 + �21 + �0) (�12 + �0) (�21 + �0)+

p22�0(�12 + �22 + �0) (�12 + �0) (�22 + �0)

¿ 0: (2.9)

Expression (2.9) indicates that S1 and S2 are positively dependent, which is to be expected by mixtures. SeeShaked (1977) for details.

3. Competing risk models

Competing risk models arise in situations in which components fail from several di�erent causes. In suchsituations each system failure is caused by only one of the competing risks. Here we describe our modelswhen each competing risk has a mixture of Weibull or exponential distribution.

3.1. Mixture of Weibull distributions

In this section, we develop the distribution of the minimum. Let Xi be a mixture of Xi1; : : : ; Xik each withWeibull (�; �ij) and the mixing probability ai1; : : : ; aik ; i.e.,

∑kj=1 aij = 1 for i= 1; 2; : : : ; r. Next, consider one


latent random variable Z ∼Weibull(�; �0) independent of X1; X2; : : : ; Xr . De�ne

T =min(X1; X2; : : : ; Xr; Z):

Then

P(T ¿ t) = P(X1¿t)P(X2¿t); : : : ; P(Xr ¿ t)P(Z ¿ t)

=

{r∏i=1

k∑j=1

aije−t�=�ij

}e−t

�=�0

=r∏i=1

{k∑j=1

aije−t�((1=�ij)+(1=r�0))

}

=r∏i=1

P(Ti ¿ t)

where Ti ∼ Mixture[Weibull(�; �ij) | ai1; : : : ; aik ] and �−1ij =((1=�ij)+(1=r�0)). Thus it immediately follows that

ST (t) =r∏i=1

STi(t):

Therefore, the density and the hazard functions of T are given as

fT (t) =ddtP(T¿t)

=−P(T ¿ t)r∑i=1

1P(Ti ¿ t)

ddtP(Ti ¿ t)

=r∑i=1

P(T ¿ t)P(Ti ¿ t)

fTi(t)

and

hT (t) =fT (t)ST (t)

=r∑i=1

hTi(t):

The expectation of T is then given as

E(T ) =∫ ∞

0ST (t) dt =

∫ ∞

0

r∏i=1

STi(t) dt

=∫ ∞

0

r∏i=1

e−HTi (t) dt =∫ ∞

0e−

∑iHTi (t) dt;

where HTi(t) is the integrated hazard of Ti; i = 1; : : : ; r.


3.2. Mixture exponential

In this section we develop the distribution of the minimum and the corresponding hazard function for themixture of exponential distributions. De�ne S =Min(S1; : : : ; Sr). Then

S(s) = P(S ¿s) = P(Min(S1; : : : ; Sr)¿s)

= P(Z ¿s)r∏i=1

P(xi ¿ s)

=r∏i=1

k∑j=1

aije−(s�ij�ij+S�0=r);

=r∏i=1

P(Ti ¿ s) =r∏i=1

Si(s) (say);

(3.1)

where fTi(t) =∑k

j=1 aij(�ij�ijt�ij−1 + �0=r)e−(�ij�ij t

�ij+t�0=r); t ¿ 0. Thus, the density of S reduces to

fS(s) =S(s)r∑i=1

fTi(s)Si(s)

: (3.2)

Thus the hazard function corresponding to S reduces to

hS(s) =r∑i=1

hTi(s)

where

hTi(s) =

∑kj=1 aij(�ij�ijs

�ij−1 + �0=r)e−(�ijs�ij+s�0=r)∑k

j=1 aije−(�ijs�ij+s�0=r)

is the hazard of Ti; i = 1; : : : ; r:Let us now consider the special cases when �ij = 1 for all i; j; k = 2 and r = 2. Then

fT1 (t) = a1

(�11 +

�02

)e−(�11+(�0=2))t + (1− a1)

(�12 +

�02

)e−(�12+(�0=2))t ;

fT2 (t) = a2

(�21 +

�02

)e−(�21+(�0=2))t + (1− a2)

(�22 +

�02

)e−(�22+(�0=2))t

and

S(t) = p11e−(�11+�12+�0)t + p12e−(�11+�22+�0)t + p21e−(�12+�21+�0) + p22e−(�12+�22+�0) t ¿ 0:

Thus, in particular, under Marshall–Olkin model, we have

a1 = a2 = 1; i:e:; p11 = 1; pij = 0 ∀i; j 6= 1and

�11 = �1; �21 = �2; �0 = �0:


Then the joint pdf of S1 and S2 is

fS1 ; S2 (s1; s2) =

(�1 + �0)e−(�1s1+�2s2+�0s1) s1¿s2¿ 0

(�2 + �0)e−(�1s1+�2s2+�0s2) s2¿s1¿ 0

�0e−(�1+�2+�0)s0 s1 = s2 = s0¿ 0:

It follows that the joint moment generating function is

Ms1 ; s2 (t1; t2) =1

(�1 + �2 + �0 − t1 − t2)[�2(�1 + �0)(�2 − t2) +

�1(�2 + �0)(�1 − t1) + �0

]:

Then simple calculation produces

Cov(S1; S2) =�0

(�1 + �2 + �0) (�1 + �0) (�2 + �0);

Var(S1) =1

(�1 + �0)2; Var(S2) =

1(�2 + �0)2

;

and

�(S1; S2) = Corr(S1; S2) =�0

�1 + �2 + �0¿ 0:

Now, under the set up of Lee (1979), we have k =1 and r=2. Thus the joint survival and density functionsare given as

S(s1; s2) = exp[− �1s�11 − �2s�22 − �0s0]and

fs1 ; s2 (s1; s2) =

(�1�1s

�1−11 + �0)�2�2s

�2−12 e−(�1s

�11 +�2s

�22 +�0s1) s1¿s2¿ 0

(�2�2s�2−12 + �0)�1�1s

�1−11 e−(�1s

�11 +�2s

�22 +�0s2) s2¿s1¿ 0

�0e−(�1s�10 +�2s

�20 +�0s0) s1 = s2 = s0¿ 0

:

Thus, the marginal pdf of the minimum is

fS(s) = p11(�11 + �12 + �0)e−(�11+�12+�0)s + p12(�11 + �22 + �0)e−(�11+�22+�0)s

+p21(�12 + �21 + �0)e−(�12+�21+�0)s + p22(�12 + �22 + �0)e−(�12+�22+�0)s; s¿ 0:

Now, the moment generating function of the minimum is

Ms(t) =∫ ∞

0e−tsfs(S) ds

=p11(�11 + �12 + �0)(�11 + �12 + �0 − t) +

p12(�11 + �22 + �0)(�11 + �22 + �0 − t)

+p21(�12 + �21 + �0)(�12 + �21 + �0 − t) +

p22(�12 + �22 + �0)(�12 + �22 + �0 − t) :


It follows that

E(Sr) = r![

p11(�11 + �12 + �0)r

+p12

(�11 + �22 + �0)r+

p21(�12 + �21 + �0)r

+p22

(�12 + �22 + �0)r

]r = 1; 2; 3; : : :

In particular,

E(S) =p11

�11 + �12 + �0+

p12�11 + �22 + �0

+p21

�12 + �21 + �0+

p22�12 + �22 + �0

and

E(S2) =2p11

(�11 + �12 + �0)2+

2p12(�11 + �22 + �0)2

+2p21

(�12 + �21 + �0)2+

2p22(�12 + �22 + �0)2

:

Finally, the expected residual life is given as

E(S − s | S ¿s) =

∫∞s S(u) du

S(s)

where ∫ ∞

sS(u) du =

p11e−(�11+�12+�0)s

(�11 + �12 + �0)+p12e−(�11+�22+�0)s

(�11 + �22 + �0)+p21e−(�12+�21+�0)s

(�12 + �21 + �0)+p22e−(�12+�22+�0)s

(�12 + �22 + �0):

4. Conclusion

The class of models developed in this paper provide methodology to analyze many interesting real dataarising from industrial and medical �eld. Our proposed models can be generalized in many directions. First,�i (i = 1; : : : ; r) may depend on a covariate vector x through a functional form such as �ig(xt�), where g(·)is some known non-negative function and � is a vector of coe�cients. We can allow �i to vary over time as�i(t) and then consider some reasonable prior process on �i(t); i=1; : : : ; r. We can further assume some otherdistribution for Z , beside exponential. We can still handle the situation using our data augmentation scheme.In this paper we describe our methodology using frequentist approach. A Bayesian approach using sampling-

based technique will be pursued elsewhere.

Acknowledgements

The authors would like to thank the referee for many helpful comments which improved the presentationof the paper.

References

Block, H.W., Basu, A.P., 1974. A continuous bivariate exponential extension. J. Amer. Statist. Assoc. 69, 1031–1037.Clayton, D.G., 1978. A model for association in bivariate life tables and its application in epidemiological studies of familial tendencyin chronic disease incidence. Biometrika 65, 141–151.

Freund, J.E., 1961. A bivariate extension of the exponential distribution. J. Amer. Statist. Assoc. 971–977.Gumbel, E.J., 1960. Bivariate exponential distributions. J. Amer. Statist. Assoc. 55, 698–707.Lee, L., 1979. Multivariate distributions having Weibull properties. J. Multivariate Anal. 9, 267–277.Lu, J., 1989. Weibull extensions of the Freund and Marshall–Olkin bivariate exponential. IEEE Trans. Reliability 38, 615–619.


Lu, J., Bhattacharya, G.K., 1990. Some new constructions of bivariate Weibull models. Ann. Inst. Statist. Math. 42, 543–559.Lynn, N.J., Singpurwalla, N.D., 1997. Comment: “Burn-in” makes us feel good. Statist. Sci. 12, 13–19.Marshall, A.W., Olkin, I., 1967. A multivariate exponential distribution. J. Amer. Statist. Assoc. 30–44.Marshall, A.W., Olkin, I., 1988. Families of multivariate distributions. J. Amer. Statist. Assoc. 834–841.Sarkar, S.K., 1987. A continuous bivariate exponential distribution. J. Amer. Statist. Assoc. 82, 667–675.Shaked, M., 1977. A concept of positive dependence for exchangeable random variables. Ann. Statist. 5, 505–515.

A multivariate mixture of Weibull distributions in reliability modeling

Documents

Transcript of A multivariate mixture of Weibull distributions in reliability modeling