This article was downloaded by: [North West University]On: 22 December 2014, At: 16:50Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Statistical Computation andSimulationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/gscs20

Analysis of strength distributions ofmulti-modal failures using the EMalgorithmChanseok Park a & W. J. Padgett ba Department of Mathematical Sciences , Clemson University ,Clemson, SC, 29634, USAb Department of Statistics , University of South Carolina ,Columbia, SC, 29208, USAPublished online: 01 Feb 2007.

To cite this article: Chanseok Park & W. J. Padgett (2006) Analysis of strength distributions ofmulti-modal failures using the EM algorithm, Journal of Statistical Computation and Simulation,76:07, 619-636, DOI: 10.1080/10629360500108970

To link to this article: http://dx.doi.org/10.1080/10629360500108970

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Journal of Statistical Computation and SimulationVol. 76, No. 7, July 2006, 619–636

Analysis of strength distributions of multi-modal failuresusing the EM algorithm

CHANSEOK PARK*† and W. J. PADGETT†‡

†Department of Mathematical Sciences, Clemson University, Clemson, SC 29634, USA‡Department of Statistics, University of South Carolina, Columbia, SC 29208, USA

(Revised 14 June 2004; in final form 1 February 2005)

Analysis of various multi-modal strength distributions are studied by using competing risks models.This multi-modality may arise due to several kinds of flaws in a material. The fracture of a materialis controlled by the most severe of all the flaws, the so-called ‘weakest-link theory’, which is alsocommonly referred to as ‘competing risks’ in the statistics literature. These multi-modal problems canalso be further complicated due to possible censoring. In practice, censoring is very common becauseof time and cost considerations on experiments. Moreover, in certain situations, it is observed thatthe mode of failure is not properly identified due to lack of appropriate diagnostics, expensive andtime-consuming autopsy, etc. This is known as the masking problem. Several studies have been carriedout, but they have mainly focused on bi-modal Weibull distributions with no censoring or maskingconsidered.

In this paper, we deal with the strength distribution of multi-modal failures when censoring andmasking are present. We provide the EM-type parameter estimator for a variety of strength distributionsincluding Weibull, lognormal and inverse Gaussian distributions. The applicability of this method isillustrated by several examples.

Keywords: Competing risks; Censoring; Masking; EM algorithm; MLE, Missing data; Likelihoodfunction; Weibull; Lognormal; Inverse Gaussian (Wald)

1. Introduction

Knowledge of the strength of a type of material is required for engineering design of variousstructures made from such materials in order for the structures to withstand predicted stresses.To determine the strength properties, specimens of the materials are typically tested underlaboratory conditions, and appropriate statistical models are investigated in order to predictstrengths of specimens or structures of different sizes than those tested. This approach istaken, for example, in the case of modern fibrous composite materials. Due to flaws occurringat random in the material specimens under test, perhaps from various imperfections or othercauses, the tensile strength of a single specimen must be considered as a random variablewhose probability distribution depends on the various kinds of flaws that are present. Such a

*Corresponding author. Email: cspark@ces.clemson.edu

Journal of Statistical Computation and SimulationISSN 0094-9655 print/ISSN 1563-5163 online © 2006 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/10629360500108970

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

620 C. Park and W. J. Padgett

probability distribution is used to estimate strengths for the design of larger structures madefrom the material. Thus, it is very important to find appropriate statistical models that fitobserved specimen data well.

Most statistical analyses of material properties have been studied assuming that the materialstrength follows a single Weibull distribution which gives a linear Weibull plot. In contrast, ithas been frequently reported by several authors that there are different modes of flaws whichdetermine the fracture of the material. Among them are Johnson and Thorne [1], Jones andWilkins [2], Layden [3], Boggio and Vingsbo [4], Beetz [5], Martineau et al. [6], Simon andBunsell [7], Chi et al. [8], Goda and Fukunaga [9], Wagner [10], Stoner et al. [11], and Meekerand Escobar [12], among others.

In the case where there are several potential modes of causes, statistical strength distributionsbased on ‘weakest-link theory’, which is also commonly referred to as ‘competing risks’ in thestatistics literature, have been developed by several authors. Goda and Fukunaga [9] analyzedthe strength distributions of silicon carbide and alumina fibers using a multi-modal Weibulldistribution, Wagner [10] also studied competing risks model, and Taylor [13] developeda Poisson–Weibull flaw model. In this context, end-effects (or clamp-effects) models weredeveloped by several authors to explain the strengths observed in very small fiber or compositespecimens [see refs. 11, 14, 15]. They, however, have mainly focused on Weibull distributionsand they did not consider censoring or masking problems. Although they stated that theirmethods extend to general multi-modal Weibull distributions, no explicit illustration wasprovided. The main reason, we think, is that the parameter estimation under the large numberof different failure modes is extremely difficult. In addition, it has been reported that thepopularly used Weibull distributions often do not provide good fits to the tensile strengthdata set. For example, for carbon fiber or composite tensile strengths, see Durham and Padgett[16]. These motivate the need for developing a highly stable parameter estimation methodologyunder various distribution models with both censoring and masking considered.

In this paper, we deal with multi-modal problems with censoring and masking under avariety of strength distributions including Weibull, lognormal, and inverse Gaussian (Wald)distributions. We provide the EM-type parameter estimator, which is fairly stable in estimationand can handle any number of failure modes. The rest of the paper is organized as follows. Insection 2, we introduce the competing risks model. We provide the general likelihood methodin section 3. Parameter estimation using the EM algorithm is described in sections 4 and 5followed up with illustrative examples in section 6.

2. Competing risks model

The analysis of lifetime data has been of considerable interest in many branches of statisticalapplications such as reliability engineering, electrical engineering, industrial engineering,biological sciences, etc. Consider a system of multiple components in a series. In this system,the failure of the whole system is caused by the earliest failure of any of the components,which is commonly referred to as competing risks. In certain situations, it is observed thatthe determination of the cause of failure may be expensive or may be very difficult to observedue to the lack of appropriate diagnostics. Therefore, it might be the case that the failure timeof an individual is observed, but its corresponding cause of failure is not fully investigated.This is known as masking. We consider that the cause of the ith system failure may or maynot be exactly identified, so the cause-of-failure leads to non-empty subset of labels definingthe component in the module. For example, if the ith system with J components fails due tothe j th component, then the set of labels is Mi = {j} (no masking); if its failure is completely

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 621

unknown, then Mi = {1, 2, . . . , J } (complete masking); and if its failure is identified by themodes containing more than one failure but not all failures, then Mi = {j1, . . . , ji} (partialmasking). Moreover, this competing risks problem is further complicated due to possiblecensoring. In practice, censoring is very common because of time and cost considerations onexperiments.

The traditional approach when dealing with competing risks is to consider the hypotheticallatent lifetimes corresponding to each cause in the absence of the others [see ref. 17]. Letthere be a finite number of independent causes of failure indexed by j = 1, . . . , J . Let T

(j)

i

denote the continuous lifetime of the ith subject due to the j th cause, where i = 1, . . . , n.It is assumed that T

(j)

i are independent for all i, j and are identically distributed for alli for given j . We denote the cdf, pdf, survival function, and hazard function of T

(j)

i byF (j)(·|θ(j)), f (j)(·|θ(j)), S(j)(·|θ(j)), and h(j)(·|θ(j)), respectively, where θ(j) is a vector ofreal valued parameters for each j . The observed lifetime of the ith subject is given by therandom variable

Ti = min{T (1)i , T

(2)i , . . . , T

(J )i }.

Typically, in reliability analysis problems, complete observation of Ti may not be possibledue to various censoring schemes that can be inherent in data collection. It is further assumedthat each Ti can be randomly right-censored by censoring times Ci which are independent oflifetimes Ti for all i. Thus, one observes triplets (Xi, �i, Mi), where Xi = min{Ti, Ci}, Mi isthe set of labels defining the components that failed and �i is a censoring indicator variabledefined as

�i =

−1 if masked

j if failed with j th cause

0 if censored.

(1)

We denote a realization of the random variable (Xi, �i) as (xi, δi).The competing risks model has been studied by several authors including Cox [18],

Herman and Patell [19], Miyakawa [20], Usher and Hodgson [21], Usher and Guess [22], Guesset al. [23], Reiser et al. [24], and Kundu and Basu [25]. Most works, however, have mainlyfocused on exponential models. Ishioka and Nonaka [26] presented a technique to stably esti-mate the common Weibull shape parameter with two causes using a quasi-Newton method.Their method is very limited to only two causes and a common shape parameter. Recently,Park and Kulasekera [27] extended the previous works to the case of multiple causes withpartial masking and provided the closed-form maximum likelihood estimators (MLEs) forthe exponential and Weibull models after the common shape parameter is estimated by thelikelihood estimating equation. However, this is limited to the exponential and Weibull modelsand it requires a restrictive common shape parameter condition.

Another approach using the EM algorithm was considered by Albert and Baxter [28]. Theyfound the EM sequences for the exponential model with multiple causes, censoring, andgeneral masking. However, unless one assumes an exponential distribution for the lifetimes,it is extremely difficult or impossible to apply their idea because it requires that the hazardand survival functions have nice closed forms. Recently, Park [29] presented a new methodfor deriving the EM sequences to estimate parameters even when the hazard and survivalfunctions are not of closed forms. This method can be easily implemented to find the EMsequences when the pdf of the distribution has nice closed forms. In many cases, the pdf ofthe distribution is in closed form, whereas the hazard and survival functions have no closed

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

622 C. Park and W. J. Padgett

forms such as for the normal (lognormal) distribution. In this paper, using this new method,we analyze the strength models of multi-modal failures.

3. Strength distribution and likelihood function

3.1 Strength distribution

Most multi-modal strength analyses of materials have been studied on the basis of so-called‘weakest-link theory’ which requires two assumptions [5, 9]:

A1. The material contains inherently many strength-limiting flaws, and its strength dependson the weakest defect of all of them.

A2. There are no interactions among the flaws.

These assumptions exactly match with the competing risks model under the assumption ofthe hypothetical latent lifetimes. Using the observed material strengths instead of lifetimes,we can adopt the competing risks model theory in this context. Assume that there are a finitenumber of independent flaws in the material specimen, indexed by j = 1, . . . , J , and letT

(j)

i denote the strength of the ith material specimen due to the j th type of defect, wherei = 1, . . . , n. Similarly as before, the observed strength of the ith material specimen is givenby Ti = min{T (1)

i , . . . , T(J )i }. Then, we have the following strength distribution of Ti

F (t) = 1 −J∏

j=1

{1 − F (j)(t)},

where F (j)(·) is the strength distribution due to the j th type of defect. In what follows, weconstruct the general likelihood function of the parameters. This likelihood function alsoconsiders masking and censoring problems.

3.2 Likelihood function

The likelihood function presented here is detailed by Park [29] and Park and Padgett [30].Here, we briefly summarize their results. Let I[A] be the indicator function of an event A. Forconvenience, denote Ii (j) = I[δi = j ] and � = (θ(1), θ(2), . . . , θ(J )). The likelihood functionof the censored sample is

L(�) ∝J∏

j=1

n∏i=1

Li(θ(j)), (2)

where

Li(θ(j)) = {f (j)(xi)}Ii (j)

J∏�=0��=j

{S(j)(xi)}Ii (�). (3)

Maximizing L(�) with respect to � is equivalent to individually maximizing L(θ(j)) foreach cause j . Thus, we have reduced the joint maximum likelihood problem for a set of J

parameters to J separate estimation problems for the single parameter θ(j). This simplifies thenumerical work considerably.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 623

Next, we consider a lifetime of a subject Ti due to an unknown cause of failure (masking),but its cause is known up to being one in a set Mi . Denote δi = −1 if the cause of failure isunknown. Then, the overall likelihood of the censored and masked data is given by

L∗(�) ∝n∏

i=1

L∗i (�),

where

L∗i (�) =

J∏j=1

Li(θ(j)) × {f (Mi)(xi)}Ii (−1) (4)

and

f (Mi)(t) =∑j∈Mi

h(j)(t)

J∏�=1

S(�)(t).

In general, the closed-form MLE from the likelihood function above is not available andnumerical methods are required to maximize L∗(�). One popular method that is often used isthe Newton–Raphson method, but a problem with this method is that it can be very sensitiveto the choice of starting values and therefore can often fail to converge to a solution. Also,in the case of the likelihood function (4), if the number of causes is large, the likelihood canbecome overparameterized and the Newton–Raphson method becomes totally ineffective. Thedifficulty with using direct maximization of the likelihood is overcome through the use of theEM algorithm discussed in the following section.

4. EM algorithm and likelihood construction

In this section, we introduce the EM algorithm and develop the likelihood functions whichcan be conveniently used as inputs in the E-step of the EM algorithm.

4.1 EM algorithm

The EM algorithm consists of an expectation step (E-step) and maximization step (M-step).The advantage of the EM algorithm is that it solves a difficult incomplete-data problem byconstructing two easy steps. The E-step only needs to take the conditional expectation of thecomplete-data log-likelihood with respect to the missing part of the complete data, whilethe fully observed data are assumed to be given. The M-step needs to find the maximizer ofthis expected likelihood. Let θ be the vector of unknown parameters. Then, the complete-datalikelihood is

LC(θ|x) =n∏

i=1

f (xi).

Denote the fully observed part of x = (x1, . . . , xn) by y = (y1, . . . , ym) and the missingpart by z = (zm+1, . . . , zn), and denote the estimate at the sth EM sequence by θs . The EMalgorithm consists of two distinct steps:

• E-step: Compute Q(θ|θs),where Q(θ|θs) = ∫

log LC(θ|y, z)p(z|y, θs) dz.• M-step: Find θs+1

which maximizes Q(θ|θs) over θ.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

624 C. Park and W. J. Padgett

4.2 Application of the EM to competing risks model

The question is whether we can apply the EM algorithm to the competing risks problem.When the data are masked, this is equivalent to the cause of failure being missing, so we canconstruct the complete-data likelihood, LC

i (�), by treating the cause of failure as missingdata. Constructing the complete-data likelihood is not difficult once we introduce an indica-tor variable. Define U

(j)

i = I[�i = j |Xi = xi] for j = 1, . . . , J . Then U(j)

i has a Bernoullidistribution with Pr{U(j)

i = 1} = Pr{�i = j |Xi = xi}. It follows that

E[U(j)

i ] =

h(j)(xi)∑�∈Mi

h(�)(xi)if j ∈ Mi

0 if j /∈ Mi.

Replacing f (Mi)(xi) with∏J

j=1{f (j)(xi)}U(j)

i {S(j)(xi)}1−U(j)

i in equation (4), we have thecomplete-data likelihood of the censored and masked data as follows:

LCi (�) =

J∏j=1

LCi (θ(j)),

where

LCi (θ(j)) = {h(j)(xi)}Ii (j)+U

(j)

i Ii (−1)

J∏�=−1

{S(j)(xi)}Ii (�). (5)

If δi = j , then clearly Mi = {j} and thus E[U(j)

i ] = 1. It follows that

Ii (j) + U(j)

i Ii (−1) = U(j)

i .

Using this and∑J

�=−1 Ii (�) = 1, we can simplify (5) as

LCi (θ(j)) = {h(j)(xi)}U(j)

i × S(j)(xi). (6)

Now, because the likelihood LCi (�) is fully factorized by LC

i (θ(j)), the estimation problemcan be solved individually for each single parameter θ(j). Therefore, although the likelihood inequation (4) is not easy to solve because of numerical difficulties, considering the masked dataas missing data and applying an EM framework allows one to obtain a likelihood which consistsof individual likelihoods for each parameter θ(j). Therefore, the transformation of the problemto a missing-data problem simplifies the numerical difficulties considerably. Nevertheless, itstill may not be obvious how the EM algorithm is implemented in the missing-data case andthis is discussed in the next section.

4.3 EM implementation issues

When the distribution for the lifetimes is assumed to be exponential and the data are censoredand masked, we can easily implement an EM algorithm using (6) because the hazard andsurvival functions are of closed forms. In contrast, consider the case where the lifetimes have

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 625

the normal distribution and the data consist of both censored and masked observations. Theapplication of equation (6) is clearly not straightforward because the hazard and survivalfunctions do not have closed forms and the overall likelihood cannot be written as a product ofindividual likelihoods each with a single parameter.Yet, by treating the censored observationsas missing data, it is possible to write the complete-data likelihood in equation (6) in terms ofclosed-form pdf’s. We show later just how to obtain (6) as closed-form pdf’s by treating thecensored data as missing data.

Let Zi be a truncation of Xi at xi with Zi > xi . Then, we have the complete-data likelihoodcorresponding to equation (6)

LCi θ(j) = {f (j)(xi)}U(j)

i {f (j)(Zi)}1−U(j)

i , (7)

where the pdf of Zi is given by

f(j)

Z (t |θ(j)) = f (j)(t)

1 − F(j)

(xi)

for t > xi .

5. Parameter estimation

In this section, we develop the EM-type MLE of the parameters ofWeibull and inverse Gaussian(Wald) distributions. The parameter estimates of other distributions including exponential,normal (lognormal) are detailed by Park [29] and Park and Padgett [30].

5.1 Weibull distribution model

In the case of the Weibull models, the EM sequence can be obtained by either equation (6)or equation (7). For this model, we used equation (6). In the M-step, we need to estimatethe shape parameter α(j) numerically, but this is only a one-dimensional root search and theuniqueness of this solution is guaranteed. Lower and upper bounds for the root are explicitlyobtained, so with these bounds we can find the root easily. We provide the proof of theuniqueness under quite reasonable conditions and give lower and upper bounds of α(j) inAppendix A.

We assume that T(j)

i is a Weibull random variable with the parameter vector θ(j) =(α(j), λ(j)). Thus, the pdf and cdf of T

(j)

i are

f (j)(t) = α(j)λ(j)tα(j)−1 exp(−λ(j)tα

(j)

)

F (j)(t) = 1 − exp(−λ(j)tα(j)

).

First, we obtain an EM sequence using equation (6). Using h(j)(xi) = α(j)λ(j)xα(j)−1i and

S(j)(xi) = exp(−λ(j)xα(j)

i ), we have the complete-data log-likelihood of λ(j):

log LCi (λ(j)) = U

(j)

i {log α(j) + log λ(j) + (α(j) − 1) log xi} − λ(j)xα(j)

i .

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

626 C. Park and W. J. Padgett

Let � = (α(1), λ(1), . . . , α(J ), λ(J )) and denote the estimate of � at the sth EM sequence by�s = (α(1)

s , λ(1)s , . . . , α(J )

s , λ(J )s ).

• E-stepIt follows from Qi(λ

(j)|�s) = E[log LCi (λ(j))|�s] that

Qi(λ(j)|�s) = ϒ

(j)

i,s {log α(j) + log λ(j) + (α(j) − 1) log xi} − λ(j)xα(j)

i ,

where

ϒ(j)

i,s = E[U(j)

i |�s] =

α(j)s λ

(j)s x

α(j)s −1

i∑�∈Mi

α(�)s λ

(�)s x

α(�)s −1

i

if j ∈ Mi

0 if j /∈ Mi.

• M-stepDifferentiating Q(α(j), λ(j)|�s) = ∑n

i=1 Qi(λ(j)|�s) with respect to α(j) and λ(j), and

setting this to zero, we obtain

n∑i=1

∂Qi

∂α(j)=

n∑i=1

ϒ(j)

i,s

{1

α(j)+ log xi

}− λ(j)

n∑i=1

xα(j)

i log xi = 0

n∑i=1

∂Qi

∂λ(j)=

n∑i=1

ϒ(j)

i,s

λ(j)−

n∑i=1

xα(j)

i = 0.

Rearranging for α(j), we have the equation of α(j) as

1

α(j)

n∑i=1

ϒ(j)

i,s +n∑

i=1

ϒ(j)

i,s log xi −n∑

i=1

ϒ(j)

i,s

∑ni=1 xα(j)

i log xi∑ni=1 xα(j)

i

= 0. (8)

The (s + 1)st EM sequence of α(j) is the solution of the earlier mentioned equation. Afterfinding α

(j)

s+1, we obtain the (s + 1)st EM sequence of λ(j) as

λ(j)

s+1 =∑n

i=1 ϒ(j)

i,s∑ni=1 x

α(j)

s+1i

. (9)

5.2 Inverse Gaussian (Wald) distribution model

For the inverse Gaussian distribution, it is extremely difficult or impossible to obtain the EMsequences using equation (6) because finding the closed-form maximizer is not feasible in theM-step. Using equation (7), we can avoid these difficulties to obtain the EM sequences.

We assume that T(j)

i is an inverse Gaussian random variable with the location and scaleparameter θ(j) = (µ(j), λ(j)). Then, the pdf of T

(j)

i is

f (j)(t) =√

λ(j)

2πt3exp

(−λ(j)(t − µ(j))2

2µ(j)2t

),

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 627

and its cdf is

F (j)(t) =

√λ(j)′

t

(t − µ(j)

µ(j)

) + exp

(2λ(j)

µ(j)

)

−

√λ(j)

t

(t + µ(j)

µ(j)

) ,

where (·) is the standard normal cdf.We have the complete-data log-likelihood of θ(j)

log LCi (θ(j)) = U

(j)

i log f (j)(xi) + (1 − U(j)

i ) log f (j)(Zi),

where Zi is the truncated inverse Gaussian random variable with the pdf given by

f(j)

Z (t |θ(j)) = f (j)(t)

1 − F (j)(xi), t > xi.

We denote the estimate of θ(j) and � at the sth EM sequence by θ(j)s and �s , respectively.

• E-stepWe have

log f (j)(Zi) = C + 1

2log λ(j) − 3

2log Zi − λ(j)

2µ(j)2 Zi + λ(j)

µ(j)− λ(j)

2

1

Zi

E[log f (j)(Zi)|θ(j)s ] = C + 1

2log λ(j) − 3

2m

(j)

Ai,s − λ(j)

2µ(j)2 m(j)

Bi,s + λ(j)

µ(j)− λ(j)

2m

(j)

Ci,s ,

where m(j)

Ai,s = E[log Zi |θ(j)s ], m(j)

Bi,s = E[Zi |θ(j)s ], and m

(j)

Ci,s = E[1/Zi |θ(j)s ]. Here, m

(j)

Ai,s ,

m(j)

Bi,s , and m(j)

Ci,s can be obtained by numerical integration. Using these results, we have

Qi(µ(j), λ(j)|�s) = C + ϒ

(j)

i,s

2

{log λ(j) − 3 log xi − λ(j)

µ(j)2 xi + 2λ(j)

µ(j)− λ(j) 1

xi

}

+ ϒ(j)

i,s

2

{log λ(j) − 3m

(j)

Ai,s − λ(j)

µ(j)2 m(j)

Bi,s + 2λ(j)

µ(j)− λ(j)m

(j)

Ci,s

},

where

ϒ(j)

i,s = E[U(j)

i |�s] =

h(j)(xi |θ(j))∑�∈Mi

h(�)(xi |θ(�)s )

= f(j)

Z (xi |θ(j)s )∑

�∈Mif

(�)Z (xi |θ(�)

s )if j ∈ Mi

0 if j /∈ Mi

ϒ(j)

i,s = 1 − ϒ(j)

i,s .

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

628 C. Park and W. J. Padgett

• M-stepDifferentiating Qi(µ

(j), λ(j)|�s) with respect to µ(j) and λ(j), we obtain

∂Qi

∂µ(j)= λ(j)

µ(j)3 {ϒ(j)

i,s xi + ϒ(j)

i,s m(j)

Bi,s − µ(j)}

∂Qi

∂λ(j)= 1

λ(j)− ϒ

(j)

i,s

2µ(j)

(xi + 2µ(j) + µ(j)2

xi

)− ϒ

(j)

i,s

2µ(j)

(m

(j)

Bi,s + 2µ(j) + µ(j)2m

(j)

Ci,s

).

Solving∑n

i=1 ∂Qi/∂µ(j) = 0 and∑n

i=1 ∂Qi/∂λ(j) = 0 for µ(j) and λ(j), we obtain the(s + 1)st EM sequence in the M-step as follows:

µ(j)

s+1 = 1

n

n∑i=1

{ϒ

(j)

i,s xi + ϒ(j)

i,s m(j)

Bi,s

}

1

λ(j)

s+1

= 1

n

n∑i=1

{ϒ

(j)

i,s

1

xi

+ ϒ(j)

i,s m(j)

Ci,s

}− 1

µ(j)

s+1

.

It is of interest to look at the role of ϒ(j)

i,s and ϒ(j)

i,s when an observation is incomplete. If an

observation xi is right-censored, then ϒ(j)

i,s = 0, which results in the full weight (i.e. ϒ(j)

i,s =1) toward m

(j)

Bi,s and m(j)

Ci,s . If an observation xi is masked, then ϒ(j)

i,s has a value between 0and 1 whose value is determined by the extent of masking. That is, as the number of indicesin the set Mi = {j1, . . . , ji} gets larger, the value ϒ

(j)

i,s becomes smaller, which results in

more weight on m(j)

Bi,s and m(j)

Ci,s .

6. Examples

In this section, we illustrate the method with several examples. The data analysis is per-formed using codes in the R language [31]. The R codes which were used to analyzethe data in the examples are presented by Park and Padgett [30] and also are available athttp://www.ces.clemson.edu/∼cspark/.

To compare the fits of the strength distribution models, the mean square errors (MSEs)from the fitted model to the empirical distribution were used. The MSE for the fitted model iscalculated as

MSE(F (·; θ)) = 1

n

n∑i=1

{F(ti, θ) − Fn(ti)}2,

where Fn(ti) is the empirical cdf and F(ti; θ) is the fitted cdf using the MLE of θ. If the dataare not censored, the empirical cdf Fn(·) can be easily calculated. Several versions of theseempirical estimates Fn(·) have been suggested in the statistics literature, but the most popularone is (j − 1/2)/n (also known as median rank method) for n > 11 and (j − 3/8)/(n + 1/4)

for n ≤ 10, due to Blom [32] and Wilk and Gnanadesikan [33]. However, if the data set hascensoring, then the empirical cdf Fn(t) can be estimated by the well-known product limit

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 629

estimator of S(t) = 1 − F(t) originally attributed to Kaplan and Meier [34], which is definedhere as

Sn(t) =

1 if 0 ≤ t ≤ t1k−1∏i=1

(n − 1

n − i + 1

)I(δi>0)

if tk−1 < t ≤ tk, k = 2, . . . , n

0 if t > tn

so that we have Fn(t) = 1 − Sn(t).An alternative way to compare the fits of the proposed models for each specific failure

mode is to compare the empirical cumulative incidence function (CIF), proposed by Aalen[35], with the parametric CIF. Here, we are interested in the strength distribution of a wholesystem or a material specimen, not in the distribution due to each specific failure mode, sowe do not consider the CIF in this paper. If one is interested in the lifetime distribution due toeach failure mode, one should look at the CIF. For applications of the CIF with the maskeddata, the reader is referred to Park and Kulasekera [27] and Park [29]. In particular, Park [29]deals with the partially masked data for a variety of distributions.

6.1 Device-G from a field-tracking study

The data set in table 1 was illustrated by Meeker and Escobar [12]. They analyzed the datausing the Weibull model. The data set gives times of failure and running times for a sample ofdevices from a field-tracking study of a larger system. At a certain point in time, 30 units wereinstalled in normal service conditions. The failure cause was investigated for each unit thatfailed. Mode S denotes the failure caused by an accumulation of randomly occurring damagefrom power-line voltage spikes during electric storms. Mode W denotes the failure caused bynormal product wear. Here, the failure modes are coded as follows: δi = 1 (mode S), δi = 2(mode W), and δi = 0 (censored).

From our results summarized in table 3, the lognormal model outperforms the Weibullmodel. For the Wald model, the EM algorithm is very slow because of the large proportionof censoring. In the E-step of the algorithm with the Wald model, we have to use numericalintegrations which cause slowdown in the estimation. It is a challenging future work to speedup the EM of the Wald model. But for other models, the EM algorithm works fairly fast. Theparameter estimates and MSEs of the models are summarized in tables 2 and 3, and their fitsusing these estimates are superimposed on the Weibull plots in figure 1.

Table 1. Device-G failure times and cause of failure.

Cycle × 10−3 Mode Cycle × 10−3 Mode Cycle × 10−3 Mode Cycle × 10−3 Mode

275 2 300 0 300 0 300 013 1 173 1 2 1 23 1

147 2 106 1 261 1 300 023 1 300 0 293 2 80 1

181 2 300 0 88 1 245 230 1 212 2 247 1 266 265 1 300 0 28 110 1 300 0 143 1

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

630 C. Park and W. J. Padgett

Table 2. Parameter estimation under the models considered.

Weibull Lognormal Wald

Mode λ(j) α(j) µ(j) σ (j) µ(j) λ(j)

S 1.6598 × 10−2 0.6710 5.5728 2.1830 10886.6 32.6W 1.0426 × 10−11 4.3373 5.7706 0.3760 346.6 2209.3

Table 3. MSEs under the models considered.

Model Weibull Lognormal Wald

MSE × 103 0.8314317 0.5407669 24.1763249

6.2 Microbond testing of pitch-based fibers

An experiment was performed at Clemson University by Harwell [36] to study the strength ofthe interfacial bond of a carbon fiber (whose average diameter is approximately 8 ∼ 12 µm)and matrix material. Ribbon fibers, i.e. flat-shaped rather than round-shaped fibers, were usedin Harwell’s ‘microbond tests’. In the experiment, a droplet of the epoxy resin was placed ona fiber and cured by heat treatment. The fibers were coated with SiC because it was thoughtthat such a coating would improve the interfacial bond. Microbond tests were performed on‘uncoated’ fibers and on fibers with either a ‘thin’ SiC coating or a ‘thick’ SiC coating. The

Figure 1. Weibull probability plot.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 631

fiber-in-droplet specimen was then placed in a ‘micro-vise’, and the fiber was placed undertensile load in an attempt to force it to debond from the matrix droplet. The applied stressrequired to debond the fiber from the droplet was recorded by a load cell. However, for someof the specimens tested, due to inherent flaws in the fiber which tend to decrease its tensilestrength, the fiber broke before debonding occurred. Kuhn and Padgett [37] analyzed thisdata set using the kernel density estimation with the main interest in comparing the debondingstrengths of ribbon fibers. Hence, the breakdowns due to the internal flaws were treated as right-censored. In this example, we consider all kinds of flaws to estimate the strength distributionof the specimen. So the internal flaws (mode B), debonding (mode D) and coating (mode C)are considered to be causes of failure.

We consider the data for the thick-coated fibers. For other examples with the data foruncoated and thin-coated fibers, the reader is referred to Park and Padgett [30]. The data areprovided in table 4, and tables 5 and 6 summarize the parameter estimates and MSEs. On thebasis of MSE criterion, the Weibull model is the best. It seems that the failure mode of thickcoating is more dominant and its strength distribution is very closely modeled by a Weibulldistribution. Figure 2 indicates that all the models are relatively good.

6.3 Strength data with censoring and masking

In many tensile strength experiments, specimens tested are broken down due to several causes,with the cause of fracture not properly identified along with censoring due to time and costconsiderations on experiments.

Table 4. Strength of the interfacial bond (thick coating).

Strength Mode Strength Mode Strength Mode Strength Mode

0.232 C 0.412 C 0.275 C 0.171 C0.335 C 0.355 C 0.242 C 0.392 B0.120 C 0.425 D 0.075 C 0.080 C0.073 C 0.332 D 0.319 C 0.121 C0.134 C 0.399 D 0.231 C 0.347 C0.176 C 0.110 C 0.173 C 0.263 D0.276 C 0.492 D 0.356 D 0.353 D0.065 C 0.257 C 0.373 C 0.332 C0.414 D 0.114 C 0.273 C 0.190 C0.573 B 0.044 C 0.035 C 0.066 C

0.289 C 0.397 B

Table 5. Parameter estimation under the models considered.

Weibull Lognormal Wald

Mode λ(j) α(j) µ(j) σ (j) µ(j) λ(j)

Debonding 58.5486 5.5510 −0.8345 0.2227 0.4453 8.7143Break 190.4963 8.4643 −0.6959 0.1636 0.5042 18.9087Coating 5.2106 1.5031 −1.4169 0.8789 0.3636 0.3034

Table 6. MSEs under the models considered.

Model Weibull Lognormal Wald

MSE × 103 0.992133 1.311246 1.970773

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

632 C. Park and W. J. Padgett

Figure 2. Weibull probability plot.

The strength data in table 7 were obtained using the lognormal random number generatorof R language to illustrate the use of the proposed method. Here, we assume that the fracturecauses are due to a surface defect (mode 1), an inner defect (mode 2), and an end effect atthe clamp to hold the specimen (mode 3). The censored observations are denoted by 0. Toillustrate the applicability of the partial or complete masking along with censoring, the datawere censored at 150 and 10% of the data were randomly masked.

Table 7. Simulated strength data for three fracture causes with censoring and masking.

Strength Mode Strength Mode Strength Mode Strength Mode

54 {3} 7 {1, 2, 3} 86 {2} 104 {1}143 {2} 81 {3} 141 {1} 89 {3}97 {3} 52 {3} 79 {3} 9 {3}

104 {3} 40 {3} 23 {3} 111 {1, 2, 3}71 {1, 2} 82 {2} 8 {3} 150 098 {1} 3 {3} 17 {3} 79 {2}24 {2} 130 {2} 41 {2} 94 {2}

138 {3} 5 {3} 43 {2, 3} 150 038 {3} 32 {2} 9 {3} 77 {2}78 {3} 16 {3} 92 {2} 76 {3}

150 0 33 {3} 80 {2} 100 {2}46 {3} 137 {1, 2} 92 {3} 108 {2}

109 {1} 71 {1} 60 {2} 88 {1}7 {3} 11 {3} 150 0 150 0

42 {2} 6 {3} 43 {3} 124 {1, 2}

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 633

Table 8. Parameter estimation under the models considered.

Weibull Lognormal Wald

Mode λ(j) α(j) µ(j) σ (j) µ(j) λ(j)

Surface defect 2.6150 × 10−10 4.3078 5.0390 0.3599 165.4 1,172.5Inner defect 8.9161 × 10−6 2.3295 4.8525 0.6732 169.4 261.6End effect 1.1309 × 10−2 0.8802 4.6931 1.7034 7478.7 30.2

Table 9. MSEs under the models considered.

Model Weibull Lognormal Wald

MSE 0.1322911 0.1314757 0.1396442

The parameter estimates and MSEs of the models are summarized in tables 8 and 9 and theirfits using these estimates are superimposed on the Weibull plots in figure 3. Note that unlikethe preceding examples, the cause of fracture cannot be classified separately due to masking.Thus, we only mark the data with either ‘failure’ or ‘censored’ in the figure. Comparing theMSEs, we come to the conclusion that the strength distribution can be modeled by a lognormaldistribution.

Figure 3. Weibull probability plot.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

634 C. Park and W. J. Padgett

7. Concluding remarks

This paper developed a highly stable parameter estimation methodology using the EMalgorithm. It has been frequently reported that the EM-type parameter estimation has severaladvantages over ordinary numerical methods such as the Newton–Raphson gradient methodand the Gauss–Seidel iterative method. One drawback of the EM algorithm is that it is slowerthan the gradient method in some cases. For example, the EM algorithm of the Wald modelis very slow, especially when a large portion of the data are censored. In the E-step, for thecensored observations, we need to use numerical integration for the expectation, which slowsdown in the estimation process. It is a challenging future work to speed up the EM algorithmwith the heavily censored data under the Wald model. Nevertheless, in an era of powerful andaccessible computers, a computer-intensive method, such as the EM algorithm, promises tobe one of the keystones of applied parametric modeling and data analysis in the years ahead.

Further, we want to stress that in the case of the exponential and Weibull distributions, h(·)and S(·) are of closed forms, so applying the EM algorithm using equation (6) is straightfor-ward, so there is no need to treat the censored data as missing data. In contrast, for the normal,lognormal, and inverse Gaussian distributions, it is either impossible or very difficult to obtainclosed forms for h(·) and S(·), so applying the EM algorithm through the use of equation (6)is quite difficult. However, equation (7) involves only the corresponding pdf f (·), so apply-ing the EM algorithm using equation (7) can be thought of as a straightforward generalizedapproach to the competing risks problem as illustrated clearly by the examples in section 6.

Acknowledgements

W. J. Padgett’s work was partially supported by the National Science Foundation grant DMS-0243594 to the University of South Carolina. The authors thank Michael Harwell of ChemicalEngineering at Clemson University for providing the bond strength data set.

References

[1] Johnson, J.W. and Thorne, D.J., 1969, Effect of internal polymer flaws on strength of carbon fibres preparedfrom an acrylic precursor. Carbon, 7, 659–660.

[2] Jones, B.F. and Wilkins, J.S., 1972, A technique for the analysis of fracture strength data for carbon fibres. FibreScience and Technology, 5, 315–320.

[3] Layden, G.K., 1973, Fracture behaviour of boron filaments. Journal of Materials Science, 8, 1581–1589.[4] Boggio, J.V. and Vingsbo, O., 1976, Tensile strength and crack nucleation in boron fibres. Journal of Materials

Science, 11, 273–282.[5] Beetz, C.P., 1982, The analysis of carbon fibre strength distributions exhibiting multiple modes of failure. Fibre

Science Technology, 16, 45–59.[6] Martineau, P., Lahaye, M., Pailler, R., Naslain, R., Couzi, M. and Cruege, F., 1984, SiC filament/titanium matrix

composites regraded as model composites: Part 1 filament micro-analysis and strength characterization. Journalof Materials Science, 19, 2731–2748.

[7] Simon, G. and Bunsell, A.R., 1984, Mechanical and structural characterization of the nicalon silicon carbidefibre. Journal of Materials Science, 19, 3649–3657.

[8] Chi, Z., Chou, T.-W. and Shen, G., 1984, Determination of single fibre strength distribution from fibre bundletestings. Journal of Materials Science, 19, 3319–3324.

[9] Goda, K. and Fukunaga, H., 1986, The evaluation of the strength distribution of silicon carbide and aluminafibres by a multi-modal Weibull distribution. Journal of Materials Science, 21, 4475–4480.

[10] Wagner, D.H., 1989, Stochastic concepts in the study of size effects in the mechanical strength of highly orientedpolymeric materials. Journal of Polymer Science, B-27, 115–148.

[11] Stoner, E.G., Edie, D.D. and Durham, S.D., 1994, An end-effect model for the single-filament tensile test.Journal of Materials Science, 29, 6561–6574.

[12] Meeker, W.Q. and Escobar, L.A., 1998, Statistical Methods for Reliability Data (NewYork: John Wiley & Sons).[13] Taylor, H.M., 1994, The Poisson–Weibull flaw model for brittle fiber strength. In: J. Galambos, J. Lechner and

E. Simiu (Eds) Extreme Value Theory (Amsterdam: Kluwer), pp. 43–59.[14] Phoenix, S.L. and Sexsmith, R.G., 1972, Clamp effects in fiber testing. Journal of Composite Materials, 29,

1873–1884.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

Analysis of strength distributions 635

[15] Padgett, W.J., Durham, S.D., and Mason, A.M., 1995, Weibull analysis of the strength of carbon fibers usinglinear and power law models for the length effect. Journal of Composite Materials, 29, 1873–1884.

[16] Durham, S.D. and Padgett, W.J., 1997, A cumulative damage model for system failure with application to carbonfibers and composites. Technometrics, 39, 34–44.

[17] Moeshberger, M.L. and David, H.A., 1971, Life tests under competing causes of failure and the theory ofcompeting risks. Biometrics, 27, 909–933.

[18] Cox, D.R., 1959, The analysis of exponentially distributed lifetimes with two types of failures. Journal of theRoyal Statistical Society B, 21, 411–421.

[19] Herman, R.J. and Patell, R.K.N., 1971, Maximum likelihood estimation for multi-risk model. Technometrics,13, 385–396.

[20] Miyakawa, M., 1984, Analysis of incomplete data in competing risks model. IEEE Transactions on Reliability,33, 293–296.

[21] Usher, J.S. and Hodgson, T.J., 1988, Maximum likelihood analysis of component reliability using masked systemlife-test data. IEEE Transactions on Reliability, 37, 550–555.

[22] Usher, J.S. and Guess, F.M., 1989, An iterative approach for estimating component reliability from maskedsystem life data. Quality and Reliability Engineering International, 5, 257–261.

[23] Guess, F.M., Usher, J.S., and Hodgson, T.J., 1991, Estimating system and component reliabilities under partialinformation of the cause of failure. Journal of Statistical Planning and Inference, 29, 75–85.

[24] Reiser, B., Guttman, I., Lin, D.K.J., Guess, F.M. and Usher, H.S., 1995, Bayesian inference for masked systemlifetime data. Applied Statistics, 44, 79–90.

[25] Kundu, D. and Basu, S., 2000, Analysis of incomplete data in presence of competing risks. Journal of StatisticalPlanning and Inference, 87, 221–239.

[26] Ishioka, T. and Nonaka, Y., 1991, Maximum likelihood estimation of Weibull parameters for two independentcompeting risks. IEEE Transactions on Reliability, 40, 71–74.

[27] Park, C. and Kulasekera, K.B., 2004, Parametric inference of incomplete data with competing risks amongseveral groups. IEEE Transactions on Reliability, 53, 11–21.

[28] Albert, J.R.G. and Baxter, L.A., 1995, Applications of the EM algorithm to the analysis of life length data.Applied Statistics, 44, 323–341.

[29] Park, C., 2005, Parameter estimation of incomplete data in competing risks using the EM algorithm. IEEETransactions on Reliability, in press.

[30] Park, C. and Padgett, W.J., 2004, Analysis of strength distributions of multi-modal failures using the EMalgorithm. Technical Report no. 220, Department of Statistics, University of South Carolina.

[31] R Development Core Team, 2004, R: A language and environment for statistical computing (Vienna, Austria:R Foundation for Statistical Computing) (ISBN 3-900051-07-0).

[32] Blom, G., 1958, Statistical Estimates and Transformed Beta Variates (New York: Wiley).[33] Wilk, M.B. and Gnanadesikan, R., 1968, Probability plotting methods for the analysis of data. Biometrika, 55,

1–17.[34] Kaplan, E.L. and Meier, P., 1958, Nonparametric estimation from incomplete observations. Journal of the

American Statistical Association, 53, 457–481.[35] Aalen, O.O., 1978, Nonparametric estimation of partial transition probabilities in multiple decrement models.

Annals of Statistics, 6, 534–545.[36] Harwell, M., 1995, Microbond tests for ribbon fibers. MS thesis, Clemson University.[37] Kuhn, J.W. and Padgett, W.J., 1997, Local bandwidth selection fro kernel density estimation from right-censored

data based on asymptotic mean absolute error. Nonlinear Analysis, Theory, Methods and Applications, 30,4375–4384.

[38] Farnum, N.R. and Booth, P., 1997, Uniqueness of maximum likelihood estimators of the 2-parameter Weibulldistribution. IEEE Transactions on Reliability, 46, 523–525.

Appendix A

Sketch Proof of the Uniqueness and the Bounds

Analogous to the approach of Farnum and Booth [38], the uniqueness of the solution ofequation (8) is proved as follows. For convenience, omitting the failure mode index (j) andthe step index s, and letting

g(α) = 1

α

n∑i=1

ϒi,

h(α) =n∑

i=1

ϒi ·∑n

i=1 xαi log xi∑n

i=1 xαi

−n∑

i=1

ϒi log xi,

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014

636 C. Park and W. J. Padgett

we rewrite (8) by g(α) = h(α). The function g(α) is strictly decreasing from ∞ to 0 onα ∈ [0, ∞] unless ϒi = 0 for all i, whereas h(α) is increasing because it follows from theJensen’s inequality that

∂h(α)

∂α=

∑ni=1 ϒi{∑ni=1 xα

i

}2

n∑i=1

xαi log2 xi

n∑i=1

xαi −

(n∑

i=1

xαi log xi

)2 ≥ 0.

Now, it suffices to show that h(α) > 0 for some α. Denote xmax = maxi (x1, . . . , xn). Because

limα→∞ h(α) =

n∑i=1

ϒi(log xmax − log xi),

we have h(α) > 0 for some α unless ϒi = 0 or xi = xmax for all i. This condition is unrealisticin practice.

Next, we provide upper and lower bounds of α. These bounds guarantee the solution in theinterval and enable the root search algorithm to find the solution very stably and easily. Sinceh(α) is increasing, we have g(α) ≤ limα→∞ h(α), that is,

α ≥∑n

i=1 ϒi∑ni=1 ϒi(log xmax − log xi)

.

Denote the earlier mentioned lower bound by αL. Then, because h(α) is again increasing, wehave g(α) = h(α) ≥ h(αL). If h(αL) > 0, then we have an upper bound

α ≤∑n

i=1 ϒi

h(αL).

If h(αL) < 0 (it is extremely rare in practice though), then the upper bound can be obtainedby

k · max

(αL,

n∑i=1

ϒi

|h(αL)|

),

for some large k. This can be easily found by increasing k (say, k = 2, 3, . . .). Because h(α)

is increasing and h(α) > 0 for some α, this method guarantees an upper bound.

Dow

nloa

ded

by [

Nor

th W

est U

nive

rsity

] at

16:

50 2

2 D

ecem

ber

2014