[Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...

14 Mixed Models 11: The AVE Method with Balanced Data

In this chapter we describe an alternative method for computing the estimates of variance components for mixed models with balanced data. The method is motivated by a description of the model in terms of the covariance structure implied by the experiment. The method leads to valuable diagnostic information about the data, the model, and the design. The computations and diagnostic analyses are illustrated with numerical examples.

14.1 INTRODUCTION

In Chapter 13, the variance components were modeled as variances of terms in the linear model. In some of our examples, the estimates were not admissible since they were negative. Attempts to account for this phenomenon have offered the following explanations:

1. They are a consequence of highly variable data. 2. They are the result of unusual or incorrect data. 3. The model is incorrect. 4. The experimental design is not sufficient.

Questions with regard to unusual observations, design limitations, and model specification should be investigated in any analysis, not just situations in which negative estimates arise. To provide answers to these questions, Hocking (1983a, 1985) introduced an alternative approach for describing the model and for computing the estimates for the balanced, two-factor, mixed model. The method was extended to the general class of factorial models and illustrated by Hocking, Green, and Bremer (1989), and then generalized to include models with nested effects by Hocking (1990).

489

Methods and Applications of Linear Models: Regression and the Analysis of Variance, 2nd Edition. Ronald R. Hocking

Copyright 0 2003 John Wiley & Sons, Inc. ISBN: 0-471-23222-X

490 Chapter 14 Mixed Models It: The AVE Method with Balanced Data

The description of the model, in Chapter 13, in terms of the covariance structure imposed on the data, suggests that some functions of the variance components may be viewed as covariances rather than variances. We will see that the estimates of the variance components, or linear hc t ions of them, are given by averages of certain sample covariances or sample variances. This observation led to the acronym Am, (Average Variance Estimator) for the method. The method does not require the fitting of linear models or the computation of AOV tables but requires the computation of sample covariance matrices.

In this chapter, we describe the method using the notation introduced in Chapter 13 and illustrate the value of the diagnostic analysis with numerical examples. Following the usual pattern, we first develop the analysis for specific models, we then describe the general method and some of its properties. We then provide additional illustrative examples. For emphasis, we use some of the examples from Chapter 13. A general approach to performing the computations is given in Section 14.7.

14.2 TWO-WAY CROSSCLASSIFICATION MODEL

To introduce the concepts we consider the two-way cross-classification model with n = 1 and n > 1 for both random and mixed models. We introduce the model formulation, develop the estimates and illustrate the diagnostics.

Example 14.1. n = 1. In Example 13.5, we considered an experiment designed to compare al = 5 fertilizers. The design was a randomized block with a2 = 6 blocks and the data are found in Table 13.12. The model was given in (13.83) and the estimates were obtained f?om the AOV in Table 13.12. We noted that with n = 1, we could only estimate the linear combination 4e + and adopted the convention, = 0. We now give an alternative formulation of the model that we call covariance specification.

Mixed Model. We assume that observations on the ith fertilizer have mean pi and variance a* and that observations in the same block are correlated with covariance, &. Observations in different blocks are uncorrelated. The model is summarized as

E [ ~ i j l = Pi var[3/pj] = 2

cov[yij, = 4 2 i # i' j = f = o j # Y e

(14.1)

In matrix form this covariance structure is written as

14.2 Two-way Cross-Classification Model 491

v = u21a + 42(U1 - I , ) €3 1 2 - (14.2)

(Recall our convention that Ii denote the identity matrix of size ai and I , denotes the identity of size a = a1u2.) For notational convenience and to compare with the linear model formulation, we introduce the parameter, (Pl2 = u2 - q52, and write the covariance matrix as

V = q512Ia + 42u1 €3 1 2 . (14.3)

Comparing this matrix with that in (13.15) with n = 1 and

u," = (pe = 0 L72 = 0 u; = 4 2 ump 2 - - 412, (14.4)

we see that this model agrees mathematically with that used in Example 13.5. Note that in this formulation, there is no implicit assumption that 42 and 412 are non-negative but their sum has that requirement.

The interpretation of the parameter functions q52 + q512 and q52 as a variance and a covariance, respectively, suggests that their estimates should depend on sample variances and covariances. The AOV method focused on the expected mean squares, A , and the means squares from the AOV table provided estimates of these functions. The estimates of and q512 are then determined by solving the implied system of linear equations. Typically, these computations are done numerically. The AVE method identifies a different set of linear functions of the variance components, denoted by -yt. In this case, the functions are

Y2 = 42 7 1 2 = 4 2 + 412 (14.5)

We now show that there are simple and informative expressions for the estimates of these new parameter functions. From our discussion of the AOV method, we see that the estimates of X2 = a142 + 412 and A12 = $12 are given, in terms of the sums of squares st, by

It follows that the estimate of q52 is given by

1 -

a1 8 2 = --(A2 - A121

1 1 = YT(-C1 a1 rl €3 -s2)*. r2

(14.6)

(14.7)

492 Chapter 14 Mixed Models 11: The AVE Method with Balanced Data

We have introduced the cross-products matrix,

ca = ui - Ii

and the simplification in (14.7) is a consequence of the relation

1 1 1 - u - - s = - c . a r r

(14.8)

(14.9)

To appreciate the role of the cross-products matrix, we write the expression for the estimate of 42 in algebraic form as

To examine this expression, assume that the data are arrayed as in Table 13.12 with al columns and a2 rows. The quantity in braces is the sample covariance computed for the data in columns i and i* of the data table. The rest of the equation tells us to compute the average of these sample covariances for all pairs of columns. This expression for the estimate is consistent with the interpretation of 42 as the covariance between pairs of observations in the same row and different columns.

Operationally, we compute the estimate of q& by forming the sample covariance matrix W2, treating the columns as variables and the rows as observations. The estimate of & is the average of the off-diagonal elements of this matrix.

It is natural to ask if the diagonal elements of W2 are of interest. These elements are the sample variances for the data in a given column. Summing the expressions for the estimates of d12 and g5z fiom (14.6) and (14.7), we see that the AOV estimate of ylz is written in matrix and algebraic form as

(14.1 1)

The expression in braces is the sample variance computed for the ith column of the data table, that is, the ith diagonal element of W2. The remainder of the equation tells us to average these sample variances.

Thus the estimates of the parameter functions y2 and yI2 are obtained by forming the sample covariance matrix W2 and computing the averages of the off- diagonal and the diagonal elements of this matrix. The estimate of 412 is obtained by taking the difference. The A YE method thus replaces the fitting of a linear model and formulation of the AOVtable by the computation of this sample


covariance matrix. We emphasize that these estimates are identical to the AOV estimates hence enjoy the same statistical properties.

At this point the difference between the two methods seems to rest on the choice between developing the AOV table or computing a sample covariance matrix. We now examine the AVE form of the estimates to appreciate the advantages of this method with regard to a diagnostic analysis.

It is of interest to examine the individual elements of W2. To do so, let ul(i) denote a column vector, of length alp with one in the ith position and zeros elsewhere. Thus the ith diagonal element of W2 is written

(14.12)

The average of the elements in positions ( i , i*) and (1" , i ) is written

1 1

r2 y2(i, i ') = #( (ul(i)uT(I") + ul(r)uT(i)) 8 -S,>y. (14.13)

For the purpose of computing moments, we consider the average of these two identical elements so that the matrix of the quadratic form is symmetric. It follows that

E[%2(9l = 4 2 + 4512

v ? 2 ( i , ?>I = 4 2 . (14.14)

Thus each diagonal element of W2 is an unbiased estimate of y12 and their average is the A VE estimate. The analogous comment applies to the off-diagonal elements with regard to estimating -y2-

If the model is correct and we have enough data containing no aberrant observations, the individual estimates should vary in a reasonable way about their average. Diagnostic procedures based on the AVE method are apparent. An unusual individual element of W2 suggests an examination of the associated data. Far the off-diagonal elements a scatter-plot of the data for the two columns used to compute the sample covariance may reveal outlying observations or some other unusual behavior of the data. Box-plots of the data in a column may explain unusual diagonal elements. Note that the sample variances and covariances have a2 - 1 degrees of freedom hence, if a2 is small, a single observation has high leverage on the estimate. This reveals a limitation in the design. Patterns in the covariance matrix W2 may suggest that the model is incorrect. For example, the covariance may be small for certain pairs of columns and large for others.

It is now clear how negative estimates can arise. Since sample covariances can be negative, a negative estimate of 42 can arise if the sum of the negative off- diagonal elements exceeds the sum of the positive ones. Examination of the scatter-plot matrix associated with W2 may reveal the cause. For example, the negative estimates may not reflect a negative trend in the data but may be caused


by one or more outliers. Note that an outlier can impact several of the individual estimates. While negative estimates of d2 attract our attention, the examination of the individual elements and the scatter-plot matrices should be a routine part of the analysis.

To illustrate these ideas, we use the data from Example 13.5. The sample covariance matrix is shown in Table 14.1. The sample variances are on the diagonal and the covariances below the diagonal. To give an indication of the strength of the relations, we show the sample correlations (in italics) above the diagonal.

We recall, from Example 13.5, that the estimates of the components are q512 = 24.65 and c j 2 = - 0.21. Averaging the diagonal and off-diagonal elements of Table 14.1 yields the estimates y2 = - 0.21 and TI2 = 24.44 in agreement with the AOV estimators. Examination of the off-diagonal elements in Table 14.1 reveals the source of the negative estimate of 4,. That is, some of the individual estimates are negative and the negatives dominate. It is of interest to note that all of the sample covariances involving fertilizer 3 are negative. The associated scatter-plot matrix is shown in Figure 14.1.

Typically a negative covariance implies a negatively sloped orientation of the data in the scatter-plot, but in mixed model applications this negative covariance is often caused by outlying observations. Examination of the scatter- plots associated with fertilizer three (D) reveals that the large negative covariances for fertilizer three with fertilizers four and five are primarily caused by the data in block six. In particular, the smallest observation for the third fertilizer and the largest observations for the fourth and fifth fertilizers occur in that block. Examination of these observations is warranted.

The box-plots associated with the diagonal elements are shown in Figure 14.2. Examination of these plots shows that the observations for fertilizers three and four, in the sixth block are potential outliers. The outlying observation for fertilizer three in block two also contributes to the other negative covariance, but the effect is not as strong. The large negative covariance between the first and third fertilizers is the result of a small correlation ( - 0.30) associated with the large variance (6 1.5) in fertilizer one. The low mean and high variance, noted in Figure 14.2, would seem to rule out fertilizer one for future study.

h h

TABLE 14.1 The W2-matrix for Example 14.1 Fert.1 Fcrt.2 Fert.3 Fert.4 Fert.5

Fert.1 61.50 0.52 -0.30 0.10 0.39 Fert.2 14.84 13.20 - 0.41 0.50 0.77 Fert.3 - 12.80 - 8.02 29.60 - 0.93 - 0.84 Fert.4 2.40 5.56 - 15.53 9.41 0.88 Fert.5 8.84 8.20 - 13.38 7.82 8.49


64].m,

58 25 35

FI

SO{ , , :, , 58 64

F1 F2

I. A

104

981.:. , , , 5 8 6 4

F2

2535 5 8 6 4 8 0 9 0 98 104 Fl F2 F3 F4

Figure 14.1 Scatter-plot matrix for Example 14.1.

4% Chapter 14 Mixed Models 11: The AVE Method with Balanced Data

FI F2 F3 F4 F5 Fertilizer

Figure 14.2. Box-plots for Example 14.1.

Actions to be taken as a result of this analysis depend on the needs of the researcher. It may be of interest to repeat the analysis deleting one or both of the potential outliers. Ofcourse, this creates an imbalance in the data and the results of this chapter are not applicable. We return to this problem in Chapter 15.

Random Model. In this example factor one is fixed, but it is of interest to examine the model with both fhctors random. The extension of the model and the analysis is straightforward. We assume all observations have mean p and that observations in the Same column are correlated with covariance q51 The model is written as

EC7dijI = P Var[yij] = 0 2

Cov[yjj, yi.31] = 41 i = i* j # 1- = 42 i # i* j = j *

= O i # i * j # j * . (14.15)

For convenience, we define d12 = a2 - q51 - q52 and note that the resulting covariance matrix is given by (13.15) and (13.20) with the obvious notational changes. Using the same development as above, noting the symmetry of the situation with regard to rows and columns, we see that the estimate of is given bY


In algebraic form we have

(14.16)

The estimate of is obtained as the average of the off-diagonal elements of the covariance matrix Wl based on the rows of Table 13.5. The diagonal elements of this matrix estimate yi2 = d1 + 412 and it is easily shown that W1 and W2 yield the same estimate of (plz.

In the mixed model, there is no need to compute Wl for estimating variance components, but it may be done to provide further information about the data. It can be shown that each of the diagonal elements and each of the off- diagonal elements has the same expected value (see Exercise 14.2). It is suggested that these elements and the associated scatter-plots and box-plots be examined for consistency and unusual observations.

Inference on Parameters. The test statistics and interval estimation methods discussed in Chapter 13 are used. The only issue is computation since we have not fit the linear model to compute the relevant sums of squares. Given that we have performed the AVE computations, we have all the needed information. For example, using the mixed model to illustrate, the F-statistic for the hypothesis, H2 : 42 = 0, as described in Table 13.13, is obtained by using the parameter estimates from W2 and computing

(14.18)

To test the hypothesis, Hl : pi - p. in the mixed model, recall that the numerator sum of squares is the same as that for the mixed model with expected value A1 = 412 + as&. Thus the numerator sum of squares can be computed by using the estimates of 41 and q512 in this expression as if both hctors are random. The test statistic is thus given by

(14.19)

where 0 is defined in Example 13.5. Thus the AVE method yields all of the information given in the AOVmethod plus the important diagnostic information. The amount of computation for the AVE method is also slightly


less, but in this era of high-speed computing, this is not an important concern.

Exampfe 14.2. n > 1. In Example 13.2, we considered an experiment designed to examine the production of al = 3 machines when run by a2 = 6 operators on n = 3 occasions. The sample cell means are given in Table 13.3. It was assumed that factor one is fixed and factor two is random. To develop the model we assume that the response on the ith machine has mean pt and variance c2 and that observations on the same operator on a given machine are correlated with covariance ?,/J,~. We allow for a different correlation, with covariance 42, when the operator is using a different machine. Letting yt,, denote the response on the rthoccasion with the jth operator on the ith machine, the model is described as

EEY/,,,J = P*

VarEv,,,l = fY2

C O V f Y y r > Y,*j*,*I = $12 i = i ' j=j* r f r * = 4 2 i f ? j=j*

= O i f i ' j # j * . (14.20)

For notational convenience and to compare with the linear model formulation, we introduce the parameters, 412 = $32 - 49, and & = o2 - 42 - 412. It follows that this covariance structure is the same as (1 3.14) with gk = 0 and the appropriate notational changes. The covariance matrix can be written as in (13.15) or (13.20). Thus this model agrees mathematically with that used in Example 13.2, but the only restriction is the matrix be positive dehite. This requires that the eigenvalues noted in (13.20) are positive but does not require that all of the individual components are positive as in the linear model formulation.

We will fmd it convenient to work with the vector of cell means, g. Note that the cell means have Var[jj,,.] = 42 + +12 + $,,/n. Cell means for the same machine have covariance 952 and cell means for different machines are not correlated. In matrix form we write

(14.21)

Using the expression for the sums of squares f?om Table 1 I .4 and the expected mean squares &om Table 13.4, (62 is estimated by


Note that the estimate of q52 can be computed as before as the average of the off- diagonal elements of the sample covariance matrix W2, where this matrix is computed fiom the two-way array of cell means using machines as variables and the operators as observations..

The average of the diagonal elements of this matrix is written

with expected value

(14.23)

(14.24)

We will use this general definition of y12, noting that the definition for the case n = 1 is a special case using the convention q& = 0. We also note that y,2 # $ J ~ ~

the covariance defined in (14.20). The residual mean square estimates rPe , written in matrix form as

1 1 1

a1 a2 n - 1 3, = $( -1, €3 -1, @ -s+. (14.24)

The diagnostic analysis on the estimatesof y2 and y12 follows as in the previous case, except that we now use the cell means in the box-plots and the scatter-plots. Diagnostic information on the individual elements is found in g5e, since it is the average of the within cell sample variances. These individual estimates should vary reasonably about the overall estimate, and box-plots can be used to identifj. unusual data. The estimate of Cp12 follows fiom (14.24) and can be negative as can the estimate of 42.

The W2 matrix is shown in Table 14.2, where we show sample correlations in italics above the diagonal.

A

TABLE 14.2. The Wz-matrix for Exampie 14.2 Mach.1 Mach.2 Mach.3

Mach. 1 16.95 0.790 0.612 Mach2 28.24 74.70 0.763 Mach.3 11.15 29.18 19.57


The averages of the off-diagonal elements, the diagonal elements and the within cell variances yield the estimates

3, = 22.86 y12 = 37.07

q5e = 0.93

h

h

and the definition ofy,, in (14.24) gives &, = 13.91. In Example 13.2 there were no negative estimates to raise questions about

the data. The estimates indicate that the population of operators is highly variable and might suggest that a new training program be instituted. Before reaching this conclusion, a detailed examination of this experimental data is recommended. The A VE method provides a means of examining these estimates.

The estimate of 4, is computed as the average of the within-cell estimates. Since this represents a small portion of the total variability for this example, we make only a few observations. The average of the within-cell variances for the three machines are (1.32, 1.00,0.45). The large estimates for the first two machines are a result of the sample variances for cells (1, 6) and (2, 3), which are 4.96 and 4.04, respectively. With only three observations per cell, such estimates are not too surprising and we will comment later on operators three and six. For now we note that they represent the extremes in average performance.

The estimate of the variance of the cell means 175~ + q512 + (I/n)& is the average of the sample variances (16.95, 74.70, 19.57). Machine two contributes a major portion of this and an examination of the data for that machine is warranted. In Figure 14.3, we show the box-plots for the data for each of the machines fkom Table 13.3. It is clear that the low productivity of operator six on machine two is the primary reason for this large variance. That operator also did poorly on machine one. We note that operator three performs consistently well.

The estimate of d2, the covariance between observations (or cell means) of a given operator on different machines, is the average of the sample covariances (28.24, 1 1.15,29.18). The higher estimates based on the covariances of machine two with machines one and three attract our attention, and we are interested in an examination of the data leading to these estimates.

In Figure 14.4 we show the scatter-plot matrix associated with the pairs of machines. The effect of operators three and six is dramatically illustrated. Without these two operators, these covariances would be much smaller. For example, deleting operator six, the sample variances are (1 1.92, 9.82, 17.05), and the sample covariances are (7.49, 5.65, 11.59), yielding estimates of & = 8.24 and &, = 4.45 with 9, = 0.72.

h

14.2 Two-way Crass-Classification Model 501

- MAC1 MAC2 MAC3

Machine

Figure 14.3 Box-plots for Example 14.2.

0) m l t.

4 6 5 2 5 8

MAC1

4 6 5 2 5 8

MAC1

4 5 5 5 6 5

MAC2

Figure 14.4 Scatter-plot matrix for Example 14.2.


The effect of sample size on the estimation of variance components is demonstrated by noting that, although we have 54 observations, our covariance estimates are based on only six pairs of observations. (Of course, these points are based on means of three observations.) It follows that each pair of points has considerable leverage in determining the estimate. An obvious design improvement would be to increase the number of operators in the experiment.

Further examination of the data for operator six shows that the performance on machine three was much better. A tentative conclusion based on these data is that, not only does machine three have a higher average productivity, but also that it may be robust to operator dif€erences. Perhaps it is easier to operate. Further expwimentation to examine this feature is suggested.

In this example our diagnostic analysis was not motivated by the occurrence of a negative estimate but was very informative. Such an examination should be a routine part of the overall analysis of the data.

Random Model. The extension to the random model follows the same pattern. We now assume that E[y,,,] = p and that observations on the same machine are correlated with covariance 41. Using the parameter definitions in (14.20), the covariance structure is written as

Var[Pijvl = 4 e + dl + 42 + 412

C4Y,,,, n*,-,*l = 41 -t 4 2 + 4 1 2 i = z * j = j " r # r' = #, i = i * j # 3" = 42 i#i' j = j 3 = o i#i' j # i . (14.25)

Again, we will work with the cell means with covariance structure

In matrix form, we have

1 n V ~ ~ G I = ( - 4 e + 412)1+ c ~ l ( l 1 ~ ~ i ) + 4 2 ( ~ 1 @ 1 2 ) . (14.27)

The estimates of d2 and yI2 = + q512 + #2. are obtained fi-om WZ and the estimate of 4, from the residual mean square as in the mixed model. The estimates of and yZl = ide + 412 + d1 are obtained as the averages of the off-diagonal and diagonal elements of the sample covariance matrix Wl. This matrix is computed using operators as variables and machines as observations.


Examination of this matrix and the associated scatter-plots and box-plots provides valuable information about the data.

Inference on Parameters. The computation of the test statistics follows the same pattern as in Example 14.1. That is, for hypotheses on the ratio of eigenvalues of V, the test statistic is just the ratio of estimates as in (1 4.18).

For the mixed model, testing the hypothesis, HI : pt - p. we use the same approach as in Example 14.1. That is, the numerator sum of squares is computed as if both factors are random. Using the expected mean square under the random model and inserting the estimates, we have

3, + n3I2 + na231 F = ,, h

(14.28)

Here, the estimate of estimate of 412 fiom W2.

is obtained fiomthe covariance matrix W1and the

Am Table. It is convenient to describe the AVE estimation procedure in tabular form analogous to the AOV table. The table gives a symbolic representation of the procedure indicating the matrices of the quadratic forms and their expected values. The computations are done in terms of sample covariance matrices. The general issue of computation is addressed in Section 14.7.

The rows of the A VE table are similar to those in the AOV table but since they do not correspond to effects and interactions, we use the numerical labels that we associated with the parameters. The labels are the elements of the index set T defined in Section 12.2. Note that this same labeling could have been applied to the AOV table. For estimation of the components, we only use the rows corresponding to random effects. (See Exercise 14.2 for an interpretation of the role of the rows corresponding to the fixed effects.) The quadratic forms, for all except the residual row, are expressed in terms of the vector of cell means. In AVE tables we usually show only the matrices of the quadratic forms on the cell means. To express them in terms of the original data, simply post-multiply each matrix by @ (l/n2)Un. The residual row is usually omitted since c # ~ is always estimated as the average of the within cell variances. We adopt the notation "lt, t E T, for the expected value of the quadratic forms. We do not include the redundant row associated with the diagonal elements of Wl since it is not needed for estimation. Table 14.3 give the information for the random, two-&ictor model with replicates.

Note the relation of the labels to the matrices in the quadratic forms. Thus, the largest integer in the label is associated with the matrix S, the other integers in the label are associated with I, and integers not present in the label are associated with C. Rows with the same maximum integer are computed

SO4 Chapter 14 Mixed Models 11: The AVE Method with Balanced Data

TABLE 14.3. A YE table far tbe twefactor model Label Matrix Expected Value

1 is1 c3 &C2 7 1 = $*

2 L C l @ a1 r1 is2 7 2 = $2

8'1 €3 3 712 = 4 2 + $12 + 54, 12

fiom the same covariance matrix. For the mixed model we ,,Ae the row labeled 1. If n = 1, we adopt the convention that (6, = 0.

14.3 TWO-FOLD NESTED MODEL

Ejcampie 14.3. The two-fold nested model was illustrated in Example 13.4, where we described a study with al = 4 vessels, az = 4 loads, that are nested in vessels, and a3 = 2 tests performed on each load. We will see that, with klly nested models, the A VE and AOV analyses use the same quadratic forms. The diagnostic analysis in such cases does not involve the computation of sample covariances or the examination of scatter-plots but uses within-cell or marginal variances. This example introduces the effect on the A VE table when there are nested hctors.

Recall that the model was written

Yijk = P + vt + l(v)ij + t(l(v))ijk 7 (14.29)

where the random effects have variances, dl, c$2(1), and (63(2(1)). The estimates of these parameters were given by

31 = 0.0682 &(I) = - 0.0077 33(2(1)) = 0.0872 (14.30)

with total variance estimated as var[yiJk] = 0.1477. Implicit in these assumptions is the covariance structure

Var[%jkl = dl + dZ(1) + d3(2(1))

C04Yijk7 Yi'fk'l = (61 + (62(1) i = i*, j = f , k # k* = dl i = i ' , j # $ , k # k* = o i# i " . (14.31)

Thus, we are assuming that observations on the same vessel are correlated, allowing for different correlations if they are on different loads.

The AVE method is summarized in the Table 14.4. In this case the quadratic forms are for the response vector, not the vector of cell means. The labels are given by the index set T defined in Section 12.4. The expected values

14.4 Three-Factor, Nested-Factrial Model 505

in this table are identical to those in Table 13.4, apart .from division by the coefficient on the leading component, and the quadratic forms are similarly related. The A YE method does suggest a diagnostic analysis. The estimate of y1 is the sample variance computed fiom the four vessel means j j t . . , (5.38,5.57,4.94,4.79), yielding the estimate = 0.077. The estimate of y2(1) is the average of the four sample variances, computed for each vessel using the marginal means &. These four sample variances are (0.033,0.087,0.015, 0.009), yielding the estimate y2(1) = 0.03585. The estimate of (63(2(1)) is the average of sample variances computed for the two tests within each load (vessel) combination. Examination of these single degree-of-fieedom estimates shows that the estimate .from vessel C, load four is 0.898. This cell represents 64% of the sum of squares for this effect in the AOV table.

As suggested by the diagnostic analysis, the data for vessel C were re- examined and it was found that the observation for test one on load four in vessel C was recorded in error. The reading of 5.61 should have been 4.61. Repeating the analysis yields the estimates

A

41 = 0.0818 &(I) = 0.0267 $3(z(l)) =z 0.0347

with total variance estimated as vur[yijk] = 0.1432. Note that most of the variation comes fiom differences in the testing vessels. An examination of performance of these vessels was recommended. It would have been better to run more than two tests on each loaded vessel to obtain better estimates.

The reader should become familiar with the relation of the matrices to the computation of the estimates. In this case, they are just averages of sample variances computed for various marginal means as indicated by the U matrices.

14.4 THREE FACTOR, NESTED-FACTORIAL MODEL

To provide Wher motivation for our general presentation of the ACE method, we describe the analysis of a model with both nested and crossed factors.


Example 14.4. For this example, we use the data from Example 13.3 for the experiment designed to test u2 = 2 types of aerosol on filters fiom al = 2 manufacturers who each provided a3 = 3 filters. The data are found in Table 13.6. The model, assuming factors one and two are fixed, was described in (13.27) with implied covariance matrix given in (13.28). To develop the covariance specification model, we assume that the response on the (ij)th manufacturer-aerosol combination has mean ptJ. Let u2 denote the variance of an observation and assume that observations on the same manufacturer-filter combination are correlated allowing for a different correlation for different aerosols. Letting y+..denote the observation for the rth replicate for the kth filter €torn the ith manufacturer using the jth aerosol. the model is described as

E[a,krI = Dl3

Var[%,,l = rY2

cov[y,jk,, Yz*j*k’r*l = $123 = i* j = j * k = k * T # T *

= 4 3 ( 1 ) i = i* j # j * k = L ‘ = O i f ? k # k*. ( I 4.32)

To relate to the covariance structure in (13.28), we define (623(1) = ‘$123 -

and $e = c2 - 4 3 ( 1 ) - (623(1).

covariancematrix is given by(13.29) or (13.31).

that

With the obvious notational changes, the

In the AVE method we work with the vector of cell means, & , noting

Var[~zjL 1 = 4 3 ( 1 ) -t $23(1) + 4 e / n cov[?l%jk 7 ?lz*j”k* 1 = $3(1) i = c j # ~ j k = k *

= o i # i* or k # k*. (14.33)

To relate to our general discussion, we define the parameter functions

Using (1 3.29), we write this in matrix form as

Using the expressions for the sums of squares in Table 12.9 and the expected mean squares from Table 13.7, the AOV estimate for $3(1) is given by


(14.36)

This result follows from the definition of C in (14.7). Comparing this expression with (14.7) and the algebraic expression in (14.10), we see that this estimate is written in algebraic form as

To interpret this expression, we arrange the cell means as in Table 14.5. The inner summation computes the sample covariance for pairs of columns. The next two sums direct us to do this for all pairs of aerosols and the outer sum says to do this for each manufacturer. Thus we compute al = 2 covariance matrices (one for each manufacturer) and sum the off-diagonal elements of these matrices to form the estimate. These covariance matrices are shown in Table 14.6 where, for later reference, we give them in block diagonal form. correlations are shown above the diagonal.)

1 TABLE 14.5. Cell means for Example 14.4 Manufacturer

1 2 Aerosol Aerosol

Filter 1 2 1 2

1 0.787 0.717 1.113 0.897 2 0.088 3.630 0.177 0.700 3 0.078 1.830 0.130 1.757

TABLE 14.6. Covariance matrices for Example 14.4 Manufacturer 1 Manuhctwer 2

Aero.1 Aer0.2 Aero.1 Aer0.2

Aero. 1 0.165 - 0.783 Aer0.2 - 0.468 2.161 Aero. I 0.304 - 0.353 Aero.2 -0.118 0.318

(The sample


The average of the diagonal elements of these two matrices is the estimate Of 7(23(1) given

(14.38)

Let W3 denote this matrix and let W3(1)( i ) denote the diagonal blocks.

estimates, we have the estimates Using these results and the estimate of 4e as the average of the within cell

h 8, = 0.30

723(1) = 0*74

(b3(1) = - 0.29 h

h

423(1) = O.93 *

These estimates agree with those obtained in Example 13.3 and we are now prepared to explain the negative estimate of 41~(~)and to do a general diagnostic analysis on the data. The negative estimate arises because both off- diagonal elements are negative. In Figure 14.5 we show scatter-plots of the cell means for each manufacturer using the filter number as the plotting symbol. At first glance the primary cause of the negative estimate seems to be filter one in each case, even though these filters are not necessarily related for the two manufacturers. However, firrther inspection of the data for manufacturer one shows that aerosol two has a much higher mean for filters two and three and this feature is equally responsible for the negative estimate. A similar comment applies for manufacturer two. The essential point to note here is that the covariances are based on only three pairs of observations, hence each pair has high leverage. It would have been better if each manufacturer could have contributed more filters to the experiment.

The estimate of ~ 2 3 ( 1 ) as the average of the four estimates (0.165,2.161, 0.304,0.3 18) suggests that the estimate is inflated by observations for aerosol two fiom manufacturer one. The estimate of (be is the average of the within-cell variances. The residual sum of squares for the cell, (1, 1, 2,) is 5.45 which contributes a substantial portion of the pooled residual sum of squares, RSS= 7.24. An examination of the observations in this cell, for possible

A1 i:: I 1 A1 :\! 0.4

0.0 0.0 0.0 1.2 2.4 3.6 0.0 0.6 1.2 1.8

Manufacturer 1 Manufacturer 2 A2 A2

Figure 14.5 Scatter-plots for Example 14.4.


recording or other errors, is suggested. Further diagnostic discussion is not warranted for this example, but the design implications should be clear.

Random Model. To provide more insight into the A VE method, we extend this example by assuming that all effects are random. Thus, E[yjk,] = pand we extend the covariance structure on the cell means (14.33) as

Here, we have introduced the parameters 412and (p3(1) to allow for different correlations on observations fiom the same manuhcturer. The expression of the first covariance as the sum 4, + 42 + I # J ~ ~ is only for notational convenience and places no restrictions on the parameters. The parameter 423(1) arises in the covariance between replicates on the same manufacturer-aerosol-filter combination. This structure is written in matrix form as

1 var[g1 zz ( k 4 e -t 4 2 3 ( 1 ) ) I + h ( I 1 @ u 2 '8 v3) -k 42(v1 @ I 2 @ u3)

+ $12(Il @ I 2 @ u 3 ) + &(1)(11 @ u 2 '8 13) . (14.40)

The development of the A YE estimates follows in the usual way. We give the details of the development in Section 14.5. For now, we simply give the formulas in Table 14.7. The labels are defined by the index set T* described in Section 12.6. Note that the expected values are similar to those for the two-way cross classification in Table 14.3 but modified due to the nesting as suggested by Table 14.4. The last two rows represent the A VE table for the mixed model and


have been described. To understand the first three rows, we note that the matrices are similar to those in Table 14.3 for the two-factor, cross-classification. The difference is the addition of the factor, U3/a,2. The effect of this is to compute the marginal means averaging over the third subscript. For example, the quadratic form in the second row is

It follows that these quantities are obtained from the covariance matrices, Wl and W2as described for Example 14.2. The expected values are similar apart fiom the addition of 43(l)/a3 in the first row and 423(l)/a3 in the third row. The diagnostic analysis follows the usual pattern. The row associated with the sum of the diagonal elements of W, is not included in this table since it is not used for variance component estimation, but it is of interest. (See Exercise 14.2.)

lnferencs on Parameters. To describe the test statistics in the mixed model we recall the hypotheses on the variance components and the marginal means as described in Table 13.7.. Again, the form of the test statistic is known but the issue is computation since we have not fit the linear model to compute the relevant sums of squares. Given the A VE computations for the random model in Table 14.7, we have all of the needed information. Using the estimates of the variance components from the AVE table and recalling the expected mean squares from Table 13.7, the test statistics for the variance components are

To describe the test statistics for the fixed effects, we write the expected mean squares for the random model in Table 14.8, using the same format as Table 13.7. In this table, we use the labeling as in Table 14.8 and show only the expected mean squares. The sum of squares and degrees of freedom are given in Table 12.9.

Using the ratios of mean squares suggested by Table 13.7, the expected mean squares for those rows in Table 14.8 and the estimates of the variance components from Table 14.7 for the random model, the test statistics for the marginal means hypotheses given by

14.5 General Description of the AVE Table 51 I

- pz.-ii.j + ii.. (1 4.43)

We emphasize that the denominators in (14.43) are computed fiom the AVE estimates of the variance components and the numerators are computed fi-om the appropriate diagonal and off-diagonal elements of Wl and W2. A general description of the computational procedure is given in Section 14.7.

14.5 GENERAL DESCRIPTION OF THE Am TABLE

The A VE method illustrated in Examples 14.1 - 14.4 extends to a general class of mixed models that we call mixed-factor models. To motivate the definition of this class of models, we note that Examples 14.1 - 14.4 all had the property that one of the factors was random. The restriction in Section 13.5 implies that any interaction effect involving that factor is random. Further, we saw in those examples that if an interaction effect was random, then at least one of the hctors associated with that interaction was random. Our discussion of the A VE method considers this restricted class of models. Formally, we state the restriction as follows:

Restriction: In a mixed factor model, if t E T,, then at least one integer in r must correspond to a random fixtor.

Our primary interest in variance component analysis will be applied to mixed- hctor models, since we are interested in assessing the sources of variability in the process we are examining.

To illustrate a mixed effects model that is not a mixed fhctor model, consider a two-hctor, cross-classification model with only one experimental unit

512 Chapter 14 Mixed Models II: The AVE Method with Balanced Data

per factor combination and assume that we make m measurements on each experimental unit. If we assume that both factors are fixed, we note that we must assume the no-interaction model on the cell means. The variability on the observations has two components, one due to differences in experimental units and the other due to variability in the measurements. The model in over- parameterized form is written

gijk = p + ai + Pj + S,j + e i j k . (14.44)

We emphasize that this situation differs from that in which we have m replicates for the factor combinations. In this case, both Si j and ejjk are assumed to be random variables with variances bI2 and be, respectively. The interaction row in the AOV table provides the denominator for the test statistics. This is a mixed effects model, but it is not a mixed factor model. It is models of this type that we initially exclude from our general presentation of the A VE method. We will see that the method is easily extended to cover such models, but it complicates the notation to include them at this point.

The development of the AVE method depends on the description of the A VE table that is the focus of this section. We will describe it for the model with all random factors. We have seen that the rows corresponding to fixed effects in the mixed model are used for making inferences in fixed effects. For notational simplicity, we separate the discussion of cross-classification models and those with nested factors.

14.5.1 A VE Table for Factorial Models

Consider a factorial model with c crossed factors where the ith factor has ai levels and assume n replicates for each hctor combination. Assume that all effects are random, and the rows of the A YE tabIe are labeled by t E I: where the index set was defined in Section 12.2. In the mixed model, we only use the rows that are labeled by t E T2 for estimating the variance components.

The estimate of q5e is given by the pooled, within-cell sample variances. Written as a quadratic form on the vector of observation, we write the estimate in matrix form as

(14.45)

The remaining components are estimated as linear combinations of a set of linear functions, T ~ . The estimates of these functions, qt, are described as quadratic forms on the vector of cell means using the matrix Qt. This matrix is defined as Kronecker products of matrices that we have encountered in our previous discussions. The estimates are given by

14.5 General Description of the AVE Table 513

(14.46)

where the matrices in the Kronecker product are defined by

LSq if i = i,,

'Iai if i E t, i # im, (14.47) ri

ai

airi "Cai if i e t .

Qi =

We have used the subscript ai, for clarity, to indicate the dimension of the matrices. As has been our practice, we will just use i to simplify the notation. To write the estimate in terms of 9 we simply adjoin the matrix Qefl = (l/n2)Un to the Kronecker product. The convention of associating the residual matrix Si with the largest integer in the subscript t provides a unique set of parameter functions for estimating the variance components.

To determine the expected values of the quadratic forms, we use (13.36) to write the covariance matrix of the vector of cell means as

where

I , if i E t

U, i f i 4 t. La =

(14.48)

(14.49)

It may be verified that the expected values of the quadratic forms are given bY

1 n

+ 6-&. (14.50)

The sum in this equation is over all elements, m E T, such that, (1) the integers in m are in t and (2), the maximum integer in t i s also in m. The coefficient 6 on the last term is defined to be one if t contains all of the integers 1, s.1, k and zero otherwise.

These results are summarized symbolically in Table 14.9. In the random model we include all rows for t E T, as indicated in the table. For the mixed model only the rows for t E T2 are necessary for estimating the components.

$14 Chapter 14 Mixed Models [I: The AVE Method with Balanced Data

TABLE 14.9. A VE table for random factorial models Matrix Expected Value

To illustrate, we show in Table 14.10, the AVE table for the three-way, cross-classification model. The row associated with #e is not included. To show that these quadratic forms provide estimates of the variance components, let y and (b denote the vectors of expected values and variance components in the usual order with ye = de in the last position. From (14.50), it follows that we may write

r = m , (14.51)

where

B = [ r ;] (14.52)

In (14.52), is a lower triangular matrix with ones along the diagonal, and a is a vector of zeros apart fiom the last element which is l/n. For example, with c = 2, we have

B = 0 1 0 . (14.53) [ a : PI It follows that h B is non-singular, and the estimates of the variance components are given by (b = B-'?, with T t given by the quadratic form, qt. The same


argument applies to the mixed model using only the rows for t E T2. If n = 1 as in Example 14.1, we note that we cannot estimate (6,.

In our introduction to the A VE method, we derived the quadratic forms as linear combinations of the mean squares from the AOV table. To establish the equivalence of the two methods, we now describe these relations for the general factorial model. To motivate the discussion, we first describe the relations for the three-way, cross-classified model.

Recall that the AOV mean squares are defined by the quadratic forms on the cell means, it = r&, where xt is defined by (12.28) and (12.29) with G,+l replaced by n. The expected values, A, are defined by (13.43). The estimates of the A VE parameter functions in Table 14.10, expressed in terms of the AOV fbctions, are given by

Here we have used the notation, a12 = al@ as defined in (12.24). It is easily verified that these relations reduce to the quadratic forms in Table 14.10. (Notee: The same relations hold for the parameter functions.)

The pattern established for the relations with main effects and interactions should be noted. In particular, the y-functions associated with the main effect functions, yE are differences of differences of the form (A, - Xyk) for i = l , . - + , c , j # i a n d r = l , * - - , c excepti a n d j . Thetwo-hctorinteraction terms, yE3, are differences of sums of the form (Xk3 + T , X , ~ ~ ) and the three-factor interaction is the linear function indicated in (14.54). To indicate how these definitions extend to higher-way classification models, note that, with four factors, we have


This example should indicate the definition of the functions. Rather than describe the general relations algebraically, we do so in matrix form. Writing the relation between paratpeter functions as 7 = LA, we define the matrix L inductively as follows:

For c = 2:

(14.56)

For c = 3:

In general:

( 1 4.58)

where


(14.59)

The 0 matrices have dimension as needed. The limits on the Kronecker product imply that the index is decreasing fiom left to right. This matrix is non-singular and gives the unique relation between the two sets of parameter functions and the two sets of quadratic forms. As a practical matter, it is not used since the y- functions and their expected values are easily defined as in (14.45)- - (14.46) and (14.50). However, if the A-functions are computed for other reasons, this is a convenient way to compute the expected values of the y-functions. The quadratic forms are not computed directly but obtained fiom the appropriate covariance matrix. The details of the computation are described in Section 14.7.

14.5.2 A YE Table for Nested Models

As noted in Example 14.3, the A VE method does not differ significantly fiom the AOV method. The only contribution is to suggest that the AOV sums of squares be viewed as averages of sample variances. It is of interest to spell out the AVE table as it will help us to understand the table when we have both crossed and nested kctors. The labels in the A VE table are defined by the index set in Section 12.4. The quadratic forms are given by

(14.60)

where the matrices in the Kronecker product are defined by

'S, if i = i,,

Q,. = {: lIai if i < i,, #Uq if i>i,,.

We note that these are related to the AOV quadratic forms defined in (12.69) - (12.71). In particular, we have the relation

(14.61)

h at- Yt = --At, na

(14.62)

where Xt denotes the AOVmean square. Table 14.4 suggests the AVE for the general, hlly nested model.


14.5.3 A VE Table for Nested-Factorial Models

To generalize to the situation in which we have combinations of crossed and nested factors, assume that factors 1,2, . . ., c have the factorial structure and that the remaining factors, c + 1, c + 2, .-., mr are nested in some way. The A VE table is described for the case in which all factors are random, hence the rows of the table are labeled by the elements of the index set T* defined in Section 12.6. The table for the mixed model follows by including only the rows labeled by q, Recall that a typical element of I" was written as t = t2(t1), where tl represented all integers that are in parentheses and t2 represents those that are not. The definition of the quadratic forms in (14.47) is extended by noting that the matrices in the Kronecker product are

( :st if i = i,,

"I , if i E t , i f i,,

'C, if i $ t 9 f o r i = 1 , - - . , c

$Ut if i @ t , f o r i = c + l , - - - , m .

(14.63) 4

4r,

The expected values for these quadratic forms follow fiom the covariance matrix as defined in Section 13.5. The rows corresponding to the crossed factors are given by (14.50), as if there are only c factors, with modifications due to the nesting as indicated in Table 14.4. The rows corresponding to the nested factors are similar to the analogous rows in (14.50) with 4, = 0 if appropriate and the modifications caused by the nesting. Rather than develop rules for writing these expected values,y,, we develop the relations between them and the AOV expected mean squares, At. Since (13.43) gave simple expressions for At as a h c t i o n of 4, this provides a simple way to obtain the relations that we will find usefbl when we describe the computing procedure in Section 14.7. At the same time it will establish the equivalence of the AVE and AOV estimators. To motivate these relations, we describe the relations that led to the estimators for Example 14.4 as described in Table 14.7.

From the Table 14.7, we write the relations


(14.64)

Note that: these relations follow the pattern described in (14.54) with 3(1) treated as a main effect and 23(1) treated as a two-kctor interaction. Additional examples in Section 14.6 will illustrate the general situation.

14.5.4 AVE Method for General Mixed Effects Models

For convenience of exposition, we have restricted our attention to the mixed factor model, but the procedure also applies to the general mixed effects model. Exactly the same computations can be used to estimate the variance components. The only modification is in the interpretation of the rows of the A VE table. In particular, the specification of the expected values of the quadratic forms. To illustrate, consider the two-factor, fixed effects model with one experimental unit per cell but with T measurements on that unit.

Example 14.5. Two-Factor, Fixed Effects Model with Sub-Sampling. The model is written in (14.44). To estimate the variance components, we use the quadratic forms fkom the last two rows of Table 14.3. Under the sub-sampling model the expected values of the quadratic forms are given by

7 2 = QAP) 1

712 = 412 + 4 e + Q20r)7 (14.65)

where

(14.66)

The estimate of q$12 + (l/n)~$, is obtained as the difference between these quadratic forms. The estimate of q5e is obtained as usual fiom the within-cell estimates. As noted in Example 14.2, the average of the sample covariances eom W2 provides the estimate of Q2(p). As usual, the individual covariances and the associated scatter-plots should be examined for consistent behavior.

The method implied by this example extends to the general case of mixed effects models which are not mixed tictor models. That is, we follow exactly the same procedure to estimate the variance components as in the corresponding random model. The role of the rows of the AVE table that correspond to fixed effects are used to determine the estimates, but they do not estimate variance components.


14.6 ADDITIONAL EXAMPLES

To further illustrate the AVE method, we present several numerical and conceptual examples. The examples will iilustrate the computations as well as the diagnostic information contained in the method.

Example 14.6. Three-Factor, Cross-CIassification. To illustrate the A VE method for a higher-ordered kctorial model, consider the experiment described by Hocking, Green, and Bremer (1 989). The experiment was run on a chemical- industry process to identi@ sources of variation with the objective of improving the quality of the product. A continuous medium, made fiesh each day, is sent through a conduit to a machine for processing. From the machine, two smaller conduits direct the product to opposite sides of the machine. On each side there are four packaging units for final processing. The response is a measure of the property of the product. The four packaging units are side-by-side, with units one and four on the outside and units two and three on the inside. Like- numbered units are opposite each other on the two sides. The location of the unit is of primary importance. The experiment was modeled as a three-factor cross- classification. The three factors are side, al = 2, unit, a2 = 4, and day, a3 = 60. The side and unit factors are assumed to be fixed and kctor three is assumed to be random. Data are taken over 60 days with n = 4 packages taken on each side-unit combination on each day.

The linear model is given by

where p i j denotes the mean response for the (ij)th side-unit combination and the remaining terms are random variables with variances, 43, 413, 423, $123 and &, respectively. The covariance specification model is determined by assuming different correlations for observations on (1) the Same side, the same unit and the same day, (2) the same side, the same day and different units, (3) the same unit, the same day and different sides and (4) the same day but different sides and units. All other correlations are zero. Using notation to correspond to the linear model, the covariance structure is given by

14.6 Additional Examples 521

= 6 3 + 6 2 3 i # i * j r f k = k *

= 43 i # i " j # j " k = V . (14.68)

The matrix expression for the sums of squares for the AOV method were given in Table 12.1. The AOV table for their mixed model was developed in Exercise 13.1 1 . Since our primary interest in this example is to illustrate variance component estimation, we show the part of the AOV table associated with the random efEcts in Table 14.1 1. The AOV estimates of the variance components are

h h

qh3 = 17.1 4 2 3 = 2.1 h h A

4 1 3 1.3 $123 = 16.1 4e = 1.48 .

The AVE table is given in Table 14.10. As suggested by the matrices of the quadratic forms, we compute a covariance matrix as follows: Array the cell means in a two-way table, with al x a2 columns indexed in the usual way (in this case 11, 12, 13, 14, 21, 22, 23, 24) and with q rows. The sample covariance matrix W3 computed using the columns as variables yields the estimates as follows:

1. The estimate of 7123 = (p3 + dI3 + $23 + 4123 + is the average of the diagonal elements. 2. The estimate of 73 = d3 is the average of the off-diagonal elements for which i # 1- and j # j". 3. The estimate of 713 = 43 + 6 1 3 is the average of the off-diagonal elements for which i = i" andj #f. 4. The estimate of 723 = q53 + $23 is the average of the off-diagonal elements for which i # J* andj =I*.

1 TABLE 14.11. AOV table for random effects in Exnmde 14.6 I


The data for this example are not given to protect proprietary information, but the W3 matrix, not including the correlations, is shown in Table 14.12. For reference we have separated the lower left-hand corner sub-matrix. The average of the twelve, off-diagonal elements of this sub-matrix is the estimate of 4 3 . The estimate of yz3 is the average of the four diagonal elements of this submatrix. The average of the twelve remaining off-diagonal elements of W3 is the estimate of yI3, and the estimate of 7123 is the average of the diagonal elements of W3.

(The covariance matrix for Example 14.4, shown in Table 14.6, was constructed in the same way. In that example only the diagonal blocks of W3 were relevant.)

The y-functions are estimated as h h

7 3 = 17.1 7 1 3 =z 18.4

7 2 3 = 19.2 7123 = 37.0

h h

and, using the expected values fkom Table 14.10, we obtain the AVE estimates of the components in agreement with the AOV estimates.

The objective of this study was to try to improve the quality of the product by identifylng the sources of variation. The estimated value of the total variance was deemed to be too large. The diagnostic analysis, which is inherent in the A VE method, is now utilized to examine the data.

Examination of the within-cell estimates of Cp, reveals no abnormalities, so we focus on the estimates of the remaining parameters. Inspection of Table 14.12 reveals an interesting pattern. That is, the large elements of this matrix are all associated with the interior packaging units two and three on either side of the machine. In particular, the main diagonal elements (54.4,47.1,48.5,56.0) used in the estimate of 7123, the off-diagonal elements (34.9,38.8) used in the estimate of yI3, the subdiagonal elements (31.5,27.1) used in the estimate of

TABLE 14.12. Ws matrix for Example 14.6 Side

1 2 Unit

Side Unit 1 2 3 4 1 2 3 4

1 23.2 1 2 16.8 54.4

3 14.6 34.9 47.1 4 11.7 24.3 17.9 29.0

I 6.8 16.7 14.6 9.7 18.1 2 2 11.7 31.5 33.2 12.2 13.4 48.5

3 10.5 31.1 27.1 17.3 13.3 38.8 56.0 4 8.1 16.9 22.6 11.3 9.5 14.2 11.0 19.5


Y23,and the suboff-diagonal elements (33.2,3 1.1) used in the estimate of +3 are much larger than the remaining elements. A detailed examination of the data $om the interior packaging units is suggested.

In Figure 14.6, we show the scatter-plot for the data &om side one, unit two, plotted against the data fiom side one, unit three. The sample covariance is 34.9 and the sample variances are 54.4 and 47.1, respectively. At first glance, the plot did not appear to be unusual, with no obvious outlying observations contributing to the large values of these estimates. However, there is an explanation. The plotting symbol in Figure 14.6, is an indicator of the day of observation, with V denoting the first 30 days and A the second 30 days. We see that observations taken early in the experiment had generally smaller responses than those taken later. This results in an inflation of the marginal variances and induces a stronger correlation than would be obtained if the data were separated into two groups.

Further examination shows that a similar pattern exists for all interior units (two and three) but not for the exterior units (one and four). Examination of the records shows that the source of raw material was changed after the first 30 days of the experiment. Subsequent studies have showed that the nature of the change in the properties of the raw material was such that it did not affect the exterior units but had a dramatic effect on the interior units. The change is indicated in Table 14.13, where we show for each side-unit combination the mean response using all of the data, and the mean responses when the data are partitioned into the first and second 30 days of the experiment. The mean shifts for the interior units is noticeably greater. Recognizing this feature of the processing units enabled the company to improve the quality of the product, a result that would not likely have been achieved without the AVE analysis.

8 60 70 80 90 100

Side 1 - Side 3

Figure 14.6. Scatter-plot for side-unit 1,2 and side-unit 1,3.


TABLE 14.13. Mean responses for Example 14.6 Side-Unit Alldata First30 Second30

11 76.2 75.7 76.7 12 85.1 81.0 88.9 13 84.4 79.7 88.2 14 76.1 75.0 77.5 21 76.6 75.7 77.4 22 84.9 81.4 88.3 23 84.8 81.3 88.7 24 74.8 74.1 74.4

The following examples do not contain data, but will illustrate the AVE method for a variety of nested-factorial models.

Ejcample 14.7. To illustrate the effect of multiple levels of nesting, consider a model with four hctors. Factors one and two are crossed, hctor three is nested in factor one, and fhctor four is nested in factor three. The index set is defined by T* = {1,2,12,3(1),23(1),4(3(1)),24(3(1))}. For reference, the AOV expected mean squares are shown in Table 14.14. The sums of squares are obtained fiom the definition in Chapter 12. The relations between the AOV expected mean squares, A, and the y-functions for the AVE method follow the pattern described earlier. For the fhctorial terms, they are the Same as in the two factor model. For the functions involving nested factors, we have

(14.69)

We note that the relations follow the pattern described in Section 14.5 with the indices, 3(1) and 4(3(1)) treated as main effects and the indices 23(1) and 24(3(1)) treated as two-factor interactions. The AVE is shown in Table 14.15, where the expected values follow the pattern noted earlier for crossed and nested factors. The matrices for the quadratic forms are given by (14.63). (For ease of reference, we separate the terms for crossed and nested effects.) The estimates of the y-functions are obtained by summing the relevant elements of the covariance matrices, W , , W2, W3, and W4. The diagnostic analysis follows as usual. (See Exercise 14.1 1 .)


TABLE 14.14. Expected mean sqoares for Example 14.7 Label df EMS

TABLE 14.15b. AVE table for Example 14.7 (nested effects) Label Matrix Expected Value


Example 14.8. To illustrate a different situation with two nested fictors, consider a model with three hctors, where factors two and three are crossed and nestedinbctorone. Theindexset isdefined byT" = {1,2(1),3(1)12(1)3(1)}. The AOV expected mean squares are shown in Table 14.16.

Since there are no effects crossed with factor one, y1 = Xl/naZ3. The remaining relations follow from our general discussion with 2(1) and 3(1) treated as main effects and 2( 1)3( 1) treated as a two-factor interaction. Thus

1

a3 %(l) = - -02(1) -

( I 4.70)

The A VE is shown in Table 14.17 with quadratic form matrix defined by (14.63) and expected values determined &om (14.70). The estimates of the y- functions, and subsequently the variance components, are obtained from sums of appropriate elements of the matrices Wl, W2, and W,. The diagnostic analysis follows as usual. (See Exercise 14.12.)

TABLE 14.16. ExDtetcd meam sauires for Examde 14.8

TABLE 14.17. AVE table for Example 14.8 Label Matrix Expected Value


h q l e 14.9. Exception to the General Rules. A four-factor model with two crossed and two nested Fdctors is defined by the index set, T' = {1,2, 12, 3(1),23(1),4(2), 14(2), 3(1)4(2)}. The AOV table follows as usual and the relations between the y- and A-functions follows the usual pattern with a slight modification. This occurs in the highest ordered interaction component since we have an odd number of A-functions to combine. Thus, we have

1 73(1)4(2) = -(A4(2) i- T1x14(2) + alT3A3(1)4(2))*

a13 (14.71)

Note that the anticipated coefficients, r3 and ~ 1 3 for the third and fourth terms have been replaced by their sum on the last term.

Example 14.10. Kirk (1995) described an experiment designed to evaluate a new drug education program for high school students. The new program and a control program (al = 2) were used at two schools(az = 2). A total of eight teachers are randomly assigned to the four treatment combinations. Thus factor three is nested in the interaction of factors one and two. The index set for his example is T" = (1, 2,12,3(12)].

We extend this example to provide a more interesting application of the AVE method. Thus, suppose that there are four factors with the first three having the factorial structure and factor four nested in the interaction of factors one and two. The index set is given by T* = {1,2,12,3,13,23,123,4(12), 34(12)). The relation between the 7- and A-functions is given by (14.54) for the crossed factors. The remaining relations are given by

(14.72)

The expected values in the AVE table follow fi-om these relations and the quadratic forms are defined by (1 4.63).

h q I e 14.11. SpIit-PIot Design. In Example 13.7 we discussed a split-plot design in which the whole-plot treatments were arranged in a randomized block design. The model was written in (13.90) and the covariance structure was given by (13.91). We noted that the model was similar to the mixed, three-factor model in which the first two hctors, whole-plot and split-plot treatments, are fixed, and blocks are considered as the random third factor. Recall that the model was the same as that used in Example 14.6 with the exclusion of the sub- plot by block interaction term, (ed)j, . (Equivalently, setting 423 = 0.) We are now ready to examine the rationale for that decision.


Comparing the covariance structure in (13.91) with that of the three-hctor model in (14.68) provides information. It is informative to describe the covariances in terms of this example. We see that $3 + q513 is the covariance between observations with the same whole-plot treatment but different subplot treatments in the same block. Similarly, $3 + $23 is the covariance between observation with the same sub-plot treatment but different whole-plot treatments in the same block. Finally, 43 is the covariance between observations with different whole-plot and sub-plot treatments in the same block. In our analysis in Example 13.7, we assumed that the latter two covariances were the same. In Exercise 13.19, the reader was asked to analyze the data assuming these covariances differed. The AVE analysis under the assumption $23 # 0 is given in Table 14.10 and the parameter estimates are obtained from the W3 matrix, as described in Example 14.6. The estimates ofthe four y-functions are:

h

y 3 = 0.057 7 2 3 = 0.061

T i 3 = 0.085 7123 = 0.113. h

If we wish to make the assumption that 723 = 0, we can use this same covariance matrix to compute the revised estimates. Under this assumption, the alternative estimate of 453 is obtained &om the relation

(14.73)

We note that this is just the average of all elements in the 3 off-diagonal 4 x 4 matrices of the W3 matrix. Using the data in Table 13.14, the relevant part of the covariance matrix is shown in Table 14.18. The estimates of the components are obtained as in Example 14.6. Under the assumption $23 = 0, the estimate 7, = 0.058, as obtained in Example 13.7, is given by averaging all of the elements in these three off-diagonal blocks.

TABLE 14.18. Part of W, matrix for Example 14.11 Temperature

Electrolyte LOW Med.

A B C D A B C D A 0.0765 0.0649 0.1323 0.1273

Med. B - 0.0225 0.0024 0.0358 0.0348 C 0.0260 0.0582 0.1031 0.1232 D 0.0406 0.0591 0.1465 0.1388 A 0.0289 0,0312 0.0724 0.0882 0.0727 0.0772 0.0769 0.1149

High B 0.0108 0.0082 0.0443 0.0363 0.0461 0.0647 0.0327 0.0847 C 0.0242 0.0298 0.0632 0.0736 0.0564 0.0604 0.0666 0.0916 D - 0.0099 0.0079 0.0258 0.0402 0.0248 0.0604 0.0552 0.0675


In Table 14.19 we show a summary of the information in the W3 matrix. The six cells of the array show the estimates of the indicated parameters obtained from that matrix. The average of these estimates yields the overall estimates. The estimates are reasonably stable, but some observations are in order. First, we note that the estimate of 7123 for the high temperature is about half that for the low and medium temperatures. Recall that this estimate is the average of the four sample variances for the data from the four electrolytes. The estimate of yI3 from that high-temperature matrix is also somewhat smaller than the other estimates. Recall that the means for the four temperatures did not differ significantly; it might therefore be best to operate at the high temperature for most reliable performance. The estimate of r3 fkom the low-high covariances is about half of the other two estimates. The small variances in the high- temperature matrix are primarily responsible for this estimate and for the low estimate of 7 1 3 from that matrix. The estimates of the variance components are given by

h

cP3 = 0.0567

&3 = 0.0044

$13 = 0.0283

$123 = 0.0236.

The diagnostic analysis inherent in the A VE method is different ftom that used in regression analysis. In the latter we used deletion diagnostics, that is, statistics reflecting the effect on estimation or prediction as a result of fitting the model with and without an observation. The problem with deletion diagnostics in mixed model analysis is primarily computational. We will see in Chapter 15 that repeating the analysis after deleting each observation is not feasible. The AVE diagnostics do not involve refitting but are merely methods of examining the data as suggested by the estimation method. Beckman, Nachtsheim, and

TABLE 14.19 Diagnostic summary for Example 14.11 Temp.

LOW Medium High

h

LOW rlZ = 0.123 .. 713 = 0.098


Cook (1987) proposed a perturbation scheme for examining the data and model assumptions using the concept of likelihood displacement to assess the effect of the perturbation. They considered various perturbations such as perturbing the data or the variances of the random effects. We will not discuss their method but we have examined their illustration in Example 14.4.

14.7 COMPUTATIONAL PROCEDURE FOR THE AYE METHOD

The description of the AVE method suggests a general computing procedure. The computations are summarized as follows:

1. the covariance structure.

The model is specified, either in linear model form, or by speci@ing

2. The index set, T or T', is determined fiom the model statement.

3. (14.61) for nested models, or in general, by (14.63).

The quadratic forms are specified by (14.47) for factorial models, by

4. The quadratic forms determine the covariance matrices to be computed. The position of the S matrix determines the variable to be used for the rows of the data matrix from which the covariance matrix is computed. The U matrix tells us to use marginal means for the data, averaging over the indicated variable. The I and C matrices indicate the elements to be averaged to estimate the associated 7-hnction.

5. indicated in Chapter 12. The y-functions are described in terms of the X- functions as indicated in Section 14.5. This can be done numerically.

The X-functions are described in terms of the variance components, as

6. estimates of the y-fimctions.

The estimates of the variance components are obtained from the

7. elements of the covariance matrices and plotting the appropriate covariance matrices or box-plots.

The diagnostic analysis follows by examining the relevant individual

8. The test statistics for hypotheses on the fixed effects or variance components are suggested by the X-functions for the appropriate mixed model. The numerator and denominator are computed as the implied linear functions of the variance component estimates. The computation of the numerator sums of squares for the fixed effects are computed as the linear functions of the variance components as described for the random

14.8 Properties of tbe Estimators 531

model as indicated in Examples 14.1 and 14.2. Interval estimates follow as in Chapter 13.

We emphasize that the computations do not require the fitting of a linear model with the implied matrix inversion. An interactive computer program is indicated, allowing the user to perform the diagnostic analyses as needed.

14.8 PROPERTIES OF THE AVE ESTIMATES

The diagnostic analysis described is based on a comparison of the relative magnitudes of the individual estimates without regard to the statistical properties of these quantities. Often they are based on small amounts of data and hence are highly variable. Further, the individual estimates are often correlated. A formal diagnostic analysis should include tests for outlying estimates, and such tests require a knowledge of the distribution of the estimators. To indicate the situation, we develop the details for the two-way classification random model. The extension to other models is straightforward.

14.8.1 Diagnostic Analysis for the Two-way Classification Model

Expressions for the individual estimates of yl, y2, and?,, in the two-hctor, random model with n replicates per cell follow from (1 4.17), (14.9), and (14.1 1). The analysis of the estimates of T~ follows fiom that for yl, so we only discuss the estimates of y1 and ylz. For the purpose of computing moments, we write the individual estimates of yl as the average the two symmetric off-diagonal elements. The individual estimates of yI2 are given by (14.12) and

As anticipated, these individual estimates are correlated, with second moments

2

r2 Vam12(i)l = - $2


4; Case 1

(14.75) { r1 (4; + 41712) Case 2- C O V l & O , 3*), &(r, r*)I =.

In case 1 there are no m m o n subscripts in (j, f) and (r, r*), and in case 2 there is a common subscript.

Inferences on the individual estimates may be based on contrasts on these estimates. Thus if we expect that TI2( 1) is an outlier, we might compare it with the average of the remaining estimates by examining the contrast

( 1 4.76)

Under the null hypothesis that the variance is the same for each row of the two- way table, it follows that E [ q = 0, and fiom (14.75) we have

(14.77)

Using the final estimates of rI2 and q51 to estimate this variance, large sample properties imply that

(14.78)

and Tl2( 1) is indicated as an outlier if this statistic is extreme. With the dimensions of typical experiments, the normality assumption is

not justified. Since the variance of the T12(i) , given in (14.74), is a function of the mean, a variance stabilizing transformation is recommended. Using the results fiom Chapter 3, we let

(14.79)

and note that

wi - fi Z71(T12) A N ( 0 , l )

COV[Wi, Wl'] = - 4; . (14.80)

The outlier test is conducted by using the contrast in (14.76) with T:2(i) replaced by w,. The variance of the contrast is obtained by dividing (14.77) by rE.

A similar appoach can be applied to the diagnostic analysis of the off- diagonal elements q51(j, f). A suggested transformation to normality is given by

$2

and

Exercises 533

(14.8 1)

(14.82)

Outlier tests can be conducted in terms of contrasts defined in terms of the zj3+.

14.8.2 Confidence Intervals

These same transformations can be used to develop confidence intervals for the 7-functions. Thus a confidence interval for (bl can be obtained as follows:

1. Use the average of the zj3* to develop a confidence interval for their expected value. 2. Invert this relation to obtain an interval estimate of (bl.

A study by Ou (1994) indicates that such intervals compare fivorably with those proposed by Graybill and Wang (1980) as described in Section 13.5. The Graybill-Wang intervals are favored because of their computational simplicity.

EXERCISES

Section 14.2

14.1 a. Verify the relation in (14.9) and the expression for 3, in (14.7). b. VerifL the algebraic expression for d2 in (14.10). c. Show that y12 = (b2 + (b12 is given by (A, + r1A12)/nal. Verify the expressions for $12 in (14.1 1) is given by ( s 2 + r1i12)/nal.

d. Verify the expressions for the individual elements of W2 in (14.12) and (14.13).

A


e. Use the general results on the expected values of quadratic forms to establish the expected values in (14.14). f. Use the expected mean squares from the AOV table for the random model to verify the expression for g. Verify, using the quadratic forms for the estimates, that (14.18) and (14.19) give the correct numerator sums of squares for the test statistics.

A

in (14.16).

14.2 a. In Example 14.1 determine the matrix Wl as if the fertilizer factor was random. Examine this matrix and the associated scatter-plot matrix for any unusual features. b. Describe the average of the diagonal elements of W , and determine the expected value under the mixed model with hctor one fixed. c. Determine the expected value of the quadratic form in (14.16) under the mixed model with factor one fixed.

14.3 Apply the A VE method to examine the variance component estimates in the Thompson-Moore, two-ktor data in Exercise 13.5. In particular, note how W . explains the source of the negative estimate. Note also the unusual structure of w,.

Section 14.3

14.4 Write the AOV table for the two-fold nested model and compare the quadratic forms and the expected values with the A YE in Table 14.4.

Section 14.4

14.5 a. Verify algebraically, using the AOV mean squares and expected values, that (14.38) is the estimate ofy23(1). b. Write the test statistics for the marginal means hypotheses for main effects for Example 14.4 in terms of the AVE estimates for the random model using Table 14.8 and verify (14.43).

14.6 a. Apply the AVE diagnostic analysis to the Littell, nested-factorial data in Exercise 13.7. In particular, compute the estimate of &(,) for each of the four modes. b. Repeat the analysis if factor two, the position on the wafer, is assumed to be random.

Exercises 535

14.7 Brownlee (1960) described an experiment designed to study two different annealing methods used in making of cans. Three coils of material were selected from the populations of coils made by each of the two methods. From each coil two samples were taken from each of two locations on the coil. The response is the life of the can. Let ,uv denote the mean response for the ith method of annealing and thejth location, and let yzJk be the response for the kth coil for that combination of factors. The data are shown below. a. Justi@ the choice of the linear model

Yxjk = k j + C ( Q ) ~ k + l c ( u ) ~ j k + elJkT

where c ( 4 W O , d3(1)), I 4 4 m +23(1)), and e N O , A). b. Fit the linear model, complete the analysis of variance table, test the usual fixed effect hypotheses, and estimate the variance components. c. Apply the AVE method to estimate the variance components. Note the difference in the estimates of based on the two different annealing methods. Comment on Brownlee's statement that more coils would be required for a definitive study. d. In the row of the A O V table for the coils within methods effect, the row labeled 3(1) in our notation, Brownlee gives EMS = ~5~ + nu2q53(1). Compare this with the expression for the expected value in your table. Comment on the effect of this difference with respect to parameter estimation and to testing the hypothesis H: +33(1) = 0.

Section 14.5

14.8 a. Verirjl that (14.54) provides the relation between the AOV expected mean squares and the expected values of the A VE quadratic forms. b. Use the Kronecker product definition of the mean squares in the AOV table to veritjl that (14.54) yields the quadratic f m s as described in Table 14.10.

Data for Exercise 14.7 Annealing method

1 2 Coils within anneals

Loc. Dup. 1 2 3 1 2 3

1 1 288 355 329 310 303 299 2 295 369 343 282 321 328

2 1 278 336 320 288 302 289 2 272 342 315 287 297 284


14.9 Use the general description of L@) in (14.58) to describe the relation between 7 and X for the four-way classification model.

14.10 a. Apply the A VE method to the Brownlee, three-hctor data in Exercise 13.18. b. Determine the expected values of the AVE quadratic forms under the alternative definition of the variance components. Show that the diagnostic fxitures of A VE are also informative under this model.

Section 14.6

14.11 Recall Example 14.7 with four factors such that the first two factors are crossed, the third factor is nested in the first, and the fourth is nested in the third. Assume that the first two factors are fixed with cell means piJ and that the remaining f$ctors are random. a. Justify the choice of the model

~ i j h = pij +A(fi)a + f & ( f i ) i j k

+h&(fi))ikt +fif4&v;)>ljkt + e i jh

where the last five terms are random with variances ~ 3 3 ( 1 ) , ~ 2 2 3 ( 1 ) , ~ ~ ( 3 ( i ) ) ,

b. Describe the covariance structure for this model. c. Describe the analysis of variance table for this model, including the Kronecker product expressions for the sums of squares and the expected mean squares. d. Construct the A VE table. Describe the relevant covariance matrices and how they are used to determine the estimates. Describe the diagnostic analysis. c. Use the data given below, taken fiom Brownlee (1960), to demonstrate the AOVand AVE estimation methods and the diagnostic analysis. Note that al = 3, a2 = a3 = a4 = n = 2.

424(3(1))7 and @e*

14.12 Develop the analysis for the situation described in Example 14.8. In particular, a. Write the appropriate linear model and describe the implied covariance structure. b. Write the mean squares for Table 14.16 in Kronecker product form and verifL that (14.70) yields the expected values and matrices in Table 14.17.

Exercises 537

Replicates Fac. 1 Fac. 3(1) Fac. 4(3( )) 1 2 1 2

3

1 2 1 2 1 2 1 2 1 2 1 2

11 16 0 0 17 16 14 13 12 12 6 6 1 0 8 1 3 9 9 3 1

19 18 6 6 8 9 6 8

14 13 6 7 15 17 21 20 22 21 13 13 18 18 6 6 19 21 6 4

14.13 Develop the analysis for the situation described in Example 14.9. In particular: a. Write the appropriate linear model and describe the implied covariance structure. b. Write the AOV table, including the Kronecker product expressions for the mean squares and the expected mean squares. c. Use the general relations between the y- and A-functions &om Section 14.5, noting the exception in (14.71) to determine the matrices and expected values for the A VE table.

14.14 For the four-kctor model described in Example 14.10, suggest a possible third kctor that might have been considered in the Kirk (1995) example. a. Write the linear model and describe the implied covariance structure. b. Write the AOV table, including the Kronecker product expressions for the mean squares and the expected mean squares. c. Use the relations fiom Section 14.5 to determine the matrices and expected values for the A VE table.

14.15 Write the linear model for the situation described by the index set T* = {1,2,12,3(1),23(1), 4(1),24(1), 3(1)4(1),23(1)4(1)}. a. Describe the implied covariance structure. b. Write the AOV table, including the Kronecker product expressions for the


mean squares and the expected mean squares. c. values for the A VE table.

Use the relations from Section 14.5 to determine the matrices and expected

14.16 Write the linear model for the situation described by the index set T' = {1,2,12,3(12), 4(12), 3(12)4(12)}. a. Describe the implied covariance structure. b. Write the AOV table, including the Kronecker product expressions for the mean squares and the expected mean squares. c. Use the relations fiom Section 14.5 to determine the matrices and expected values for the A VE table.

[Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...

Documents

Transcript of [Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...