[Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...

11 Fixed Effects Models 11: Two- Way Classification of Means

In this chapter we continue the discussion of classification models with a scalar covariance matrix. We extend the discussion to allow the populations to be described by two factors. This two-factor model is used to illustrate the basic issues of parameter estimation and inference that will apply for multi-factor models, to be discussed in Chapter 12. The cases of balanced and unbalanced data are separated. The discussion is also partitioned depending on the assumption of the no-interaction constraints. The analyses are described in terms of the cell means for ease of interpretation. Model reparameterization is introduced for computational convenience and for reference to the classical mathematical expression for this model. Some of the confhsions associated with the over-parameterized model with unequaI, and especially zero, cell frequencies are resolved.

11.1 UNCONSTRAINED MODEL: BALANCED DATA

The two-way, cross-classification (two-factor) model, introduced in Section 9.3.2, is a special case of the cell means model in which the populations can be described by two factors. The means of these populations could be indexed by one subscript and the results of Chapter 10 could be applied to develop the analyses. The use of a single subscript to describe the cell means is notationally clumsy and so we use two subscripts to refer to the populations. This will be especially usefbl in the description of hypotheses.

The model, with a levels of factor A, b levels of factor B, and n observations on each combination of these factors is repeated here for reference as

(11.1) y.. rjr = p . . 23 + e . 23T

with i = I , a * - , a, j = I,..., b, andr = 1, - - - , n. In matrix form we write

343

Methods and Applications of Linear Models: Regression and the Analysis of Variance, 2nd Edition. Ronald R. Hocking

Copyright 0 2003 John Wiley & Sons, Inc. ISBN: 0-471-23222-X

344 Chapter 11 Fixed Effects Models 11: Two-way Classification of Means

g = W p + e , (11.2)

where p denotes the vector of cell means in the order implied in Table 9.1 with elements lists by rows. The cell frequency matrix is W = I, @ I b €3 J, and we assume, e N N(o, u'I).

There are two general situations in which this model arises. The first is the natural extension of the one-way classification model in which we have two treatment hctors and we apply all combinations of the levels of these treatments to a homogeneous set of experimental units in a completely randomized design. In the second situation the populations are not created by applying treatments, but it is possible to observe them. In either case it is assumed that a random sample is taken fiom each of the ab populations, hence the same model is applicable in both cases. The following example is a combination of these two situations and will be used to illustrate the analysis. The discussion follows the outline used in Chapter 10.

Ejcample 11.1. A study was conducted to see if there are differences in the quality of steel produced by b = 3 different types of rolling machines. It is also felt that there may be differences in the feed stock obtained fiom a = 3 different suppliers. For the experiment, nine samples of feed stock were selected fiom each supplier, and n = 3 samples were randomly assigned to each machine. In Table 1 1.1 we show the response, ductility, a measure of a quality of the product, as a function of the type of machine and the supplier. Note that hctor B (machines) represents a treatment which can be applied to the experimental units. Factor A (suppliers) represents populations that can be observed. The nature of the experiment is such that we are able to observe the populations defined by these two fixtors.

TABLE 11.1. Data for Example 11.1 .- 1 8.03 7.76 8.17

7.55 6.36 8.52 8.50 7.12 7.91

2 7.26 7.90 7.26 6.05 7.79 7.18 7.97 8.13 8.58

3 8.65 8.21 9.64 8.29 7.39 8.78 8.55 8.01 9.04

11.1 Unconstrained Model: Balanced Data 345

The sample cell means, gij, and sample variances, s ; ~ , for each combination of factors are shown in Table 11.2. In addition, we show the marginal means ‘iii.. and g.?, that is, the average of all observations for each supplier and for each machine. We will see that the analysis of machine and supplier effects is based on these marginal means. Considering the small sample size, the

rn assumption of equal variance for each population is reasonable.

11.1.1 Parameter Estimation

The cell means are estimated by the sample cell means, the residual mean square is given by the average of the sample variances in each cell, and these statistics are mutually independent. The estimates and their distributions are given in (11.3). The Kronecker product expression for the residual sum of squares follows directly from the Kronecker product expression for the model and the general expression for the residual sum of squares.

a b n

(11.3)

TABLE 11.2. Summary statistics for Example 11.1 Machine 1 2 3 y ,

S u 1 yl3 8.03 7.08 8.20 7.77

P s ; ~ 0.23 0.49 0.09

1 s13 0.94 0.03 0.62

e 0.04 0.18 0.19

7.87 7.63 8.34 r y3

P 2 & 7.09 7.94 8.20 7.57

~i 3 &j 8.50 7.87 9.15 8.51

l -


11.1.2 Tests of Hypotheses

The hypothesis of equal means and mean comparisons discussed in Chapter 10 are of interest and the methods developed for the one-way classification are directly applicable. Typically we are concerned with hypotheses that examine the different levels of the two factors and the interaction between the factors. The analyst is fiee to formulate these and other hypotheses as dictated by the nature of the application. We will discuss typical formulations of such hypotheses.

Hypothesis of Equal Means. The hypothesis of equal means may be stated as an extension of any of the expressions in (10.7) with appropriate notational changes. The simplest expression for the hypothesis is Ho: pil = p... The numerator sum of squares follows directly fi-om (10.12) and (10.19) and is given bY

a h

N a2X2(ab - 1 , AH). ( 1 1.4)

The non-centrality parameter is the generalization of (10.2 l ) , given by

(1 1.5)

This hypothesis can be considered as a preliminary test which, if not rejected, might preclude hrther examination of relations between the means.

Main Eflects Hypotheses. The two-factor structure of the means suggests that we attempt an evaluation of the effects of different levels of the factors. We develop the analysis for factor A, the row effect. The results for factor B, the column effect, follow with obvious notational changes. A typical situation with three levels of factor A and four levels of factor B is illustrated in Table 11.3. The average of the cell means in a given row or column, the marginal means, are also shown. The main effect hypotheses described below are concerned with comparisons of these marginal means. and are called the marginal means hypotheses.


TABLE 1 13. Display of cell and marginal means

Main Effect A. The problem with assessing the effects of the different levels of fkctor A is that the effects of the different levels may not be the same for all levels of factor B. As a preliminary test we consider the effects of the levels of factor A when averaged over all levels of factor B. That is, we base the test on the marginal means. The hypothesis may be written in several forms as the obvious analogs of the hypothesis statements in (10.7). For example, we may use either of the forms

(11.6)

for i = 1, .-., (a - 1). H p = 0, are obtained by using the matrices

Matrix expressions for these hypotheses in the form

1

b HA2 =s -@-J: , (11.7)

where A, and s, are defined in (1 0.9) with p replaced by a. This hypothesis, called a main effect hypothesis, requires some

discussion. First, the hypothesis may not be rejected, even when there are differences in the levels of factor A, if these differences depend on the levels of hctor B. Further, rejection of the hypothesis might be caused by the behavior of hctor A in only some of the levels of factor B. In this sense we view this as a preliminary test.

In some applications the hypothesis may be if direct interest. To illustrate, suppose the two factors were varieties and fertilizers. If factor A corresponds to the different fertilizers, this main effect test is comparing the effect of fertilizers when averaged over all varieties. It may well be that the h e r would prefer to buy only one type of fertilizer to use on all varieties. A fertilizer which is best for all varieties would be preferable, but lacking this, one that is best on the average


would be acceptable. A comparison of average performance is then appropriate. The proper interpretation of the hypothesis is clearly important.

To develop the test statistic, we use the results of the Lagrange multiplier method and appeal to the development in Section 10.2.2. (The model reduction method is clumsy in this case.) Using H = HA^, it follows fiom (10.16) - (10.18) that

(1 1.8) 1 b

H ~ ( H H ~ ) - ' H = sa B -u, .

The numerator sum of squares is then given by

a

2 2 0 x ( (a - I), X A ) .

Here, the non-centrality parameter is given by

The expected value of the mean square, EMSA = E[NA/(cJ - l)] is

2a2 a - 1 EMSA = a' -I- - X A .

The test statistic for the hypothesis HA is then

(1 1.9)

(11.10)

(1 1.11)

(11.12)

It is of interest to examine the estimate of /A when constrained by this hypothesis. The constrained estimator (see Exercise 1 1.2) is

g.. '3 = g . . 2.J. - (D. I.. -- Y...). (11.13)

Note that the estimators will be similar to the unconstrained estimators if the hypothesis is acceptable but not otherwise.

Main Effecr B. The results for factor B are developed in Exercise 1 1.3.


Interaction Hypothesis. The evaluation of the effects of the different levels of hctor A is much clearer if the effect of changing levels is the same for all levels of factor B. For levels i and P of fsctor A and levels j and7 of factor B, consider the relation among the means given by

Ptj - Pz? = Pi3* - Pa-3- (11.14)

If this relation is satisfied, the difference in the means for levels i and i* of factor A is the same for levelsj andj" of factor B. Similarly, by rewriting (1 1.14) as

Pr3 - Pi3- = Pi*j - Pz*3- (11.15)

we see that the change fiom level j to level j* of fkctor B is the same for both levels of factor A. If this relation holds, we say that levels i and i* of kctor A do not interact with levelsj andj* of factor B.

The hypothesis of no interaction, or the interaction hypothesis, is that the relation holds for all pairs of levels of factor A and all pairs of levels of factor B. Two non-redundant forms of this hypothesis are given by

( l ) ' P~t3 - PRb - PQ3 + = (2) H A B ~ : pt3 - P a - P , + P = O (11.16)

Matrix expressions for these for i = I , - - - , (a - 1) and j = 1, .-., ( b - 1). hypotheses are obtained by using the matrices

It follows that

and hence the numerator sum of squares is given by

(11.17)

(11.18)

(11.19)


The non-centrality parameter is

(1 1.20)

The expected value ofthe mean square, EMSAB = .!?[NAB/(. - l)(b - l)],

2ff EMSAB = U* + AAB (a - l)(b - 1)

(11.21)

The test statistic for the hypothesis, HA^, is then

NAB/(a - 1)(b - 1) S2

FAB = - F((a - I)(b - l), ub(n - I), AAB). (1 1.22)

The estimate of p, when constrained by this hypothesis, is

p - . = r3 gz.. +g, -g... . ( I 1.23)

Analysis of Variance Table. The information for conducting these tests is summarized in the analysis of variance table shown in Table 11.4. The mean square column for the main effects and interaction rows gives the numerator for the test statistics, that is, NH/df. The expected values for M S A and M S A B are given in (11.11) and (11.21) and that for M S , is found in Exercise 11.3. These expected values provide a convenient reminder that the divisor for the F-ratios is the residual mean square, since under the null hypothesis the ratio has a central F distribution. The relations between the expected mean squares and the non- centrality parameters, described in (1 1. I 1) and (1 1.21) should be noted.

TABLE 11.4. Analysis of variance for twefactor model Description aj- SS MS EMS

HO Ub-1 No MSo EiVfSo HA a - 1 NA MSA EMSA HB b - 1 NB MSB EMSB HAB ( ~ - l ) ( b - l ) NAB MSAB EMSAB

Residual &(n-1) RSS RMS i?

C-Total abn-1 TSS TMS


The sums of squares for the interaction and the two main e&cts add to the sum of squares for the hypothesis of equal means, providing another illustration of Cochran's theorem (see Exercise 1 1.7). Thus these three hypotheses represent an orthogonal decomposition of the hypothesis of equal means. It will be argued in Chapter 18 that the testing of multiple, orthogonal hypotheses does not warrant the use of simultaneous methods and we feel free to conduct each test at level a.

The results of these hypotheses may be viewed separately with the appropriate interpretation as contained in the hypothesis statement. However, it is natural to consider the interaction hypothesis first. If this hypothesis is rejected then the main effect hypotheses must be interpreted in the marginal means sense as described above. At the other extreme, if the interaction constraints are satisfied then the main effect hypotheses are equivalent to the stronger hypotheses

H A o : p v = p a j f o r i = l , . , ( a - l ) a n d j = l , . , b

HBo : pt3 = &b for i = 1, . , a a n d j = 1, - , ( b - l ) . (11.24)

To verify this observation, use the second form of the interaction hypothesis in (1 1.16) written as jiv - p.. = pr3 - The right side of this expression implies that all means in a column are equal. HAo expresses thissame hct in a non- redundant form. Thus the marginal means form of the main effect hypothesis for factor A is equivalent to this stronger form under the no-interaction constraints. In this form we are hypothesizing that the effects of the different levels of factor A are the same for all levels of factor B, not just in the sense of marginal means. The result for factor B follows similarly.

In Section 11.3 we will discuss the analysis when the interaction constraints are known to be satisfied exactly. In the current model we do not have this information and can only infer the presence or absence of interaction from the test statistic. Acceptance of the hypothesis of no interaction for a typical value of Type I error, say, 0.05, means that we do not have sufficient information for rejection at that level, but does not imply that the constraints are satisfied. The extent to which we believe this depends on the P-value associated with the test. If this is large, say, P > 0.25, some authors suggest that the model be modified by imposing the no-interaction constraints on the model. (See Bozivich, Bancroft, and Hartley (1956) for a discussion.) If P > 0.25 for the interaction test, a compromise between modifying the model and insisting on the marginal means interpretation is to compute the test statistics for the main effect hypotheses as in Table 1 1.4 but interpret the results as in (1 1.24).

The analysis of variance for Example 1 1.1 is shown in Table I 1.5. We see a moderately strong indication of interaction, and our conclusions with regard to main effects must be made in the sense of the marginal means hypotheses.


TABLE 11.5. Analysis of variance for Example 11.1 Description df SS MS F P-value

Ho 8 10.17 1.27 4.07 0.006 Suppliers 2 4.39 2.20 7.02 0.006 Machines 2 2.36 1.18 3.78 0.043 Interaction 4 3.42 0.85 2.73 0.062

Residual 18 5.63 0.31 C-Total 26 15.79 0.61

This preliminary analysis raises questions about the nature of the interaction and the source of the differences in suppliers and machines. This leads naturally to the multiple comparisons that we will examine in Section 1 1. I .3. To motivate that discussion, we examine a graphical interpretation of the analysis of variance.

A Graphical Interpretation of the Analysis of Variance. A convenient way to visualize the presence or absence of interaction is to plot the sample cell means as a function of one of the hctors, say, fictor B, using an identification of factor A as the plotting symbol. From ( I 1.14), we see that no-interaction is indicated if

(1 1.25)

The extent to which these differences are not equal leads to the magnitude of the interaction sum of squares. In Figure 11.1 we show a plot of the sample cell means for Example 11.1. In Figure 1l.l.a the means are plotted against machines using the supplier number as the plotting symbol and in Figure 1 1 . I .b, the roles are reversed. To help with the visualization, it is usefbl to connect the points for a given pair of levels of factor B for each level of factor A. If these line segments are nearly parallel for all pairs of levels of factor B, no interaction is indicated. Lack of parallelism suggests the presence of interaction.

- - yii - g,j+. 1 jji.,. - v~.~.+, for all i, j , i*, 3.

Figure 11.1. Interaction plots for Example I 1. I.


The test of the interaction hypothesis assesses the apparent interaction as a function of the degrees of fieedom and the observed variability. The sou~ce of the interaction is clearly indicated in this figure. Thus, from Figure 1 1.1 .a, we see that the pairs of line segments for suppliers 1 and 3 are nearly parallel but this is not the case for supplier 2. In particular, the mean response for supplier 2 on machine 2 is higher than would have been anticipated in the case of no interaction. Further examination of that combination of factors is suggested.

From these plots we also see the sense in which the main effect hypotheses are rejected. From Figure 1 1. la we see that mean responses for supplier 3 are higher than those of supplier 1 for a11 machines. Further, the average of the means for suppliers 1 and 2 are about the same but less than that for supplier 3. This is confirmed by the marginal means in Table 11.2. The situation is similar for the machine effects. Thus if a single supplier is to be selected and we have an equal number of each type of machine in our plant, we would select supplier 3. Similarly, we would choose machine 3 if we have to purchase equally fiom all suppliers. Other considerations such as the price of the material and the inconvenience of purchasing from more than one supplier might influence our conclusions.

11.1.3 Simultaneous Inference

Depending on the results of the analysis of variance, we may wish to conduct a more detailed examination of the cell means. The methods, used in the one-way classification model, for examining linear functions of the cell means are applicable. However, it is important to emphasize the scope of the simultaneous inference. To illustrate, we discuss the Scheffk method for constructing confidence intervals. The corresponding acceptance intervals follow as usual.

Inference on Marginal Means. Suppose we are interested in making inferences on linear functions of the marginal row means, pi, . Let p R denote the a-vector of these marginal means and consider the linear function, cTpR. The methods discussed in Chapter 10 for the one-way classification model apply, by noting that estimates of the marginal row means are independently distributed as

- yi.. - N(Fi,., g). (1 1.26)

The Scheffe intervals follow, by noting that such linear functions can be described as a linear combination of the rows of the A-effed hypothesis matrix. Thus the intervals are given by

354 Chapter 11 Fixed Effects Models 11: TweWay Classification of Means

The simultaneous property applies only to functions of the row marginal means. If we also write intervals for the column marginal means, the simultaneous property also applies to such intervals but not to both sets of intervals.

For Example 11.1, with a! = 0.05, the half-width of the Scheffi intervals for pairwise comparisons of marginal means is 0.703 for both the supplier and the machine effects. This leads to the conclusion that supplier 3 has a higher mean ductility, when averaged over all machines, than either of the other suppliers. When applied to the marginal means for machines, machine 3 differs fiom machine 2 but not fiom machine 1 . Machines 1 and 3 do not differ significantly. (The half-width for the Tukey intervals is 0.672, leading to the same conclusions.) The concept of simultaneous acceptance and confidence ellipses, introduced in Section 10.2.4, can be applied to clarifL the interpretation. (The Studentized range intervals follow in a similar way.)

Inference on Interaction Contrasts. If we are interested in confidence intervals on interaction contrasts such as (1 1.14), the Scheffe intervals can be based on the interaction hypothesis. Letting cTp denote a linear combination of interaction contrasts, the intervals are

cTp : cTc f J(a - l)(b - l)F(a; (a - l)(b - I), ab(n - l))war[cT$] . (1 1.28)

Again the simultaneous property applies to linear combinations of interaction contrasts but not to intervals for marginal means.

With a! = 0.05, the half-width of the Scheffe intervals for the interaction contrasts is 2.22. If we write the interaction hypothesis in the first form of (1 1.16), we can use the projection ellipses to clarifL the interpretation.

General Contrasts on the Cell Means. For general contrasts on the cell means, the Scheffi intervals are based on the hypothesis of equal means and have the same form as (1 1.28) with (a - I)(b - 1) replaced by ab - 1 as the coefficient and numerator degrees of freedom of the F-ratio. This allows us to write intervals for marginal means, interaction contrast, and any other linear functions of the cell means and still retain the protection of these simultaneous intervals. The price paid for this generality is that the intervals are quite conservative. In practice, the Scheffk intervals for marginal means are written using (1 1.27) and for interaction contrasts using (1 1.28) without regard to the loss of overall protection. This is justified by the orthogonality of the hypothesis matrices for marginal means and interaction.

Other Hyputheses. Depending on the results of the hypotheses described in the analysis of variance table, we may consider other hypotheses. If the interaction hypothesis is rejected, we might study the effects of the factors by testing other hypotheses. For example, we could compare the means for all levels of factor A at a given level of hctor B. The hypothesis is stated for columnj as


HAj : p i j = j i . j for i = 1, - - - , (a - 1). (1 1.29)

The numerator sum of squares for this test, with (a - 1) degrees of fieedom, follows in the usual way, yielding (see Exercise 11.9)

U

(11.30)

The residual mean square @om the AOV table provides the denominator for the F-statistic. This test would generally be made for each level of factor B and we again raise the question of testing multiple hypotheses. The size of these tests can be protected, using the Scheffe concept, by comparing the F-ratios to F(a; (ab - I ) , ub(n - 1)). In practice, this protection is often ignored.

11.1.4 Reparameterizations of the Two-Factor Model

As in the one-factor model, we consider a reparameterization of the two-factor model for convenience in computing test statistics. We seek a simple structure for the new form of the model. The change in parameters is based on the main effect and interaction hypothesis matrices and HAB, defined in (1 1.6) and (1 1.16) with the analogous form for factor B. We define the new parameter vector 0 = M p , where

and

(11.31)

(1 1.32)

The inverse of this transformation matrix is

The vectors a, p, and (ap) have dimensions (a - I), (b - 1) and (a - I)(b - l), respectively.

We define the basic design matrix

XO = ( Ja @ Ja I 10 €3 Jb 1 Ja @ r b I Za @ Ib ) (1 1.35)


and the parameter matrix

(1 1.36)

Noting that M-' = XoP, the reparameterized form of the cell means model is then written as

9 = WXoP8 + e . (1 1.37)

It is informative to write the relation between these sets of parameters in algebraic form. Thus the relation 8 = M p implies that

/.Lo = p.. I (a - 1)

j = l , . . . , ( b - 1) a, = pi. - p.. i = 1, ... pj 1 F , - F..

(a& = ,uij - pi. - ,ii, - p.. i = 1, . --, (u - 1)

j = l , . - * , ( b - 1). (1 1.38)

The relation p = M-'8 implies that

pij=,u0+cui+fl j+(a& i = l , . . - , a (11.39) j = 1 , - * * , b,

where, for convenience, we have written

0- 1

1 x 1

6- 1

Qa = - p i

P b = - CPj /= 1

b-l

J= I

a- 1

(Q.P)~~ = -C(Q@),~ i = I,--.,(u-- I )

(a&= -C(ap)aj j = l , . . . , (b - 1) i= 1

(1 1.40)

We have not defined additional parameters in (1 1.40) but have merely introduced these expressions for convenience in writing (1 1.39).


Non-Full Rank Model. We may now examine the relation W e e n the reparameterized form of the cell means model ( 1 1.37) and the classical, over- parameterized, or non-hll rank model written as

Yijr = Po + ai + Pj + ( 4 i j + eijr * (11.41)

In this form of the model there are I + a + b + ab parameters used to describe the mean responses, but the design matrix has rank ab. If we let 0, denote the augmented parameter vector, this model can be written in matrix form as

g = WXoO, + e. ( I 1.42)

It follows that the coefficient matrix in the least squares equations is singular, and hence there is not a unique solution. There are two general methods for obtaining a solution: (1) finding a generalized inverse for the coefficient matrix and (2) imposing conditions on the parameters to remove the degeneracies. The generalized inverse method has the effect of removing the redundancies by imposing conditions on the parameters but typically the user is not aware of the conditions implicit in the choice of inverse. A common set of conditions is given by a. = /3. =(a& = (a@).j = 0, for i = l , . . . ,a and j = l , . . . , b . Using (11.40) to remove the redundancies, we see that this approach is equivalent to the reparameterized model in (1 1.37).

We have arrived at (11.37) fiom two directions: (1) a model expressed in terms of parameters that are carehlly defined as in (1 1.38) and (2) a model that arose by eliminating redundancies flom the over-parameterized model in (1 1.4 1) without explicitly defining the parameters in the resulting model. To emphasize this point, consider another set of conditions commonly used to remove the redundancies, that is, a, = pa = (a& = = 0 for i = 1, . . -, a and j = 1, e - . , b. This corresponds to a common method for finding a generalized inverse by applying the sweep method to solve the normal equations (see Exercise 11.10). To see the implications of this, note that this is equivalent to using the parameter matrix

(1 1.43)

where L, denotes the identity matrix of size a with the last row deleted. Jt follows that the transformation is defined by the matrix

358 Cbapter 11 Fixed Effects Models 11: Two-way Classification of Means

(1 I .44)

where ua and Ub are vectors of length a and b with one in the last position and zeros elsewhere. It follows that this method of removing the redundant parameters corres?onds to defining the parameters as

= kab

= ktb - kab i = I , - a a , (u - 1)

p; = ka3 - kab j = 1, * * * , ( b - 1)

( @ 0 ) t 3 = kt3 - ktb - Po, - kab = I ' ' I (. -

j = l,...,(b- 1) ( 1 1.45)

This is a valid definition of the parameters, since the matrix M is non-singular, but it is important to appreciate how the parameters are defined. The interaction parameters are defined as in the first form in ( 1 1.16) and the main effect parameters do not correspond to the marginal means hypotheses. In particular, this choice of parameters corresponds to defining main effects as row or column differences in only the last row and column of Table 11.3. While it may be of interest to examine those differences, it does not seem reasonable to define the main effects in this way. The consequences with regard to testing hypotheses are discussed in the next paragraph.

Parameter EMmation and Hypothesis Testing. To appreciate the role of the reparameterized model defined by the parameter matrix in (1 1.36), note that the orthogonality of the hypothesis matrices used in (1 1.33) yields a block diagonal coefficient matrix in the least squares equations. That is, letting

we see that

xTx = n

x = W X , P = (J I XA I X B I XAB) (1 1.46)

(1 1.47)


Here

with XzXB defined similarly. It is easily verified that the solution to the least squares equations is given by substituting the cell mean estimates, Ci j = Jjii,into the relations given in (1 1.38). (See Exercise 11.10.)

It is clear that parameter estimation in the reparameterized model requires more computation than in the cell means model. The advantage lies in developing the test statistics. In this form of the model, the hypotheses are given bY

HA : a = o H B : P = O HAB: (aP)=O. (1 1.49)

The numerator sums of squares for testing these hypotheses are easily developed by fitting the model with and without the appropriate set of parameters. The sweep algorithm is well suited to this computation. In the next paragraph, we introduce a common notation to describe these sums of squares.

R-Notation. To develop a convenient notation for describing sums of squares, consider a linear model with parameter vector 0 partitioned into two subvectors

and 02. Let RSS(0) denote the residual sum of squares for fitting the full model and RTS(B1) the residual sum of squares for the model under the constraint O2 = 0. The difference in these sums of squares is described by the R- Notation,

R(02 I 01) = RSS(01) - RSS(0). (1 1 SO)

Thus R(02 10,) is the numerator sum of squares for testing the hypothesis H : O2 = 0. If follows that the numerator sums of squares for the hypotheses in (1 1.49) may be described by

N A = R ( a I Po, P, (@)) NB R(P t Po, a, (afl))

NAB = R ( ( 4 I PO, Q, PI- (1 1.51)

To contrast the analysis using the cell means and the classical model, we see that the cell means model yields trivial estimates of the parameters and simple formulas for the test statistics. The reparameterized model requires a matrix inversion to determine the parameter estimates and standard regression computations to determine the test statistics. It is not clear that the reparameterized model has a computational advantage over the cell means model


in this situation but we will reap the benefits of this development when we consider the unbalanced case.

The block diagonal structure noted in (1 1.47) for the model defined by P offers a computational advantage with balanced data. The effect of adding one of the sets of parameters to the model does not depend on which parameters are already in the model. Thus we see that

Na I Po) = W a I P0,P) = R(a I POI P , (a@>) (11.52)

with the analogous expressions for ,# and (a@. Thus the numerator sums of squares can be computed as the forward sweep steps are performed to fit the model and there is no need to perform the reverse sweep implied by (1 1.51). We will see that the quantities in (11.52) are not equal if the data are unbalanced. The Edilure to appreciate this fact is a source of some of the conhsion in hypothesis testing with unbalanced data.

We close this discussion by noting the consequences of applying this computational procedure when the parameters are defined as in (11.45). Computing sums of squares as in (1 1.5 1) yields the correct sum of squares for the interaction hypothesis. However, the sums of squares for testing the main effect hypotheses are appropriate for comparing means in the last row and column of Table 1 1.3, as indicated in (1 1.45), and are not the sums of squares for testing the marginal means hypotheses. Further, the rows of these hypotheses matrices are not orthogonal, hence XTX is not block diagonal, and the relations in (1 1.52) are not valid.

It is of interest to examine the effects of using combinations of the conditions used to remove redundancies. For example, suppose that we assign (a@)& = (c.P),~ = 0 for all i andj but use a. = @. = 0 to remove the remaining redundancies. It can be shown that computing sums of squares as in (11.51) yields the same results as if we used aa = pb = 0. (See Exercise 11.12.) This is a special case of a general result by Hocking, Hackney, and Speed (1978), which shows that the hypotheses tested by (1 1.5 1) are determined by the conditions imposed on the interaction terms and do not depend on the conditions on the main effects.

11.1.5 Test for Interaction with One Observation per Cell

In many applications we do not have the luxury of replicate observations on any of the factor combinations. Consider an experiment to study the yield of a particular variety of tomatoes as a function of temperature and humidity. To control the environment, the plants are placed in a growth chamber and then temperature-humidity combinations are assigned at random to the chambers. If we have only 15 growth chambers, we might consider three different

11.1 Unconstrained Model: Batanced Data 361

temperatures and five levels of humidity, but we cannot provide replicates of these treatment combinations. It is clear &om (1 1.3) that the residual sum of squares is zero and that no inferences are possible. We will see in Section 11.3 that assuming the no-interaction constraints are satisfied will enable us to complete the analysis. The lack of replicates does not justiij that assumption and we ask if there any way that we can test the interaction hypothesis. The interaction plots described in Section 11.1.2 give a visual indication of the presence or absence of interaction, but we seek a test of the significance of any observed lack of parallelism. Tukey (1949) proposed a test for this situation. The concept is based on a Taylor series approximation to the response function (see Graybill 1976) that leads to the approximation

paj Ti.. + (Pi. - Ti..) + (j3.j - P..) + 6(&. - P..)@.j - Ti..). (1 1.53)

Using the marginal means parameterization described in (11.38), the model is approximated by

(1 1.54) gij po + ai + Pj + 6zij + e i j ,

where

zij = &. - ii..)(p.j - F..) . (11.55)

The test for interaction in this approximate model is then given by

H;: 6 = 0 . (11.56)

The model in (1 1.54) is not linear since 6 and zij are unknown parameters. To develop an approximate test, we temporarily assume that the zij are

known and apply standard techniques. It can be shown (see Exercise 11.14) that the estimate of 6 is given by

a b

i=l j=1 c c(z2j-z~. -2.j+2..)(plj-gi. -gj+g..)

c C ( Z i j -22. -2.j +z..)2

6= (1 1.57) a b

r = l j = 1

and that the numerator sum of squares for testing H; is given by

n h

1 4 j=l

The residual sum of squares for the model in (1 1.54) is then given by

(1 1.58)


( I 1.59)

The test statistic for the null hypothesis 6 : S = 0, under the assumption that the zij are known, is given by

NH6 RSS/(ub - a - b) .

F = ( I 1.60)

Under this null hypothesis, F N F( 1, ab - a - b) conditional on the ziJ but, since the distribution does not depend on these quantities, this is also the unconditional distribution. If follows that (1 1.60) provides an exact test in the approximate model for this null hypothesis.

The zzj are not known and must be estimated. To do so we replace pZJ in (1 1.55) by the estimate of the mean response when 6 = 0. That is,

( I 1.61) A, = y,. + D, - 5..

and it follows that Ti. = 2.j = 2.. = 0 and

2 . . zj - - ( gi. - B..)@.j - g..) .


, a b

(11.62)

(1 1.63)

Using this expression in (1 1.60), the hypothesis of no interaction is rejected if F exceeds F(a; I, (ab - a - b)).

This test is often referred to as a test for additivity. This expression is based on the reparameterized model statement that describes the cell means as in (1 1.41). If the interaction term is not present, the mean response is given by a constant plus the sum of the effects of the two factors. Functions of several variables, which are the sum of functions of a single variable, are known as additive functions. The term additive model is thus applied to the model in which the interaction terms have been deleted. A detailed analysis of this model is given in Sections 1 1.3 and 1 1.4.

This test, based on the approximate model in (1 1.54), seems to perform reasonably well in practice and provides statistical support for the observed interaction plot. Snee (1982) noted that this model is a special case of a more general model proposed by Mandel (1971). Snee noted that the apparent non- additivity may be due to non-homogeneous variance or to interactions between

11.2 Unconstrained Model: Unbalanced Data 363

the row and column factors and provides two illustrative examples. Often a transformation of the data to correct for variance inhomogeneity will remove the apparent interaction. Box, Hunter, and Hunter (1978) provide an example illustrating this situation. (See Exercise 1 1.13.)

11.2 UNCONSTRAINED MODEL: UNBALANCED DATA

If all treatment combinations are observed, the imbalance in the data has a minor effect on the analysis when developed in the cell means context. Parameter estimation is simple and standard methods may be applied to develop test statistics and interval estimates. If some cells are not observed, we may not be able to conduct the planned analysis. We will discuss the way in which alternative analyses arise. The model defined in terms of the main effect and interaction parameters is stiIl defined by (1 1.37) and the cell frequency matrix reflects the imbalance.

11.2.1 Discussion in Terms of the Cell Means Model

Parameter Estimation. With cell fiequencies nij # 0 and total sample size N = n.., the estimates of the cell means are given by

(1 1.64)

Clearly, pij is not estimable if that treatment combination is not observed. The residual mean square for, ni, # 0, is given in (1 1.65). Other than the obvious changes caused by the unequal sample sizes, the primary effect on estimation is the loss of the convenient Kronecker product notation for the residual sum of squares. (See Exercise 11.15 for a discussion of the case when some of the cell frequencies are zero.)

a h

s2 = c ( n i j - l)si/(N - ab) l= l J=1

(1 1.65)


Testing Hypotheses. The situation with regard to testing hypotheses is equally straightforward. That is, given a hypothesis stated in terms of the observed populations, we apply the standard results to determine the numerator sum of squares. In particular, if the cell frequencies are non-zero, we consider the marginal-means, main effect, and interaction hypotheses as described in Section 11.1.2. The expressions for the numerator sums of squares are algebraically complex, since WZW = Diag(nij) but they are easily computed. In particular, if all nij # 0, the computations can be done in terms of the appropriate reparameterized model.

The only problem encountered is that a hypothesis that was formulated prior to the data collection may not be testable when some of the cell frequencies are zero. For example, the hypothesis H: pll = p12 cannot be tested if either rill or nI2 is zero. We now consider the implications of this with regard to the interaction and the marginal-means, main effect hypotheses. The stated hypotheses cannot be tested, so we seek an alternative. In some sense we wish to test as much of the original hypothesis as possible. We define the effective hypothesis that meets this objective. We will see that this philosophy is commonly used with the interaction hypothesis but rarely used for main effects.

Effective Hypothesis for Interaction. If we wish to test the interaction hypothesis with missing (empty) cells, it should be clear that the best we can do is to consider all constraints of the form (1 1.14) that do not involve the means for which nt3 = 0. While these constraints may be obvious for simple situations, their identification may be tedious, so we describe a general method for identifLing them. To this end, let HAB denote any convenient form of the hypothesis matrix. Suppose that the vector of means p, is partitioned into pa and pm corresponding to the observed and missing cells. Assume that there are m empty cells hence pa and pm have lengths ab - m and m, respectively. Let the hypothesis matrix be similarly partitioned and perform row operations to eliminate pm fiom as many rows as possible. Symbolically we write this as

HAB = ( ~ o I H m )

(1 1.66)

where H,, is t x m of rank 1. The effective interaction hypothesis is then given by

Hmpo = 0 . (1 1.67)

For reference we define the effective model as the cell means model in which we delete the populations with nij = 0. Thus, the model is given by

(1 1.68)


Fitting this model, subject to the constraints in (1 1.67), provides the appropriate residual sum of squares for computing the numerator sum of squares for the test statistic. The degrees of fieedom associated with the numerator sum of squares are (a - 1Xb - 1) - I, and the denominator is given by (1 1.65) with N - ab + m degrees of fieedom.

Eflective Hypothesis for Main Effects. The effective hypotheses for main effects are developed in the same way and involve only the observed cells. it is assumed that we are interested in the hypothesis that would have been tested with no empty cetls. The main effect hypothesis are determined by applying the same row reduction concept to the associated main effect hypothesis matrices. It is easily seen that this results in comparing marginal means for all rows in which there are no missing cells. These ideas are illustrated in the following example.

Exumpfe 11.2. The concept of effective hypotheses is illustrated by the following matrices of cell frequencies, known as incidence matrices:

For incidence matrix (a), the effective interaction hypothesis is given by

(.) H A B ~ ' p 1 2 - p 1 3 - p 2 2 -k p23 = 1121 - p 2 2 - p 3 1 $- p32 = 1122 - 1123 - p 3 2 - 1133 = O*

This hypothesis is easily verified by examining the incidence matrix or applying the row-reduction procedure. The effective hypothesis for comparing marginal row means, with one degree of fieedom, is

(a) HA^ : ji2. = Ji3.

with the analogous result for comparing column means. For incidence matrix (b) there are no effective row or column main effect

hypotheses since all rows and columns have at least one empty cell. At first glance there are no obvious interaction constraints, but, in fact, the affective hypothesis for interaction is given by

(b) : pll - k 2 - p2l = p33 - p23 - p32 '

This hypothesis is easily verified using the row-reduction method. Note that it can be obtained by writing the interaction constraints for the 2 x 2 matrices in the upper left and lower right of (b) and eliminating pLZz between these two equations.

For incidence matrix (c) there are no effxtive row or column hypotheses. The single interaction constraint is written using the first two rows and columns.


Thus,

(c) H A B ~ : PII - ~ 1 2 - ~ 2 1 f ~ 2 2 0 .

The significance of this example is that the number of interaction constraints is not given by (a - I)(b - 1) - m as in the first two examples. That is, the matrix H,, in (1 1.66) is 3 x 4. The importance of this rank condition will be apparent as we develop the analysis.

It is of interest to ask what main effect hypotheses would arise if we had based the analysis on the main effects implied in (1 1.45). It seems clear that the same hypothesis would arise in (a) since all cells are observed in the last column. (See Exercise 11.16.)

11.2.2 Reparameterized Model

It seems logical to assume that we are still interested in the main effect and interaction hypotheses defined for the balanced data situation and hence the parameterization described in (1 1.38) is still appropriate. In particular, the model is written as (1 1.37) with Xo and P as defined in (1 1.35) and (1 1.36). The cell frequency matrix reflects the imbalance in the data.

Parameter f i t i t i o n . The effect of the differences in cell fiequencies is that the coefficient matrix for the least squares equations is not block diagonal nor algebraically invertable. Thus parameter estimation requires the numerical inversion of the non-singular coefficient matrix to solve this system of ub equations. If all cell fiequencies are non-zero, then the cell means are estimable, and the solution of this system of equations is given by % = m. That is, the estimates are easily obtained, without solving this system of equations, by substituting the cell mean estimates 6-om (1 1.64) into the defining relations in (1 1.38).

Case 1, nij # 0.

Parameter Estimation. If some cell frequencies are zero, we clearly cannot estimate the corresponding cell means. h the reparameterized model, the rank of the coefficient matrix in the least squares equation is ab - m, where rn is the number of empty cells. It follows that there is not a unique solution to these equations. A solution can be obtained by using a generalized inverse or by deleting columns from the design matrix to remove the redundancies. A common procedure is to apply the sweep algorithm to generate a generalized inverse that will solve for ub - m of the parameters and, implicitly, set the remaining parameters to zero. The danger with this procedure is that if we use (1 1.39) to recover the estimates of the cell means, it appears that we can now estimate the means for unobserved cells. The following example will illustrate this phenomenon.

Case 2, some nij = 0.


Evample 11.3. Suppose that the incidence matrix is given by the first two rows of (a) in Example 11.2. Thus a = 2, b = 3, nll = 0 and, for notational simplicity, we assume that all other cells have n observations. With the parameter vector, BT = (po a1 0, ,B2 ( ~ , f 3 ) ~ 1 ( C K P ) ~ ~ ) , the design matrix for the reparameterized model is given by

J J O J O J J - J - J - J - J

J - J 0 J 0 - J I J - J - J - J J J

X = J - J J 0 - J 11. The degeneracy in this design matrix is defined by noting that Xa = 0, where a* = (1, 1 , - 1, - 1,2,2). Deleting one of the columns of X will yield a matrix of rank five. A standard procedure is to set to delete the interaction column corresponding to the missing cell. In this example we set = 0. The resulting parameter estimates are algebraically messy but, since Gij = g i i , they satisfy the relation

h ,. gif = E0 + Gi + Pj + (a,B)ij for (iJ # (I, 1).

Unfortunately, there is no warning to say that this relation should not be applied to estimate p1 1 . Thus, using the estimates of po, al , and ,B1 and noting that (a& = 0, we obtain the apparent estimate

h

$11 = 20 + 81 + P1

- 1 - = YZl. + i ( 3 1 2 . + 313. - 322. - 323.1.

Since pll is not estimable, this result is misleading, and we ask for an explanation. The answer is that, by setting ( C K P ) ~ ~ = 0, we impose a constraint on the cell means. In particular, this condition is equivalent to setting

= pll - pl. - ,iz1 + p.. = 0. Solving for pll and replacing the remaining parameters by their correct estimates yields the above estimate. In general, this procedure yields the correct estimates of the means for observed populations and meaningless estimates for the unobserved populations. The reparameterized model must be used with caution in parameter estimation when there are missing cells.

Testing Hypotheses. Case 1, nij # 0. An advantage of the reparameterized model is that the computation of test statistics is easily accomplished using the sweep algorithm. Much of the confusion in this area is caused by differences in the application of this procedure and we now describe some of these methods in terms of the R-notation. The differences lie in the main effect sums of squares that are computed and hence the hypotheses that are being tested. The methods


agree on the interaction sums of squares. The situation in which m e cells are not observed is more complex and is treated under Case 2.

For main effect and interaction hypotheses described in (11.6) and (11.16), the test statistics are computed as described by the R-notation in (1 1.51). In particular, we use the parameterization in (1 1.38) and compute sums of squares according to (1 1.5 1). That is, we use the full-rank models with the computations, ~ ( a I pO,P, (d)), R(P I PO,^, (aP)l7 and R( (aP) I POYQ, P). We will refer to this as the marginal means method. In Exercise 11.21, we examine the hypotheses resulting fiom these computations if we use the parameterization in (1 1.45).

The R-notation computing procedures, shown in (1 1.52), for the main effect sums of squares are no longer equivalent since the columns of the design matrices are not block-orthogonal and hence the coefficient matrix of the normal equations is not block-diagonal. Many computing algorithms compute the sum of squares for the hctor A main effect using either R ( a I po) or R(a I po, /3) with the analogous choices for factor B. It is of interest to describe the hypotheses associated with these sums of squares. The results, given by Hocking, Hackney, and Speed (1978) are as follows:

(1) Ifthe sum of squares is computed as NA* = R(a I po, p), the hypothesis being tested is given by

(11.69)

for i = 1 , "-, (a - l ) , with the analogous result for the column hypothesis.

(2) If the sum of squares is computed as NA** = R(a I po), the hypothesis being tested is given by

(11.70)

for i = 1 , . . ., (u - I ) , with the analogous result for the column hypothesis.

Two common procedures use combinations of these sums of squares in the analysis of variance table. The sequential method uses R(a 1 po), R(P I po, a), and R((aP) I po,a , @) to compute the row, column, and interaction sums of squares. These sums of squares are indicated symbolically in Table 11.6. Note that the sums of squares in this table have the additive property noted in the balanced case. That is, the sums of squares for the two main effects and the interaction add to the sum of squares for the hypothesis of equal means.


A variation on this method, the partially sequential method, uses the symmetric choice, R(a I po, p) and R(j3 I po, a) for main effects and R((ap) I po, a, p)) for the interaction sums of squares.

Speed, Hocking, and Hackney (1978) discussed these methods, relating them to existing computer packages and to discussions in various texts. They noted the use of such descriptive terms as least squares, complete least squares, the experimental design method, and the method of fitting constants to describe the methods. PROC GLM of the SAS@ statistics package (see SAS/STAl@ (1 992)) refers to the sequential, partially sequential, and marginal means methods as types I, 11, and 111, respectively. Explicit expression for the hypotheses being tested are clearly preferable.

Our primary objection to the sequential and partially sequential methods is that the hypotheses involve the cell fkequencies. In most cases the cell fkequencies differ because of circumstances unrelated to the population means and should not influence the hypotheses.

For example, in the classical problem of comparing two means with different sample sizes, the standard null hypothesis is Ho : p1 = p2. A hypothesis such as Ho : nlpl = n2p2, weighted by the cell frequencies, might be appropriate if the fiequencies reflected population sizes but would not be considered as the standard hypothesis. The analogous hypothesis in the two-hctor model might consider HA- as a weighted version of the marginal means hypothesis. The hypothesis HA* seems difficult to justig.

The partially sequential method is often justified if the interaction hypothesis is not rejected. The motivation for this is that the sums of squares R(a 1 po, p) and R(P) I po, a) are appropriate in the model without interaction, as we shall see in Section 1 1.4. Proponents of this method still use the residual mean square fiom the two-factor model in the denominator of the test statistic. The alternative would be to refit the model. This would pool the interaction and residual sums of squares to compute the new residual sum of squares. Not refitting is apparently a concession to the fact that the deletion of the interaction terms is based on a test rather than on a priori knowledge.

The following example illustrates a special case of unequal cell frequencies that is algebraically tractable and allows us to explore the relation between parameter definitions and the analysis of variance.

TABLE 11.6. Scqucntial sums of squares Description df ss

HO ab-1 R(*., 0, (*.P)1160)

A-Effkt (a- 1) R(aI160) B-Effect ( b - 1) R(PlPOl4 Interaction (a - I)@ - 1) R((aO)Ip,, a, p)

Ejuunple 11.4. Ostle and Malone (1988) describe an experiment to measure the yield of three varieties of oats when treated with three different types of fertilizer. The variety-fertilizer combinations are assigned at random to experimental plots, but as indicated by the data shown in Table 1 1.7, the design is not balanced.

In this case the imbalance is not due to chance but was intentional as more information was desired for some treatment combinations. In particular, the varieties were assigned to plots in the ratio 2: 1: 1, and the fertilizers were assigned in the ratio 3: 2: 2. Denoting the variety ratios as r l : r2: r3 and the fertilizer ratios as cl: c2: c3. the cell fiequencies are then given by ntJ = rzc3. This is known as the case of proportional cell frequencies. While this example is not typical of the general situation, it will illustrate the problems with the analysis of unbalanced data.

If there is no reason to incorporate the cell fiequencies into the hypotheses, it is appropriate to consider the marginal means hypotheses HA, HB, and HAB. We emphasize that these sums of squares may be computed using our general results for sums of squares, stated in terms of the cell means model. The simple formulas given in Table 1 I .4 do not apply. Alternatively, the sums of squares can be obtained by using the marginal means method described above. The results are shown in Table 11.8. Note that the sums of squares for varieties, fertilizers, and their interaction do not add to the sum of squares for testing equality of means.

In Table 1 1.9 we show the analysis presented by Ostle and Malone This analysis is often presented for the proportional fi-equency model but it is not necessarily appropriate. Examination of the sums of squares shows that (1) the main effects differ fiom those in Table 11.8 and (2) the main effects and interaction sums of squares add to that for Ho. This analysis of variance table can be generated by using the sequential method with sums of squares defined by R(a I po), R(P I po, a), and R((aP) 1 p0, a, p). (In the case of proportional frequencies, this analysis is identical to the partialiy sequential analysis which uses R(a 1 ,uo, p) for the A-effect.) This accounts for the additivity of the sums of squares and reveals that these sums of squares are appropriate for testing the hypotheses, HA** and HB** (equivalently HA* and Hp) described in (1 1.70) or (1 1.69), simplified by setting ntJ = rtc3. Since we have argued that

TABLE 11.7. Data for Example 11.4 Fertilizer 1 2 3

A 50,5132 42,40 55,56 56,60,55 38,38 5638

Variety B 65,69,67 50,50 62,62 I C 67,67,69 48,50 65,64


I TABLE 11.8. Mareinal means analvsis for Examde 11.4 - Description 4f SS MS F P-value

HO 8 2326.26 290.78 55.02 0.0001 Varieties 2 729.67 364.84 69.03 0.0001 Fertilizers 2 1349.97 673.98 127.53 0.0001 Interaction 4 52.23 13.06 2.47 0.0795

Residual 19 100.42 5.29 C-Total 27 2426.68 89.88

TABLE 11.9. Sequential analysis for Example 11.4 Description df SS MS F P-value

HO 8 2326.26 290.78 55.02 0.0001 Varieties 2 818.89 409.45 77.47 0.0001 Fertilizers 2 1455.14 727.57 137.66 0.0001 Interaction 4 52.23 13.06 2.47 0.0795

Residual 19 100.42 5.29 C-Total 27 2426.68 89.88

the unequal cell fiequencies were only for the purpose of obtaining more precision for some treatment combinations, it does not seem reasonable to prefer these weighted hypotheses over those that compare marginal means.

Historically, the choice of this analysis was not based on the preference of these hypotheses but on convenience is doing the analysis. To see this, recall that the marginal means analysis arose by using the parameterization in (1 1.38) and applying the regression analysis described by (1 1.5 1). Alternatively, we could have arrived at the marginal means model by writing the model in over- parameterized form as in (11.41) and imposing the conditions, a. = 0. = (a& = = 0 to remove the redundancies. With proportional fiequencies the redundancies can be removed by imposing the conditions

These conditions imply a different definition of the parameters and yield a set of normal equations that has a simple solution. Application of the computing procedure in (1 1.51) in terms of these parameters yields the analysis in Table 11.9.

This example emphasizes that the use of this computing procedure is dependent on the parameter definition. The fact that the row descriptions in Tables 11.8 and 11.9 are the same belies the fact that these generic descriptions


do not reflect the parameter definitions or the hypotheses being tested. In this case the two analyses led to essentially the same conclusions, but it is not unusual that an effect is significant in one analysis and not in the other.

Testing Hypotheses. Case 2, Some nij = 0. Since the desired main effects and interaction hypotheses cannot be tested when some cell are empty, the researcher should examine alternatives that can extract as much information as possible about the factors being considered. The desire for simple answers often leads researchers to use standard linear model programs and, unfortunately, most computer packages will provide an analysis of variance table. It is of interest to examine the meaning of these tables. That is, to associate the sums of squares with statements of hypotheses. We summarize the results reported by Hocking et al. (1978).

Consider first, the sum of squares R((ap) I p,, a, p) which is almost always used for the numerator in the test for interaction. This sum of squares tests the effective hypothesis given in ( I 1.67). The associated degrees-of- fieedom is (a - I)(b - 1) - r, where r 5 m is the rank of the matrix H,, in (1 1.66).

The main effat sums of squares described by R(a I po, p, (a@) depend on the definition of the parameters, the number and the location of the missing cells, the rank of the matrix Hmm, and how we perform the sweep operation when computing this quantity. First, suppose we use the standard form of the reparameterized model in (1 1.37), assume that H,, has rank m, and that there are no missing cells in the last row and column of the incidence matrix. Applying the sweep procedure to solve the normal equations, we will find that if n,3 = 0, the corresponding (a?p),j parameters cannot enter the model. In our first computing method, we permanently exclude these parameters €torn entering the model. To generate the sum of squares for the A-effect, we sweep again on the a-columns and the numerator sum of squares is obtained as the difference in residual sums of squares. The associated hypothesis, with (a - 1) degrees of fieedom, is

(1 1.72)

where ci3 = 1 if nij # 0 and Cij = 0 otherwise. (Note the similarity of this hypothesis and HAS in ( 1 1.69).) Hocking (1985) notes that there are other computing methods that lead to this same hypothesis.

In the second computing method, we allow the previously excluded (a?p),j-columns to reenter the model after deletion of the a-columns, the sum of squares defined by R(a I j ~ , , p, (ap)) now tests the effective hypothesis with fewer, perhaps zero, degrees of fkeedom.


If we use the standard form of the reparameterized model, with missing cells in the last row and column, the main effect hypotheses are not given by (1 I .72) and some effort is required to determine the hypothesis.

The condition that the last row and column have no empty cells can be removed if we consider an alternative form of the reparameterized model. In particular, suppose we use the conditions (crp),. = (a& = 0 to eliminate redundancies by solving for (a& only if nij # 0. We will illustrate this approach in Example 11.5.

If H,, has rank t < m, there do not appear to be any general results. The following example illustrates typical situations.

Example 11.5. Consider the incidence matrices given in Example 11.2. The sum of squares R((ag) 1 po, a, p) is appropriate for testing the effective hypotheses with degrees-of-fieedom = (a - I)(b - 1) - 1. The row effect hypotheses associated with the sum of squares R ( a I p0, a, (a@)) are described as follows:

For the incidence matrix in (a) the row e f f i hypothesis given by (1 1.72), obtained by using the first computing method, has two degrees of fieedom and is described as

p 2 . = p 3 .

The second constraint describes the one degree-of-fieedom, effective row hypothesis noted in Example 11.2. The first constraint does not compare marginal means, but it attempts to compare row one with the other two rows by making the comparison using only the last two columns. While this is a reasonable comparison, our point is that the user should be aware of the Sense in which the row effect is being tested. In the case of missing cells, it is recommended that we do not consider a general row effect hypothesis but consider sub-hypotheses, perhaps single degree-of-fi-eedom contrasts, in an effort to salvage as much as possible &om the original intent of the experiment.

The danger of using automated procedures is more clearly illustrated by the incidence matrix in (b). The row effect hypothesis (1 1.72) is described by the two constraints,

(b) H A : 1111 -1121 = o 1112 - p32 = p33 - p 2 3 .

Note that neither of these constraints describes a marginal means comparison. The fist constraint compares rows one and two but only in column one. This is a reasonable contrast that might be examined as a single degree-of-fieedom contrast but it should not be viewed as a part of a general row hypothesis on marginal means. The second contrast seems difficult to justify. It would seem


more informative to examine contrasts comparing rows one and three in column two and rows two and three in column three. These single degreeof-freedom tests are suggested as an attempt to learn something from the experiment given that the original hypotheses cannot be tested. Recall that there is no effective hypothesis for this example.

As a hrther illustration, suppose we remove the redundancies fiom the over-parameterized model by solving for (a /3 )12 , (aP)zl , (Q/3)23, ( a / 3 ) 3 2 , and (a/3)33 and then set ( 0 / 3 ) ~ , = 0 if the cell is empty. The first computing method, when applied to the incidence matrix in (b), then corresponds to testing the hypothesis

(b) H A : PI1 - p21 = p32 - p12

P12 - p32 = p33 - p23 *

Note that the second constraint is the same as in the previous method, but the first constraint differs. While it is unlikely that a computer program would be designed to adjust the reparameterization as a h c t i o n of the missing cell pattern, this example illustrates the fallacy of using standard procedures when we have zero cell frequencies.

The incidence matrix in (c) illustrates the situation in which the rank of H,, is less than the number of missing cells. The single interaction constraint is defined in terms of the first two rows and columns of the incidence matrix, and the correct sum of squares for testing this hypothesis is given by &((as) I po,a,j3). Using the first computing procedure, the sums of squares R(a I po, p, (as)) and R(P 1 po, a, (ap)) test the single degree-of-fteedom hypotheses

(c) HA' : Pl2 = P22

HB* 1-121 = P22.

Again, these are reasonable contrasts to examine but should be examined as single degree-of-6eedom contrasts rather than being considered as a general marginal means hypotheses. Examination of this incidence matrix reveals that there is no way to make comparisons of the first two levels of either Edctor with the third level. In this sense there are two separate, or unconnected designs, and the best we can do is analyze them separately. With this interpretation more reasonable hypotheses are given by

(4 ffx : P11 + P12 = P21 + P22

4 : Pll + P21 = P12 + P22.

The purpose of Example 11.5 is to illustrate the potential confusion that may arise when using standard procedures with the reparameterized model when there are zero cell ftequencies. The proper approach is to examine the incidence matrix and develop comparisons that are appropriate for the particular

11.3 No-Interaction Model: Balanced Data 375

combination of empty cells and not try to rely on a general hypothesis statement. These comparisons are simply formulated in terms of the cell means.

11.3 NO-INTERACTION MODEL: BALANCED DATA

We now consider a constrained model in which it is assumed that the cell means satisfj. the no-interaction constraints given, for all pairs of rows and columns, by (1 1.14). Non-redundant forms of these constraints are given algebraically in (11.16) and in matrix form in (1 1.17). For reference we state the constrained model as

y.. = p . . + e . . %JT 23 f j T

subject to : pij - pi. - p. j + p.. = 0 for all i and j. (1 1.73)

11.3.1 Parameter Estimation

From the development in Section 1 1.1, it follows that the constrained estimates are given by (11.23). Writing this expression in matrix form as F = (I - (S, 8 S b ) ) c , it follows that p N N(p, v), where

(1 1.74)

Note that the elements of are not independent. However, the estimates of the marginal means have the same properties as in the constrained case. That is, the estimates of &.are independent with variance a 2 / b n with the analogous result for the column means.

Rewriting the algebraic form of the estimator in (1 I .23), we have

p . . *J = 3 . . z j . - (- Vl j . - Ba.. - TIJ. f g... 1. (1 1.75)

Using this expression for the estimator, or appealing directly to the results of Section 11.1, we see that the residual sum of squares for the constrained model may be written

a h


a b

Note that the residual sum of squares is given by summing (pooling) the residual and interaction sums of squares &om the unconstrained model. In the special case n = 1, the first term is zero, but we still obtain an estimate of u2 with (a - l)(b - 1) degrees of fieedom fiom the second term. This estimate is a direct consequence of the assumption of no interaction. We emphasize that having only one observation per cell does not justify the assumption of no interaction. Such an assumption must be based on the researchers knowledge of the populations being examined.

11.3.2 Tests of Hypotheses

A general expression for the test statistic in a constrained model is developed in Chapter 17, but it is informative to determine the numerator sum of squares directly by computing the residual sums of squares under the two competing models. The test statistics for the hypotheses of equal means and for main effects are described in the following paragraphs.

Hypothesis of Equal Means. The hypothesis is given by Ho: pij = ji,., and our task is to compute the residual sum of squares when the means are constrained by this hypothesis and the constraints in (1 1.73). Note that the no-interaction constraints are implied by the hypothesis constraints, and hence it is sufficient to determine the residual sum of squares for the unconstrained model under the assumption of equal means. This residual is just the corrected total sum of squares given by

(1 1.77)

After some simplification, the numerator sum of squares is


N a2x2((a + b - 2), Ao) (1 1.78)

with Xo obtained as usual. Note that NO differs from the analogous quantity in the unconstrained model.

Main Effect Hypotheses: Main Effect A. stated as in (I 1.6) for the unconstrained model. That is,

The row effect hypotheses can be

HA : pi. = p,. . (1 1.79)

However, in view of the constraints in (1 1.73), this hypothesis reduces to

H A : p i j = p . j for anyj. (1 1.80)

The implications of this observation should be emphasized. The no-interaction constraint implies that the difference between means for any two levels of factor A is the same for all levels of factor B. Thus acceptance of the hypothesis in (1 1.79) is equivalent to acceptance of the hypothesis in (1 1 30). This hypothesis implies that there are no differences in the levels of factor A at any level of factor B as opposed to the marginal means interpretation in the unconstrained model. Rejection of this hypothesis may lead, after further examination, to a conclusion that, for example, one level of factor A is superior at all levels of factor B. This stronger conclusion makes the no-interaction model very appealing but does not justify the assumption of these constraints.

To develop the test statistic, note that the constraints of no-interaction are implied by the constraints in (I 1 .SO). Thus it is sufficient to develop the residual sum of squares under the reduced model

The residual sum of squares for this model is given by

n h n

(11.82)



(11.83)

The non-centrality parameter, XA, is given by (1 1.10). We also note that N A is identical to (1 1.9).

Main Effect Hypotheses: Main Eflect B. hypotheses follow in the same manner. unconstrained model is equivalent to the stronger hypothesis

The results for the column effect The main effect hypothesis for the

HB : p , j = pi. for any i . (11.84)

The numerator sum of squares and the non-centrality parameter NB and XB are the same as in the unconstrained model and were developed in Exercise 1 1.3.

Analysis of Variunce Table. These results are summarized in the analysis of variance table shown in Table 1 1.10. The main effects sums of squares in this table are identical to those in Table 11.4. In the residual row the degrees of freedom and sum of squares are obtained by pooling the interaction and residual rows fiom Table 1 1.4. The denominator for the test statistics is the residual row.

11.3.3 Simultaneous Inference

If we are interested in general linear contrasts on the cell means, the %he% method can be applied based on the hypothesis of equal means. The confidence intervals are given by

TABLE 11.10. Description df SS MS EMS

Analysis of variance for the no-interaction model

Ho a + b - 2 No MSo EMS0 HA a - 1 N A M S A E M S A H B b - 1 NB MSB EMSB

Residual n a b - a - b + 1 RSSc RMS d C-Total abn-1 T S S T M S


T c p : cTp f J(a + b - 2)F(a; (a + b - 2), c$)uur(c*ji) , (1 1.85)

where df = nub - a - b + 1. The Bonferroni method can be applied in the usual way, but note that the Tukey method is not applicable because the elements of ji are correlated.

In view of the interaction constraints, we can make inferences on row effeds in a given column since they are assumed to be the same for any column. Further, from (1 1.73), we have

a 0

i= 1 i=l

If follows that the Tukey and Scheffe confidence intervals are given by (1 1.27) and (1 1.28), with ab(n - 1) replaced by abn - a - b + 1 as the second degrees- of-freedom parameter. The analogous results holds for column effects.

11.3.4 Reparameterization of the No-Interaction Model

We have noted that a convenient way to incorporate constraints into the model is to express the model in terms of a new set of parameters. Imposing the constraints is then equivalent to setting certain parameters equal to zero. The parameterization defined in Section 1 1.1 achieves this since the no-interaction constraints are equivalent to setting (ap) = 0 in (1 1.37). Further, the remaining parameters are defined by the row and column effect hypotheses, making it convenient to compute the test statistics. These observations are summarized in the next paragraph.

Stutemnt ofthe ModeL The no-interaction model is written in matrix form as

y = WXoPB+e , (1 1.87)

where W is the usual cell fi-equency matrix. The basic design matrix is

xo = ( Ja 63 Jb I I ,@ Jb I Ja 63 IL, ) (1 1.88)

and the parameter matrix is

0 0 A:

The parameter vector, 6, is defined as in (1 1.3 1) with (a@ = 0.

( 1 1.89)


Algebraically the model is written

(1 1.90)

where the parameters po, ai, and Pj are defined as in (1 1.38) with redundancies removed as in (1 1.40).

The model is often written in this form without specifj4ng how the redundancies are removed. In some computer programs a full rank model is obtained by setting aa = pa = 0. While it is important for us to be aware of what parameter functions are being estimated, this does not affect the computation of the test statistics as it did in the unconstrained model. The no- interaction model is often referred to as the additive model.

Parameter Estimation. The parameters in this model are estimated by solving the normal equations where XTX is given by (1 1.47) after deletion of the rows and columns associated with (a@.

Hypothesis Testing. The row and column hypotheses are given by

(1 1.91)

(1 1.92)

It should be clear that these sums of squares do not depend on the choice of conditions to remove redundancies fiom the non-full rank version of the model.

Some references advocate the sequential approach to computing sums of squares. Thus, after computing NB as in (1 1.92), a row effect sum of squares is computed as

NA* == R(a 1 Po)- ( I 1.93)

The orthogonality noted in the columns of X ensures that in this balanced data situation the column effect sums of squares computed by these two methods are identical. This is not the case with unequal cell &equencies.

11.4 NO-INTERACTION MODEL: UNBALANCED DATA

When the cell fiequencies differ in the no-interaction model, there do not exist simple algebraic expressions for the estimates of the cell means and the test statistics except in some special cases. We can apply our general results but it is simpler to use the reparameterized model. If the design matrix has full column rank, the estimates of the cell means are given by

11.4 No-Interaction Model: Unbalanced Data 381

(11.94) - p.. = Po + Bi + pj

13

and the residual mean square is computed in the usual way as

(1 1.95)

with degrees of freedom given bydf = n.. - a - b + 1. The numerator sums of squares for testing the row and column hypotheses

are computed as in (1 1.92) and the results are summarized in an analysis of variance table such as Table 11.10.

The requirement that the design matrix has full column rank is satisfied if all cell kequencies are different ffom zero and may be the case if some cell frequencies are zero. We examine the case of missing cells in the next section.

11.4.1 Missing Cells: Estimation

In the constrained model it may be possible to estimate a cell mean even if the associated cell frequency is zero. For example, suppose that rill = 0 and we have estimates of p12, pql, and p22. It follows from the interaction constraint that the estimate of pll is given by

P11 = P I 2 + P21 - P 2 2 . ( 1 1.96)

We now examine the general problem of estimating parameters with missing cells.

Eflective Model. The situation with regard to estimation is best explained in terms of the cell means model. For later reference the concepts are developed in terms of the general constrained cell means model, written as

y = W p + e subject to: Gp = 9. (1 1.97)

Let the vector of cell means be reordered so that it may be written in partitioned form as

P = [;;I. (1 1.97)

Here po and pm denote the vector of cell means associated with the observed and missing cells, respectively. Let m also denote the number of missing cells. The cell ffequency matrix is then written as W = (W, I Wm), where W, is a matrix of zeros of size n.. x m. The interaction constraints are also written in

382 Chapter 11 Fixed Effects Models 11: TwwWay Classification of Means

partitioned form as G = Go I G, . Multiplication of the constraint equations

by a non-singular matrix yields an equivalent set of constraints. In particular, we can perform row operations on G to eliminate 1.1, from as many rows as possible. Thus we can assume that G and g have the form

0

(1 1.98)

where C,, is t x m of rank t. If follows that the constraints on the means in the observed cells are given by G-p, =go. We refer to these as the effective constraints. These results are summarized in the following definition.

Definition 11.1. When some cell fiequencies are zero, we let po denote the means associated with the observed cells. Then the eflective cell means model is given by

Y = w o p o 4- e subject to: Gmpo = go.

The ranks of Wo and G, are p - m and q - t, where t is the rank of C,, in (1 1.98). The dimension of the model, when reduced by these constraints, is dim = ( p - q ) - (m- t), and the residual degrees of freedom are df = (n.. - dim), where n.. is the number of observations.

The estimability of the remaining cell means depends on the structure of

(1 1.99)

the equations

GmoPo + GrnrnPm = g m .

If G,, is non-singular, we have t = m, and we may estimate pm by

P , = GGLkm - G,oPo). (1 1.100)

If t < m, these equations cannot be solved uniquely for p, and the best that we can do is estimate linear hc t ions of the unobserved cell means. That is,

GmmP, = gm - GrnoPO. Y (1 1.101)

These results are summarized in the following theorem.

Theorem 11.1. In the constrained cell means model, the parameter vector p is estimable if and only if the matrix G,, in (1 1.98) is non-singular. If G,, is singular, then some of the parameters in p, cannot be estimated.

Reduction of the constrained model to the effective model to establish estimability is tedious. Further, it is common to use the reparameterized model


to impose the constraints. In that model the structure of the equations (1 1.99) is not evident. The fbllowing theorem establishes an equivalent condition for estimability. (For simplicity we assume g = 0.)

Theorem 11.2. The constrained cell means model, described in (1 1-97), can be reduced to an unconstrained model by a transformation of the form 8 = Mp. The resulting model is given by

y = X,B,+e,

where X , has T = p - q columns. The cell means are estimable if and only if X, has rank r. In that case the estimate of p is given by

where

Proof We introduce a transformation matrix of the form

where the last q rows of M are the constraint matrix G and the first p rows, described by the matrix G*, are chosen so that M is non-singular. (This is a model reduction transformation and is discussed in detail in Chapter 17.) Writing the model in terms of the new parameter vector 8 with components 8, = G*p and eP = Gp, we note that the constraints imply that 6, = 0. The resulting unconstrained model is p = XJ?, + e. If X, has rank r, 8, is estimable and the estimability of p follows .from the non-singularity of M.

Conversely, if p is estimable, if follows that 6, and hence 6,, are estimable. This implies the existence of a matrix, say, T, such that

E[TgJ = TX,8, = 6,.

If follows that TX, = I , . 4x7) 5 r and, !?om Appendix A.I.2, r(TX,) 5 r(X,).

Note that the ranks are given by r(TX,) = r, This implies that

0 r(Xr) = rand establishes the theorem.

Applying this theorem to the two-factor, no-interaction model, it follows that if the design matrix in the reparameterized model has full column rank, then all parameters are estimable, and there is no need to examine the effective model. The following example will illustrate this concept.

384 Chapter 11 Fixed Effects Models 11: Tw+Way Classification of Means

Erample 11.6. Consider the three incidence matrices in Example 11.2, but now assume that the no-interaction constraints are appropriate. Inspection of the incidence matrices shows that all parameters are estimable in cases (a) and (b) but not in (c). If follows that the design matrix has full column rank in the first two cases but not the third. For case (c), the design matrix is given by

Jn11 0 Jn11 0

X = J%l 0 Jn12 0 Jw 0 Jn21 : j, Jw 0 Jnzz 0 J%2 ;i: Jn3, - Jn3, - Jn33 - Jn33 - Jn3,

where the columns are associated with the parameters p o , al , a2, pl, and ,B2. Note that the sum of columns two and three is equal to the sum of columns four and five, hence the degeneracy. It follows that XTX is singular, and the normal equations do not have a unique solution.

A common prccedure is to apply the sweep method to invert as much of this matrix as possible. In this case the last column will not enter into the inverse. Using this partial inverse (a generalized inverse as described in Appendix A.1.12) is equivalent to setting p 2 = 0. Using (11.94) with the computed values of the remaining parameters and C3 = - 6 1 - 6 2 and p 3 = - p , appears to yield estimates of all of the cell means. In fact, the estimates of p I 3 , pZ3 , p3,, and pU are meaningless. To see this, we reduce the interaction constraint matrix to the form in (1 1.98). Assuming the parameters are reordered as (1 1,12,21,22,33 I I3,23,3 1,32), we obtain

1 - 1 - 1 0 - 1 0 1 0 1 ’

1 - 1 - 1 1 0 0 0 0 0 - 1 0 0 - 1 1 0 0 :I G = [

-1 1 0 0 0 0 0 1 - 1

The degeneracy in X follows fiom Theorem 11.2, which shows that dim = ab - (a - l ) (b - 1) - (m - t), where a = b = 3, m = 4 and t = 3. The first row of this matrix gives the effective constraint, and the remaining rows define the functions of the unobserved cell means that are estimable. For example, using the second row of this matrix, we see that the estimate of p13 + p32 is given by p12 + p33, but the individual parameters are not estimable.

To identi@ the source of the estimates of these parameters in the above application of the sweep method, recall the definition of ,B2 under the no- interaction constraints, that is p2 = p.2 - %.. = pE2 - &. If follows that assuming p2 = 0 is equivalent, with i = 1, to assuming that

p13 = 2p12 - pll .

The apparent estimate of p I 3 is based on substituting the valid estimates of p12 and pll into this expression. Since this is not a valid constraint, the estimate is


meaningless. A similar analysis reveals the source of the apparent estimates of the remaining unobserved cell means.

In general, the practice of using such a partial inverse on the reparameterized model will yield the correct value for the residual sum of squares, and correct estimates of any cell means that are estimable, whether or not that cell is observed. The estimates for the remaining cell means are meaningless. Unfortunately, the procedure does not distinguish valid estimates fiom invalid ones. The effective model analysis is a simple method for making this identification.

11.4.2 Missing Cells: Testing Hypotbeses

It may be possible to test the row and column hypotheses HA and HB even though not all cells are observed. For example, if there are no missing cells in column one, we can test the row hypothesis in that column. We now examine conditions under which the desired hypotheses can be tested.

Eflective Hypotheses. The concept of an effective hypothesis is also applicable with constrained models. To identi@ this hypothesis, suppose that in the model with no missing cells, we are interested in testing the hypothesis Ho: H p = 0. We ask if it is still possible to test this hypothesis or, if not, how should we proceed. To examine this situation, consider the cell means model in the effective form given in Definition 11.1 and the relations identified in (1 1.99). The objective is to express the hypothesis constraints in terms of means corresponding to observed cells. Thus, suppose the hypothesis constraints are written as

Hop0 + H m P m = h. (1 1.102)

There are two cases to consider depending on the rank of C,,. First, suppose that C,, is non-singular. Then, solving (1 1.99) for pm and substituting into ( I 1.102) yields the effective hypothesis

(Ho - Hrncrno)po = h - Hmgrn. (11.103)

Note that this relation simply expresses the original constraints as constraints on the observed cell means. It is easily verified that the rank of the coefficient matrix in (1 1.103) is the same as the rank of H and that this hypothesis is equivalent to the original hypothesis.

If Gmrn has rank t < m, write C,, =(I t I C) and partition pm conformably into pml and pmz. We then write (1 1.99) as

(1 1.104)

For simplicity, assume that C,, = I,.

GmoPo + ItPml + CPm2 = g m .

386 Chapter 11 Fixed Effects Models XI: TwcbWay Classification of Means

The hypothesis matrix is similarly partitioned as H = (Ho I Hml I Hm2). Using (1 1.104) to eliminate pml fiom the hypothesis constraints yields the relation

If the coefficient of pm2 is the zero matrix, this is the effective hypothesis and it is equivalent to the original hypothesis. If not, we apply row operations to reduce (1 1.105) to the form

(11.106)

and the effective hypothesis is given by

Hoopo = h,. (11.107)

Thus, in the case in which not all cell means are estimable, we may not be able to test the original hypothesis. The rank of H,, is less than the rank of H, and we are testing only a part of the original hypothesis. A feature of the effective hypothesis is that if these constraints are not satisfied, then the original hypothesis is not true, and this is the hypothesis of maximal rank satisfying that condition (see Hocking, Speed, and Coleman (1 980)).

We thus see that the ability to test a given hypothesis depends on the rank of G,, and the form of the hypothesis. The first observation leads to the following extension of Theorem 1 1.2.

Theorem 11.3. Suppose that we use a model reduction transformation B = Mp, to reduce the constrained, cell means model to an unconstrained model given by

y = X,B, + e ,

where X, has T = p - q columns and that we wish to test the hypothesis

H p = HM-'B= H,B, = h .

If X, has rank T , we are assured that this hypothesis can be tested and the test statistic is developed in the usual way. If X, does not have full column rank, we may not be able to test the hypothesis.

The following example will illustrate these issues.

Eumple 11.7. To illustrate these ideas, consider the incidence matrix

7111 7212 n13

0 n33


Case 1. The main effect hypothesis H A : pi. = p. . is not testable in the unconstrained model, but it is testable in the no-interaction model. This is a direct consequence of Theorem 11.3, since the cell means are estimable and the hypothesis, in terms of the observed cells, is given by (1 1.103). Note that this is obvious because the main effect hypothesis, under the no-interaction constraint, compares means in a column and any of these columns can be used. Thus this main effect hypothesis is equivalent to H A : pI1 = pZ1 and ~ 1 3 = p33.

n13 # 0.

Case 2. ~ 1 3 = 0. From Example 11.6c, the cell means are not all estimable and Theorem 11.3 implies that the hypothesis HA may not be testable. To examine this, we write the main effect hypothesis as

H A : Pi. = p 3 . 112. = 113. .

We then use the constraint matrix G &om Example 11.6 to write the constraints as in (1 1.104) with pm, = 0113, ~ 2 3 , p31) and pm2 = (1132). Substituting into the hypothesis matrix, we obtain

1112 - p32 =

- pll + 2pI2 + 1121 + ~2~ - 31132 = 0.

The reduction in (1 1.106) requires the elimination of pL3* between these two equations and yields the effective hypothesis as in (1 1.107). That is,

HA,jf : = /La1 f1122 .

Note that, in view of the no-interaction constraints, this hypothesis may be written ,ull = p21.

In practice, the analysis is done in terms of the reparameterized model, and the sum of squares for the row main effect hypothesis is generated by setting a1 = a2 = 0. Recall ffom Example 11.6 that we had to set Pz = 0, that is, we must delete the last column of the design matrix to achieve full column rank. The consequences of then deleting the second and third columns depends on how we treat that last column. The two computing methods described for the unconstrained model may be applied.

Method 1. Suppose that the P2-column is permanently deleted. It follows that setting al = a2 = 0 reduces the rank of the matrix by two, and hence the numerator sum of squares corresponds to a two degree-of-ffeedom hypothesis. To determine that hypothesis, recall that setting P2 = 0 corresponds to imposing the additional constraints pi2 = pi . for i = 1,2,3. Using these relations, we may express the means for the unobserved cells in terms of the means of the observed cells. Using these relations to express the hypothesis constraints HA: pi. = p.. for i = 1,2 in terms of the observed cell means we see that the hypothesis tested by this procedure is


The first constraint provides a natural comparison of rows one and two, but the second constraint provides an unusual comparison of rows one and two with row three.

Method 2. Now suppose that the p2-column is not permanently deleted. W i n g o1 = o2 = 0, we see that the remaining matrix, consisting of the p,,, PI and p2 columns, has rank three. This procedure is testing a one degree-of- fieedom hypothesis. To determine this hypothesis, note that setting a2 = 0 in the original design matrix has no effect, since this simply removes the degeneracy. This is equivalent to adding the constraint plj = p. j forj = 1,2,3. Using these constraints to eliminate p m fiom the relation a1 = p1. - pi.. = 0 yields the hypothesis

which we see is identical to the effective hypothesis. w

The purpose of Example 11.7 is not to suggest a procedure for determining what hypothesis is being tested by a given computing procedure. It is designed to warn the user that if the cell means are not estimable, that is t < m or, equivalently, X, does not have full column rank, then the hypothesis of interest may not be testable and the analyst should consider other hypotheses. Perhaps the best approach is to use single-degree-of-fieedom contrasts to recover as much information as possible from the experiment. The effective hypotheses are a possible alternative, but surely leaving the choice of hypothesis to the whims of a particular computing algorithm is not acceptable.

11.4.3 Connected Designs

The question of estimability of the cell means led us to the examination of the effective model and ultimately to Theorems 11.1 and 11.2. An alternative condition for estimability is based on the direct examination of the incidence matrix to see if the cells are connected. Consider any two observed cells, say (i, j) and (r, t). These two cells are said to be connected if we can move from one cell to the other along a row-column path of observed cells. Thus the path might be described as

( I 1.108)

where all cells in the path are observed. If this condition holds for all pairs of observed cells, the design is said to be connected (see John (1971)). Graybill

11.5 Non-Homogeneous Experimental Units 389

(1976) shows that, in the two-factor, no-interaction model, the cell means are estimable if and only if the design is connected. It is assumed that there is at least one observed cell in every row and column. To illustrate the relation of connectedness to estimability, suppose that cell (r ,s) is not observed but cells (r, t) and (k, s) are observed. In the simplest case suppose that cell (k, t ) is also observed. We then have the connecting path (r, t)+(k, t)-*(k, s) and the estimate of prs follows &om the interaction constraint as

izr, = iz, -I izrt - izkt . (1 1.109)

If cell ( R t ) is not observed, we apply the same idea to obtain an estimate of ,uM in terms of observed cells and then substitute that estimate into (11.109). The connectedness ensures that this is possible.

We see that the equivalence of estimability and connectedness depends on the model assumptions. Recalling the incidence matrices fiom Example 11.2, we see that (a) and (b) are connected and (c) is not. Thus, in the no-interaction model, all parameters are estimable in (a) and (b) but not in (c). In the unconstrained model we cannot estimate all means for any of these designs. It is clear that the constraints are used to estimate means for the unobserved cells. The concept of estimability seems to be more general and is easily verified using either the effective model or the rank of X,. (For a discussion of this topic, see Murray and Smith (1985.))

11.5 NON-HOMOGENEOUS EXPERIMENTAL UNITS: THE CONCEPT OF BLOCKING

In the development of the analysis in this chapter and in Chapter 10, we have viewed the data as arising by sampling each of p populations. In the experimental setting the populations were defined implicitly and were sampled by applying the treatments (or treatment combinations) to homogeneous experimental units. We referred to the experiment as a completely randomized design.

In other situations the treatments may be applied to experimental units that are known to be different. Specifically, there may be groups of experimental units such that, within a group, the units are assumed to be homogeneous but that there are differences between groups. The groups are generally referred to as blocks and the experimental units within a block are often referred to as plots. This terminology reflects the fact that early applications of this concept were in agriculture. To illustrate, suppose that we are interested in comparing the yields of several varieties of cotton. The experimental unit is a oneacre plot of land. The experiment is run in several different fields, perhaps in different regions of the country. It seems reasonable that plots within a field might be homogeneous but that there may be differences in the fields due to fertility. In this case the fields are the blocks and fertility is called the blocking factor. If we plant only

390 Chapter 11 Fixed Effects Models 11: TweWay Classification of Means

variety one in one field and only variety two in another field, the apparent variety differences might be due to field differences. A natural procedure is to plant each variety in each field so that we can obtain a measure of variety differences fiee from field differences.

The blocks may be defined by many different blocking factors. Thus, in an experiment in which the units are animals, the blocks might consist of animals from a given litter since they are genetically similar. The experimental units may be a product produced at different factories or laboratories. When comparing reagents used to purifjl swimming pools, the block may be a day since the reagents may respond differently on sunny days than on cloudy days. The idea is that units from a given source are expected to respond similarly but perhaps differently from those from another source.

There are two general reasons for using non-homogeneous experimental units:

1. experiment and are forced to use units from different blocks.

We may have an insufficient number of units for the planned

2. since the results are to be applied more broadly.

We may not want to restrict the experiment to units of one description

For the experiment, it is assumed that there are t experimental units in each block, where t is the number of treatments under consideration, and the treatments are randomly assigned to the units within a block. The design is complete in that each treatment OCCUTS once in each block, hence we get a measure of differences in the treatments from each block. The experimental design is known as a randomized, complete block design. Note that we have only one observation on each treatment in each block. If we have more than t experimental units in a block, it is natural to consider replicating the treatment. This situation is known as a generalized, randomized block design, and we comment on it later in this section.

In many situations the block differences are not of interest, and we are only interested in removing the effect of these differences from our analysis. On other occasions, such the swimming pool study, the blocking factor may be of interest. We will examine these situations in terms of a model developed in the next paragraph.

11.5.1 Model for the Randomized, Complete Block Design

Let gij denote the response for the ith treatment when applied to an experimental unit in thejth block, let pi denote the mean response for the ith treatment and let pj denote the effect of thejth block. Assuming that the response is the sum of


these two quantities plus a random error, the model is written

y.. = pi + pj + eij (1 1.1 10) '3

with i = 1 7 . . - 7 t, and j = 1 , . . -, b. As usual, the errors are assumed to be independent, N(0 ,02) . Without loss of generality we assume that the block effects sum to zero, that is, p. = 0. This follows since the treatment means are inherently measured relative to some standard block effect, and it is natural to use the average block effect as this standard.

Equivalently, we may write this model with pE = po + a,, where a. = 0. In that form we see that the model statement is identical to that given for the two-factor, no-interaction model in (1 1.90). The first impact of this observation is that we are assuming no interaction between blocks and treatments. That is, we are assuming that differences in the mean response to two treatments is the same in every block. Further, it would appear that the analysis developed in Section 11.3 would be appropriate. We now examine the inferences based on this model.

11.5.2 Inferences on Parameters

Proceeding as usual, we see that

(1 1 . 1 11)

and that these sample means are independent of the residual sum of squares,

t b

Rss = c c (y, - 31. - 3.j + s..)2 (11.112) J=l J=1

distributed as a2x2((t - 1)(6 - 1)). It follows that confidence intervals on linear hnctions of the means are developed in the usual way and that the hypothesis of equal treatment means, Ht : pi = p. is tested by

N F(( t - l), (t - l)(b - 1)). (11.113) i= 1 F = RTSI(0 - 1x6 - 1))

The quantities needed to form this test statistic are presented in tabular form, as shown in Table 1 1.1 1. The test statistic in (1 1.1 13) is just the ratio MSt/RMS. Note that this table is algebraically identical to the last four rows of Table 1 1.10, which described the analysis for the two-factor, no-interaction model. The row descriptions have been changed to reflect the current discussion.


TABLE 11.11. AOVfor the randomized bluck design

Treatments t - 1 br (&. - g..)' MSt 1=1

Blocks f x ( g . j - g..)' h.Sb 1 Residual '-' (t - l ) ( b - 1) is RSS R M S C-Total tb - 1

The assumption of no block-by-treatment interaction is implicit in the model. If this assumption is in doubt, we can examine the interaction plot described in Section 1 I . 1.2 and apply the Tukey test described in Section 1 1. I .5.

It is natural to ask if the randomized block design provided a better comparison of these treatments than if the treatments had simply been allocated at random to the tb experimental units. Comparing the test statistic in (1 1.1 13) with that in (10.23) for the completely randomized design, we see that the numerator sums of squares and degrees of f i d o m are identical but the denominators differ.

Comparing Table 11.1 1 with Table 10..3, we see that the residual sum of squares in Table 10..3 is the sum of the blocks and residual sums of squares in Table 1 1.11 and that the degrees of keedom are related in the same way. The comparison of the two designs rests on whether the decrease in residual sum of squares in Table 11.1 1 offkets the loss in the denominator degrees of fieedom. The block sum of squares in Table 1 1.1 1 provides a measure of the differences in the blocks and we then ask how to interpret that information.

One approach is to compare the efficiency of the two designs by comparing the variance of treatment contrasts under the two designs. We define the relative efficiency (RE) as the ratio of the variance of a treatment contrast under the completely randomized design, with b replicates for each treatment, to that of the randomized block design. It is shown by several authors (see, e.g., Harville (1991)) that an estimate ofthis quantity is given by

b(r- 1) ( b - I ) (br - 1) (bt - 1) +- Fb I

- - (1 1.1 14)

where Fb = hf&,/ws. Note that, if Fb = 1, the ratio in (1 1 . I 14) is equal to one, and the designs are equally good. Further, the efficiency of the randomized block design increases with Fb. A simple interpretation of this ratio is that it


would take b ( E W replicates of the completely randomized design to achieve the same variance as the blocked design.

The ratio Fb used in the efficiency computation is the F-statistic used to test for column effect differences in Table 11-10 and it is natural to ask if this ratio is appropriate for testing for block differences. The question of whether such a test is appropriate has been debated for many years with no resolution. (See Lentner, Arnold, and Hinkelmann (1989) and Samuels, Casella, and McCabe (1991) and the discussion in the latter reference.) The issue revolves around the fact that treatments are assigned at random to units within a block as opposed to the unrestricted randomization in the two-factor model where treatment combinations are assigned at random to experimental units. (See Kempthorne (1955) for a discussion of the concept of randomization.)

No attempt will be made here to resolve the controversy, but some comments are in order. First, we note that in many cases, the test for block differences is not relevant. Thus the fields, litters, or days that defined the blocks are of no interest and often may be viewed as a random sample of blocks from a large population of blocks. (In Chapter 13 we discuss this situation in which the block effects are considered to be random variables.) The real issue is whether we obtained more precise comparisons of treatments by blocking. Thus a large value for F b simply implies that the blocking has been effective. Note that a small P-value corresponds to an F-ratio >> 1, hence highly significant F-ratios correspond to highly efficient blocking. It seems more informative to present this in terms of the relative efficiency than in terms of a test of hypothesis.

In other situations the difference in blocks may be of interest. For example, consider a comparisons of medications for high blood pressure. The results are to be used on both males and females, but it is expected that the level of response may be different for the two groups. In this case we are interested in testing for differences in gender as well as for differences in medications. This is a two-factor problem, and we would like to use the methods of Section 1 1.3 if the no-interaction constraint is justified. Unfortunately, we cannot conduct a completely randomized experiment. To do so, we would need a collection of homogeneous experimental units to which we randomly allocate combinations of gender and medication. We are forced to treat the genders as blocks and randomly allocate medications within the sexes. The issue of restricted randomization would preclude a test for sex differences.

In this case it seems reasonable to make an argument for testing block differences. Consider the conceptual population defined by the application of a given treatment to males. We sample this population by selecting a male and administering the medication. In this setting we may view the data as a sample of size one ti-om each of 2t populations. It follows that under the assumption of no medication-by-gender interactions, we can test for both medication and gender differences.

In some cases, such as the last example, the block size is not restricted, and it makes sense to replicate the treatments within each block. This situation led


Addelman (1960) to define the generalized, randomized block design. The model is identical in appearance to the unconstrained, two-factor model. We can now test for interaction, but the observations made above with regard to testing for block differences are still of concern. Addelman stresses the distinction between the case where we replicate each treatment in each block and the case where we make several measurements on each experimental unit. The latter case does not permit a test for interaction, and the interaction term in the model statement is interpreted as a random variable. This situation is discussed in Section 13.6.2 under the heading of mixed linear models.

EXERCISES

Section 11.1.2

11.1 a. Verify the Kronecker product expression for the residual sum of squares in (11.3). b. Veri& the Kronecker product expression for NH" in (1 1.4), either by using the algebraic expression or by writing the hypothesis in matrix form and applying the general expression in (10.14).

11.2 Verify the expression for the constrained estimator in ( I 1.13). Hint: Use the LaGrange multiplier approach. Give an intuitive argument for the result.

11.3 For the two-factor model, develop the expression for the main effect hypothesis, HB, test statistic for this hypothesis, the expected mean square and the associated constrained estimator.

11.4 a. Verify the matrix expressions for the hypotheses, H A B ~ and H A B ~ in (1 1.17). b. Verify ( 1 1.18) and the relations in (1 1.19).

11.5 Use the expression for the expected value of the non-central x2 distribution to verifL the relation between the non-centrality parameter and the expected mean square in (1 1.21).

11.6 Verify the expression for the constrained estimator in ( I 1.23). Hint: Use the Lagrange multiplier approach. Give an intuitive argument for the result.

11.7 Show that the sums of squares for the main effects hypotheses and the interaction hypotheses add to the sum of squares for the hypothesis of equal means. This provides an illustration of Cochran's theorem, described in Chapter

Exercises 395

16. verification.

Use both the algebraic and the Kronecker product expressions for the

Section 11.1.3

11.8 a. For Example 1 1.1 use the Scheffk half-width to test the significance of the interaction contrasts that are suggested by Figure 1 1 . 1 ~ . Contrast this with the conclusions using the separatst method. b. Apply the expression for the most significant linear hct ion fiom Chapter 18 to identify the primary source of significance in the test for interaction. You may prefer to use the first form in (1 1.16) to express the interaction hypothesis. c. Develop the projection ellipse to study the source of significance.

11.9 Show that the numerator sum of squares for testing the hypothesis H j , defined in (1 1.29), is given by (1 1.30). Do this algebraically and in matrix form by writing the appropriate hypothesis matrix and applying the general result.

Section 11.1.4

11.10 a. Use (1 1.42) to write the normal equations for the over-parameterized, two- way classification model. b. Show that sweeping on the diagonal elements in the natural order corresponds using the parameter matrix in (1 1.43). c. Use the parameter matrix defined in (11.36) to transform the normal equations in part a to a full rank set. VerifL the form of the coefficient matrix in

d. Write out the matrices X,, XB, and X A ~ . e. Verify that the solution of the equations in part c is given by substituting the cell means estimates fiij = Sij. into the defining relations in (1 1.38).

(1 1.47).

11.11 Use the parameter matrix in (11.43) to transform to a 111 rank set of equations. Note the structure of the coefficient matrix. Verify that the solution of this set of equations is given by substituting the cell means estimates into (1 1.45).

11.12 model by imposing the conditions a. = p. = 0 and ( ~ $ 3 ) ~ ~ = and j . parameter matrix P' defined in (1 1.43).

Suppose that we remove the redundancies in the over-parameterized = 0 for all i

Show that this leads to the same test statistics as if we had used the


11.13 Box, Hunter, and Hunter (1978) describe a two-factor experiment in which all combinations of three poisons (factor A) and four antidotes (factor B) are considered. Four replicates (laboratory animals) are randomly allocated to each treatment combination. The survival times are table.

1 0.31 0.82 0.43 0.45 0.45 1.10 0.45 0.71 0.46 0.88 0.63 0.66

P 0.43 0.72 0.76 0.62 o 2 0.36 0.92 0.44 0.56 I 0.29 0.61 0.35 1.02 s 0.40 0.49 0.31 0.71 o 0.23 1.24 0.40 0.38 n 3 0.22 0.30 0.23 0.30

0.21 0.37 0.25 0.35 0.18 0.38 0.24 0.31 0.23 0.29 0.22 0.33

shown in ;he following

a. Complete the analysis of variance table, test for interaction, and test the marginal means hypotheses for main effects. b. Construct the interaction plots and identify any possible sources of interaction. c. Use the Scheffe criterion to examine differences in the marginal treatment means. Contrast these results with those obtained by using the separate-t criterion. d. Prepare a plot of residuals against predicted values. e. The residual plot suggests the reciprocal transformation, zijr = l / y i ~ . Make this transformation, and repeat the analysis, noting the effect on the interaction plots.

Section 11.1.5

11.14 a. Verify the estimate of 6 in (1 1.57). Hint: Recall the development of the analysis of covariance described in Exercise 10.15. b. Verify the expression for N6 in (1 I .58). c. Verify the expression for @h in (1 1.63).

Exercises 397

Section 11.2.1

11.15 Describe the estimate of o2 in (1 1.65) if some cell have nij = 0. Note the associated degrees of freedom.

11.16 Suppose the main effect hypothesis is given by the relations in (11.45). Determine the effective hypotheses based on these hypotheses for the three incidence matrices in Example 1 1.2.

Section 11.2.2

11.17 Write normal equations for the marginal-means reparameterized, unconstrained, unbalanced two-hctor model, noting the lack of orthogonality. Note that the solution is given by substituting the cell means estimates into the defining relations in (1 1.38).

11.18 a. Verify the degeneracy in the design matrix for Example 1 1.3. b. Write the normal equations after setting (a@)11 = 0, and determine the solution. e. Verify that the apparent estimate of p1 follows kom the constraint imposed on the cell means.

11.19 a. hypotheses HA* and HA** are identical. b. describe the new parameters in terms of the cell means. c. obtain the solution.

In the proportional frequencies model from Example 11.4, show that the

Write the parameter matrix corresponding to the conditions in (1 1.71) and

Write the normal equations in terms of these parameters and use (1 1.71) to

11.20 Verify that the sequential method tests the hypotheses HA**.

11.21 Suppose we use the parameterization in (11.45). Determine the hypotheses associated with the sums of squares defined by R(aIpO, 0, (a@) and R((~P)IPo, Q Y P) .

Section 11.3

11.22 Consider the no-interaction model in (1 1.73) with cell means estimates given by (1 1.75). Show that the sample cell means gij. are unbiased for pij in this model but have greater variance than iiij.


Section 11.4

11.23 For the incidence matrices in Example 11.2, use the matrix reduction method implied in ( 1 1.98) to show that for the no-interaction model all parameters are estimable for (a) and (b) but not for (c). Show that the design matrix for the first two cases have full column rank.

11.24 Consider the two-factor, no-interaction model with one observation per cell except that nll = 0 for the case a = 2 and b = 3. a. Show that the estimate of p1 is given by

b. data analyzed as if it was the actual observation with the following exceptions:

It is suggested that the missing observation yI1 be replaced by Fll and the

( 1 ) The residual degrees of fieedom is reduced by one, and

(2) The sum of squares for testing the A-effect is reduced by

(8.1 - (a - W1d2 a(a - 1)

A =

Show that this leads to the correct test statistic for testing the hypothesis HA: p . . = pTj for i # i* and any j . c.

t3

Generalize for any a and b.

11.25 For the missing cell problem of Exercise 1 1.24, consider the model

gij = po + oi + Pj + 6s, + eij ,

where we have set yIl = 0, sll = - 1 and sij = 0, otherwise. a. Using any scheme to remove the degeneracy in the design matrix, show that the estimates of the design parameters are correct. b. Show that the estimate of 6 is as defined in Exercise 11.23. Refer to the discussion of the analysis of covariance in Exercise 10.15.

11.26 Verib the hypotheses implied by the two computing methods in Example 1 1.7.

11.27 In Exercise 10.14 we discussed a comparison of fluids used to combat the buildup of lactic acid. Those data can be modeled as a two-factor experiment with hctor one, with two levels, being the type of drink, A or B. Since the drinks are prepared by adding a powder to water, we can consider adding different

Exercises 399

amounts, factor two, to achieve different concentrations. In such experiments it is common to use zero concentration to develop a standard of comparison. This situation was studied by Addelman (1 974), who called it the zero-level problem. a. Show that the two-factor cell means model, with the constraint pll = pzl is appropriate. b. Discuss the problem with using the two factor analysis if the constraint is ignored. c. Describe the contrasts in Exercise 10.14 in terms of the cell means pij and interpret them in the context of main effects and interaction.

11.28 factor model

Suppose you have the following incidence matrix, and assume the two-

1 1 1 0 [; p /I. a. Determine the A-effect hypotheses corresponding to (1 1.69), (1 1.70), and ( I 1.72). b. Determine the effective interaction hypothesis. c. Assuming the no-interaction model, determine the effective model and verify that all means can be estimated. Determine the effective hypothesis for the A- effect. d. Suppose the no-interaction model is reparameterized as

a i = p i b - p a 6 i = l , . . . , ( a - l ) j = 1, ' - ' , b, 'Yj = Paj

where i indexes the rows of the incidence matrix. Write the model in terms of these parameters. Determine the parameter matrix. e. For the model in part d, determine the parameter estimates, and show that Vat-[@, - zkj] is the same for all pairs of rows.

Application

11.29 An experiment was conducted to determine the weight gains of laboratory animal under nine feeding regimes in a completely randomized design. The treatments were defined as all combinations of factor A, three sources of protein, beef, pork, and grain, and factor B, three amounts of protein, low, medium, and high. It was initially planned to have six replicates of each treatment combination, but for various reasons some observations are not available. The cell means, with cell fiequencies in parentheses, are shown in the


following table. The residual sum of squares is RSS = 1888.

I Data for Exercise 11.28 Amount

Low Medium High

S Beef 76.0 86.6 101.8 0 (4) ( 5 ) (6) u Pork 83.3 89.5 98.2 r (3) (4) ( 5 ) c Grain 83.8 83.5 86.2 e (4) (6) (6)

a. Determine the analysis of variance table for the analyses described by ( I ) the sequential method, R(a I po), R(@ I po, a), (2) the partially sequential method, R ( a I po,P), R(P 1 pa, a), and (3) the marginal means hypotheses.

Compare the conclusions with respect to the source hypotheses. b. Prepare the interaction plots. c. Describe contrasts on the cell means for the following:

C1 : Beef vs. grain at the low and medium amounts. C2: Beef vs. pork averaged over all amounts. C3: High vs. low amounts, averages over meat sources.

Write the analogous contrasts using cell fi-equency weighted averages for C2 and c3 *

[Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...

Documents

Transcript of [Wiley Series in Probability and Statistics] Methods and Applications of Linear Models (Regression...