•
•
•
COLLINEARITY IN MIXED MODELS
by
Sandra S. Stinnett
Department of BiostatisticsUniversity of North Carolina
Institute of StatisticsMimeo Series No. 2125T
December 1993
•
•
•
COLLINEARITY IN MIXED MODELS
by
Sandra S. Stinnett
A dissertation submitted to the faculty of The University of North Carolina atChapel Hill in partial fulfillment of the requirements for the degree of Doctor ofPublic Health in the Department of Biostatistics.
Chapel Hill
1993
Approved by:
I/} ;J j, I / (;;/~~/~ Advisor
Reader
Reader
Reader
Reader
•
. ..
SANDRA S. STINNETT Collinearity in Mixed Models (Under the direction of
RONALD W. HELMS)
ABSTRACT
Methods for detection of collinearity in the GLUM are well established,
however no previously published research has been directed toward extending
these methods for use in the mixed model. In the mixed model, collinearity in
the fixed effects arises from ill-conditioning of (X't"'X), leading to inflated
elements of vl/J) =(X't"'xr'. Mixed model diagnostics, analogous to those
used in ordinary least squares regression, were defined using (X'I-'X) in place
of X'X. A procedure for their use was specified and illustrated by applying the
diagnostics to a data set. Preliminary analyses revealed that variation in two
factors, the number of variables in the random effects and constraints on the
covariance of the random effects, produced different collinearity diagnostics for
models with the same variables in the fixed effects.
In order to generalize the behavior of the diagnostics, they were applied
to experimental data with two types of known predetermined collinearities with
increasingly tighter dependencies. The focus of the research was the behavior
of the diagnostics when different random effects were in models containing the
same fixed effects, for a given type and level of dependency and a specified
covariance matrix 11. A procedure, developed for computing (X'I-'X) directly
rather than through fitting of actual models, permitted a pure assessment of the
impact of varying the random effects since the "noise" of the estimation
ii
process was bypassed. The diagnostics were able to pinpoint the
dependencies in the experimental data. The results demonstrated that adding
variables, especially collinear variables, to the random effects diminished the
impact of collinearity in the fixed effects. Repetition of the experiment using
a difference covariance structure produced similar results.
These experimental results were explored in actual data by creating
similar dependencies and varying the variables in the random effects. The
pattern of diminished collinearity was seen when collinear variables were added
to the random effects, though some departure from the experimental results
was seen and attributed to the differences in dependencies, covariances and
the estimation process.
iii
•
•
•
ACKNOWLEDGEMENTS
My sincerest thanks go to my advisor, Ron Helms, for the extended
discussions and helpful advice and support that not only made this dissertation
possible, but also made the task of creating it enjoyable. I have learned a great
deal from him both during my tenure as a student and during this endeavor.
I thank the other members of my committee for their keen interest in the topic
and their responsive feedback. Thank you, Keith Muller, Gerardo Heiss, Gary
Koch and Lloyd Edwards.
Since a dissertation is a culmination of many years of education, I thank
those who have contributed to mine and enabled me to get to this point.
Special thanks go to Dennis Gillings, under whose influence I learned many
analysis strategies and practical aspects of statistics. My work, under his
direction, in the Biometric Consulting Laboratory was a focal point in shaping
my career. For those years, I am grateful. I also especially thank Gary Koch
for ten years of advice, friendship and shared endeavors. He has been a true
mentor, permitting me to work beside him as I was able. His tutelage in
statistics and his professional advice have been invaluable. I also give a special
thanks to Barry Margolin for supporting my pursuit of a grant for developing the
department's statistical consulting course and for allowing me to teach it for
two years after the grant was received. This experience has impacted the
iv
direction of my career enormously. I am very appreciative for the opportunity
he gave me.
I also thank my Chapel Hill friends for sustaining friendships during this
work. The Chambless and Schlitt families have been extensions of my own
and have nurtured my progress. I thank my good friend Ellen Sim Snyder for
much encouragement. Our shared experience of creating dissertations has
made the path to the finish easier. I also thank my friends elsewhere and my
family for their enduring support for many years.
I am very grateful to my mother, Sudie Stinnett, who has taught me,
especially by example, the importance of education. As I become more like her
in pursuit of knowledge and filled with curiosity, I become more grateful for her
example. She has been my greatest supporter throughout the pursuit of this
degree in every way possible. Thank you, Mother, for everything.
v
•
To my mother, Sudie
and
To the memory of my father, Lee
vi
..TABLE OF CONTENTS
Chapter Page
I. REVIEW OF LITERATURE 1
1.1 Introduction.................................. 1
1.2 Collinearity in the General Linear Univariate Model (GLUM) .. 2
1.2.1 General Linear Univariate Model (GLUM) Notation ... 3
1.2.2 Detection and Diagnosis of Collinearity . . . . . . . . . .. 4
1.2.2.11.2.2.21.2.2.31.2.2.41.2.2.5
Foundation for Collinearity Assessment . . .. 5Measures of Collinearity . . . . . . . . . . . . .. 5Steps in the BKW Diagnostic Procedure 11Computation of Collinearity Diagnostics 14Impact of Collinearity . . . . . . . . . . . . . . .. 15
1.2.3 Related Issues 16
1.2.3.11.2.3.21.2.3.31.2.3.4
Parameterization 16Scaling of Columns 16Mean Centering . . . . . . . . . . . . . . . . . . .. 17Distinctions and Additional Aspects 22
Collinearity and Correlation (22); Collinearity,Conditioning, Weak Data and Short Data(23); Assessment of Damaging Collinearity(24); Collinearity-Influential Observationsl27); Maverick Interlopers and RagingControversies (28)
1.2.4 Procedures for Resolution of Collinearity . . . . . . . . .. 29
1.2.4.11.2.4.2
1.2.4.31.2.4.4
Column Scaling .. . . . . . . . . . . . . . . . . . . 30Deletion of Variable(s) Involved in
Dependencies . . . . . . . . . . . . . . . . . . 30Introduction of New Data 31Bayesian-type Techniques . . . . . . . . . . . .. 31
vii
Pure Bayesian (31); Mixed-Estimation (32)
1.2.4.5 Biased Regression Techniques 33
Ridge Regression (34); Principal ComponentsRegression (36)
1.3 The Mixed Effects Model (MIXMOD) 39
1.3.1 Introduction 391.3.2 Definition, Notation and Assumptions 411.3.3 Features of the Covariance Structure 431.3.4 Estimation of Parameters 45
1.3.4.1 Estimation Principles 451.3.4.2 Computing Algorithms 461.3.4.3 Extant Procedures for Estimation of
p, A, and 02 49
Estimation Steps (50)
A
1.3.5 Variance of IJ 511.3.6 Prediction 521.3.7 Objectives of Mixed Model Analysis 53
1.4 Collinearity Diagnostics for Mixed Models . . . . . . . . . . . . .. 53
II. COLLINEARITY DIAGNOSTICS FOR THE MIXED MODEL:AN OVERVIEW 55
2.1 Introduction 552.2 Type of Data 552.3 Specifics of the Mixed Model 562.4 The Diagnostic Measures 562.5 Methods of Computation 602.6 Employing the Diagnostic Procedure 60
2.6.1 Examine Condition Indexes 612.6.2 Look for Gaps in "10/30 Progression" of Cis 632.6.3 Examine the Variance Decomposition Proportions 632.6.4 Determine Involved Variables 642.6.5 Perform Auxiliary Regressions 652.6.6 Determine Uninvolved Variables 66
2.7 Illustration of ,the Collinearity Diagnostics 67
viii
..
•
2.7.1 Description of the Data 672.7.2 GLUM Example and Diagnostics 692.7.3 Mixed Model Examples and Diagnostics 76
2.7.3.1 A Simple Mixed Model 762.7.3.2 GLUM for Longitudinal Data 812.7.3.3 Mixed Model with Multiple
Independent Variables . . . . . . . . . . .. 84
2.8 Factors Impacting Collinearity in the Mixed Model 92
2.8.1 Number and Nature of Variables in Z 93
2.8.1.1 Two variables, Pair-wise collinear . . . . . . .. 932.8.1.2 One Variable 932.8.1.3 Summary 94
2.8.2 Structure of A 102
2.8.2.1 Constrained, One Off Diagonal ElementEqual to Zero 102
2.8.2.2 Constrained, All Off Diagonal ElementsEqual to Zero 102
2.8.2.3 Summary 102
2.8.3 Different Response, Same Fixed andRandom Effects 105
2.9 Summary and Implications of Results . . . . . . . . . . . . . . .. 108
2.9.1 Initial Conclusions . . . . . . . . . . . . . . . . . . . . . . .. 1082.9.2 Implications for Subsequent Research 110
III. APPLICATION OF DIAGNOSTICS TO EXPERIMENTAL DATA .... 111
3.1 Introduction................................. 111
3.2 The GLUM Experiment 112
3.2.1 The Basic Data 1133.2.2 The Dependency Sets 1133.2.3 The Data Series 1143.2.4 The Issues Addressed 1143.2.5 Results 114
3.2.5.1 Simple Dependency: Two Variables. . . .. 1153.2.5.2 Simple Dependency: Three Variables . . .. 115
ix
3.2.5.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) ... 116
3.3 The Mixed Model Experiment 122
3.3.1 The Basic Data . . . . . . . . . . . . . . . . . . . . . . . . .. 1223.3.2 The Dependency Sets 1233.3.3 The Data Series .........•.••............ 1233.3.4 The Issues Addressed 1243.3.5 Mixed Model Matrices and Parameters . . . . . . . . .. 1333.3.6 Values for 4, 0'-, and V" . . . . . . . . . . . . . . . . . . .. 1353.3.7 Results 136
3.3.7.1 Simple Dependency: Two Variables . . . .. 137
Baseline (137); Adding Variables to RandomEffects (137)
3.3.7.2 Simple Dependency: Three Variables .... 139
Baseline (139); Adding Variables to RandomEffects (140)
3.3.7.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) ... 142
Baseline (142); Adding Variables to RandomEffects (145)
•
3.4 Effect of Adding Variables to Random Effects 153
3.5 Summary of Results 157
IV. IMPACT OF RANDOM EFFECTS COVARIANCEON COLLINEARITY 158
4.1 Introduction................................. 1584.2 The Mixed Model Experiment 1594.3 Results 160
4.3.1 Simple Dependency: Two Variables 160
4.3.1.1 Adding Variables to Random Effects. . . .. 1604.3.1.2 Comparison to Experiment 1 162
4.3.2 Simple Dependency: Three Variables 162
x
..
4.3.2.1 Adding Variables to Random Effects . . . .. 1624.3.2.2 Comparison to Experiment 1 164
4.3.3 Coexisting Dependency: Two Variables,Three Variables (Nonoverlapping) 164
4.3.3.1 Adding Variables to Random Effects . . . .. 1644.3.3.2 Comparison to Experiment 1 171
4.4 Summary of Results 172
V. DATA EXAMPLES OF THE IMPACT OF RANDOMEFFECTS ON COlliNEARITY 174
5.1 Introduction 1745.2 The Data 1765.3 Example 1: One Simple Dependency 1765.4 Example 2: Overlapping Dependencies 1815.5 Summary 185
VI. EXAMPLE MIXED MODEL DATA ANALYSISUSING COlliNEARITY DIAGNOSTICS 187
6.1 Introduction................................. 1876.2 Model Fitting Strategy 1886.3 Model Fitting Example 1896.4 Summary................................... 212
VII. SUMMARY AND RECOMMENDATIONS FORFUTURE RESEARCH 213
7.1 Summary of Research 2137.2 Directions for Future Research 217
APPENDIX 1: GENERAL LINEAR UNIVARIATE MODEL (GLUM)COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 220
APPENDIX 2: MIXED MODEL BASELINECOlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 227
APPENDIX 3: MIXED MODEL EXPERIMENT 1COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 234
APPENDIX 4: MIXED MODEL EXPERIMENT 2COlliNEARITY DIAGNOSTICS . . . . . . . . . . . . .. 269
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
xi
LIST OF TABLES
Table 1.1: GLUM Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 4
Table 1.2: The BKW Diagnostic Procedure ..... . . . . . . . . . . . . .. 11
Table 1.3: Details of the BKW Diagnostic Procedure 13
Table 1.4: Selected Collinearity Measures 20
Table 1.5: Structures and Models for the Components ofV(Yk) =I.t=~4~· + u2vk •••••••••••••••••••• 45
Table 1.6: Aspects of Mixed Model Estimation 45
Table 2.1: Row-oriented Format for Presentation of CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 61
Table 2.2: Descriptive Statistics for 34 Black Female Childrenwith 527 Total Pulmonary Function Studies 69
Table 2.3: Correlation Coefficients of GLUM Regressors 70
Table 2.4: FVC: GLUM Results and Collinearity Diagnostics 71
Table 2.5: GLUM Auxiliary Regressions for Pulmonary Data 73
Table 2.6: Number of Pulmonary Studies for Each Subject 76
Table 2.7: A Simple Mixed Model: FVC as a Function of Age 79
Table 2.8: FVC: GLUM Results and Collinearity Diagnosticsfor 527 Observations on 34 Subjects . . 83
Table 2.9: FVC: MIXMOD Results and Collinearity Diagnostics. . . .. 86
Table 2.10: FVC (Z = Int,Ht,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
xii
Table 2.11: FVC (Z = Int,Age,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Table 2.12: FVC (Z = Int,Age,Ht): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Table 2.13: FVC (Z =Int,Age): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Table 2.14: FVC (Z = Int,Ht): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Table 2.15: FVC (Z =Int,Wt): MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Table 2.16: Summary of Results of Altering Z Matrix by DeletingVariables from FVC Mixed Model With All ThreeVariables in Z 101
Table 2.17: FVC(One element lJ. =0): MIXMOD Results andCollinearity Diagnostics . . . . . . . . . . . . . . . . . . . .. 103
Table 2.18: FVC(AII off diag lJ. =0): MIXMOD Results andCollinearity Diagnostics . . . . . . . . . . . . . . . . . . . .. 104
Table 2.19: Summary of Results of Altering Z Matrix byConstraining lJ. Compared to FVC Mixed ModelWith Unconstrained lJ. . . . . . . . . • • . . . . . . • • • . .• 105
Table 2.20: VMAX50'l6: MIXMOD Results and CollinearityDiagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107
Table 3.1: The GLUM Experiment, Series X1{i}, ;=0, ..., 4(One Contrived Near Dependency): ConditionIndexes and Variance-Decomposition Proportions . .. 118
Table 3.2: The GLUM Experiment, Series X2{j}, j=O, n., 4(One Contrived Near Dependency): ConditionIndexes and Variance-Decomposition Proportions 119
Table 3.3: The GLUM Experiment, Series X3{i,j}, ;=2;j=0, ..., 4(Two Contrived Near Dependencies): ConditionIndexes and Variance-Decomposition Proportions . .. 120
Table 3.4: The GLUM Experiment, Series X3{i,j}, ;=0, ..., 4; j=2(Two Contrived Near Dependencies): ConditionIndexes and Variance-Decomposition Proportions 121
xiii
Table 3.5: Variables in Fixed and Random Effects of MixedModel Experiment 126
Table 3.6: Impact of Collinearity in Fixed Effects for DifferentCombinations of Variables in Random Effects forX1{4} 154
Table 5.1: Impact of Collinearity in Fixed Effects for Different.Combinations of Variables in Random Effects forExample 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 178
Table 5.2: Impact of Constraining Covariance of Random Effects Ii.for Model with Intercept, Height, and Height2 inFixed Effects . . . . . . . . . . . . . . . . . . . . . . . . . . .. 180
Table 5.3: Impact of Collinearity in Fixed Effects for DifferentCombinations of Variables in Random Effects forExample 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 182
Table 5.4: Impact of Constraining Covariance of Random Effects Ii.for Model with Intercept, Age, Height, and Heighein Fixed Effects 184
Table 6.1:
Table 6.2:
Table 6.3:
Table 6.4:
Table 6.5:
Table 6.6:
Table 6.7:
Table 6.8:
Results for Model 1 in Step 1 of Model Fitting
Results for Model 2 in Step 1 of Model Fitting
Results for Model 3 in Step 1 of Model Fitting
Results for Model 1 in Step 2 of Model Fitting
Results for Model 2 in Step 2 of Model Fitting
Results for Model 1 in Step 3 of Model Fitting
Results for Model 1 in Step 4 of Model Fitting
Results for Model 2 in Step 4 of Model Fitting
191
192
193
195
196
198
200
201
Table 6.9: Results for Alternative Model 2 in Step 1 203
Table 6.10: Summary of Model Fitting . . . . . . . . . . . . . . . . . . . . .. 204
xiv
..
LIST OF FIGURES
Figure 2.1: Employing the Diagnostic Procedure 62
Figure 2.2: Values of FVC Plotted Against Age 74
Figure 2.3: Values of FVC Plotted against Height . . . . . . . . . . . . . . . 75
Figure 2.4: Values of FVC Plotted Against Weight 75
Figure 2.5: Values of FVC Predicted from Mixed Model withAge in X and in Z . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Figure 2.6: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted Against Age . . . . . . . . . . . . . . . . . . . . . . . . 88
Figure 2.7: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted against Height 89
Figure 2.8: Values of FVC Predicted from Mixed Model withAge, Height and Weight in X and in Z,Plotted Against Weight . . . . . . . . . . . . . . . . . . . . . . 90
Figure 3.1: Condition Index for BX3 and Wi byNumber of Random Effects Variables 138
Figure 3.2: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables 140
Figure 3.3: Condition Index for BX3 and Wi byLevels of Wi and Zj' No Random Effects Variables . .. 143
Figure 3.4: Condition Index for BX1, BX2 and Zj byLevels of Wi and Zj' No Random Effects Variables .. 144
Figure 3.5: Condition Index for BX3 and Wi byNumber of Random Effects Variables, at Zo . . . . . .. 146
xv
Figure 3.6: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z, · ...... 146
Figure 3.7: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z2 · ...... 147
Figure 3.8: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z3 · ...... 147
Figure 3.9: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z4 · ...... 148
Figure 3.10: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at Wo · ..... 148
Figure 3.11 : Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W, · ..... 149
Figure 3.12: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W2 · ..... 149
Figure 3.13: Condition Index for ax1, aX2, and Zj byNumber of Random Effects Variables, at W3 · ..... 150
Figure 3.14: Condition Index for aX1, aX2, and Zj byNumber of Random Effects Variables, at W4 · ..... 150
Figure 4.1: Condition Index for aX3 and Wi byNumber of Random Effects Variables ........... 161
Figure 4.2: Condition Index for ax1, aX2, and ~ byNumber of Random Effects Variables ........... 163
Figure 4.3: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Zo · ...... 166
Figure 4.4: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z, · ...... 166
Figure 4.5: Condition Index for aX3 and Wi by ..Number of Random Effects Variables, at Z2 · ...... 167
Figure 4.6: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z3 · ...... 167
Figure 4.7: Condition Index for aX3 and Wi byNumber of Random Effects Variables, at Z4 · ...... 168
xvi
•
Figure 4.8: Condition Index for BX1, BX2, and ~ byNumber of Random Effects Variables, at Wo 168
Figure 4.9: Condition Index for BX1, BX2, and ~ byNumber of Random Effects Variables, at W 1 •••••• 169
Figure 4.10: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W2 •••••• 169
Figure 4.11: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W 3 •••••• 170
Figure 4.12: Condition Index for BX1, BX2, and Zj byNumber of Random Effects Variables, at W 4 •••••• 170
Figure 6. 1: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Weight, Plotted Against Age . . . . . . . . . .. 210
Figure 6.2: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Weight, Plotted Against Weight . . . . . . . .. 210
Figure 6.3: Values of FVC Predicted from Final Mixed Modelwith Intercept, Age and Weight in X and in Zat Mean Age, Plotted Against Height 211
xvii
•
..
1.1 Introduction
CHAPTER I
REVIEW OF LITERATURE
•
•
The evolution of computers has enabled investigators to statistically
analyze large data sets with great facility. However, the analysis of such data
brings with it the concern of how to diagnose data-related problems. It is
difficult to examine interrelationships among variables in large data sets and
thus to become aware of ill-conditioned data. III-conditioning or coJlinearity
among the variables may produce model results that are incorrect. These
considerations have led, in recent years, to the development of diagnostic tools
for data analysis, primarily in the context of ordinary least squares regression.
However, other types of statistical analyses also require tools for assessing
conditioning, especially when large data sets are analyzed in which the
interrelationships among variables are not easily seen. Recently, the
diagnostics used in least square regression have been extended to other types
of modeling. (See, for example, Schindler (1986) for extensions to logistic
regression, the Buckley-James model and Cox's proportional hazards model.)
However, little if any research has been conducted for extensions of
diagnostics to the mixed model. Initially, this chapter presents an overview of
collinearity, its assessments and remedies, in the context of the general linear
univariate model. Then the mixed model, its notation and recent development,
is presented. Finally, the extension of collinearity assessment to the mixed
model is discussed.
1.2 Collinearity in the General Unear Univariate Model (GLUM)
According to Belsley, Kuh and Welsch (BKW) (1980), "no precise
definition of collinearity exists in the literature." They state (BKW 1980, P 85)
that two variates are exactly collinear if the data vectors representing them lie
on the same line (Le., in a subspace of dimension one). Further, k variates are
exactly collinear if the vectors that represent them lie in a subspace of
dimension less than k, Le., if one of the vectors is an exact linear combination
of the others, or equivalently, if the vectors are linearly dependent. In practice
and in this dissertation, we are concerned with the situation in which there are
~ collinearities among variables rather than exact collinearities. Statisticians
use the terms collinearity and multicollinearity to designate cases in which the
data vectors are approximately, but not exactly, collinear.
Collinearity may occur for several reasons. Rawlings (1988, p 274)
states that geometrically, collinearity results when at least one dimension of the
X-space is very poorly defined in the sense that there is almost no dispersion
among the data points in that dimension. A single variable that has little
dispersion will be collinear with the intercept term in the model. Other reasons
are listed by Rawlings (1988, pp 327-328). One cause of collinearity is due to
2
•
•
..
..
"
•
•
mathematical constraints on variables; another is due to the transformations of
original variables to create new ones. Other causes are related to study design
and sampling. Inadequate study designs may produce levels of experimental
factors that are not orthogonal, i.e., are nearly collinear. Inadequate sampling
may create dependencies which are an artifact of the data collection process
and may not be present in other samples. Collinearity also may be due to the
system or process being studied and thus will be present in all data from the
system.
BKW (1980) emphasize that collinearity is not a "statistical" problem (in
the sense of probability distributions, etc.), but rather a data problem.
Nevertheless, collinearity in the data impacts the statistical model in several
ways. First, due to redundancies, the contributions of collinear variables are
often inseparable. Second, estimates of model parameters are often imprecise,
Le., the estimated coefficients of all variables involved in the collinearity have
high variances. This is in comparison to variances of estimates in a model with
nearly orthogonal variables. Finally, predicted values can be inconsistent with
underlying science.
1.2.1 General Linear Univariate Model (GLUM) Notation
After discussing the concepts of collinearity in general terms, we now
introduce notation and define terms mathematically. The general linear
univariate model of full rank (GLUM-FR) is represented by the model equation
Y = X/J + e, (1.2.1)
where Y is an Nx 1 response vector, X is a full column rank Nxp matrix of
3
explanatory variables, Il is a
p x 1 vector of parameter
estimates and e is an N x 1
vector of unobservable errors.
The random errors have
E[e] =0 and V[e] =021.
Table 1.1 GLUM NOTATIONModel equation, data from the kth subject:Yk = XJl + " k= 1, 2, ..., NwhereYk :0: measurement of a dependent variable
from the kth subject, k =1, 2, ... , Nx.: =~ of the design matrix for the kth subjectII = vector of primary parametersek - an unobservable error term
Ele] =0, Vle] - crtAssumptions lead to:
ElY] =X/l, VlYl -crt
Notation for the kth subject is presented in Table 1.1 above. The least squares
estimate of Il, the estimate that minimizes the sum of the squared error terms,
is
(1.2.2)
A.
The variance of Il is
(1.2.3)I
In the GLUM, collinearity arises from ill-conditioning of X and X'X, leading toA.
inflated elements in the variance of /1...
1.2.2 Detection and Diagnosis of Collinearity
Historically, many procedures have been used to detect the presence of
collinearity. Several are reviewed by BKW (1980, pp 92-98) and Belsley
(1991, pp 26-37) with respect to their deficiencies; these are the forerunners
of currently used diagnostic tools. Numerical analysts have used techniques
to assess collinearity with a view toward obtaining a matrix A which is
conditioned well enough for the solution to the equation Az = c to be obtained
with numerical stability. This is relevant to the statistical process of obtainingA.
a solution to (X 'X)/1 = X'V. BKW (1980) apply the techniques of numerical
4
..
..
analysts to obtain indices of conditioning that 1) indicate the presence of
collinearity, 2) detect the variables that are involved in particular dependencies,
and 3) assess the degree to which the estimates of the coefficients are
degraded by the presence of the collinearities.
1.2.2.1 Foundation for Collinearity Assessment
Informative collinearity diagnostic tools can be derived from the singular
value decomposition of X or the spectral decomposition of X'X. In the singular
value decomposition of X,
(1.2.4)
where U'U =V'V =Ip, 1\ = Diag(A, ~A2 ~ ... ~Ap) contains the singular values of
X on its diagonal, X has full column rank and p =rank(X). Note that Ap> 0 is the
smallest singular value.
The spectral decomposition of X'X (eigenanalysis) is related to the
singular-value decomposition of X. In the spectral decomposition of X'X,
(1.2.5)
•
where V is an orthogonal matrix that diagonalizes X'X. The diagonal elements
of 1\2 = Diag(A,2 ~A22~ ... ~Ap2) are the eigenvalues of X'X and the columns of
V are the eigenvectors of X'X. The matrix of right singular vectors of X is also
the matrix of eigenvectors of X'X. The positive square root of the jth
eigenvalue A/ is the jth singular value Aj •
1.2.2.2 Measures of Collinearity
Several measures useful in diagnosing collinearity are described in many
textbooks on linear regression [Kleinbaum, Kupper and Muller (1988); Myers
5
(1990); Neter, Wasserman, and Kutner (1985); Rawlings (1988); and
Chatterjee and Price (1991)] and are available in most software packages in
current use. Related matrix concepts are presented in Strang (1980). The
primary measures used for diagnosing collinearity are summarized in this
section. The first involves the eigenanalysis of X'X from which is obtained the
condition index (CI) and condition number (CN) of the matrix and another
related measure, the multicollinearity index (MCI). Other measures described
in this section are the variance decomposition proportions (VDP) and the
variance inflation factor (VIF). These diagnostics are usually applied after the
X matrix has been scaled to have equal column lengths. This is accomplished
by dividing the elements of each column vector by the square root of the sum
of squares of the elements of that column. There are varying views on the
issue of centering the data, in addition to scaling it. Centering and scaling are
discussed in section 1.2.3. [Also, Chatterjee and Price (1991) state that
principal components analysis and ridge regression can be used to detect and
pinpoint collinearity. The use of these procedures as alternative estimation
techniques is discussed in section 1.2.4.5.]
Eigenanalysis of X'X' Definitions As described above, the matrix X'X
can be decomposed as a· product of orthogonal and diagonal matrices that
contain its eigenvectors and eigenvalues. Using the singular values of X,
several measures of conditioning are defined. The condition index (CI) is
defined as the ratio of the largest singular value to the jth singular value,
CI = A,IAj • (1.2.6)
The condition number (CN) is defined as the ratio of the largest singular value
6
..
to the smallest singular value,
CN = A,IAp ' (1.2.7)
The multicollinearity Index (MCI) is defined as the sum of the ratios of the
square of the smallest eigenvalue to the square of each eigenvalue:
p A4
MC/=r..!- .j-' A~'J
(1.2.8)
..
Eigenanalysis ofX'X' Utility According to Rawlings (1988, p 276), the
eigenvalues, A/, measure dispersion in dimensions corresponding to principal
component axes of the X-space. The last principal component axis identifies
the dimension with least dispersion. Most authors state that "small"
eigenvalues indicate singularities in the X matrix. However, BKW (1980, p 96
and 104) state that few can agree on how small "small" is and that zero is the
wrong standard of comparison. According to BKW (1980, p 101), the
condition number (the largest condition index) of a matrix provides more useful
summary information on potential problems in calculations involving the matrix
than do the eigenvalues. The larger the condition number, the more iII-
conditioned the matrix. Specifically, it is a measure of the sensitivity of the
matrix to small changes in the components of the equations for which solutions
are sought. Belsley (1991, p 71) claims that the condition number gives a
multiplication factor by which imprecision in the data can be inflated to produce
even greater imprecision in the solution to a linear system of equations. He
states that it is a factor by which a 1% relative change in the data X could
effect a relative shift in the least squares estimates (Belsley 1991, 184). Also,
Belsley (1991, p 51) states that it measures the distance of a matrix from
7
singularity (or collinearity). Further, the condition number gauges the ability of
a matrix to be inverted and gives an upper bound on the "elasticity" of the
diagonal elements of the matrix (X'Xr' with respect to any element of the X
matrix. (The diagonal elements are proportional to the variances of the least
squares estimates.) (Belsley 1991, P 51). Weak dependencies are indicated by
condition indexes of 5 to 10, moderate to strong dependencies by values of 30
to 100, and serious dependencies by values greater than 100 (BKW 1980, P
101). In addition, the number of large condition indexes indicates the number
of contributing dependencies. According to Rawlings (1988), values of the
multicollinearity index (Mel) near 1 indicate high collinearity; values near 2
indicate no collinearity.
Variance Decomposition Proportions (VDP): Definition The estimate of
/1 in the GLUM was given in (1.2.2) and its variance in (1.2.3). Using the
singular value decomposition of X, (1.2.3) can be rewritten asA.
V(/l(p xPi) =u2(X '(p XN,X(N xpi)"' = u2v(px PII\-2(p xpIV'(PX pi
where
(1.2.9)
1/A~ 0
o 1/A~
o 0A.
Then for the kth component of /1,
0 v,
0 V2 (1.2.10).
1/A;.
vp
~
(1.2.11)
The k,jth variance decomposition and the sum of the p components of the jth
8
decomposition are
k=1, ... ,p. (1.2.12)
Since t/Jk is the variance of the kth regression coefficient, the variance
proportions are
k,j=1, ... ,p. (1.2.13)
•
"it< is the proportion of the variance of 1Jk attributable to the collinearity
indicated by AtVariance Decomposition Proportions (VDP): Utility The components
involved in the dependencies will have small eigenvalues in the denominators
of (1.2.12) and thus, large variances. A large "it< indicates that the kth
independent variable is a major contributor to jth principal component. A
subset of regressors with large variance proportions associated with the same
small eigenvalue indicates dependencies in that subset. Usually, the variance
proportions are displayed in a table that identifies the associated eigenvalue
and/or condition index.
Two qualifications are described by Belsley (1991, P 60-61). First, if
some collinear variables, say Set 1, are orthogonal to all other variables in a
model, say Set 2, then the collinearity may affect only the variables in Set 1.
Thus, all variances may not be affected by the collinearity. Second, collinearity
is present only if two or more variables are involved in the dependencies.
Variance Inflation Factor (VIF): Definition If the data are centered and
scaled, then the diagonal elements of the inverse of the correlation matrix of
9
A
X (which corresponds to the correlation matrix of Il, except for the intercept) are
called the variance inflation factors. The relationship between the variance of
the regression coefficients and the variance inflation factors is given by
(1.2.14)
where Xk is the kth column of X, centered and scaled to unit length (Rawlings
1988, p 277).
Variance Inflation Factor (VIF): Utility The diagnostic value of the
variance inflation factor is evident when it is written as follows:
_ 1VIFk - -- ,
1-Rf(1.2.15)
where Rk2 is the multiple correlation coefficient of Xk regressed on the other
variables. A "high" VIF indicates an R/ near one and thus, a collinearity. If Xk
is orthogonal to the other variables, VIFk will be 1.0. Thus, the VIF is a
A
measure of how many times larger the V(fJk) will be for collinear data than for
orthogonal data (Mansfield and Helms, 1982). This measure is not as useful
as the other diagnostic tools, however, because it cannot identify multiple
dependencies and because there is no clear-cut definition of what values of the
VIF are "high" and which are "low." It is of some use in detecting overall
problems not involving the intercept. Further, if collinearity is present, the VIFs
will be high, but the converse is not necessarily true. Because lack of a high
VIF often occurs when there are variables collinear with the intercept, Belsley
(1991, P 29) suggests using a VIF computed from uncentered data to enhance
10
..
detection of collinearities involving the intercept. The uncentered form of the
VIF is less general since it only applies when there is an intercept term in the
model.
1.2.2.3 Steps in the BKW Diagnostic Procedure
•
A process for diagnosing collinearity has developed empirically, based on
the work of many researchers. Belsley, Kuh and Welch summarized much of
this research in their book in 1980. Based on a series of experiments with
varying levels of contrived collinearity, they proposed a process for detecting
and assessing the extent and impact of collinearity. Belsley (1991) provided
an extension of the previous work based on another decade of research, his
own and that of others. Additional experimentation provided more information
regarding the diagnostics, especially when more than one dependency is
present. The steps in the procedure advocated by these authors are abstracted
here.
Table 1.2 The BKW Diagnostic Procedure·
1. Determine X.2. Scale the columns of X to equal length.3. Obtain condition indexes and variance
decomposition proportions.4. Determine the number of near dependencies5. Determine which variables are involved.6. Determine auxiliary regressions.7. Determine unaffected variables.
·Source: Belsley 1991, pp 134-135.
According to BKW (1980, p 112), there are two conditions which
should be satisfied in order to identify a "degrading" collinearity. First, a
11
singular value must have a high condition index. Second, condition one must
be associated with high variance decomposition proportions for two or more
estimated regression coefficient variances. After identifying the variables
involved in dependencies, the nature of the dependencies can be explored by
regressing the variates implicated on the others. Specific steps to be employed
for a given model are listed by BKW (1980, pp 157-158) as follows:
12
.,
,
OJ
Table 1.3 Details of the BKW Diagnostic Procedure"
1. Scale the data matrix X to have unit column length.2. Obtain the singular-value decomposition of X, and from this calculate:
a. the condition indexes fill.b. the n matrix of variance-decomposition proportions ...
3. Determine the number and relative strengths of the near dependencies bythe condition indexes exceeding some chosen threshold fl·, such as fI"=10, or 15, or 30.
4. Examine the condition indexes for the presence of competing dependencies(roughly equal condition indexes) and dominating dependencies (highcondition indexes---exceeding the threshold determined for step 3--coexisting with even larger indexes.)
5. Determine the involvement (and the resulting degradation to the regressionestimates) of the variates in the near dependencies. For this step, somethreshold variance-decomposition proportion, n°, must be chosen (n" =0.5has worked well in practice). Three cases are to be considered.Case 1. Only one near dependency present. A variate is involved in, andits estimated coefficient degraded by, the single near dependency if it isone of two or more variates with variance-decomposition proportions inexcess of some threshold value n°, such as 0.50. Presumably, if only onehigh variance-decomposition proportion is associated with this singlehighest condition index, no degradation is exhibited.Case 2. Competing dependencies. Here involvement is determined byaggregating the variance-decomposition proportions over the competingcondition indexes .... Those variates whose aggregate proportions exceedthe threshold n" are involved in at least one of the competingdependencies, and therefore have degraded coefficient estimates. In thiscase, it is not possible exactly to determine in which of the competing neardependencies the variates are involved.Case 3. Dominating dependencies. In this case (1) we cannot rule out theinvolvement of a given variate in a dominated dependency if its variance isbeing greatly determined by a dominating dependency, and (2) we cannotassume the noninvolvement of a variate even if it is the only one with ahigh proportion of its variance associated with the dominated conditionindex--other variates can well have their joint involvement obscured by thedominating near dependency. In this case additional analysis, such asauxiliary regression, is warranted, directly to investigate the descriptiverelations among all of the variates potentially involved....
6. Form the auxiliary regressions. Once the number of near dependencies hasbeen determined, auxiliary regression among the indicated variates can berun to display the relations....
7. Determine those variates that remain unaffected by the presence of thecollinear relations. ...
"Source: Belsley, Kuh and Welch 1980, pp 157-158.
13
According to BKW (1980, pp 158-159), the quality of the regression can
be analyzed after the diagnostic steps have been conducted. They suggested
that the following can be learned: 1) how many near dependencies plague a
given set of data and what they are; 2) which variates have coefficient
estimates adversely affected by the presence of those dependencies; 3)
whether estimates of interest are included among those with inflated
confidence intervals, and therefore whether corrective action is warranted; 4)
whether prediction intervals based on the estimated model are greatly inflated
by the presence of ill-conditioned data; 5) whether specific coefficient
estimates of interest are relatively isolated from the ill effects of collinearity and
therefore trustworthy in spite of ill-conditioned data.
1.2.2.4 Computation of Collinearity Diagnostics
The review of diagnostic procedures described thus far assumes that the
analyst will compute them from scratch. Computation can be done easily using
the SAS IMl procedure; this was the path taken for this dissertation.
Alternatively, Velleman and Welsch (1981) present formulas useful in
computing diagnostics when the regression itself is computed by a previously
written or packaged program and illustrate computations using results from a
computer package. In addition, the article alerts the reader to nuances in
computations and in interpretation of diagnostics. Several points are
noteworthy in regard to this dissertation: 1) computation and interpretation of
the VIF is different for regression through the origin; 2) the spectral
decomposition of X'X is not as computationally stable as the singular value
14
..
•
decomposition of X in the computation of eigenvalues and eigenvectors; and
3) rescaling the columns of X can change conditioning. They advocate an
alternative to BKW's unit scaling, proposing that X be scaled such that the
decimal point separates the digits trusted by the analyst from those likely to be
in error.
1.2.2.5 Impact of Collinearity
Computational According to BKW (1980, P 114), the condition number
reflects the meaningfulness of the digits in the solution of the normal equations.
The rule of thumb given is: • .. if data are known to d significant digits, and the
condition number of the matrix A of a linear system Az = c is of the order of
magnitude 10r, then a small change in the data in its last place can (but need
not) affect the solution z= A-'c in the (d-r)th place.· (BKW 1980, P 114). Thus
as the condition number increases, the trustworthiness of the digits, i.e,
computational precision, in the least squares estimates decreases. In addition,
Belsley (1991, p 178) states that in ill conditioned data, small relative changes
in X and Y can produce large relative changes in the least squares estimates.
Statistical As mentioned previously, collinearity causes variances of the
least squares estimates to be unduly large. When this is true, the value of the
estimates for estimation, hypothesis testing and prediction is attenuated. Some
tests may not attain significance due to large variances of estimates and the
separate impact of important variables may be missed if they are collinear.
[See Willan and Watts (1978) for a discussion the impact of collinearity on
parameter confidence regions, tests of hypotheses, effective sample size and
15
predictability.J
1.2.3 Related Issues
1.2.3.1 Parameterization
A linear statistical model Y = X/J + ~ may be rewritten in a parallel form
as Y =(XG-')G!l + ~ • Z6 + ~ where G is a p x p nonsingular matrix.
According to BKW (1980, P 178), the collinearity diagnostics of Z do not
change substantially from those of X, if at all, however their composition may
be altered. Belsley (1991, p 165) states that neither the condition indexes nor
the variance-decomposition proportion of Z are the same as those of X. The
transformation does not reverse the ill-conditioning of X unless the
transformation is ill-conditioned in a manner reflective of the ill-conditioning of
X and designed to offset it. Usually this would not occur since
parameterizations are chosen based on modeling considerations and not on the
ill-conditioning of X. However, even if one did choose a transformation based
on problems in X, its own ill-conditioning would render it unstable
computationally. Thus, the BKW concluded that reparameterization does not
typically remedy the collinearity in the data. However, BKW did state that
"some linear combinations of regression parameters can be estimated even if
ill conditioning prevents precise knowledge of the specific parameters
estimated." (BKW 1980, p 178). In any case, the diagnostics should be
applied to the model actually used, reparameterized or not.
1.2.3.2 Scaling of Columns
As mentioned in section 1.2.2, diagnostics are applied to the X matrix
16
..
after its columns have been scaled. The obvious reason for scaling is to
remove the effect of unequal units in the columns of the X matrix on the
assessment of collinearity. The scaling is essentially a transformation that does
not result in an intrinsically new parameterization. It merely changes the units
of the columns of X and the elements of fl. However, Belsley (1991, P 171)
states that different column scalings of the same X matrix will give different
singular values and thus cause the collinearity diagnostics to vary according to
scaling. BKW give a procedure for optimal scaling (BKW 1980, P 184), which
removes this ambiguity and gives a matrix with minimum condition number (see
also Belsley 1991, p 171-172), but state that scaling for unit length is
sufficient to approximate the optimal scaling. Scaling to unit length is a form
of the more general scaling to equal length; it is accomplished by dividing the
elements of each column vector by the square root of the sum of squares of
the elements of each column XI (the norm of XI' II XIII) so that the resulting
XII II XIII has unit Euclidean length (Belsley 1991, P 135).
1.2.3.3 Mean Centering
There are divergent points of view with respect to centering each column
of the X matrix prior to fitting models and applying diagnostics. According to
BKW, X should not be centered if the data are relevant to a model with a
constant term (BKW 1980, p 98). They state that centering can mask the role
of the constant in any underlying dependencies and produce misleading
diagnostic results. Rawlings (1988) states that centering makes all
independent variables orthogonal to the intercept column and hence, removes
17
"nonessential collinearity," a term initiated by Marquardt (1980).
The topic of centering was addressed in detail by Belsley (1984) and
discussed by Cook (1984), Gunst (1984), Snee and Marquardt (1984), and
Wood (1984). Belsley (1984) says that the least-squares solution becomes ill
conditioned if small relative changes in X and Y can result in large relative
changes in the estimates; this is a numerical issue. This problem is evidenced
by the condition number of the X matrix and manifested by large variances of
the estimates, a statistical issue. Belsley argues that centering does not affect
conditioning of the X matrix because it does not change the inflated variances
and does not change the sensitivity of the data to perturbations. Mean
centered data do have lower condition numbers and thus appear to be well
conditioned, however Belsley states that centering can remove from the data
the information needed to assess conditioning. He provided empirical evidence
of these assertions. Further, Belsley said that data must be "structurally
interpretable," a term describing "data whose form allows a given numerical
relative change also to be meaningfully assessed as unimportant or
inconsequential relative to the real-life situation being modeled." The effect of
relative change, i.e., perturbations, is the basis for Belsley's assessment of
conditioning. He states that perturbations must be carried out on data whose
form makes sense. That is, the data must have an appropriate origin. Since
centering changes the origin, it renders the assessment meaningless. In
contrast to data-dependent centering, Belsley states that model-dependent
centering is acceptable if it gives structural interpretability. That is, it would be
carried out for all such models and is not dependent on the data at hand. It
18
•
..
seems that this is the only case in which Belsley would preform diagnostics on
centered data.
In response to Belsley's article, Cook (1984) states that Belsley focuses
on numerical stability rather than statistical stability, which is related to inflated
variances measured relative to a specific standard design. He says that
choosing a centering is equivalent to choosing the center of a structurally
relevant design space. Gunst (1984) takes another view of centering. He
reacts to Belsley's "dogmatic insistence that there is one correct technique
within which discussions of collinearity must be stratightjacketed." He believes
that centering does not demean diagnostics, but rather that one must
understand the nature of the centering and know where to look for the
appropriate diagnostic. He believes that collinearity is difficult to define and
that no one measure can completely characterize the nature and effects of
collinear variables. Gunst actually redefines collinearity, based on linear
dependencies, and advocates different measures for assessing different aspects
of collinearity. In fact, he re-presents a table which captures many of the uses
of diagnostics presented by BKW (1980). The table, shown below, summarizes
many of the concepts presented thus far in this chapter.
19
Table 1.4 Selected Collinearity Measures
eetection Measures Estimator Effects Precision
Predictor-variable Condition indices Variance inflationcorrelations factors
...Variance inflation factors Estimator correlations s.e. fA), s.e·(9)
Eigenvalues, eigenvectors Curve decolletages Variance decompositionof X'X proportions
Condition indices Volumes of confidenceellipsoids
Source: Gunst (1984)
Gunst advocates a global approach to assessment. He contends that structural
interpretability is a concept to be considered without regard to collinearity. He
states that the "application of structural interpretability to regression implies
that a constant term is included in the model if appropriate and not of
necessity." Gunst states that to properly diagnose collinearity, the X matrix
should be centered or standardized as deemed necessary for correct analysis.
In general, Gunst is supportive of the arguments of Belsley, but he prefers
different collinearity diagnostics.
Snee and Marquardt (1984) are critical of Belsley's article and produce
oblique arguments to counter his points. They state that the "domain of
prediction" is the key to proper centering and that the collinearity diagnostics
must also relate to this domain. That is, if centering captures the appropriate
domain of prediction, diagnostics should be performed on the centered data.
To counter Belsley's argument that mean-centering masks the role of the
intercept, Snee and Marquardt argue that the intercept is of little interest in
most cases; prediction at the center of the data is of more interest. Much of
20
•
Belsley's arguments for not centering revolve around the empirical evidence he
presents based on "small" perturbations of data which reveal their sensitivity,
centered or not. Snee and Marquardt criticize Belsley's approach by stating
that since the "small" perturbations he produced were actually large, his
conclusions are untenable. They claim that diagnostics should be carried out
on the model being fitted.
Wood (1984) states that the decision to center data "depends solely on
the substantive meaning of the data." Further, the goal should be to estimate
a meaningful intercept; this may involve centering some variables and not
others. Reverting to a limited diagnostic, Wood states that the ~2 values can
be used to assess pair-wise collinearity. In an example model, he shows that
the variance inflation factors (1/( 1-R2i)) reveal that independent variables show
high collinearity without centering and low collinearity with centering. Like
Snee and Marquardt (1984), he counters Belsley's conclusion that small
perturbations causing large shifts in coefficients is evidence of collinearity since
the perturbations Belsley used were not small. He uses a "real" example, in
contrast to Belsley's contrived example, to demonstrate that "centering
ameliorates collinearity but does not remove it."
Belsley (1991, pp 175-183) revisits the issue of mean centering,
providing examples to illustrate his claims. He demonstrates, by way of a
counterexample, that even though centered data have better condition numbers
than uncentered data, mean centering does not reduce the impact of
collinearity. That is, from a computational standpoint, small relative changes
in X and Y still produce large relative changes in parameter estimates; from a
21
statistical standpoint, the variances remain inflated. Further, he states that
diagnosing the conditioning of the mean-centered data (which are perfectly
conditioned) overlooks these two issues whereas diagnosing the basic data
does not (Belsley 1991, pp 178-179). Even worse, conditioning diagnostics
applied to mean-centered data provide misleading information about the true
conditioning of a set of data relevant to a regression model having a constant
term and remove information needed to assess problems with variables that are
collinear with the constant (Belsley 1991, p 183). Actually, since the
condition number for mean-centered data provides information about the
sensitivity of the parameter estimates to small changes in the mean-centered
data, i.e., data without "structural interpretability," the magnitude of the
condition number is also uninterpretable (Belsley 1991, P 189).
1.2.3.4 Distinctions and Additional Aspects
Collinearity and Correlation
Often the correlation matrix of regressor variables is examined for "high"
correlations between pairs of variables, as a means of detecting collinearity.
However the two issues are distinct. When two uncorrelated collinear variables
are plotted against each other, it can be seen that a small angle between the
two vectors is not equivalent to a high correlation between them (though two
vectors with a high correlation will have a small angle). Belsley (1991, P 20)
demonstrates that the angle between two variates is small, while the angle
between the centered variables is a right angle, indicating zero correlation.
Also, Belsley (1991, P 26-27) states that 1) a high correlation indicates
22
..
collinearity, but lack of a high correlation does not imply that no collinearity is
present; 2) three or more variables may be collinear while pairs of the variables
are not collinear; 3) the correlation matrix cannot expose the existence or the
number of coexisting collinear relationships; and 4) there is no consensus on
what values of the correlation coefficient should be considered as "large."
Mansfield and Helms (1982) present an example in which regressor
variables for a multiple regression are highly collinear even though no pairwise
correlations are large. They conclude that 1) the existence of large pairwise
correlations is sufficient for determining collinearity, but not necessary; and 2)
that examination of the eigenvalues and eigenvectors of the correlation matrix
is a "necessary and sufficient means" of detecting collinearity.
Collinearity, Conditioning, Weak Data and Short Data
The terms collinearity and conditioning have been used interchangeably
to denote linear dependencies in data. However, the concept of conditioning
is more global than that of collinearity and encompasses it. Collinearity refers
to a near linear dependency among a set of variables; conditioning refers to the
sensitivity of a relationship to perturbations in the data (Belsley 1991, P 7).
Collinearity in the X's can result in sensitivity in the least squares estimates to
small changes in the data.
Belsley defines "weak data" as data that has been robbed "of the
information needed for statistical analysis to proceed in some dimensions with
adequate precision" (Belsley 1991, P 7). Collinearity is a data weakness.
Another is "short data" (defined below).
23
Assessment of Damaging Collinearity
As indicated earlier, not all collinearity is deleterious. Though the
diagnostics reveal the presence of collinearity, they cannot alone determine the
degree to which it is affecting the parameter estimates. However, Belsley
(1991) has developed a statistical test for the presence of weak data, of which
collinearity is one type. If collinearity is present and the test indicates that the
data are weak, then the collinearity is harmful. Basically, the test assesses
whether the variances of the parameter estimates are "too large" by some
standard. Belsley (1991, P 207) uses the actual component of P as the
standard and tests it relative to its estimated variance. This "signal-to-noise"
test is
(1.2.16)
given that Pi is not equal to zero. He states that a test that T is high (low) isA.
also a test that V{fJj) is relatively low (high), which in turn signals the absence
(presence) of weak data. The parameter T is related to the noncentrality
parameter of the t distribution.
In order to operationalize this test, Belsley generalizes (1.2.16) to a
subset of regressors and proposes a measure of signal-to-noise and a test for
its significance. For details, see Belsley (1991), pages 209-213. The
significant points are captured here.
Initially, the P vector is partitioned so that the subset of interest is
A.
isolated from the other components. Similarly, the least squares estimators PA. A.
are partitioned to give rJJ'"p'2r. Then the signal-to-noise of the least squaresA.
estimator P2 of P2 relative to IJ"2 (any arbitrary point) is
24
•
A A
Because Pz - Np2(IJz'V(lJz))' whereA
V(lJz) = u2(X'zM,Xzr' and
it follows that
(1.2.17)
(1.2.18)
(1.2.19)
A A A
(lJz - 'z),V·'(lJz)(/1z - 'z) - r (A). (1.2.20)P2
Here, r (A) denotes the noncentral chi-squared distribution with P2 degrees ofP2
freedom and noncentrality parameter
A
A • (/1z - 'Z)'V·'(/12)(/1z - '2)'
Then the test statistic is derived as
(1.2.21 )
(1.2.22)
where S2 is the residual sum of squares divided by its degrees of freedom (n-p)
and where (n-p)s2/u2 - r n-p' Then the test statistic (/)2 is distributed as a
noncentral F with P2 and n-p degrees of freedom and noncentrality parameter
A as in (1.2.21), which is the same as the signal-to-noise parameter -r. For the
test, .A = T. 2 and
(1.2.23)
This tests that Ao: -r =T. 2 against-r> T. 2• For test size a, calculate
(1.2.24)
•
the (1-a) critical value for the noncentral F with P2 and n-p degrees of freedom
and noncentrality parameter T. 2• If (/)2 s Fcr' accept Ao; if (/)2 > Fcr' reject Ao '
A
Belsley notes that this test requires one to know Pz and V(/1z) in order to
25
provide 7. 2• He then proposes a ·practical and intuitively appealing definition
for an adequate level of signal-to-noise· that does not require this knowledge.
Details are provided in Belsley (1991), pages 216-217. Essentially, he definesA
a ·y-isodensity ellipsoidW for the least-squares estimator Ilz of Ilz' This is
sometimes call the •ellipse ofconcentration.· The y-isodensity ellipsoid definesA
regions of likely and unlikely outcomes for the least-squares estimator Ilz (given
A
X and V(JJz) (Belsley 1991, P 215). This is the ellipsoid of smallest volume thatA A
contains any particular least-squares outcome from I z - Np2(JJz'V(JJz)) with
probability y (Belsley 1991, P 214). Belsley then defines the probabilisticA
distance between ,.z and Ilz relative to I z as the y that determines the
isodensity ellipsoid that is centered on Ilz and includes poz on its boundary
(Belsley 1991, P 215). The is the y such that
•
A A A
(Jrz-Ilz)'Y" (JJz) (Jr2-/2) = I" r p2' (1.2.25)
A level of signal-to-noise y2 is large if it corresponds to a large probabilistic
separation of /1'z from Pz, Le., if it equals a value of )<;2 in the range 0.90 to
1.0. A weak signal-to-noise indicates little separation and has a small value
«0.75) (Belsley 1991, p 215). The magnitude is called
(1.2.26)
or the threshold of adequacy at level y. Then the signal-to-noise r2 of the least-A
squares estimator Ilz of 12 relative to"z is called adequate at level y if
(1.2.27)
..Further details and critical values for testing are provided in Belsley (1991, pp
216-244).
26
•
The pertinence of these results is that a low value of r indicates that
there are inflated variances and data problems either in the form of collinearity
or short data. Belsley clarifies the distinction between collinearity and short
data. -Harmful collinearity is defined as inadequate signal-to-noise occurring
simultaneously with collinearity, while short data is defined as inadequate
signal-to-noise without concurrent collinearity. - (Belsley (1991, p 209) It is
useful to know in which sense the data are weak in order to take corrective
action. Belsley sees this test as complementary to the collinearity diagnostics.
The test can determine the presence of weak data, but not its cause; whereas
the diagnostics can determine whether an existing data weakness is due to
collinearity or to short data.
Collinearity-Influential Observations
Influence diagnostics are aimed at the detecting the disproportionate
effect of an observation on the regression coefficients and other model
parameters. In addition, an influential observation may also induce (or mask)
collinearity. These observations are called collinearity influential (Belsley 1991 I
P 246). Influential observations can be those with outlying values of the
dependent variable Y or those with leveraging values of a row of X, or both.
Leverage indicates that a value is so divergent from that of the other
observations that it overly impacts the estimation process. Several diagnostics
for detecting these collinearity influential observations are given by Belsley
(1991, Chapter 8). One general method attributed to Chatterjee and Hadi
(1988) measures the relative change in the condition number of X that results
from the deletion of a row of X. Belsley gives several weaknesses of the
27
diagnostic, one of which is that it is computationally intensive for a large
number of observations. He also notes that diagnosing the presence of several
coexisting collinearity-influential observations becomes more difficult. Belsley
concludes that the diagnostic process can identify "good" or "bad" data points
in terms of inducing or worsening collinearity. And the decision to correct,
remove or keep them would depend on how they are classified. However,
according to Belsley (1991, P 270), this process does not add anything to
currently used influential-data diagnostics.
A contrasting view was presented by Mason and Gunst (1985). They
suggest that when outlier-induced collinearities occur in X, estimators can
exhibit effects that are associated with both collinearity and outliers. In
particular, collinearities produce large coefficient estimates while leverage
points tend to drive estimates toward zero. They recommend a careful
examination of the nature of collinearity in a data set, providing general
procedures for combining influence and collinearity diagnostics.
Maverick Interlopers and Raging Controversies
After this review, the process of detecting collinearity might seem
straight-forward. However, many researchers still disagree on several issues.
Often, an article by one author is followed by reviews and comments by others.
To read the entire set is both illuminating and entertaining. One can almost be
equally persuaded by each point of view. Stewart (1987) reviewed current
diagnostic measures for collinearity and proposed new ones called collinearity
indices, defined as the square root of the variance inflation factors. He gives
four reasons why a change to these indices is desirable: 1) the indices are
28
...
scale invariant; 2) the nomenclature reflects the purpose of the indices while
that of other measures does not; 3) the indices vary linearly with the relative
distance to exact collinearity, whereas variance inflation factors vary as the
square (This renders them more interpretable and "removes unsightly square
roots from formulas. "); and 4) the indices cast results in terms of relative
errors. Stewart addresses most other issues raised regarding diagnostics,
including the effects of centering and errors in regression variables. He
provides a diagnostic procedure and gives the properties of the indices. This
extensive and impressive article concludes with a suggested documentation for
a regression package that uses collinearity indices.
Comments following the article by Marquardt (1987), Belsley (1987),
Thisted (1987), and Hadi and Velleman (1987) are filled with praise for
Stewart's "lucid and practical article," "his substantive contribution," and his
"time and energy." The praise is followed by each reviewer's criticism, harsh
at times, and re-presentation of his own points of view, often integrated with
those of Stewart, however. Belsley (1987) concludes his review of Stewart's
article by stating that if information is needed regarding multiple dependencies,
then something more than VIFs and Stewart's indices will be needed. In this
case, he says "try mine, you'll like 'em." This is the route taken for this
dissertation.
1.2.4 Procedures for Resolution of Collinearity
.. After the presence and nature of collinearity have been determined, steps
may be undertaken to correct or remedy the ill conditioning in the data. Several
29
measures for resolution of the problems are described in this section. They
range from some simple data-related approaches to the use of Bayesian and
biased regression methods.
1.2.4.1 Column Scaling
A simple approach to improving conditioning is to scale the columns to
unit or equal length as described in section 1.2.3.2. This procedure does not
add or delete data, but transforms existing data in a manner that renders it
more computationally optimal. According to BKW (1980, P 194), column
scaling can often reduce the condition number by a factor of 103 or more.
Marquardt (1980) emphatically presents his views regarding scaling. He
states that the goals of an analyst are "facilitated when the predictor variables
are expressed in a scaling that makes the terms as nearly orthogonal and of
equal size as practicable." He claims that standardization is the only way to do
this. He provides explicit steps for scaling regression data.
1.2.4.2 Deletion of Variable(s) Involved in Dependencies
One seemingly obvious solution for improving ill-conditioning caused by
variables involved in dependencies is to delete redundant variables. When this
is done, the collinearity is removed. However, the model may no longer be
meaningful. Belsley states that this procedure should be avoided. "If an
investigator has reason for including a variate in the regression model in the
first place, there is just that much reason for not excluding it capriciously."
(Belsley 1991, P 301). If collinearity is affecting the parameter estimates, his
conclusion would be that "the data lack the information needed to accomplish
30
the statistical task at hand with precision, not that the model must be molded
into a form that looks good relative to the data." (Belsley 1991, P 304).
Hamilton (1987) shows by way of example that "omitting all but one of
the redundant variables," as is of often advocated, can amount to "throwing
out the baby with the bathwater." He warns of the dangers of discarding
variables as a cure for collinearity because correlated variables are not always
redundant. (This is the other side of the coin for the statement the
uncorrelated variables are not necessarily not collinear.) He restates the view
of several others that backward elimination variable selection procedures be
used in contrast to forward selection in the presence of correlated explanatory
variables.
1.2.4.3 Introduction of New Data
..
Another approach to improving conditioning is to collect and use new
data points in order to provide more variation than that contained in the original
data. However, this is often an impractical approach because of time, budget
or study design constraints. Even if new data are used in the model, there is
no guarantee that the problem will be completely alleviated (BKW, pp 193
194).
1.2.4.4 Bayesian-type Techniques
Pure Bayesian
BKW (1980, P 194) mention, but do not describe, a purely Bayesian
approach developed in the 1970's. In this technique, prior subjective
information about the parameters in the model is used to improve conditioning.
31
Several disadvantages of the approach are listed. First, it uses subjective
information that is either not obtainable or not trusted by many investigators.
Second, the statement of the exact distribution may be too precise to be
realistic. Third, the theory underlying the decision is neither well understood
nor accepted. Fourth, computer software needed to apply Bayesian methods
is not available widely (BKW 1980, P 194).
Mixed-Estimation
The approach called mixed-estimation, which was also developed in the
1970's, is discussed more fully by BKW (1980 pp 195-196); their description
is paraphrased below. In this technique, supplementary information is added
to the data matrix. For the linear model
Y = X/J + e, (1.2.28)
with E[e] =0 and V[e] =I" restrictions on the elements of IJ are constructed in
the form
c=R/1 + f, (1.2.29)
with E[f] =0 and V[f] =1 2 • R is a matrix of rank r of known constants, c is an
r-vector of specifiable values, and fis a random vector, independent of e, with
mean zero and variance-covariance matrix 12 which is specified by the
investigator. Y and X are augmented to give
[j = [~] P + [f]' (1.2.30)
where
According to BKW (1980, P 195), if I, and 12 are known, the solution to the
equation is obtained using generalized least squares; the unbiased mixed-
32
..
v [fl • [~ ;J.I.(1.2.31 )
estimation estimator is
bME = (X'2:;'X + R'2:2-'Rr'(X'2:;'Y + R'2:2-'c). (1.2.32)
5" an estimate of 2:" is used and 2:2 is given by the investigator as prior
information. The solution may be better conditioned than the problem without
the supplemental prior information.
1.2.4.5 Biased Regression Techniques
It is often desirable to use least squares estimators of the regression
coefficients since they are the best linear unbiased estimators. However, when
the data are collinear, the variance of the estimators may be too large. Opting
for a biased estimator which has a smaller variance is the basis of biased
regression methods. In this case, bias is exchanged for precision.
Several types of biased methods have been proposed, including Stein
shrinkage, ridge regression and principal component regression (Rawlings 1988,
p 337); there are all similar. Rawlings states that biased regression techniques
have not been universally accepted and should be used with caution. He
advises that while biased solutions may be better for estimation purposes, they
may not be better for other purposes, presumably prediction purposes
especially outside the existing X-space. Also, biased solutions may not be
useful in assessing the relative importance of independent variables involved in
a collinearity (Rawlings 1988, p 338). Mason and Gunst (1985) demonstrate
that biased estimators may not be effective alternatives when collinearities are
33
outlier-induced. Swamy, Mehta, Thurman and Iyengar (1985) state that the
determination of whether biased estimation methods can cope with
multicollinearity in least squares regression depends on knowing the estimates
of collinearity implied by both types of estimators. They developed a new
measure of multicollinearity suited to biased estimation. After developing and
applying a formula for measuring collinearity in both cases, they found that "if
one forces a biased regression estimator to satisfy the minimax conditions, then
the other goal of reducing multicollinearity will not be realized." With these
qualifications for perspective, the biased methods are now presented.
Ridge Regression
In ridge regression the ill-conditioning of the X'X matrix is reduced by
adding a small positive constant term to the diagonal elements. The parameter
estimates produced using the augmented X'X matrix are biased. According to
Rawlings (1988, p 338), ridge regression is carried out on the centered and
scaled independent variables Z. The estimate of Pis
b, = (Z'Z + kl)°'Z'Y. (1.2.33)
The variance of Pis
V[b,] = (Z'Z + kl)°'(Z'ZHZ'Z + kl)O'02.
And the bias of b, is
E(b,) - P = [(Z'Z+kl)°'Z'Z - 1]p.
(1.2.34)
(1.2.35)
The choice of a value for k involves several considerations. If k = 0, ridge
regression is equivalent to ordinary least squares regression. According to
Rawlings (1988, P 339), as k increases from 0, several quantities decrease:
1) maximum variance inflation factor, 2) the sum of the variances of the
34
•
estimated regression coefficients, 3) the length of the vector of parameter
estimates, and 4) R2• As k increases, the variance of the parameters decreases
but the bias increases. Thus, the objective of the procedure is to choose a k
that is ·optimal· in reducing variance while minimizing bias. Several choices
of k may be used. Then the estimates of individual regression coefficients can
be plotted against k to give ·ridge traces· (Rawlings 1980, p 339). According
to Rawlings, the value of k chosen is the smallest value where major changes
in the parameter estimates and their variances occur and where R2 has not
decreased too mUCh. The choice is somewhat subjective, but these are useful
guidelines. Rawlings (1988, P 339) also gives a specific equation for
computing k:
(1.2.36)
where p is the number of parameters excluding the intercept, S2 is the residual
mean square estimated from ordinary least squares regression and P(O) is
ordinary least squares regression coefficients, excluding the intercept and
computed with centered and scaled variables (Rawlings 1988, p 339).
Ridge regression has a Bayesian interpretation if the prior probability
distribution of P is assumed to have zero mean and variance-covariance matrix
1(02/k). Then the choice of k expresses the prior belief regarding the variances
of the pistributions of the true regression coefficients. The larger the value of
k, the greater the shrinkage toward zero of the ridge regression estimates from
the ordinary least squares estimates (Rawlings 1988, p 340). [See OmanA
(1982) for a discussion of the choice of the origin towards which P is shrunk.]
In summary, the ridge estimates are weighted averages of the least squares
35
estimates with greatest weight given to the regression coefficients of the
variables involved in the near-singularities (Rawlings 1988, p 341). [See Smith
and Campbell (1980) for a review and critique of ridge regression methods and
the article by Marquardt (1980) for related comments. In particular, Marquardt
discusses the relationship between predictor variable scaling and the a priori
assumptions of ridge regression.]
Two other estimators that are related to the ridge estimator are
mentioned by BKW (1980, P 196). These are the generalized ridge estimator
that has the form
bgr = (X'X + 4r'X'Y, (1.2.37)
where 4 is a positive-definite matrix, and the wedge estimator that has the
form
bw • (Z'Zr'Z'Y where Z = X +kX(X'Xr' . (1.2.38)
Principal Components Regression
In principal components regression, the dimensions of the X-space that
are causing the collinearity are eliminated by restating the model in terms of a
set of orthogonal explanatory variables. This is accomplished by forming a
linear combination of the collinear variables rather than retaining only one (or
a subset) of the collinear variables. Then a dimension is represented by the
combination. These are known as the principal components. They lack simple
interpretation since each is a mixture of the original variables. According to
Chatterjee and Price (1991 ), these new variables enable one not only to obtain
information about collinearity, but also serve as the basis of an alternative
estimation technique. Rawlings' (1988, pp 344-349) description of the process
36
of obtaining the principal components is paraphrased here. First, the singular
value decomposition of the centered and scaled variables Z is obtained. Then
the principal components are the linear combinations of the ~ (column vectors)
that are specified by the eigenvectors of Z. The matrix of sums of squares and
cross products of the principal components is the diagonal matrix of the
eigenvalues. The first principal component has the largest eigenvalue and the
principal components corresponding to the smallest eigenvalues are the
dimension of the Z-space with the least dispersion, typically those dimensions
involved in the collinearity.
Using (1.2.4), the SVD of Z is
Then the linear model
can be written as
Y=2/l+E
Y = ZW'P + E,
(1.2.39)
(1.2.40)
(1.2.41)
since VV' = I. The model can be written in terms of the principal components
as
Y = Wy + E, (1.2.42)
where W = ZV and y= V'p. Then y is the vector of regression coefficients for
the principal components and P is the vector of regression coefficients for the
Z's. Next, Y is regressed on W, the principal components, using ordinary least
squares. The regression coefficients for the principal components are
A
Y=(W'W)-1w,y = 1\2W'Y, (1.2.43)
since W'W = 1\2 =Diag(A/~A/~ ... ~Ap2) are the eigenvalues of Z'Z. The
37
A
variance of y isA
V[y] = ,,-2u2. (1.2.44)
Since the matrices involved in these computations are orthogonal, the
estimates of the regression coefficients and their variances can be computed
individually. Then, the coefficients are tested for significance and eliminated
if nonsignificant. In addition, they are eliminated if they cause a collinearity
problem. Otherwise, they are retained; retained coefficients are denoted by the
subscript g. Finally, the regression coefficients for the principal components
are converted to the regression coefficients for the original variables Z by
.
The estimated variance is
A
/l+ (gl = V(gIY(gl'
2rD+ ] _ V A-2V' 2S IP (gl - (gin S.
(1.2.45)
(1.2.46)
Rawlings suggests being conservative in eliminating principal components
since each elimination is a constraint on the estimates and another increment
of bias. He advocates not eliminating a principal component for which Yj is very
different from zero (Rawlings 1988, p 348).
Mandel (1982) gives the geometric representation and interpretation of
the singular value decomposition and its relation to principal components
regression. For a collinear data set, he illustrates the use of the SVD in
principal components, noting that both the detection and the treatment of
collinearity is greatly facilitated its use. In particular, he echoes the observation
of others that even though regression parameters cannot be estimated precisely
in collinear data, certain linear combination of the coefficients can be estimated
38
with confidence. In addition, he notes that valid predictions can be made under
collinear conditions provided that they take place in the same subspace as the
points on which the regression was computed.
1.3
1.3.1
The Mixed Effects Model (MIXMOD)
Introduction
In medical and public health research, studies involving the observation
of a longitudinal response profile for each subject in two or more groups are
common. The longitudinal dimension, or metameter, might correspond to time
or to varying conditions, such as a sequence of increasing doses of a stimulus
or treatment. Typically, interest lies in determining and modeling one or more
responses over time or under the different conditions. In these studies, the
times or conditions for observation may vary somewhat from subject to subject
and thereby complicate the analysis. Some longitudinal terminology is useful
is referring to such studies. A longitudinal study is (Helms, 1992; Grady and
Helms, 1992):
regularly timed
irregularly timed
consistently timed
if
if
if
the time interval between measurement
occasions is the same throughout the study,
e.g., each month;
the time interval between measurement
occasions differs, e.g., weeks 1, 2, 4, 8, 12
and 24;
all subjects are evaluated on the same
39
schedule, whether or not regularly timed;
inconsistently timed if subjects are evaluated on different schedules,
due to missed appointments or because the
study design called for data collection after a
specified event or episode in the subject's life;
balanced if there are no missing data in a consistently
timed study;
unbalanced if there are missing data and the study is
consistently timed, or if the data are from an
inconsistently time study;
complete if there are no missing data; and
incomplete if there are missing data values.
If longitudinal data are unbalanced or incomplete, then analysis using
standard general linear multivariate model (GLMM) methodology is difficult.
One of the assumptions of GLMM methodology is that the elements in a
column of Yare measurements of the same entity, i.e., measurements at the
same time point or under the same conditions. Thus unbalanced longitudinal
data might violate this assumption, even if the data are complete. Because
data collected longitudinally are rarely complete and because GLMM methods
do not handle missing data, several other analysis approaches have been used
to deal with incomplete and/or unbalanced data. One approach, case-wise
deletion, is to delete all of the data from any subject with any missing data and
then employ methods for analyzing complete data. Another approach is to
impute values, such as the mean of existing observations, to missing data
40
points and then to proceed with the analysis. These approaches involve
manipulation of the data that may result in loss or contamination of
information. A more desirable approach is one in. which the available
incomplete data can be utilized without the constraints imposed by the other
methods.
Recently, statistical methods, in particular the mixed model, have been
developed that permit the analysis of longitudinal data when they are affected
by unbalanced designs, missing data, attrition, time-varying covariates and
other characteristics that make standard multivariate procedures inapplicable.
[See Searle (1988) for a history of the mixed mode!.] Ware (1985) states
several advantages offered by the mixed model. Individuals need not be
observed at the same times or on the same number of occasions. In addition,
time-varying covariates can be included in the model if their contribution to the
expected response can be written linearly. Also, covariates can modify either
the expected value of the dependent variable or its rate of change. Finally,
Ware notes that the mixed model offers other generalities, such as inclusion of
trigonometric functions, that do not complicate the analysis.
1.3.2 Definition, Notation and Assumptions
For the situation in which the kth subject is observed on nk occasions,
the mixed model with fixed population effects and random individual effects is
(Helms, 1992)
y = X IJ + Z d + 8.
The model equation for the kth subject is
41
(1.3.1)
(1.3.2)
where
Y (Nx1)
X (Nxp)
Z (NxKq)
is a vector of the nk observations (dependent variable or
responses) for the kth subject, k= 1,2, ..., K;
= [Y, /I Y2 /I ... /I YK] is the vector of responses from all K
Ksubjects (N= r nk ) (-/I- denotes the vertical concatenation
k-'operator);
is a known fixed effects design matrix for the kth subject (The
columns of Xk represent the values of independent variables.);
= [X, II X2 II ... II XK] is the fixed effects design matrix for the
model;
is a known random effects design matrix for the kth subject (The
columns of Zk represent the values of independent variables.);
= Diag(Z" Z2' "', ZK) is the random effects design matrix for the
model;
..
The following assumptions are made:
/l (p x 1)
dk (q x 1)
d (Kq x 1)
is a vector of unknown constant population parameters, a vector
of fixed effect primary parameters, essentially the same as /l in
univariate models;
is a random vector of unobservable random subject effects for the
kth subject;
= [d, II d2 /I ... II dK] is a vector of random subject effects for the
model;
is an vector of unobservable within-subject random error terms;
42
dk (qx 1)
4 (qxq)
..
...
and
8 (N x 1) = [8, II 82 II ... /I 8K] is a vector of random error terms for the
model.
The following additional assumptions are made:
- NIDq (0, 4);
= V(dk) is the covariance matrix of the random effects (each dk
has the same covariance matrix);
- NIDN (0, UZVk ) independent of the dk; and for k'¢k,
Cov(dk"dk) = 0; Cov(dk,,8k) = 0; and COV(8k,,8k) = 0;
UZ is an unknown scalar within-subject error variance parameter;
Vk (nkxnk) = V(8k ) is the covariance matrix of the random deviations about
the kth subject's random regression line.
These assumptions lead to the following:
Yk - NID (Xk P, ~k);
Y - NN (X P, I);
~ (NxN)
1.3.3
is the V(Yk) =~ = ~ 4 Zk' + UZVk, a positive definite symmetric
covariance matrix for the kth subject (often, it is assumed that
Vk = I..);
= Diag(~" ~2' "., ~) is the covariance matrix of the entire
response vector, Y.
Features of the Covariance Structure
There are several aspects of the mixed model covariance that are
noteworthy. First, the measurements from each subject are correlated, Le.,
43
V(YIl) =Ek is not diagonal. Second, the diagonal elements of Ek are not
necessarily equal. Third, Ek is modeled in terms of a smaller number of
parameters, the elements of 4 and 02. An additional feature of the mixed
model is that the covariance parameters (4 and 02) of ~ = ~ 4~' + o2vll
must be estimated, since they are rarely known. Thus, ~ is estimated. Nested
within the modeling of E is an option of also modeling 4. Finally, prior to the
estimation of the covariance parameters, the structure of Vil must be specified.
The table below provides some of the possibilities for modeling 4 and
structuring Vil • The modeling of t:,. is described in section 1.3.4.3. The
structure of Vil is determined from prior information about the data. Jennrich
and Schluchter (1986) provide a menu (in Table 1) of possible covariance
structures for the incomplete data model. Grady and Helms (1992) explored
a variety of covariance structure models and illustrated methods for comparing
the models with respect to goodness of fit. Louis (1988) states that choosing
among covariance models depends on data structures, subject-area theories,
and available computer packages. Further, he states that the choice affects
estimates and standard errors of fixed effects, diagnostics, interpretations and
extrapolations.
44
..
..
Table 1.5 Structures and Models for the Components ofV(Yk) =~= ~4~' + a2vk
Variance of Random Effects Variance of ek
~4~' 02-vk
Models for 4: Structures for Vk:
(Vk is assumed to be known.)
Linear Vk is diagonalAll parameters used (uncorrelated deviations)
(unconstrained) Diagonal elements equalFewer parameters used and equal one
(constrained) Diagonal elements not equal
Nonlinear Vk is not diagona1(correlated deviations)
Diagonal elements equalDiagonal elements not equal
1.3.4 Estimation of Parameters
There are several aspects to the estimation procedure for the primary
parameters and covariance parameters (fJ, 4, and 02-) of the random effects
model. These are summarized in the table below.
Table 1.6 Aspects of Mixed Model Estimation
Parameters Estimation Principles ComputingEstimated Algorithms
p Maximum Likelihood (ML) EM4 Restricted Maximum Likelihood (REML) Newton-Raphson02- Scoring
1.3.4.1 Estimation Principles
Two principles for the estimation of the mean and covariance parameters
are maximum likelihood (ML) and restricted maximum likelihood (REMLl. [See
45
Searle (1988) for a discussion of several estimation principles and their relative
merits.] Both principles produce sets of simultaneous nonlinear estimation
equations whose solution requires iterative computations. In the ML principle,
reviewed by Harville (1977), the log likelihood function
L(,r,4,p; Y) = constant
(1.3.3)
is maximized for 02,4, and p. One criticism of the ML approach is that the ML
estimators of 02 and ~ do not account for the 10$s in degrees of freedom that
results from the estimation of p.
This "deficiency" of the ML approach can be eliminated using the
restricted maximum likelihood (REML) approach. REML estimators for 02 and
4 maximize L" where (Harville, 1977)
L, (,r,4; Y) = constant
.
(1.3.4)
Here X· is an n x p' matrix whose columns are any p' linearly independent
columns of X and p is any solution of the normal equations
1.3.4.2 Computing Algorithms
(1.3.5)
There are many iterative algorithms that can be used for computing the
ML or REML estimates. Harville (1977), in his review of several algorithms,
46
states that the choice of the "best" algorithm depends on the application and
on computational requirements.
Use of the E-M algorithm to obtain either maximum likelihood (ML) or
restricted maximum likelihood (REML) estimates of the parameters has been
described by Dempster, Laird and Rubin (1977), Laird and Ware (1982),
Fairclough and Helms (1984), Laird, Lange and Stram (1987) and Laird (1988).
The EM algorithm is used for computing estimates of parameters in incomplete
data problems. In each iteration, it consists of two steps: an expectation (E)
step and a maximization (M) step (Laird, 1988). The maximization step is
based on maximizing the likelihood of the "complete data." The rationale for
using this technique in the mixed model situation, which treats data as if it
were complete even though it may be incomplete, lies buried in the history of
the mixed model. Dempster, Laird and Rubin (1976) and Laird and Ware
(1982) describe the connection. It is worth noting at this point that for the
mixed model, the dk can be considered as "missing." It is for this reason that
the EM algorithm can be used in this situation. In effect, the dk are appended
to Y vector as "missing" data. In this manner, they are "estimated." Searle
(1988) prefers to say that they are "predicted" since "the estimation of random
variables is counterintuitive statistically." Laird and Ware (1982) state that
they regard no data as missing; they use the EM algorithm to "estimate"
unobservable (random) parameters, not missing observations. So, in the E
step, the conditional expectation of the 'complete-data sufficient statistic' is
computed based on the observed data and current estimates of the unknown
parameters. In the M-step, the maximum likelihood estimates of the
47
components of ~ are computed, with the sufficient statistics replaced by the
conditional expectations of the statistics produced in the E-step. [See
Fairclough and Helms (1984) for a more complete summary of the process.]
Laird and Ware (1982) also derived an E-M algorithm for computing ML and
REML estimates in the general linear mixed model for V. = I". and unstructured
~. According to Laird (1988), the EM algorithm may not be the most efficient
algorithm and may be slow to converge. However, it offers a general approach
that can be applied in a wide variety of settings; it is easy to implement; and
it will not converge to parameters values outside the boundary of the parameter
space.
Harville (1977) and Callanan and Harville (1991) reviewed various
algorithms for computing ML and REML estimators, including the Newton-
Raphson procedure and the method of Scoring. The use of Newton-Raphson
and related procedures to estimate parameters of random effects models also
was discussed by Jennrich and Schluchter (1986). The Newton-Raphson
procedure is a gradient procedure that utilizes second-order partial derivatives
of the log-likelihood function. This algorithm can converge in few iterations
provided that its initial values are near the maximum value. However, it may
converge to a stationary point which is not a maximum or it may not converge
at all, if the initial values are poor (Harville, 1977). Apparently, the difficulty
can be overcome by using the "extended" Newton-Raphson procedure. (See
Harville (1977) and Callanan and Harville (1991) for a more complete summary
of the procedures.) The method of Scoring, another gradient procedure, uses
the expected values of the second-order partial derivatives instead of the actual
48
derivatives (Callanan and Harville, 1991).
Forms of the maximum likelihood equations that are useful for
computations may be found in Fairclough and Helms (1986), Jennrich and
Schluchter (1986), Laird, Lange, and Stram (1987), Lindstrom and Bates
(1988) and Callanan and Harville (1991). Jennrich and Schluchter (1986) also
discuss implementation of the algorithms, considering computational efficiency,
modifications to improve convergence, methods for dealing with nonpositive
definite estimates for J:, and constraints on the covariance parameters.
1.3.4.3 Extant Procedures for Estimation of P, A, and a2
The procedures of interest for this dissertation involve maximum
likelihood and restricted maximum likelihood estimation with the E-M algorithm.
The maximum likelihood estimate of P is a solution of the Aitken estimation
equation:
(1.3.6)
A. At A A. At. At. A
where J:k = Zk 4 Zk' + o2vk and 4, 02, and Vk are maximum likelihoodA A
estimates of 4, 02, and Vk ' respectively. In order to compute p, ~ must beA A
estimated, however estimation of ~ requires knowledge of p. Thus, an
A A
iterative algorithm must be used to solve for P and for~. There are several
versions of the maximum likelihood estimation equations for the variance-
covariance parameters, depending upon the structures of 4 and Vk and the
algorithm being used to solve the equations. Fairclough and Helms (1986)
obtained the following maximum likelihood estimating equations for use with
49
A A
the E-M algorithm. These procedures usually require initial estimates of /1, 4,
A
and cr in order to set the iterative process in motion.
Estimation Steps
A A A
1. Estimate /1 using (1.3.6) (For first /1, let E.t =1; this is the OLS
estimate of /1.)A
2. Solve the following equation for each dk, k =1, 2, ..., K: (For the
A
first iteration, let 4 =0 and cr =1.)
[b24-' + z:V;'z.]a.=z:v;'(Y.-X.h).
A
3. Compute cr as
U' =~ [t. (Y.-X~ - Z,.d.) I V;l(Y.-X~-Z,.d.)J.
A
4. Compute 4.
(1.3.7)
(1.3.8)
If 4 does not have a linear covariance structure, Le., is not
modeled, the estimator of 4 is
K.. 1 r .... I4 = K4JdP•.
k-1
(1.3.9)
If 4 ~ have a linear covariance structure, then its model
equation is:
where
H
4 = V(d.) =E ThGh ,h-1
(1.3.10)
Gh denotes a known, constant, symmetric matrix, h =1, 2, ...
50
H;
Th denotes an unknown variance-covariance parameter, h = 1,
2, ... H; and
T = (T1, T2, •••, TH)' denotes the H x 1 vector of variance-
covariance parameters.
Then, the estimator of 4 is
A-
T is the solution of
(1.3.11 )
where the notation <> gil denotes a matrix whose (g,h)-element is
the scalar inside the angled brackets ( <>)and the notation <>g
..
~ [ (Trac~A -1 G,A -1 G" ) ),h] t =
(t Trac~A-'G,A-'iljJ~ ) ), ,k-'
(1.2.12)
denotes a vector whose gth element is the scalar inside the angled
brackets.
5. For each k = 1, 2, ... , K, compute
(1.3.13)
1.3.5
A A
(If 4 has a linear covariance structure, use ~ in place of 4.)
6. Repeat the entire process until the estimates have converged.A A A
Then, the final estimates of the parameters P, 4, and if are used.A
Variance of /l
Exact small-sample expressions for variances and covariances of
MIXMOD maximum likelihood or REML estimators are not available (Helms,
51
1992). Instead, asymptotic expressions for variances, covariances andA.
standard errors are used. The asymptotic variance of /l is given by
(1.3.14) ..
or
(1.3.15)
1.3.6 Prediction
Harville (1977, P 322) states that predicting a future data point from
data to which the mixed model applies can be formulated as problem of
estimating a linear combination of the components of the fixed and randomA. A.
effects, Le., of IJ and d. Helms (1992) gives a "convenient" estimator of a
future data point as
,
K
W = LrJ + E L.dt '*-1
(1.3.16)
where Lk , k =1, 2, ..., K are known, constant matrices, each with a rows. (to is
a xp; Lk is a x q, k ~ 1.) Helms (1992) gives the variance and standard error
of the predicted value as
and
v(w) = L(JL I(1.3.17)
(1.3.18)
A.
with vEH =N-Rank(X II Z) d.f., where Q is a generalized inverse of the coefficient
matrix in the mixed model equations,
52
(1.3.19)
1.3.7 Objectives of Mixed Model Analysis
In general, the objectives of a mixed model analysis are (Hold itch-Davis,
•
Helms and Edwards, 1992) 1) to estimate the fixed effects component and testA
hypotheses about the population regression coefficients, p; 2) to simplify the
fixed effects component by eliminating variables with nonsignificantA
coefficients (/1); 3) to estimate and test hypotheses about Ii, the covariance
matrix of the random effects; 4) to simplify the random effects component by
eliminating variables with small coefficients (6jj); and 5) to simplify the model
for Ii by including large covariances (6•.) and deleting small ones. Methods for
4) and 5) have not been reviewed for this dissertation because they are not
particularly relevant.
1.4 Collinearity Diagnostics for Mixed Models
In the mixed model, collinearity in the fixed effects arises from ill-
conditioning of (t·' /2X) and (X't·'X), leading to inflated elements of
V(/J) =(X'r'xr'. In the GLUM, collinearity stems from X; in the mixed model,
collinearity might stem from X, from Z, or from both. The objective of this
dissertation is to explore the aspects of collinearity described in this chapter for
the GLUM in the context of the mixed model. Some research has been carried
out on influence diagnostics, residual plots and model adequacy for the mixed
model, though often in the context of ANOVA models. [See Louis (1988);
53
Beckman, Nachtsheim, and Cook (1987); and Christensen, Pearson and
Johnson (1992).] However, at this point, no previous work on the topic of
collinearity diagnostics for mixed models has been unearthed.
54
,
..
CHAPTER II
COLLINEARITY DIAGNOSTICS FOR THE MIXED MODEL:AN OVERVIEW
2. 1 Introduction
In this chapter, the parameters of this research are specified. These
include descriptions of: 1) the type of data to which the diagnostics are applied,
2) the specific nature of the mixed model being used, 3) the formal procedure
for collinearity assessment in the mixed model, and 4) methods for fitting the
mixed model and computing the collinearity indices. The complications arising
from the involvement of the matrix I also are addressed. Initial steps in the
process are illustrated by computing the GLUM and MIXMOD collinearity
diagnostics for a data set with some collinear variables. Finally, several factors
unique to the mixed model that may impact collinearity are discussed. This is
a prelude to Chapter 3 in which the behavior of the diagnostics, determined
empirically under a wider variety of conditions, will be reported.
2.2 Type of Data
Exploration of collinearity in the mixed model will be carried out for a
specific type of data: 1) The response vector, V, for each observation is a
sequence of measurements on a continuous scale. 2) The variables comprising
both the fixed effects design matrix, X, and the random effects design matrix,
Z, are also measurements on a continuous scale. 3) The response vector, V,
and the design matrices, X and Z, reflect assessments made on nk occasions
for the kth subject.
2.3 Specifics of the Mixed Model
For the analyses in this dissertation, specific aspects of the mixed model
will be featured; the conclusions reached apply to these circumstances.
Specifically, A is model~d linearly. The structure for Vk is Vk =I. Maximum
likelihood equations are used for parameter estimation.
2.4 The Diagnostic Measures
The mixed model was introduced in section 1.3 and the collinearity
diagnostics for the GLUM were discussed in section 1.2. Now diagnostic
measures, analogous to those used for the GLUM, are formulated for the fixed
effects of the mixed model.
Recall that the model equation for the mixed model is
•
V = X IJ + Z d + e;the estimate of Pis
A
and the estimate of the asymptotic variance of IJ is
(2.2.1)
(2.2.2)
(2.2.3)
A
where I is the estimated covariance matrix of the response vector, Y. When
56
..
the matrix (X'!:·'X) is ill conditioned, the estimates of IJ are likely to be unstable
and their variances inflated. This matrix is analogous to the matrix X'X in the
GLUM and is evaluated similarly for ill conditioning. Collinearity diagnostics for
the mixed model can be obtained through the spectral decomposition of
A
(X'i:"'X). Since E is involved, the measures must be computed after the final
estimation of E. Thus, for the mixed model, the degree of collinearity present
in the data is assessed after the estimation of model parameters. In contrast,
the assessment in the GLUM is done before estimation.
The process of computing the diagnostic measures for the mixed model
is similar to that described in section 1.2.2.2 for the GLUM. However, there
are differences due to the different nature of the models. These are described
in this section.
Eigenanalysis of(X~'X): In order to parallel the GLUM development, an
eigenanalysis is performed on
(2.2.4)
A
which is a scaled version of (X':!:"'X) that has 1's on the diagonal. Letting
(2.2.5)
then W'W is the matrix defined in 2.2.4. The spectral decomposition of
W'W = V1\2V' can be used to obtain its eigenvalues and eigenvectors, as was
done for previously for GLUM using X'X. Using the singular values of W, the
condition indexes can be obtained as described for the GLUM in section
1.2.2.2. The condition index (CI) is defined as the ratio of the largest singular
value to the jth singular value,
57
(2.2.6)
the condition number (CN) is defined as the ratio of the largest singular value
to the smallest singular value,
CN = A,IAp ' (2.2.7)
Variance Decomposition Proportions (VDPs): Similarly, with W as
defined (2.2.5) above, and using the spectral decomposition of W'W, the
A
estimated variance of IJ =(X't"xr' can be decomposed as
•
A A
V(JJ(p>cp,) =(W'(P>cN,W(N>cp,r' = V Cp >cp,I\-2Cp >cp,V'(p>cp" (2.2.8)
A
Then the variance decomposition of IJ can be obtained as described for the
GLUM in section 1.2.2.2. Specifically Vcp>cp,J\-2Cp>CPIV'(P>CPI can be rewritten as
lIA~ 0 0 V,
VJ\-2V' = [v, VII']0 11A~ 0 V2 (2.2.9) •V2 ...
0 0 VA; vII'A.
Then for the kth component of the scaled version of P,2
A (/J P vlcj (2.2.10)V .J = I:-.j.' A~1
The k,jth variance decomposition and the sum of the p components of the jth
decomposition are
2VIcj
(/)k" = I A~
1
k=1, ... ,p. (2.2.11)
Since (/)k is the variance of the kth regression coefficient, the variance
proportions are
k,j=1, ... ,p. (2.2.12)
"ik is the proportion of the variance of lik attributable to the collinearity
58
..
indicated by AtVariance Inflation Factors (VIFs): The definitions of MixMod regression
diagnostics are based upon the following idea: (1) Extend the definition of a
diagnostic from GLUM( Y ; X I, crlN) to GLUM( Y ; X I, crY), where V is
known. (2) Use the definition in MixMod(Y; X/J, J: = UZ' +crl), acting as if
J: = crV. This paradigm is not successful for the variance inflation factor. The
VIF definition can be extended from GLUM( Y ; X I, crlN) to the weighted least
squares case, i.e., GLUM( Y ; X I, crY). (That VIF will be called the w.l.s.
VIF.) The interpretation in either model is: VIFj is the ratio of the variance of
A
Pi in the model with p regressors to the variance of the corresponding estimator
in a model with only one regressor, Xi' The attempt to use the VIF from
GLUM( Y ; X P, crY) in the MixMod fails for the following reason. V is known
in GLUM( Y ; X I, crY) and when one moves from a model with p regressors
to a model with only one regressor, the elements of V do not change.
However, in a typical mixed model, the relationships between the columns of
X and Z usually imply that when one removes columns from X (as in going from
a model with p regressors to a model with one regressor), one usually removes
corresponding columns from Z as well. Removing columns from Z changes the
structure of J: as well as its estimate. Even if one were to use the w.l.s. VIF
in the mixed model, the different structure and estimate of J: would prevent the
VIF statistic from having the same interpretation, Le., VIFj would not be theA
ratio of the variance of Pi in the model with p regressors to the variance of the
corresponding estimator in a model with only one regressor, Xi' One could, of
course, compute the mixed model VIFj directly by fitting two models, one with
59
p regressors and one with only the i-th regressor, and compute the VIFi as theA
ratio of the variances of "Pi" from the two models. Although this might be
interesting, it requires substantially more computation than in the GLUM and
one must decide if this diagnostic statistic is worth the additional work.
Belsley (1991) attaches much less importance to the VIF than to the
other diagnostic statistics. Because of this statistic's lesser importance and the
lack of an efficient algorithm for its computation, the characteristics of this
statistic will not be examined in the mixed model setting.
2.5 Methods of Computation
Computations for both the analysis of the mixed model and for the
collinearity diagnostics were carried out using the SAS IML procedure. Macros
for the mixed model analysis using the EM algorithm were developed previously
by Fairclough and Helms (1984) and are provided in Appendix 1. Macros for
computing the diagnostics were developed as part of this research and are
provided in Appendix 2. The diagnostics were computed after the final iteration
of the EM algorithm.
2.6 Employing the Diagnostic Procedure
The basic procedure, as described in Table 1.2 in Chapter 1 for the
GLUM, is paralleled for the mixed model diagnostics. Conceptually, the steps
A
are as follows. First the matrix r 1/2x is determined and its columns scaled to
equal length, i.e., W = (I-1I2X) Diag(X't.-1X)-1/2. Then the condition indices
and variance decomposition proportions are obtained as described in section
60
2.4. These diagnostics are examined in order to determine 1) the number of
near dependencies, 2) which variables are involved in the dependencies, 3)
which auxiliary regressions should be performed, and 4) which variables are
unaffected by the collinearity.
The usual manner of displaying the collinearity diagnostics is in a "row-
oriented format" (Belsley 1991, p 137). The scaled condition indexes and the
variance decomposition proportions are combined into a matrix. The ordered
Cis constitute the first column; each row corresponds to a possible near
cfependency. The VDPs constitute the remaining columns; each is associated
with a column of X or the variance of its parameter estimate. The row-oriented
format is shown in Table 2.1. The objective of the analysis is to examine the
structure and patterning in the rows of the matrix.
Table 2.1 Row-oriented Format for Presentation of Collinearity Diagnostics
Scaled Proportions ofCondition
Index X, X2 X4
V(b, ) V(b 2) ... V(b4 )
CI 1 "11 "12...
"1p
CI 2 "21 "22...
"2p
Clp "p' "p2...
"pp
Adapted from Belsley 1991, p 138
The steps in the diagnostic procedure as outlined in Figure 2.1 are
described in this section.
2.6.1 Examine Condition Indexes
First, the number of near dependencies and their relative strengths are
61
Figure 2. 1 Employing the Diagnostic Procedure
1. Examine Scaled Condition Indices
IAbsolutely
Small?(5-10)
II
No problemsSTOP
I II I
Moderate? Large?(30-100) > 100
I 11 1
II
2. Look for Gap in 10130 Progression of CIsI1 --:-
1 IGap of first kind Gap of second kindbetween small and between large Cislarge Cis separated by magnitudes(1, 3, 5, 50) along 10/30 progression
I 1I 1I Look for competing1 dependencies1 11 .,......- 1
I1
3. Examine the Variance Decomposition ProportionsII
Look for strongest near dependency firstII
Look for next strongest dependency next1I
Look for dominance and competitionII
4. Determine Involved Variables1I
5. Perform Auxiliary Regressions, if NecessaryII
6. Determine Uninvolved Variables
Source: abstracted from Belsley (1991), PP 134-142.
62
..
determined by the scaled condition indexes exceeding a chosen threshold such
as 30. The highest CI indicates the worst dependency in the data. If it is
"absolutely" small « 10), then one can stop the procedure. If, however, it is
moderate (30-100) or large (> 100), then further examination is necessary. If
it is "immense," (> 1000), then smaller indexes that would be considered
moderate or large when considered by themselves are no longer of interest.
2.6.2 Look for Gaps in "10/30 Progression" of Cis
Next, the condition indexes are examined for gaps in their progression.
These aid in determining the number of near dependencies. The relative
strengths of the indexes are determined by their position along the "progression
of 10/30" (Belsley 1991, P 136). This means a progression of 1, 3, 10, 30,
100,300,1000 and so on. Belsley (1991, p 140) defines a "gap of the first
kind" as a separation of a small CI from one that is large; he defines a "gap of
the second kind" as separations between several large Cis by several orders of
magnitude along the "10/30" progression. Gaps of the first kind indicate that
one dependency is present; gaps of the second kind indicate that several
dependencies are present. Belsley (1991, P 141) notes that picking the number
of near dependencies is an art form.
2.6.3 Examine the Variance Decomposition Proportions
Next, the VDPs that correspond to the dependencies indicated by the Cis
are examined. A variable is involved if its VDP associated with the high CI
exceeds a threshold, such as 0.50. If there is only one dependency present,
there will be only one high condition index. Then it is possible to determine the
63
variables involved directly from the variance decomposition proportions. If there
are several high condition indexes, determination of the variables involved in
each is made by aggregating the VDPs of each variable over the set of these
high Cis. Those variables whose aggregate proportions exceed the threshold
(0.50) are involved in at least one of the near dependencies and their
corresponding estimated coefficients are degraded. A variable is considered
involved in a dependency, and its corresponding regression coefficient degraded
by, at least one near dependency if the total proportion of its variance
associated with the set of high scaled condition indexes exceeds a chosen
threshold such as 0.50 (Belsley 1991, P 136).
2.6.4 Determine Involved Variables
A systematic examination begins with the strongest dependency (largest
Cl), usually in the last row of a display. One typically looks for values as large
as 0.8 or 0.9. The columns in which these values are found indicate the
variables that are definitely involved in the strongest near dependency. Then
the VDPs associated with the next largest scaled CI are examined (next to last
row of a display). Here the simultaneous involvement of variables between this
dependency and the strongest dependency is indicated. If this is true, the
VDPs are distributed across the two so that their~ is large even if no single
part is. One continues in this manner until the total collinear structure is
determined.
If there are several competing near dependencies (those with equal Cis),
the determination of the involvement of a variable in each is not always clear.
64
-,
Perform Auxiliary Regressions
The VDPs of the variables involved in at least one of the competing
dependencies can be capriciously distributed among them, confounding the
assessment. However, the VDPs can indicate which variables are involved in
at least one dependency, and thereby have degraded coefficients, even though
they cannot determine which variables are involved specifically in which
dependency (Belsley 1991, p 136).
A dominating near dependency occurs when a CI is larger along the
"10/30" progression than others that exist with it (Belsley 1991, p 132). This
near dependency can be the chief determinant of the variance of the coefficient
of a variable, obscuring the simultaneous involvement of that variable in other
weaker dependencies. In this case, there may not be two or more high VDPs
associated with the CI of the dominated (weaker) near dependency, but the
sum across the set of high Cis still will be greater than 0.50 (Belsley 1991, P
133).
2.6.5
As described for competing and dominating dependencies, if several near
dependencies are present, it is not always possible to determine which variables
are involved in which dependencies from an examination of the Cis and VDPs
alone. Though it is possible to determine which variables are involved in at
least one dependency, the nature of the involvement may be obscured due to
competing and dominating dependencies (Belsley 1991, p 144). In this
situation, the specific nature of the involvement can be clarified by performing
auxiliary regressions among the variables determined to be involved when the
65
Cis and VDPs were examined (See Belsley 1991, pp 144-147). These are
carried out by choosing Qilll variable known to be involved in u.d1 near
dependency. Then each of these variables is used as a "dependent variable"
in regressions involving the remaining variables as independent variables. The
t-tests of parameter estimates equal to zero can be used descriptively to show
the involvement of the variables. The way to choose "dependent variables" is
to start with the strongest near dependency (Cn and look for the largest VDPs.
If several are very large and have about the same value, pick the one that has
the remainder of its variance associated with more removed (smaller)
dependencies (Cis). This is done to avoid picking variables that may have VDPs
distorted by competing near dependencies (Belsley 1991, P 145). Continue in
this manner through each of the other near dependencies until a set of variables
has been picked. Then regress each of these variables separately on the
remaining variables and examine the results. When there are many
dependencies or several competing or dominating dependencies, the choice of
variables for the regressions may not be clear. Belsley (1991 , P 146) describes
extensions of this procedure for these situations.
..
2.6.6 Determine Uninvolved Variables
A variable is not involved in any near dependency if the total proportion
of its variance associated with the set of low scaled condition indexes exceeds
the threshold (0.50) (Belsley 1991, P 137).
66
..
2.7 Illustration of the Collinearity Diagnostics
The purpose of this section is to demonstrate the usage of the diagnostic
measures and procedure. This is accomplished by applying the GLUM
diagnostics and the MIXMOD diagnostics to appropriate subsets of the same
data set and comparing them.
2.7.1 Description of the Data
The data used in this example are from a subset of subjects in a study
of pulmonary function in children. The study has been described extensively
by Strope and Helms (1984) and by Fairclough and Helms (1984). A brief
description is provided here. Black and white children were selected prior to
birth to parents who were permanent residents of the Chapel Hill NC area.
Shortly after birth, the children were enrolled in the Frank Porter Graham Child
Development Center of the University of North Carolina at Chapel Hill. Their
respiratory illnesses and physiological development were studied as part of a
longitudinal investigation of social and cultural effects on development.
Pulmonary function testing began as early as two and a half years of age after
each child passed criteria for producing reliable results. Children were
scheduled to be studied at three month intervals; additional measurements of
pulmonary function were made during and one month following acute upper
and lower respiratory illnesses. The assessments made included (Fairclough
and Helms (1984):
FVC Forced Vital Capacity
the volume (liters) of gas expired after full inspiration, and with
67
expiration performed as rapidly and completely as possible;
FEV, Forced Expiration Volume (1 sec)
the volume (liters of gas that is exhaled in the first second during
the execution of a forced vital capacity;
PEF Peak Expiratory Flow (liters/sec)
V_&0'" Maximum Expiratory Flow (liters/sec)
measured when 50% of the FVC has been expired;
VlIla71'" Maximum Expiratory Flow (liters/sec)
measured when 75% of the FVC has been expired; and
FEF26.76'" Forced Expiratory Flow (liters/sec)
the mean flow rate during the middle half of the FVC.
In addition to measurements of height and weight, demographic variables (age,
race, sex) and presence of respiratory symptoms were recorded.
In this example, observations for a subject at a given assessment were
excluded for the following reasons:
1) age, weight, or height measurements were missing;
2) FVC measurement was missing;
3) Vmu:50% was less than Vmu:75%;
4) total time to complete the function test was greater than 4
seconds;
5) symptoms of lower respiratory illness were present.
After exclusions, the data file contained 85 children with 1207 pulmonary
function studies. Of the 66 black children, there were 34 females and 32
males; of the 19 white children, there were 10 females and 9 males.
68
Since the objective of this chapter is to illustrate the use of collinearity
diagnostics rather than perform a complete analysis of the data, only black
females were used for this purpose. Table 2.2 characterizes the data for black
females.
Table 2.2. Descriptive Statistics for 34 Black Female Children with 527 TotalPulmonary Function Studies
MEASURE MEAN S.D. MIN MAX
Number of Studies 16 8 1 30
Average Age (years)' 5.9 1.8 4.0 11.5Age at First Study 4.1 1.4 2.4 8.4Age at Last Study 8.3 3.0 4.0 15.6
Average Height (cm)' 116 14 99 151Height at First Study 103 12 88 131Height at Last Study 131 20 103 170
Average Weight (kg)' 25 11 15 63Weight at First Study 18 6 12 41Weight at Last Study 37 22 17 120
, Mean of individual means
2.7.2 GLUM Example and Diagnostics
For the GLUM analysis, the !M1 measurement of FVC is the response
variable and the three continuous variables, age, height, and weight at the last
measurement of FVC, are continuous regressors. The mean and standard
deviation of FVC at the last study for the 34 black females are 1.72 and 0.814,
respectively. The minimum and maximum values are 0.63 and 3.7,
respectively. Correlation among the regressors is given in Table 2.3 below.
The large correlation coefficients indicate that these three regressor variables
69
may be pairwise collinear. However, the diagnostics must be examined to
determine the degree of collinearity present and the extent of damage to
regression coefficients that might be caused by collinearity in these data.
Table 2.3 Correlation Coefficients of GLUMRegressors
AGEHEIGHTWEIGHT
AGE HEIGHT1.00 0.95
1.00
WEIGHT0.820.881.00
It is not necessary to fit a model first in order to assess collinearity in the
GLUM. However, for completeness and to compare with the MIXMOD results,
the following model is fit to these data:
y =X/J + e
where
(2.2.13)
measurement of FVC at the~ study for that subject;
design matrix of fixed effects: intercept and age, height, weight
at the last study;
vector of primary parameters;
vector of unobservable errors.
The results of fitting this model and the collinearity diagnostics are
presented in Table 2.4. Since the last measurement of FVC is the response
variable, the effects of age, height and weight are cross-sectional in nature.
A statistically significant impact on the prediction of FVC was found for both
age (p=0.015) and height (p=0.004), but not for weight (p=0.397). The
slopes for these two variables were positive, indicating that FVC increases with
70
Table 2.4 FVC: GLUM Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -2.126 0.614 -3.46 0.002Age 0.101 0.039 2.59 0.015Height 0.022 0.007 3.09 0.004Weight 0.003 0.003 0.86 0.397
Covariance Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
0.377 0.018 -0.0041 0.0010.002 -0.0002 9.608E-6
0.00005 -0.000010.00001
(X'X)
A
fil = 0.04534 281
26194465
38705
598989
124112074
17559761735
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.814 1.95 1 0.000 0.001 0.000 0.0040.160 0.40 5 0.008 0.001 0.001 0.2150.025 0.16 12 0.019 0.305 0.000 0.4980.001 0.03 62 0.973 0.693 0.999 0.283
Correlation Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight VIF
Intercept 1.00 0.74 -0.98 0.58Age 1.00 -0.84 0.07 10Height 1.00 -0.54 14Weight 1.00 4
71
increasing age and height.
Using the procedure described in section 2.6 above, collinearity in these
data is assessed. There is one clear linear dependency and it is moderately
strong. This is indicated both by the one moderately large condition index, 62,
and by a gap of the first kind between 62 and 12, the next largest CI. The
values of the elements in the last row of the VDP matrix indicate that the
variables involved in the dependency are the intercept, age, and height; each
exceeds the threshold value of 0.50.
Although the value of the second largest CI, 12, is relatively small, it
may indicate a second weaker dependency; there is a smaller gap in the
progression between the Cis of 12 and 5. This situation here may be that of
a dominated dependency as was described in section 2.6. The VDP for weight
in the row associated with the second largest CI, 12, is 0.498 (- 0.50). And,
the proportion of the variance of weight associated with the set of Cis (62 and
12) is 0.781. Thus, the coefficients of all three independent variables and
the intercept may be degraded.
The exact nature of the relationships among the variables can be
determined through auxiliary regressions. For this example, height has the
largest VDP in the last row; it is chosen as a "dependent" variable associated
with the first dependency. Weight is chosen as the primary variable involved
in the second dependency. Thus, the auxiliary regressions feature height and
weight separately regressed on the intercept and age. The results are displayed
in Table 2.5.
From these results, we verify that the dominant relationship does involve
72
Table 2.5 GLUM Auxiliary Regressions for Pulmonary Data
Coefficients ofDependentVariablesHeight
Weight
Intercept
80.72«0.001 )
-13.42(0.048)
Age
6.13«0.001 )
6.05«0.001 )
0.90
0.67
CI
62
12
Numbers in parentheses are p-values for t-statistics.
the intercept, age and height. The weaker involves at least age and weight.
Thus, we conclude that each variable is involved in one or both of these near
dependencies and the coefficient of each is degraded to some degree.
The primary focus in this section is to illustrate the detection of
collinearity and the identification of collinear variables, rather than the
resolution of collinearity. However, at this point, one or more of the techniques
described in Chapter 1 might be employed to deal with the collinearity found
in these data. Specifically, one or more of the variables might be eliminated
from the model. This suggestion is made in spite of the cautions made by
several investigators because the relationships among the variables in these
data are easily known and understa.ndable.
Several measures that may aid in determining which variable(s) to delete
are considered. The first is the correlation of each variable with FVC. The
correlations are 0.95 for age, 0.96 for height and 0.86 for weight. The second
measure is a plot of FVC against each variable. These are provided in Figures
2.2, 2.3 and 2.4 for age, height and weight, respectively. The correlation of
73
age with height is 0.95 and are redundant variables. The physical reality is that
FVC is logically more related to height than to age. Thus, the current model
could be reduced to one containing only height and weight as independent
variables. The reduced model would be fit and the diagnostic procedure
repeated.
Pulmonary Function Study....~------------------,
..
to
+..rvc-
,......
f '+ • • •.,to. +. ...
..••
t J ••• 7' '''''''11''''''.InYean
Figure 2.2 Values of FVC Plotted Against Age
74
Pulmonary Function Study
• •
... •++
rvCUi
'M••
••... + +* +.+.+ f'
+.+
+ •+ ~
•
• • ... 1M ,. ,. ,. 1. ,. '70
Height .... 11m
Figure 2.3 Values of FVC Plotted against Height
Pulmonary Function Study
• +
•••
rvCUi ...
•• +;,.t. ..... t
'M
II • • • • • ,. • • 'Ill Ii' 'JO
Weight In kg
Figure 2.4 Values of FVC Plotted Against Weight
75
2.7.3 Mixed Model Examples and Diagnostics
In the MIXMOD analysis, illl measurements of FVC for each black female
constitute the response vector and the three continuous variables, age, height,
and weight at every measurement of FVC, are continuous regressors. The
number of pulmonary studies for each subject are given in Table 2.6 below.
Table 2.6 Number of Pulmonary Studies for Each Subject
SUBJECT NUMBER SUBJECT NUMBEROF OF
STUDIES STUDIES113 1 94 15
99 2 68 1629 6 70 1644 7 64 18
106 7 69 18110 8 75 1892 9 40 1993 9 81 21
101 9 43 2298 10 59 2276 11 19 2589 11 21 2677 12 18 2783 12 32 2753 13 39 2771 13 52 2782 13 28 30
2.7.3.1 A Simple Mixed Model
Prior to fitting the mixed model counterpart of the GLUM example just
described, a simple mixed model was fit to these data. The purpose was to
demonstrate the mixed model analysis and to interpret results for a situation in
which there was no collinearity. In contrast to a GLUM model, the mixed
model accounts for the correlation among observations for each subject. In this
case, FVC is the dependent variable and age is the independent variable. The
76
following model was fit:
v = X IJ + Z d + 8.
The model equation for the kth subject is
Vk = XklJ + Zk dk + 8k, k= 1, 2, ..., 34,
(2.2.14)
(2.2.15)
where
Vk (1\ x 1) is a vector of the nk observations of FVC for the kth black female,
k = 1,2, ..., 34;
V (527 xl) = [V, 1/ V2 // ... 1/ VK] is the vector of responses from all 34 black
females;
Xk (n k x2) is a known fixed effects design matrix for the kth black female:
the intercept and values of age at every study;
X (527 x 2) = [X, // X2 // ... / / Xd is the fixed effects design matrix for the
model;
Zk (nk x 2) is a known random effects design matrix for the kth black female:
the intercept and values of age at every study;
Z (527 x 68) = Diag(Z" Z2' ..., ZK) is the random effects design matrix for the
model.
The following assumptions are made:
IJ (2 xl) is a vector of fixed effect primary parameters;
dk (2 xl) is a random vector of unobservable random subject effects for the
kth black female;
d (68 x ,) = [d, // d2 // ... // dK] is a vector of random subject effects for the
model;
is an vector of unobservable within-subject random error terms;
77
8 (527 x 11
and
= [8, II 82 /I ... /I 8K] is a vector of random error terms for the
model.
The following additional assumptions are made:
4 (2x21
- NIDq (0, 4);
= V(dk) is the covariance matrix of the random effects (each dk
has the same covariance matrix);
is an unknown scalar within-subject error variance parameter;
= V(8k) is the covariance matrix of the random deviations about
the kth subject's random regression line.
is the V(Yk) = I.t = Zk 4 Zk' + fi2vk, a positive definite symmetric
covariance matrix for the kth black female. Here, we assume that
~ (527 x 5271 = Diag(~" ~2' "., ~) is the covariance matrix of the entire
response vector, Y.
The results of fitting this model are presented in Table 2.7 and
graphically, in Figure 2.5. Here, the fixed effects have the same interpretation
A
as do those for the GLUM. FVC increases significantly with i'lcreasing age (P2
= +0.219, p<0.001). For the mixed model, the random effects also are of
interest. The random effects indicate deviations of each subject's regression
from the estimated population regression. Hence, a subject's first random
78
Table 2.7 A Simple Mixed Model: FVC as a Function of Age
Parameter Estimates of Fixed EffectsVARIABLEInterceptAge
BETA STD ERR T-0.153 0.056 -2.730.219 0.010 21.90
P-VALUE0.007
<0.001A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age
InterceptAge
0.0725 -0.00970.0020
Estimate of Correlation Matrix of the Random EffectsIntercept Age
InterceptAge
1.00 -0.811.00
A
02=0.020
Eigenvalue
1214
Collinearity DiagnosticsSingular Condition Variance
Value Index DecompositionProportions
600040054
1.8600.140
, .360.37
14
79
INT AGE0.070 0.0700.930 0.930
Pulmonary Function Study
2 3 4 5 • 7 a 9 10 11 12 13 14 15 ,.
Age In Years
Figure 2.5 Values of FVC Predicted from Mixed Model with Age in X and in Z
80
...
..
..
coefficient is that subjects's deviation from the estimated population intercept;
the subject's second random coefficient is that subject's deviation from the
population slope with respect to age. The covariance matrix of theseA.
deviations is estimated by 4. The correlation matrix of the random effects, theA. A.
scaled version of 4, is often easier to interpret than 4. Here, the correlation
between the two random effects (the intercept and age) is -0.81. This means
that, overall, subjects with lower random intercept increments tend to have
higher slope random slope increments, and vice versa. This tendency can be
seen in Figure 2.5. The magnitude of this coefficient indicates that age and the
intercept are terms worthy of being retained as "significant effects" in the
random component of the model.
In addition to the modeling results, the collinearity diagnostics are
presented in Table 2.7. The highest condition index is 4, which in the GLUM
is "absolutely small" and indicates a lack of collinearity. Assuming that the
same is true for the mixed model, we can conclude that there is no collinearity
present. Both the intercept and age have variance decomposition proportions
of 0.930 which in the presence of a large CI would indicate collinearity. In this
case, however, collinearity is not indicated.
2.7.3.2 GLUM for Longitudinal Data
..Prior to fitting the mixed model, an ordinary least squares regression
model (GLUM) was fit to these data, ignoring the correlation among
observations for each subject, i.e., there are 527 observations from 34
subjects. This was done in order to have rough estimates of fixed effects and
81
GLUM collinearity diagnostics to compare with their mixed model counterparts.
The dependent variable is FVC; the independent variables are age, height and
weight. The trends in these GLUM results, shown in Table 2.8, are similar to
those found for the GLUM fit at the last measurement of FVC (Table 2.4).
Now, however, age and weight appear to be important factors in predicting
FVC, while height appears not to be a significant factor. The slopes are still
positive, but steeper for age and weight than those found at the last
measurement of FVC. Moreover, because the observations are correlated, the
p-values in the table are not meaningful.
In this example that ignores the correlation among the observations, the
patterns in the collinearity diagnostics are quite similar to those found for the
last measurement of FVC. As before, there is one moderately strong linear
dependency and perhaps, a weaker dependency. There is one gap of the first
kind between the two largest Cis, 68 and 14; there is a smaller gap between
the Cis of 14 and 5. The last row of the VDP matrix shows that again the
intercept, age, and height are involved in the strongest dependency. The total
proportion of the variance of weight associated with the set of largest Cis is
now 0.887, suggesting that the dependencies may involve all variables in a
manner similar to that found for the GLUM analysis.
82
..
Table 2.8 FVC: GLUM Results and Collinearity Diagnostics for 527Observations on 34 Subjects
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.411 0.177 -2.32 0.021Age 0.152 0.014 11.07 <0.001Height 0.003 0.002 1.44 0.150Weight 0.012 0.001 7.89 <0.001
Covariance Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
0.031 0.002 -0.0004 0.00010.0002 -0.00003 8.4E-7
5.1E-6 -1.8E-62.2E-6
(X'X)
A
02 = 0.049527 3453
2650163820
445035
7927858
15298120527
2004029581948
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.830 1.96 1 0.000 0.001 0.000 0.0020.148 0.38 5 0.009 0.005 0.001 0.1110.021 0.14 14 0.009 0.286 0.000 0.6180.0008 0.03 68 0.982 0.708 0.999 0.269
Correlation Matrix of Parameter Estimates of Fixed EffectsIntercept Age Height Weight VIF
Intercept 1.00 0.78 -0.98 0.56Age 1.00 -0.84 0.04 15Height 1.00 -0.53 21Weight 1.00 6
83
2.7.3.3 Mixed Model with Multiple Independent Variables
(2.2.17)
After the preliminary analysis, a mixed model was fit to the pulmonary
function data, accounting for the correlation among observations for each
subject. The mixed model collinearity diagnostics were computed as described
in section 2.4. The following model was fit:
Y = X /l + Z d + e. (2.2.16)
The model equation for the kth subject is
Yk = Xk /l + Zk dk + ek , k =1, 2, ..., 34,
where
is a vector of the nk observations of FVC for the kth black female,
k = 1,2, ..., 34;
Y (527 x ') = [Y, II Y2 II ... II YK] is the vector of responses from all 34 black
females;
Xk (n.x4) is a known fixed effects design matrix for the kth black female:
the intercept and values of age, height, weight at every study;
X (527 x 4) = [X, II X2 II ... II XK] is the fixed effects design matrix for the
model;
Zk (nk x4) is a known random effects design matrix for the kth black female:
the intercept and values of age, height, weight at every study;
Z (527 x, 36) = Diag(Z" Z2, ..., ZK) is the random effects design matrix for the
model.
The following assumptions are made:
/l (4 x ') is a vector of fixed effect primary parameters;
84
..
dk (4 x 11 is a random vector of unobservable random subject effects for the
kth black female;
d (136 x 11 = [d, II d2 11·.. /1 dK] is a vector of random subject effects for the
model;
8 k (1'\ x 11 is an vector of unobservable within-subject random error terms;
and
8 (527 x 11 = [8, /I 82 II ... /I 8K] is a vector of random error terms for the
model.
The following additional assumptions are made:
dk (4x1) - NIDq (0, 4);
4 (4x4) = V(dk) is the covariance matrix of the random effects (each dk
has the same covariance matrix);
8 k (nk x 1) - NIDN (0, o2Vk ) independent of the dk; and for k' ¢ k,
Cov(dk·,dk) = 0; Cov(dk.,8k) = 0; and COV(8k.,8k) = 0;
02 is an unknown scalar within-subject error variance parameter;
Vk (nkxnk) = V(8k ) is the covariance matrix of the random deviations about
the kth subject's random regression line.
I k (nkxnk) is the V(Yk) = ~ = ~ 4 Zk' + o2Vk, a positive definite symmetric
covariance matrix for the kth black female. Here, we assume that
Vk=I...
I (527 x527) = Diag(I" 1 2, ... , I K) is the covariance matrix of the entire
response vector, Y.
The results of fitting this model are presented in Table 2.9. The fixed
effect results indicate that age and weight are significant predictors of FVC;
85
Table 2.9 FVC: MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUE
Intercept -0.127 0.359 -0.35 0.724Age 0.153 0.030 5.10 <0.001Height 0.00002 0.005 0.00 0.997Weight 0.016 0.004 4.00 <0.001
A-
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
2.3081 0.1801 -0.0313 0.00970.0154 -0.0024 0.0004
0.0004 -0.00010.0002
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.96 -0.99 0.491.00 -0.95 0.27
1.00 -0.541.00
A-
02=0.0193903 16799
96970408783
1928606
44029265
72718395852
82161441712904
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.795 1.95 1 0.000 0.001 0.000 0.0020.176 0.42 5 0.005 0.020 0.000 0.0340.027 0.17 12 0.000 0.169 0.000 0.7550.006 0.02 78 0.995 0.810 0.999 0.209
86
height is not significant. The slopes of both are positive indicating that FVC
increases with increasing age and weight. These results are presented
graphically in Figures 2.6, 2.7 and 2.8 for age, height, and weight,
respectively. The population regression line was obtained using the fixed
A
effects (/1) and minimum and maximum values (over all studies) of age, height
and weight. The predicted values of FVC for each subject were obtained using
A
the fixed effects (/1) and the random effects (dk); the values of FVC for each
independent variable were computed at the mean (over all studies) of the other
two variables.
Portions of the correlation matrix of the random effects, and the Figures,
aid in interpreting the results. In contrast to the results for the simple model
containing only age in X and in Z, the correlation between the intercept and age
in this model is high, but positive (0.96). This indicates that subjects with
positive intercept increments also have higher than average random slopes and
vice versa. The correlation between the intercept and height is high and
negative (-0.99), indicating that subjects with negative intercept increments
tend to have higher than average random slopes and vice versa. The
correlation between the intercept and weight is positive (0.49), but not as
strong as the correlations for the intercept and age and the intercept and
height.
The other components of the correlation matrix of the random effects are
noteworthy. The correlation of age with height (-0.95) indicates that positive
deviations of the slopes for age tend to be associated with very negative
deviations of the slopes for height. However, positive deviations of the slopes
87
Pulmonary Function study5r-----------------------~
2 3 4 5 I 7 a I 10 11 11 13 14 15 1&
Age In Years
Figure 2.6 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted Against Age
88
Pulmonary Function Study
10 100 110 1%0 130 140 150 180 170
Height in em
Figure 2.7 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted against Height
89
Pulmonary Function Study
10 20 30 40 50 eo 70 80 80 100 110 120
Weight in kg
Figure 2.8 Values of FVC Predicted from Mixed Model with Age, Heightand Weight in X and in Z, Plotted Against Weight
90
..
..
for age tend to be associated with very little change in the random deviations
of the slopes for weight (0.27). The deviations of the slopes of height and
weight tend to vary inversely and moderately (-0.54). It is suspected that the
collinearity in these data may be affecting the estimation of 4. This will be
explored later.
The collinearity diagnostics for this model also are presented in Table
2.9. There is one clear near dependency and it is moderately strong, as
indicated by the one moderately large condition index, 78, and the gap of the
first kind between 78 and 12, the next largest CI. The values in the last row
of the VDP matrix indicate that the variables involved in the dependency are the
intercept, age, and height; each exceeds the threshold of 0.50.
There is also indication of a second weaker dependency. There is a gap
in the progression of Cis between 12 and 5. In addition, the total proportion
of the variance of weight associated with the set of Cis (78 and 12) is 0.964.
Thus the fixed effect coefficients of all three independent variables and the
intercept may be degraded. These results are similar to those found for the
GLUMs fit previously for last measurement of FVC (Table 2.4) and for all
measurements of FVC (Table 2.6). Thus, for analogous GLUM and MIXMOD
analyses, the patterning in the diagnostics is similar and the diagnostics appear
to function comparably.
In order to determine the exact nature of the relationships among these
variables, a mixed model counterpart of the auxiliary regressions could be
carried out. However, at this point the type of weighting that should be used
is not clear, Le., whether the regression should be computed from X or from
91
2.8 Factors Impacting Collinearity in the Mixed Model
As mentioned in section 2.4, collinearity in the mixed model is affected
by several factors other than collinear variables in the X matrix. The objective
of this section is to get a glimpse of what those factors might be so that they
can be addressed in Chapter 3. Both the GLUM and the mixed model involve
assessment of collinearity in the X matrix, even though the structure of these
matrices in the two models differs. However, for the mixed model, the
additional matrix I is involved in the assessment. Thus, the factors that might
impact collinearity in the mixed model all involve the structure and modeling of
I. The following components of the estimation of I are thought to be
important:
1) the number of variables in Z,
2) the presence and nature of the collinearity in Z,
3) the structure of A,
4) the structure of V, and
5) the value of cr.
In addition to these factors, it is thought that collinearity may have a different
impact for different response variables, even if they retain the same fixed and
random effects design matrices. This is due to the fact that I, the variance of
the response vector V, will be different for models with different responses.
This is in contrast to the GLUM in which collinearity for a given X matrix is the
92
..
same regardless of the response variable Y.
Three of the issues thought to impact collinearity in the mixed model are
explored in this section:
1) the number of variables in Z, of those thought to be collinear in X,
2) the structure of 4, and
3) the effect of a different response for the same X and Z matrices.
2.8.1 Number and Nature of Variables in Z
To explore the impact of the Z matrix on collinearity, the model described
in section 2.7.3.3 was refit with different subsets of variables in the Z matrix.
The results relative to the original model (See results in Table 2.9.) are reported
in 'this section.
2.8.1.1 Two variables, Pair-wise collinear
Three models were fit in which the Z matrix contained only two of the
three variables contained in the X matrix. In turn, each of the variables age,
height, and weight, were deleted from Z and the model was refit. The results
of fitting these mixed models are summarized in Table 2.10 (height and weight
in Z), Table 2.11 (age and weight in Z) and Table 2.12 (age and height in Z).
2.8.1.2 One Variable
Three models were fit in which the Z matrix contained only one of the
three variables contained in the X matrix. In turn, pairs of variables (height and
weight, age and weight, age and height) were deleted from Z and the model
was refit. The results of fitting these mixed models are summarized in Table
2.13 (age in Z), Table 2.14 (height in Z) and Table 2.15 (weight in Z).
93
2.8.1.3 Summary
The results of fitting the models with altered Z matrices are summarized
in Table 2.16. When only one variable was deleted from Z, the most dramatic
A
changes in fixed effects and in 4 occurred when age or height was deleted;
there was no substantial change when weight was deleted. When age was
deleted from Z, the diagnostics indicated that age was less involved in the
dependencies. The greatest change in the diagnostics occurred when height
was deleted from Z; the largest CI was reduced from 78 to 38 and age and
weight were less involved in the fixed effect dependencies.
In each model in which two variables were deleted from Z, the fixed
effect for height changed; the slope was negative and the p-value was smaller.
When the pairs (height and weight) and (height and age) were deleted from Z,
the magnitude of the correlation of the intercept and the remaining random
effect changed in magnitude and sign. In all cases of deleting two variables,
the Cis were reduced to 33-38 and age and weight were less involved in the
dependencies.
94
•
Table 2.10 FVC (Z = Int,Ht,Wt): MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.321 0.256 1.25 0.210Age 0.198 0.019 10.42 <0.001Height -0.006 0.004 -1.50 0.134Weight 0.017 0.004 4.25 <0.001
A
Ii, Estimate of Covariance Matrix of the Random EffectsIntercept Height Weight
InterceptHeightWeight
0.3123 -0.0050 0.00850.0001 -0.0001
0.0003
Estimate of Correlation Matrix of the Random EffectsIntercept Height Weight
InterceptHeightWeight
1.00 -0.99 -0.921.00 -0.96
1.00
A
02=0.0203076 13065
84756321369
1560357
35015410
61861365185
72165101692195
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.728 1.93 1 0.000 0.002 0.000 0.0020.239 0.49 4 0.009 0.045 0.001 0.0180.031 0.17 11 0.001 0.414 0.000 0.5990.002 0.04 50 0.989 0.539 0.999 0.380
95
Table 2.11 FVC (Z = Int,Age,Wt): MIXMOD Results and CollinearityDiagnostics
Parameter Estimates of Fixed Effects
VARIABLE BETA STD ERR T P-VALUEIntercept 0.370 0.257 1.44 0.151Age 0.198 0.022 9.00 <0.001Height -0.007 0.003 -2.33 0.020Weight 0.017 0.004 4.25 <0.001
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight
InterceptAgeWeight
0.0309 0.0021 -0.00110.0014 -0.0004
0.0001
Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight
InterceptAgeWeight
1.00 0.31 -0.581.00 -0.89
1.00
A
a2=0.0201781 9133
72504199376
1192671
23552068
39136288969
49968261244657
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.705 1.92 1 0.001 0.002 0.000 0.0040.255 0.51 4 0.016 0.030 0.001 0.0410.038 0.19 10 0.000 0.308 0.002 0.8710.003 0.05 38 0.984 0.660 0.997 0.084
96
Table 2.12 FVC (Z = Int.Age.Ht): MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.179 0.339 -0.53 0.598Age 0.158 0.035 4.51 <0.001Height 0.0007 0.005 0.14 0.889Weight 0.014 0.002 7.00 <0.001
A-
4. Estimate of Covariance Matrix of the Random EffectsIntercept Age Height
InterceptAgeHeight
2.0758 0.2171 -0.02890.0250 -0.0031
0.0004
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height
InterceptAgeHeight
1.00 0.95 -1.001.00 -0.97
1.00
A
02=0.0204368 17613
84832447863
1907353
46714919
69519340746
75841751582084
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.758 1.94 1 0.000 0.001 0.000 0.0080.194 0.44 4 0.003 0.002 0.001 0.3690.047 0.22 9 0.005 0.133 0.000 0.6100.0006 0.03 77 0.991 0.865 0.999 0.013
97
Table 2.13 FVC (Z=lnt,Age): MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.016 0.262 0.06 0.951Age 0.186 0.023 8.09 <0.001Height -0.002 0.003 -0.67 0.505Weight 0.011 0.003 3.67 <0.001
A.
Ii, Estimate of Covariance Matrix of the Random EffectsIntercept Age
InterceptAge
0.0634 -0.00840.0016
Estimate of Correlation Matrix of the Random EffectsIntercept Age
InterceptAge
1.00 -0.841.00
A.
a2=0.0201604 8395
56393180034
1026087
20870058
33564228471
41652561096723
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.703 1.92 1 0.001 0.002 0.000 0.0090.233 0.48 4 0.014 0.007 0.002 0.2960.062 0.25 8 0.014 0.336 0.000 0.6030.003 0.05 36 0.971 0.655 0.998 0.092
98
Table 2.14 FVC (Z =Int.Ht): MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.173 0.277 0.62 0.533Age 0.204 0.021 9.67 <0.001Height -0.004 0.003 -1.33 0.183Weight 0.011 0.002 5.50 <0.001
A
A. Estimate of Covariance Matrix of the Random EffectsIntercept Height
InterceptHeight
0.4464 -0.00370.00003
Estimate of Correlation Matrix of the Random EffectsIntercept Height
InterceptHeight
1.00 -0.981.00
A
02=0.0201352 7268
54040152949910413
17918891
28388212641
36002141011044
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.643 1.91 1 0.001 0.003 0.000 0.0100.280 0.53 4 0.013 0.011 0.002 0.2600.074 0.27 7 0.009 0.345 0.000 0.6680.003 0.06 33 0.978 0.640 0.998 0.062
99
Table 2.15 FVC (Z=lnt,Wt): MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.303 0.253 1.20 0.232Age 0.191 0.020 9.55 <0.001Height -0.006 0.003 -2.00 0.046Weight 0.017 0.003 5.67 <0.001
A.
4, Estimate of Covariance Matrix of the Random EffectsIntercept Weight
InterceptWeight
0.0417 -0.00100.00004
Estimate of Correlation Matrix of the Random EffectsIntercept Weight
InterceptWeight
1.00 -0.781.00
A.
a%=0.0201741 9674
78005198969
1261025
23845622
38560288096
49222081183372
..
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.729 1.93 1 0.001 0.002 0.000 0.0050.217 0.47 4 0.020 0.038 0.001 0.0650.052 0.23 8 0.000 0.255 0.001 0.8040.003 0.05 38 0.979 0.705 0.998 0.126
100
Table 2.16 Summary of Results of Altering Z Matrix by Deleting Variables fromFVC Mixed Model· With All Three Variables in Z
Variables Impact onAlteration in Z Rxed Impa,t on Impact on
of Z (Table #) Effects 4 DiagnosticsIntercept Change in Cis: largest is 50, others are
Height and height magnitude the sameWeight have smaller and sign for
p-values, (int,wt); VDPs: age is less involved in(2.10) slope of change in dominant dependency
height is magnitudenegative for (ht,wt)Height is Dramatic Cis: largest is 38, others are
One Age significant change in the sameVariable Weight and slope is patterns, allDeleted negative correlations VDPs: age and weight are less
I(2.11 ) change in strongly involved in the
sign andlor dependenciesmagnitude
No change No change Cis: almost no changeAge in patterns in patterns
Height VDPs: age is more stronglyinvolved in the
(2.12) dominant dependency;weight is less stronglyinvolved in thedependencies
No change Magnitude Cis: largest is 36, others areAge in patterns, of (int,age) slightly smaller
but height is similar,(2.13) has smaller but sign is VDPs: age and weight are less
p-value and reversed strongly involved in theslope is dependenciesnegativeNo change No change Cis: largest is 33
Two Height in patterns, inVariables but height magnitude VDPs: age and weight are lessDeleted (2.14) has smaller or sign strongly involved in the
p-value and dependenciesslope isnegativeIntercept Magnitude Cis: largest is 38
Weight has smaller and sign ofp-value; (int,wt) VDPs: age is slightly less
(2.15) height is have strongly involved in thesignificant changed dependencyand slope isnegative
• Results from original model are in Table 2.9.
101
2.8.2
2.8.2.1
Structure of A
Constrained, One Off Diagonal Element Equal to Zero
To determine the effect of the matrix 4 on collinearity, one off diagonal
element of 4 was set to zero. The element was in the 2,4 [cov(age,weight)]
position. The results of fitting the original model, described in section 2.7.3,
with this change in 4 are reported in Table 2.17.
2.8.2.2 Constrained, All Off Diagonal Elements Equal to Zero
To further determine the effect of the matrix 4 on collinearity, all off
diagonal elements of 4 were set to zero. The results of fitting the original
model, described in section 2.7.3, with this change in 4 are reported in Table
2.18.
2.8.2.3 Summary ..A
The results of fitting the models with constrained 4 matrices areA
summarized in Table 2.19. When only one off diagonal element of 4 was set
to zero, the fixed effect for height changed, relative to the original results, from
highly non-significant to significant and the slope was negative. The patterns
A
among the remaining elements of 4 were similar to the original model, but
smaller in magnitude. A similar change in the fixed effects occurred when all
A A
diagonal elements of4 were set to zero. In both cases of constraining 4, the
diagnostics change dramatically. The largest CI was reduced from 78 to 23-24
and age and weight were less involved in the dependencies.
102
Table 2.17 FVC(One element 4 = 0): MIXMOD Results and CollinearityDiagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUE
Intercept 1.026 0.812 1.26 0.207Age 0.266 0.083 3.20 0.001Height -0.020 0.011 -1.82 0.070Weight 0.030 0.014 2.14 0.033
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
12.8782 1.1571 -0.1758 0.03300.1366 -0.0156 0.0000
0.0024 -0.00070.0036
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.87 -0.99 0.151.00 -0.86 0.00
1.00 -0.241.00
A
02=0.020140 19
89010513
7578
840237
8031922
7423813557
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT2.661 1.63 1 0.000 0.005 0.001 0.0391.073 1.04 2 0.002 0.101 0.001 0.0260.261 0.51 3 0.002 0.141 0.003 0.9090.005 0.07 23 0.995 0.752 0.995 0.025
103
Table 2.18 FVC(AII off diag 4 = 0): MIXMOD Results and CollinearityDiagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.433 0.259 1.67 0.095Age 0.214 0.023 9.30 <0.001Height -0.007 0.003 -2.33 0.020Weight 0.014 0.003 4.67 <0.001
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
0.0206 0.0000 0.0000 0.00000.0001 0.0000 0.0000
3.7E-7 0.00003.0E-5
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.00 0.00 0.001.00 0.00 0.00
1.00 0.001.00
A
a2=0.020945 2945
3374392358
446145
10139655
10896115121
1605670518648
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.154 1.78 1 0.001 0.005 0.001 0.0180.709 0.84 2 0.009 0.016 0.001 0.0890.132 0.36 5 0.002 0.203 0.001 0.8770.005 0.07 24 0.987 0.776 0.997 0.016
104
..
Table 2.19 Summary of Results of Altering Z MaVix by Constraining ACompared to FVC Mixed Model- With Unconstrained A
Elements Impact on5etto Impact on Remaining
Constraint Zero Fixed ElemlV'ts of Impact ononA (Table ') Effects A Diagnostics
Height Similar2,4 changes pattern, but Cis: dramatically reduced,
from highly smaller largest is 23(2.17) non- values for all
significant VDPs: age is slightly lessto highly involved in the
Off- significant dependencydiagonal and slope isElements negativeSet to InterceptZero All approaches Not Cis: dramatically reduced,
significance; applicable largest is 24(2.18) height (all set to
changes zero) VDPs: age and weight arefrom highly slightly less stronglynon- involved in thesignificant dependenciestosignificantand slope isnegative
• Results from original model are in Table 2.9.
2.8.3 Different Response, Same Fixed and Random Effects
In order to obtain an indication of what the impact of a different
response might be, a second mixed model was fit using the same independent
variables, in both X and Z, with a different dependent variable. For this model,
the response VMAX50% was fit using the same regressors, age, height and
weight, as were used for fitting the model for the response FVC. The
specifications are the same as those given in section 2.7.3, with VMAX50%
substituted for FVC in the description. The results of fitting this model are
presented in Table 2.20.
These results indicate that even though the same X and Z matrices were
105
used in this model as were used to model FVC, the collinearity diagnostics are
somewhat different. The general patterns found previously are also found in
this analysis. However, now the largest condition index is 38 and the next
largest is 7. This gap of the first kind indicates that one moderately strong
dependency is present, since 7 is -absolutely small. - The intercept, age, and
height are involved in the dependency. A second weaker dependency involving
weight may exist, as was seen for the original model.
106
..
Table 2.20 VM~o",: MIXMOD Results and Collinearity Diagnostics
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 0.429 0.725 0.59 0.553Age 0.075 0.068 1.10 0.271Height 0.005 0.009 0.56 0.579 .Weight 0.022 0.010 2.20 0.028
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.8769 0.1623 -0.0221 -0.01800.0221 -0.0018 -0.0032
0.0003 0.00020.0006
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.80 -0.99 -0.521.00 -0.74 -0.84
1.00 0.441.00
A
02=0.165282 874
563427466
103742
2805154
387922073
44263798908
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.505 1.87 1 0.000 0.003 0.000 0.0080.428 0.65 3 0.006 0.025 0.001 0.0510.065 0.25 7 0.000 0.253 0.001 0.9400.002 0.05 38 0.993 0.719 0.997 0.001
107
1)
2.9 Summary and Implications of Results
In this chapter, collinearity diagnostic measures for the mixed model have
been proposed. Computation of the measures and application of the diagnostic
procedure, developed by Belsley (1991) for the GLUM, have been extended to
the mixed model and demonstrated for longitudinal data with collinear variables.
The purpose of this research was to obtain initial impressions of the behavior
of the collinearity diagnostics in the mixed model situation, with a view toward
refining the subsequent research.
2.9.1 Initial Conclusions
Several impressions have emerged from these analyses:
GLUM results and MIXMOD results for similar datil appear to be
comparable. Nearly identical results were found for the GLUM at the~
pulmonary study (n =34, Table 2.4) with age, height, and weight as
independent variables and the ordinary least squares model for the data
for .all visits (n = 527, Table 2.8) with age, height, and weight as
independent variables. The results indicated one strong near dependency
and a possible weaker near dependency; all variables appeared to be
involved. The mixed model results (n =527, Table 2.9), using age,
height, and weight as both fixed and random effects, were similar,
though a slightly greater degree of collinearity was indicated. Thus, we
believe that the diagnostics operate similarly for the GLUM and the
mixed model.
108
.,
2) Deletion of collinear variables from Z affects the degree of collinearity
present. More dramatic changes occurred in the condition indexes with
the deletion of two variables from Z than with the deletion of one
variable from Z. In each case, however the largest CI was smaller (38-
77) relative to the largest CI for the original model (78). The least
reduction in the largest CI (from 78 to 77) occurred when weight was
removed from Z. Weight not involved in the dominant dependency.
Thus, it appears that removing such a variable from Z has little impact
on the diagnostics. Contrastingly, when those variables (age and height)
more strongly involved in the dominant dependency were deleted from
Z, a dramatic reduction in the Cis was found (from 78 to 33-50). This
seems remarkable since all collinear variables were retained for the fixed
effects. Thus, we believe that deleting collinear variables from Z, while
retaining them in X, affects the degree of collinearity for the model.A.
3) The structure of 4 affects the degree of collinearity present. The issueA.
of constraining 4 and that of deleting variables from Z are related.
Usually, when the components of the covariance or the correlation of the
random effects are small, the corresponding variables are removed from
Z. However, for these examples, the variables were not removed.
Nevertheless, a dramatic reduction in the Cis (from 78 to 23-24) wasA. A.
found when one diagonal element of 4 or all off-diagonal elements of 4A.
were set to zero. Thus, we believe that smaller elements of 4, in the
presence of all collinear variables in both X and Z, affect the degree of
collinearity for the model.
109
4) Different dependent variables affect the degree of collinearity present.
When a mixed model with a different dependent variable, VMAX50 'l6' was
fit, a striking reduction in the largest CI (from 78 to 38) was found. The
significance of this is that the same variables (age, height, and weight)
that were used to model FVC as were used in X and Z in the VMAX50'l6
A.
model. This finding is not unexpected, since the variance of V, I, is
involved in the computation of the diagnostics. However, this condition
is unique to the mixed model. In the GLUM, the degree of collinearity
present for a given X is the same regardless of the dependent variable
in the model. Thus, we believe that a different dependent variable, with
the same collinear variables in both X and Z, impacts the degree of
collinearity for the model.
2.9.2 Implications for Subsequent Research
Initial impressions as to the types of factors impacting the diagnostics
have been reported in this chapter based on results from one data set with
three collinear variables. This has been useful as a point of departure in
designing the subsequent research. However, in order to fully characterize the
behavior of the diagnostics, an experimental approach is required. The
collinearity-impacting factors identified in this chapter, along with several
others, will be subjected to more rigorous scrutiny using simulated data. In this
way, the factors will be artifically controlled in known ways and the results will
be more definitively stated.
110
CHAPTER III
APPLICATION OF DIAGNOSTICS TO EXPERIMENTAL DATA
3. 1 Introduction
Because collinearity is data dependent, behavior of the diagnostics must
be determined empirically. Therefore in order to describe their behavior more
comprehensively, the diagnostics must be applied to data whose level of
collinearity has been predetermined. In addition, a range of severity of
collinearity must be examined. Then under known circumstances some
implications of collinearity can be determined and described, including the
viability of the diagnostics developed and the effects on the variances of
parameter estimates.
In this chapter, the diagnostics defined and used in Chapter 2 are applied
to contrived data. Several types of dependencies of varying levels are created.
Then the behavior of the diagnostics is assessed for carefully selected sets of
design matrices and parameter values. Initially, diagnostics are applied in the
context of the GLUM. The GLUM results then provide a background for
examining the diagnostics applied to the mixed model. For the GLUM, only an
X matrix (and X'X) are needed in order to evaluate the dependencies. For the
mixed model, however, the process has additional levels of complexity, arising
from the covariance structure and random effects design matrix.
In this chapter, experiments for the GLUM and the mixed model are
described. First, for the GLUM, the types of artificial dependencies in X and
the procedures for creating them are specified. Then, the results of applying
the collinearity diagnostics to the simulated X'X matrices for the GLUM are
portrayed. Next, for the mixed model, the procedures for creating the artificial
dependencies in X and Z are described. Then the procedures used to obtain the
matrices and parameters needed for the mixed model experiment are described.
Finally, the results of applying the collinearity diagnostics to the simulated
matrix (X'E"X) for the mixed model are summarized. Based on the results of
these experiments, some general guidelines for the behavior of the diagnostics
under the conditions examined are presented.
3.2 The GLUM Experiment
The procedure for using the collinearity diagnostics, described in
Chapters 1 and 2, was based on a series of experiments in which the behavior
of the diagnostics was determined empirically. The six sets of experiments are
described by Belsley (1991, Chapter 4) in which different types of near
dependencies with different types of variables are explored in the GLUM
context. Each experiment began with a "basic" data set of n = 24 to 30
observations on p = 3 to 5 variables. From the basic data sets, additional
collinear data series with increasingly tighter linear near dependencies with the
basic series were constructed (Belsley 1991, P 79). His basic data consisted
112
of actual economic time series or random series.
In this section, one of Belsley's experimental series was emulated.
However, in contrast to Belsley's experiments, the basic data used here were
totally random and an intercept term was added to the set of basic variables.
Data were chosen for the basic variables such that they represented three
continuous variables from a typical data set. Since the basic data were chosen
randomly, they were well-conditioned. Thus, the ill-conditioning assessed was
totally attributable to the contrived dependencies that augmented the basic
variables.
Dependencies for the experiment were created as described below. Then
the GLUM diagnostics were applied to them. The results, even with the
intercept term included, are similar to those obtained previously by Belsley and
are a basis of comparison for the subsequent mixed model experiment.
3.2.1 The Basic Data
The basic data set used in the experiment is
X == [lNT, aX1, aX2, aX3], (3.2.1 )
where INT is the intercept term and aX1, aX2, and aX3 are each generated
from a uniform integer distribution, on the interval 0-1. Sixty values of each
variable were generated. The range of potential values for aX1 and aX2 is
from 0 to 10; the range for aX3 is from 0 to 100.
3.2.2 The Dependency Sets
From the basic data, two dependency sets were created. In the first,
Wi = aX3 + Si,
113
i=O, ..., 4, (3.2.2)
j =0, ..., 4, (3.2.3)
with 8 j generated from a normal distribution with mean zero and variance
02 =10-j x [var(BX3)1]. In the second,
~ = 0.SBX1 + 0.2BX2 + 8j'
with 8 j generated from a normal distribution with mean zero and variance
02= 10-j x [var(0.SBX1 + 0.2BX2)1].
3.2.3 The Data Series
The ~ and ~ dependency sets were used to augment the basic data set
to produce three series of matrices:
X1{i} • [X Wi]
X2{j} • [X Zj]
X3{i,j} • [X Wi Zj]
i =0, ..., 4,
j =0, ..., 4,
i,j =0, ..., 4.
(3.2.4)
(3.2.5)
(3.2.6)
For each series, collinearity diagnostics were computed.
3.2.4 The Issues Addressed
The dependency created by the Wj is a simple relationship between two
variables that might be detectable through examination of a correlation matrix.
The dependency created by the Zj is a simple relationship among three
variables, but it is not easily detectable through examination of a correlation
matrix. These are sets of coexisting dependencies, Le., they are non
overlapping. As the variance of the error term becomes smaller and smaller
with increasing i and j, the dependency becomes tighter and tighter.
3.2.5 Results
All of the diagnostics for the GLUM experiment are presented in
Appendix 1. The primary findings for the three data series are summarized
114
here.
3.2.5.1 Simple Dependency: Two Variables
•
For the X1{i} series, results are shown in Table 3.1. The variable in the
fifth column (labeled WO-W4) is related to the variable in the fourth column
(BX3) by (3.2.2). This is the only contrived dependency in this series. As
expected, there is one high condition index and a large proportion of the
variance of BX3 and the variances of WO-W4 are associated with it. As the
dependency between the two variables increases (from Set 1 to Set 5, i.e.,
from X1 {O} to X1 {4}), the pattern of the relationship becomes clearer. The one
condition index indicating the dependency becomes larger and the variance
decomposition proportions of the two variables becomes larger.
3.2.5.2 Simple Dependency: Three Variables
For the X2{j} series, results are shown in Table 3.2. The variable in the
fifth column (labeled ZO-Z4) is related to the variables in the columns 2 and 3
(BX1 and BX2) by (3.2.3). This is the only contrived dependency in this series.
As expected, there is one high condition index and a large proportion of the
variances of variables involved in the dependency are associated with it. As
the dependency between the three variables increases (from Set 6 to Set 10,
i.e., from X2{0} to X2{4}), the pattern of the relationship becomes clearer.
The one condition index indicating the dependency becomes larger and the
variance decomposition proportions of the two variables becomes larger. For
X2{0), the dependency is not yet apparent; for X2{1), the involvement of BX1
and Z1 is obvious. Thereafter, the dependency is clearly detectable.
115
3.2.5.3 Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)
The X3{i,j} series contains a combination of the two dependencies in
X1{i} and X2{j}; these are coexisting dependencies. Twenty-five sets of
matrices result from this series; selected results are shown in Table 3.3 for;= 2
and j=O, ..., 4. The dependency in (3.2.2) is fixed and strong while the
dependency in (3.2.3) varies. There are two high condition indexes indicating
the two dependencies, though for j = 0 the second dependency is dominated by
the first and is not yet apparent. The effects become separable when j = 1.
Thus the procedure can clearly detect two dependencies and the variables
involved in each. When the two dependencies are of nearly the same strength,
their effects become confounded. For X3{2,2}, the condition indexes are of
similar magnitude and the involvement of variables in the two dependencies is
somewhat obscured. However, it is possible to determine that there are two
dependencies present and that five variables are involved in them. As the
second dependency (BX1, BX2, ~) becomes strong relative to the first
dependency (BX3, W2), the effects are again separable.
The X3{i,j} series can be examined from the other direction by holding
j constant and varying;. Selected results are shown in Table 3.4 for ;=0, ...,
4 and j= 2. The dependency in 3.2.2 varies while the dependency in 3.2.3 is
held constant. For ;=0, the dependency involving BX3 and WO is relatively
weak; the one high condition index and the variance decomposition proportions
indicate the strong dependency in BX1, BX2, and Z2. When;= 1, the two
dependencies are distin~t. When;= 2, the two dependencies are of similar
116
strength and the variables involved in each are somewhat obscured. However,
when i =3, their separate identities emerge again and remain distinct for i =4.
Again, the procedure can clearly detect two dependencies and the variables
involved in each.
117
Table 3.1 The GLUM Experiment, Series X1{i}, ;=0, ..0' 4 (One Contrived NearDependency): Condition Indexes and Variance-Decomposition Proportions
118
Table 3.2 The GLUM Experiment, Series X2{j},j=O, ..., 4 (One Contrived NearDependency): Condition Indexes and Variance-Decomposition Proportions
D{O} Set 6: ZO
CI variance Proportions for Coefficients ofIHT aX1 aX2 aX3 ZO
1 0.005 0.006 0.011 0.010 0.0084 0.009 0.066 0.001 0.426 0.1564 0.003 0.030 0.905 0.100 0.0417 0.159 0.445 0.019 0.151 0.7448 0.824 0.453 0.063 0.313 0.051
D{l} Set 7: Zl
CI VarIance ProportIons for Coefficients ofINT aX1 aX2 aX3 Zl
1 0.005 0.001 0.007 0.009 0.0014 0.012 0.015 0.026 0.521 0.0075 0.002 0.027 0.523 0.006 0.0037 0.958 0.008 0.024 0.418 0.012
21 0.023 0.949 0.421 0.046 0.977
%2{2} Set 8: Z2
ofZ2
0.0000.0010.0000.0010.998
CI
1457
69
variance Proportions forINT aX1 aX2
0.005 0.000 0.0010.012 0.001 0.0030.003 0.002 0.0950.977 0.001 0.0050.003 0.995 0.896
CoefficientsaX3
0.0090.5120.0110.4570.010
D{3} Set 9: Z3
CI
1457
251
CI
1457
723
variance Proportions for Coefficients ofINT aX1 aX2 aX3 Z3
0.005 0.000 0.000 0.009 0.0000.012 0.000 0.000 0.503 0.0000.003 0.000 0.008 0.011 0.0000.978 0.000 0.000 0.443 0.0000.003 1.000 0.991 0.035 1.000
%2{4} Set 10: Z4
variance Proportions for coefficients ofINT aX1 aX2 aX3 Z4
0.005 0.000 0.000 0.009 0.0000.012 0.000 0.000 0.499 0.0000.003 0.000 0.001 0.011 0.0000.980 0.000 0.000 0.444 0.0000.000 1.000 0.999 0.037 1.000
119
Table 3.3 The GLUM Experiment, Series X3{i,j}, ;=2; j=O, ..., 4 (TwoContrived Near Dependencies): Condition Indexes and Variance-DecompositionProportions
D{2,0} Set 21: W2, ZO
CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 ZO
1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.031 0.001 0.002 0.0785 0.000 0.054 0.837 0.000 0.000 0.0977 0.282 0.285 0.047 0.000 0.000 0.7018 0.666 0.613 0.063 0.000 \ 0.001 0.119
67 0.049 0.002 0.013 0.998 0.997 0.001
D{2,1} Set 22: W2, Zl
CI variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Zl
1 0.003 0.001 0.004 0.000 0.000 0.0013 0.000 0.009 0.034 0.001 0.002 0.0055 0.001 0.028 0.506 0.000 0.000 0.0048 0.926 0.010 0.033 0.001 0.001 0.013
22 0.025 0.891 0.417 0.001 0.000 0.91470 0.045 0.061 0.006 0.997 0.997 0.064
D{2,2} Set 23: W2, Z2
CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001
65 0.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844
X3{2,3} Set 24: W2, Z3
CI Variance Proportions for Coefficients ofINT aXl aX2 aX3 W2 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.002 0.002 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.945 0.000 0.001 0.001 0.001 0.000
68 0.048 0.000 0.000 0.996 0.997 0.000274 0.003 1.000 0.991 0.002 0.001 1.000
X3{2,4} Set 25: W2, Z4
CI Variance Proportions for Coefficients of •INT aXl aX2 aX3 W2 Z41 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.001 0.002 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.947 0.000 0.000 0.001 0.001 0.000
68 0.048 0.000 0.000 0.986 0.980 0.000795 0.000 1.000 0.999 0.012 0.017 1.000
120
Table 3.4 The GLUM Experiment, Series X3{i,j}, ;=0, ..., 4; j=2 (TwoContrived Near Dependencies): Condition Indexes and Variance-DecompositionProportions
D{0,2} Set 13: WO, Z2
CI VarIance Proportions for CoeffIcIents ofINT BX1 BX2 BX3 WO Z2
1 0.004 0.000 0.001 0.004 0.005 0.0004 0.000 0.001 0.004 0.087 0.127 0.0015 0.004 0.002 0.094 0.010 0.000 0.0007 0.359 0.001 0.000 0.190 0.603 0.0008 0.632 0.001 0.006 0.671 0.232 0.000
76 0.002 0.995 0.895 0.038 0.033 0.998
D{1,2} Set 18: W1, Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2
1 0.003 0.000 0.001 0.001 0.001 0.0003 0.000 0.001 0.005 0.013 0.013 0.0015 0.001 0.003 0.092 0.000 0.000 0.0008 0.993 0.001 0.006 0.008 0.007 0.001
23 0.000 0.000 0.001 0.956 0.963 0.00076 0.002 0.995 0.896 0.023 0.016 0.998
D{2,2} Set 23: W2, Z2
CI variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001
65 0 •.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844
X3{3,2} Set 28: W3, Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.090 0.000 0.000 0.0008 0.978 0.001 0.007 0.000 0.000 0.001
75 0.003 0.939 0.879 0.001 0.001 0.954229 0.015 0.056 0.018 0.999 0.999 0.044
D{4,2} Set 33: W4, Z2
CI variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.989 0.001 0.006 0.000 0.000 0.001
76 0.003 0.982 0.890 0.000 0.000 0.989807 0.003 0.013 0.007 1.000 1.000 0.009
121
3.3 The Mixed Model Experiment
The primary goal of this dissertation is to determine the patterning of the
collinearity diagnostics in the mixed model context. In this section, the
procedures for creating the artificial dependencies in X and Z are described.
Then the approach used to obtain the matrices and parameters needed for the
mixed model simulations is summarized. Although the approach involves
varying some of the factors previously identified in Chapter 2, accomplishing
this directly in simulations is difficult. The problems encountered en route to
a final strategy are described; they pinpoint the difficulties of collinearity
assessment in the mixed model. Then the factors varied in the simulations are
described. Finally, the results of applying the collinearity diagnostics to the
simulated matrix (X'I-'X) for the mixed model are summarized. Based on the
results of these experiments, some general guidelines for the behavior of the
diagnostics under the conditions examined are presented.
3.3.1 The Basic Data
Analogous to the GLUM data, the basic data set used for the mixed
model simulations is
X .. [INT, BX1, BX2, BX3], (3.3.1)
where INT is the intercept term and BX1, BX2, and BX3 are each generated
from a uniform distribution. However, for the mixed simulations, data for 30
subjects with 10 observations each were generated, i.e., 300 values of each
variable were generated. The variable BX1 was made a "longitudinal variable"
within a given subject's data by sorting it in ascending order and rounding it to
122
one decimal place prior to generating the dependencies. The potential range of
values for BX1 is from 0.0 to 10.0; the potential range for BX2 is the set of
integers from 0 to 10; and the potential range for BX3 is the set of integers
from 0 to 100. The actual ranges for a specific subject may be shorter.
3.3.2 The Dependency Sets
From the basic data, the same two dependency sets used in the GLUM
experiment were created for the mixed model experiment. In the first,
Wi = BX3 + 8 i, i =0, ..., 4, (3.3.2)
with 8 j generated from a normal distribution with mean zero and variance
02 = 10-i X [var(BX3)1]. In the second,
Zj = 0.SBX1 + 0.2BX2 + 8;, j =0, ..., 4, (3.3.3)
with 8 j generated from a normal distribution with mean zero and variance
02 = 10·; x [var(0.SBX1 + 0.2BX2)1). These dependencies were created for the
entire data set (300 observations) rather than on a by-subject basis; i.e., the
overall theoretical population variance from the uniform distribution was used
for the variances of BX3 and (0.SBX1 + 0.2BX2), rather than the sample
variance of these variables, considered either overall or separately for each
subject.
3.3.3 The Data Series
As in the GLUM context, the Xi and Zj dependency sets were used to
augment the basic data set to produce three series of matrices:
X1{i} == [X Wi]
X2{j} == [X Zjl
i=O, ..., 4,
j =0, ..., 4,
123
(3.3.4)
(3.3.5)
X3{i,j} • [X Wi ~] i,j =0, ..., 4. (3.3.6)
The additional complexity in the mixed model case is that these dependencies
may exist in the fixed effects, in the random effects or in both. Thus, for each
series, collinearity diagnostics for the fixed effects of the mixed model were
computed separately for several cases of variate involvement in the random
effects.
3.3.4 The Issues Addressed
The basic issues addressed are the same as those addressed for the
GLUM. The behavior of the diagnostics is assessed for 1) a simple dependency
between two variables created by the Wi' 2) the simple dependency among
three variables created by the Zj' and 3) the sets of coexisting dependencies
created by considering the Wi and the Zi simultaneously.
The reconsideration of these issues in the mixed model context
constitutes the new territory of research for this dissertation. For the mixed
model, the behavior of the diagnostics for the fixed effects can be impacted by
dependencies both in the fixed effects (the X matrix in the model) and those in
the random effects (the Z matrix in the model, not to be confused with the
vectors of dependencies Zi,i:O,4). Thus, for each dependency series, collinearity
diagnostics for the fixed effects of the mixed model were computed as the
number of variables in Z varied from 0 to 6. There were 6 runs for each
dependency; for the sixth run in each set, all variables in X were also in Z. So,
for the sixth run, there were five variables in Z for the X1 {i} series and the
X2{j} series and six variables in Z for the X3{i,j} series. Therefore, instead of
124
the 35 simulation runs required for the GLUM experiment, 210 (35 x 6) runs
were required for the mixed model experiment. Table 3.5 contains a summary
of the variables involved in the mixed model runs.
125
Table 3.5 Variables in Fixed and Random Effects of Mixed Model Experiment
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 WO 1_0 1
INT BX1 BX2 BX3 WO INT 1_1 2
WOINT BX1 BX2 BX3 WO INT BX1 1_2 3
INT BX1 BX2 BX3 WO INT BX1 BX2 1_3 4
INT BX1 BX2 BX3 WO INT BX1 BX2 BX3 1_4 5
INT BX1 BX2 BX3 WO INT BX1 BX2 BX3 WO 1 5 6
INT BX1 BX2 BX3 W1 2_0 7
INT BX1 BX2 BX3 W1 INT 2 1 8
W1INT BX1 BX2 BX3 W1 INT BX1 2 2 9
INT BX 1 BX2 BX3 W1 INT BX1 BX2 2_3 10
INT BX1 BX2 BX3 W1 INT BX1 BX2 BX3 2 4 11
INT BX1 BX2 BX3 W1 INT BX1 BX2 BX3 W1 2_5 12
INT BX1 BX2 BX3 W2 3 0 13
INT BX1 BX2 BX3 W2 INT 3 1 14
W2INT BX1 BX2 BX3 W2 INT BX1 3_2 15
INT BX1 BX2 BX3 W2 INT BX1 BX2 3 3 16
INT BX1 BX2 BX3 W2 INT BX1 BX2 BX3 3_4 17
INT BX1 BX2 BX3 W2 INT BX1 BX2 BX3 W2 3_5 18
INT BX1 BX2 BX3 W3 4 0 19
INT BX1 BX2 BX3 W3 INT 4 1 20
W3 INT BX1 BX2 BX3 W3 INT BX1 4_2 21
INT BX1 BX2 BX3 W3 INT BX1 BX2 4 3 22
INT BX1 BX2 BX3 W3 INT BX1 BX2 BX3 44 23
INT BX1 BX2 BX3 W3 INT BX1 BX2 BX3 W3 4_5 24
INT BX1 BX2 BX3 W4 5_0 25
INT BX1 BX2 BX3 W4 INT 5_1 26
W4 INT BX1 BX2 BX3 W4 INT BX1 5_2 27
INT BX1 BX2 BX3 W4 INT BX1 BX2 5_3 28
INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 5 4 29
INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 W4 5 5 30
126
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 ZO 60 31
INT BX1 BX2 BX3 ZO INT 6_1 32
ZOINT BX1 BX2 BX3 ZO INT BX1 6 2 33
INT BX1 BX2 BX3 ZO INT BX1 BX2 6_3 34
INT BX1 BX2 BX3 ZO INT BX1 BX2 BX3 6_4 35
INT BX1 BX2 BX3 ZO INT BX1 BX2 BX3 ZO 6_5 36
INT BX1 BX2 BX3 Z1 7 0 37
INT BX1 BX2 BX3 Z1 INT 7 1 38
Z1INT BX1 BX2 BX3 Z1 INT BX1 7_2 39
INT BX1 BX2 BX3 Z1 INT BX1 BX2 7_3 40
INT BX1 BX2 BX3 Z1 INT BX1 BX2 BX3 7 4 41
INT BX1 BX2 BX3 Z1 INT BX1 BX2 BX3 Z1 7_5 42
INT BX1 BX2 BX3 Z2 8 0 43
INT BX1 BX2 BX3 Z2 INT 8_1 44
Z2INT BX1 BX2 BX3 Z2 INT BX1 8 2 45
INT BX1 BX2 BX3 Z2 INT BX1 BX2 8 3 46
INT BX1 BX2 BX3 Z2 INT BX1 BX2 BX3 8 4 47
INT BX1 BX2 BX3 Z2 INT BX1 BX2 BX3 Z2 8_5 48
INT BX1 BX2 BX3 Z3 90 49
INT BX1 BX2 BX3 Z3 INT 9_1 50
Z3INT BX1 BX2 BX3 Z3 INT BX1 9_2 51
INT BX1 BX2 BX3 Z3 INT BX1 BX2 9_3 52
INT BX1 BX2 BX3 Z3 INT BX1 BX2 BX3 94 53
INT BX1 BX2 BX3 Z3 INT BX1 BX2 BX3 Z3 9 5 54
INT BX1 BX2 BX3 Z4 10_0 55
INT BX1 BX2 BX3 Z4 INT 10 1 56
Z4INT BX1 BX2 BX3 Z4 INT BX1 10 2 57
INT BX1 aX2 aX3 Z4 INT aX1 aX2 10_3 58
INT ax1 BX2 aX3 Z4 INT ax1 BX2 aX3 10 4 59
INT BX1 BX2 BX3 Z4 INT ax1 BX2 BX3 Z4 10 5 60
127
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 WO ZO 11_0 61
INT BX1 BX2 BX3 WO ZO tNT 11_1 62
WOtNT BX1 BX2 BX3 WO ZO tNT BX1 11_2 63
ZO tNT BX1 BX2 BX3 WO ZO tNT BX1 BX2 11_3 64
tNT BX1 BX2 BX3 WO ZO tNT BX1 BX2 BX3 11_4 65
tNT BX1 BX2 BX3 WO ZO INT BX1 BX2 BX3 WO ZO 11_5 66
INT BX1 BX2 BX3 WO Z1 12 0 67
INT BX1 BX2 BX3 WO Z1 tNT 12_1 68
WO INT BX1 BX2 BX3 WO Z1 INT BX1 12_2 69
Z1 INT BX1 BX2 BX3 WO Z1 tNT BX1 BX2 12_3 70
INT BX1 BX2 BX3 WO Z1 tNT BX1 BX2 BX3 12 4 71
tNT BX1 BX2 BX3 WO Z1 INT BX1 BX2 BX3 WO Z1 12 5 72
tNT BX1 BX2 BX3 WO Z2 13 0 73
INT BX1 BX2 BX3 WO Z2 INT 13_1 74
WO INT BX1 BX2 BX3 WO Z2 INT BX1 13_2 75
Z2 INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 13 3 76
INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 BX3 13_4 77
INT BX1 BX2 BX3 WO Z2 INT BX1 BX2 BX3 WO Z2 13_5 78
INT BX1 BX2 BX3 WO Z3 14_0 79
INT BX1 BX2 BX3 WO Z3 tNT 14 1 80
WO INT BX1 BX2 BX3 WO Z3 INT BX1 14 2 81
Z3 INT BX1 BX2 BX3 WO Z3 INT BX1 BX2 14 3 82
INT BX1 BX2 BX3 WO Z3 tNT BX1 BX2 BX3 14 4 83
INT BX1 BX2 BX3 WO Z3 INT BX1 BX2 BX3 WO Z3 14_5 84
INT BX1 BX2 BX3 WO Z4 15 0 85
tNT BX1 BX2 BX3 WO Z4 INT 15 1 86
WOINT BX1 BX2 BX3 WO Z4 INT BX1 15 2 87
Z4 INT BX1 BX2 BX3 WO Z4 INT BX1 BX2 15 3 88
INT BX1 BX2 BX3 wO Z4 INT BX1 BX2 BX3 15_4 89
INT BX1 BX2 BX3 WO Z4 INT BX1 BX2 BX3 WO Z4 15_5 90
128
Collinear Fixed Effects Random Effects Set &. RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 W1 ZO 16_0 91
INT BX1 BX2 BX3 W1 ZO INT 16_1 92
W1INT BX1 BX2 BX3 W1 ZO INT BX1 16 2 93
ZO INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 16_3 94
INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 BX3 16 4 95
INT BX1 BX2 BX3 W1 ZO INT BX1 BX2 BX3 W1 ZO 16_5 96
INT BX1 BX2 BX3 W1 Z1 17_0 97
INT BX1 BX2 BX3 W1 Z1 INT 17_1 98
W1 INT BX1 BX2 BX3 W1 Z1 INT BX1 17_2 99
Z1 INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 173 100
INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 BX3 17_4 101
INT BX1 BX2 BX3 W1 Z1 INT BX1 BX2 BX3 W1 Z1 17_5 102
INT BX1 BX2 BX3 W1 Z2 18 0 103
INT BX1 BX2 BX3 W1 Z2 INT 18 1 104
W1INT BX1 BX2 BX3 W1 Z2 INT BX1 18 2 105
Z2 INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 18 3 106
INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 BX3 18 4 107
INT BX1 BX2 BX3 W1 Z2 INT BX1 BX2 BX3 W1 Z2 18_5 108
INT BX1 BX2 BX3 W1 Z3 19 1 109
INT BX1 BX2 BX3 W1 Z3 INT 19 1 110
W1INT BX1 BX2 BX3 W1 Z3 INT BX1 19_2 111
Z3 INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 19_3 112
INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 BX3 19_4 113
INT BX1 BX2 BX3 W1 Z3 INT BX1 BX2 BX3 W1 Z3 19_5 114
INT BX1 BX2 BX3 W1 Z4 20 1 115
INT BX1 BX2 BX3 W1 Z4 INT 20 1 116
W1INT BX1 BX2 BX3 W1 Z4 INT BX1 20 2 117
Z4 INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 20 3 118
INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 BX3 20 4 119
INT BX1 BX2 BX3 W1 Z4 INT BX1 BX2 BX3 W1 Z4 20 5 120
129
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 W2 ZO 21_0 121
INT BX1 BX2 BX3 W2 ZO INT 21_1 122
W2INT BX1 BX2 BX3 W2 ZO INT BX1 21_2 123
ZO INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 21_3 124
INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 BX3 21_4 125
INT BX1 BX2 BX3 W2 ZO INT BX1 BX2 BX3 W2 ZO 21_5 126
INT BX1 BX2 BX3 W2 Z1 22_0 127
INT BX1 BX2 BX3 W2 Z1 INT 22_1 128
W2INT BX1 BX2 BX3 W2 Z1 INT BX1 22 2 129
Z1 INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 22 3 130
INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 BX3 22_4 131
INT BX1 BX2 BX3 W2 Z1 INT BX1 BX2 BX3 W2 Z1 22_5 132
INT BX1 BX2 BX3 W2 Z2 23_0 133
INT BX1 BX2 BX3 W2 Z2 INT 23_1 134
W2INT BX1 BX2 BX3 W2 Z2 INT BX1 23 2 135
Z2 INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 23 3 136
INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 BX3 23_4 137
INT BX1 BX2 BX3 W2 Z2 INT BX1 BX2 BX3 W2 Z2 23_5 138
INT BX1 BX2 BX3 W2 Z3 24_0 139
INT BX1 BX2 BX3 W2 Z3 INT 24_1 140
W2INT BX1 BX2 BX3 W2 Z3 INT BX1 24_2 141
Z3 INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 24_3 142
INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 BX3 24 4 143
INT BX1 BX2 BX3 W2 Z3 INT BX1 BX2 BX3 W2 Z3 24 5 144
INT BX1 BX2 BX3 W2 Z4 25_0 145
INT BX1 BX2 BX3 W2 Z4 INT 25 1 146
W2 INT BX1 BX2 BX3 W2 Z4 INT BX1 25 2 147
Z4INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 25_3 148
INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 BX3 25_4 149
INT BX1 BX2 BX3 W2 Z4 INT BX1 BX2 BX3 W2 Z4 25_5 150
130
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 W3 ZO 26 0 151
INT BX1 BX2 BX3 W3 ZO INT 26 1 152
W3INT BX1 BX2 BX3 W3 ZO INT BX1 26_2 153
ZO INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 26_3 154
INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 BX3 26_4 155
INT BX1 BX2 BX3 W3 ZO INT BX1 BX2 BX3 W3 ZO 26_5 156
INT BX1 BX2 BX3 W3 Z1 27_0 157
INT BX1 BX2 BX3 W3 Z1 INT 27 1 158
W3INT BX1 BX2 BX3 W3 Z1 INT BX1 27 2 159
Z1 INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 27_3 160
INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 BX3 27_4 161
INT BX1 BX2 BX3 W3 Z1 INT BX1 BX2 BX3 W3 Z1 27_5 162
INT BX1 BX2 BX3 W3 Z2 28_0 163
INT BX1 BX2 BX3 W3 Z2 INT 28_1 164
W3INT BX1 BX2 BX3 W3 Z2 INT BX1 28 2 165
Z2 INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 28 3 166
INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 BX3 28 4 167
INT BX1 BX2 BX3 W3 Z2 INT BX1 BX2 BX3 W3 Z2 28_5 168
INT BX1 BX2 BX3 W3 Z3 29_0 169
INT BX1 BX2 BX3 W3 Z3 INT 29 1 170
W3INT BX1 BX2 BX3 W3 Z3 INT BX1 29 2 171
Z3 INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 29 3 172
INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 BX3 29 4 173
INT BX1 BX2 BX3 W3 Z3 INT BX1 BX2 BX3 W3 Z3 29_5 174
INT BX1 eX2 eX3 W3 Z4 30_0 175
INT ex1 eX2 eX3 W3 Z4 INT 30_1 176
W3INT ex1 eX2 eX3 W3 Z4 INT BX1 30 2 177
Z4 INT ex1 eX2 eX3 W3 Z4 INT eX1 eX2 30 3 178
INT ex1 eX2 BX3 W3 Z4 INT ex1 BX2 eX3 30 4 179
INT ex1 eX2 eX3 W3 Z4 INT ex1 eX2 eX3 W3 Z4 30 5 180
131
Collinear Fixed Effects Random Effects Set & RunVariable Variables in X Variables in Z No. No.
INT BX1 BX2 BX3 W4 ZO 31_0 181
INT BX1 BX2 BX3 W4 ZO INT 31 1 182
INT BX1 BX2 BX3 W4 ZO INT BX1 31_2 183W4ZO INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 31 3 184
INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 BX3 31 4 185
INT BX1 BX2 BX3 W4 ZO INT BX1 BX2 BX3 W4 ZO 31_5 186
INT BX1 BX2 BX3 W4 Z1 32_0 187
INT BX1 BX2 BX3 W4 Z1 INT 32_1 188
W4INT BX1 BX2 BX3 W4 Z1 INT BX1 32 2 189
Z1 INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 32 3 190
INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 BX3 32_4 191
INT BX1 BX2 BX3 W4 Z1 INT BX1 BX2 BX3 W4 Z1 32 5 192
INT BX1 BX2 BX3 W4 Z2 33_0 193
INT BX1 BX2 BX3 W4 Z2 INT 33_1 194
W4INT BX1 BX2 BX3 W4 Z2 INT BX1 33 2 195
Z2 INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 33 3 196
INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 BX3 33_4 197
INT BX1 BX2 BX3 W4 Z2 INT BX1 BX2 BX3 W4 Z2 33_5 198
INT BX1 BX2 BX3 W4 Z3 34_0 199
INT BX1 BX2 BX3 W4 Z3 INT 34 1 200
W4INT BX1 BX2 BX3 W4 Z3 INT BX1 34_2 201
Z3 INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 34_3 202
INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 BX3 34_4 203
INT BX1 BX2 BX3 W4 Z3 INT BX1 BX2 BX3 W4 Z3 34_5 204
INT BX1 BX2 BX3 W4 Z4 35_0 205
INT BX1 BX2 BX3 W4 Z4 INT 35 1 206
W4INT BX1 BX2 BX3 W4 Z4 INT BX1 35 2 207
Z4 INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 35 3 208
INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 BX3 35 4 209
INT BX1 BX2 BX3 W4 Z4 INT BX1 BX2 BX3 W4 Z4 35 5 210
132
•
3.3.5 Mixed Model Matrices and Parameters
..In practice, the assessment of collinearity in the mixed model context
must be carried out after model parameters are estimated because the
A.
assessment involves the estimated matrix E. This was the procedure used in
Chapter 2 for the initial exploration of the mixed model diagnostics. However,A.
fitting a model in order to produce the matrix E greatly complicates the
experiment because it requires a Y. Recall that calculation of the GLUM
diagnostics did not require a Y.
Initially, the following strategy was conceived to generate the Y, fit theA.
model using the Y and the simulated variables in X and Z, and obtain E:
1. Values for 11 and a2 would be selected and used to compute E. Recall
that
definite symmetric covariance matrix for the kth
subject, and
E (NxN) = Diag(E" E2, "., I.r.) is the covariance matrix of the
entire response vector, Y.
2. A value for P would be selected and values for dk, and ek would be
generated from appropriate normal distributions (N(O,I1) and N(O,u2I),
respectively) and used to compute Yk' Recall that
is a random vector of unobservable random subject
effects for the kth subject;
is an vector of unobservable within-subject random
133
3.
error terms; and
The Vk would be concatenated to obtain V. ..A- A- A-
4. The mixed models would be fit and values for Il, 4, 02, I: would be
obtained.A-
S. The collinearity diagnostics would be obtained using (X'r'X).
These steps would have to be repeated for~ of the 210 simulation runs.
After several trials and some reflection, it became apparent that although
A-
this procedure was working, a different V, and hence a different 4, was
generated for every run. The simulations call for changing the number of
A-
variables in Z, which then changes 4; both of these changes also impact I:.
Thus the effect on the diagnostics due to changes in Z would be confounded
with the effects due to changes in V, making it impossible to separate their
effects. So, one problem with this strategy was obtaining the V and another
was retaining the same V for every simulation run.
In order to reduce the complications of the initial approach, a more direct
strategy was devised:
1. A 4 and 02 were specified and used to compute ~:
A-
2. Then r', instead of r' was used to compute
K
(X' r -,Xl = L X: r;'XII + crVII .k-1
3. The collinearity diagnostics were obtained from the matrix
134
3.3.6 Values for 4, tr, and Vk
To implement the direct approach described above, values for 4, u2, and
Vk were chosen. With this approach, the matrix E changed necessarily with
each run, but not because Y changed; no Y was involved. It was because the
variables in Z changed. Selecting a subset of the columns of Z required the
selection of a corresponding submatrix of 4. This procedure ensured
comparability of results from one model (X, Z, 4) to another because the only
differences between models were essential differences. This approach was
implemented by specifying an overall 4, called 4(1), and selecting rows and
columns of the matrix that corresponded to the number of variables in Z. The
overall 4 was
1 0.6 0 0 0 00.6 1 o 0 0 0
4(1) = 0 0 100 0 (3.3.7)0 0 o 1 0 00 0 001 00 0 000 1
So, for example, when the first two variables (lNT, aX1) were in Z, 4 was the
(2 x 2) matrix
4 = [1 0.6]0.6 1
When all six variables were in Z, 4 =4(1).
(3.3.8)
This particular matrix for 4(1) was chosen because we were interested in
the situation in which all of the random effects in the model, except the first
135
two, were uncorrelated. We began with a simple case in which the variances
were all equal. For the same reason, the same values of rr =1 and V" = I".
were used in each calculation.
For the special case of no random effects, i.e., no variables in Z, the
computations were modified slightly to obtain the diagnostics. In this situation,
I = IN' These calculations were included as "baseline" runs in order to more
clearly assess the impact of adding variables to Z. In addition, the "baseline"
runs were more immediately comparable to the corresponding GLUM
simulations.
3.3.7 Results
The mixed model experiment in this chapter addresses several issues:
1) the behavior of the diagnostics for Data Series X1 {i} • [X Wi] i =0, , 4;
2) the behavior of the diagnostics for Data Series X2{j} .. [X Zj] j = 0, , 4;
3) the behavior of the diagnostics for Data Series X3{i,j} • [X Wi Zj] i,j = 0,
..., 4; and 4) the effect on the diagnostics of varying the number and nature of
the variables in the random effects matrix, Z.
Following the results of the GLUM diagnostics shown in Appendix 1, the
results for the mixed model are presented completely in Appendices 2-3.
Appendix 2 contains the baseline diagnostics (35 runs); Appendix 3 contains
the diagnostics for models with 1-6 variables in the random effects. Selected
results from the experiment are presented in detail in this section in a certain
pattern. For each of the X1{i}, X2{j} and X3{i,j} data series, the baseline
results are presented first and the results of adding variables to the random
effects are presented second.
136
3.3.7.1 Simple Dependency: Two Variables
Baseline
For the data series X1{i} • [X Wi]' i =0, ..., 4, recall that the variable
BX3 is related to Wi by (3.3.2). This is only contrived dependency in this
series. The results (shown in Appendix 2, Sets 1-5) of the baseline simulations
(no variables in Z) are nearly identical to those found for the GLUM simulations
(shown in Appendix 1, Sets 1-5). As found previously, there is one high
condition index and a large proportion of the variance of BX3 and the variances
of WO-W4 are associated with it. As the dependency between the two
variables increases, i.e., as Wi increases, the pattern of the relationship
becomes clearer. The one condition index indicating the dependency becomes
larger and the variance decomposition proportions of the two variables become
larger. The diagnostics clearly point to the variables involved in the
dependency.
Adding Variables to Random Effects
For the data series X1{i} - [X Wi]' i =0, ..., 4, the results of adding
variables to the random effects can be seen in Appendix 3 (Sets 1-5) and
graphically, in Figure 3.1. The baseline diagnostics are shown in row one
(NVARS = 0). As Wi increases, the condition index increases, reaching a value
of 633 for W 4 • The patterns in the~ indicating one, two, or three variables
in Z are similar to that in the baseline row, but the values are smaller.
However, when four or five variables are in the random effects, the condition
index remains about the same (1-3) as Wi increases.
137
o
Value of Condition Index for BX3 and Wiby Level of Wi and
Number of Variables in Random Effects
WI
COlUNEAR VARIABLES: BX3. WI and lXI, BX2, ZI
Figure 3.1 Condition Index for BX3 and Wi' by Number of Random EffectsVariables
The lowest dependency is shown in column one for WOo As the number
of variables in Z increases, the condition index decreases, falling to a value of
1 when five variables are in Z. For each Wi (column), the pattern is similar;
there is a reduction in the largest condition index as variables are added to Z.
However, the reduction becomes more dramatic as the dependency increases.
The most dramatic change occurs for W4; the condition index is reduced from
271, when three variables are in Z, to 3, when four or five variables are in Z.
There appears to be a relationship between the number and the nature
of the variables in Z and the indication of collinearity by these diagnostics.
138
When three variables are in the random effects, the variables are INT, BX1, and
BX2, none of which is involved in the constructed dependency. The fourth and
fifth variables added to Z are the two collinear variables, BX3 and Wi' Thus,
it appears that when one or both collinear variables are included in Z, the
dependency in X, that clearly remains, no longer appears to adversely impactA A
the asymptotic variance of P; i.e., V(JJ) is not ill-conditioned. The condition
index no longer indicates the presence of collinearity even though the variance
decomposition proportions (see Appendix 3, Sets 1-5) still point to the variables
involved in the dependency in X. According to Belsley (1991 ), both diagnostics
must indicate collinearity for the collinearity to be degrading.
3.3.7.2
Baseline
Simple Dependency: Three Variables
For the data series X2{j} • [X Zj]' j =0, ..., 4, recall that the variables
BX1 and BX2 are related to Zj by (3.3.3). This is only contrived dependency
in this series. The results (shown in Appendix 2, Sets 6-10) of the baseline
simulations (no variables in Z) are very similar to those found for the GLUM
simulations (shown in Appendix 1, Sets 6-10). As found previously, there is
one high condition index and a large proportion of the variances of BX1, BX2
and the variances of ZO-Z4 are associated with it. As the dependency among
the three variables increases, Le., as Zj increases, the pattern of the relationship
becomes clearer. The one condition index indicating the dependency becomes
larger and the variance decomposition proportions of the three variables
become larger. The diagnostics clearly point to the variables involved in the
139
dependency.
Adding Variables to Random Effects
For the data series X2{i} • [X ~], j =0, ..., 4, the results of adding
variables to the random effects can be seen in Appendix 3 (Sets 6-10) and
graphically, in Figure 3.2. The baseline diagnostics are shown in row one
Value of Condition Index for ex 1, BX2 and Ziby Level of Zi and
Number of Variables in Random Effects
ZI
COlUNEAR VAR1A81.£S: aX3, WI CIIId IXI. 1X2. ZI
Figure 3.2 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables
(NVARS =0). As Zj increases, the condition index increases reaching a value
of 640 for Z4' The patterns in the!2M indicating one to five variables in Z are
similar to that in the baseline row, but the values are smaller. When two or
more variables are in the random effects, the condition index values are
140
substantially smaller as ~ increases.
The lowest dependency is shown in column one for Zo. As the number
of variables in Z increases, the condition index decreases, falling to a value of
1 when five variables are in Z. For each ~ (column), the pattern is similar;
there is a reduction in the largest condition index as variables are added to Z.
However, the reduction becomes more dramatic as the dependency increases.
The most dramatic change occurs for Z.; the condition index is reduced from
271, when one variable is in Z, to 63, when two variables are in Z. When
three or more variables are in Z, the condition index decreases to 27 or less.
Again there appears to be a relationship between the number and the
nature of the variables in Z and the indication of collinearity by these
diagnostics. When only one variable, INT, is in the model, the dependency is
detectable for Z2 or tighter. When two variables are in the model, one of them,
BX1, is involved in the constructed dependency; the dependency is detectable
for Z3 or tighter. When the second collinear variable, BX2, is in Z along with
INT and BX1, the condition indexes are reduced further and detectable only at
Z.. There is not much further reduction, however, when the last collinear
variable Zj is included. For the tightest dependency, Z., but not for the weaker
dependencies, the presence of collinearity is still detectable when all five
variables are in the model. Again, except for the tightest dependency, it
appears that when collinear variables are included in Z, the dependency in X no
A
longer appears to adversely impact the asymptotic variance of p. The condition
index no longer indicates the presence of collinearity. However, the variance
decompositions (see Appendix 3, Sets 6-10) do point to at least two of the
141
involved variables when the condition indexes are as low as 2. When the
values of the condition index are at least 7, the variance decomposition
proportions behave as expected.
3.3.7.3
Baseline
Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)
The X3{i,j} series contains a combination of the two dependencies in
X1{i} and X2{j}; these are coexisting dependencies. The results of the baseline
simulations (no variables in Z) are again quite similar to those found for the
GLUM simulations. The 25 sets of diagnostics produced from this series for
the baseline runs are shown in Appendix 2 (Sets 11-35). The diagnostics are
summarized in Figures 3.3 (for the relationship between BX3 and Wi) and in
Figure 3.4 (for the relationship among BX1, BX2, and Zj). Selected sets are
discussed here.
In Appendix 2, Sets 21-25, the dependency in (3.3.2) is fixed (i=2) and
strong while the dependency in (3.3.3) varies V=O, ..., 4). These results can
be seen also in Figure 3.3 (plot of CI for BX3 and Wi) and in Figure 3.4 (plot of
CI for BX1, BX2, and ~); look at the W2 column in both. There are two high
condition indexes indicating the two dependencies, though for i = 0 the second
dependency is dominated by the first and is not yet apparent. The effects
become separable when i= 1 (W2 and Z,). Thus the procedure can clearly
detect two dependencies and the variables involved in each. When the two
dependencies are of nearly the same strength, their effects become
confounded. For X3{2,2} (W2 and Z2)' the condition indexes are of similar
142
Value of Condition Index for 8X3 and Wiby Level of Wi and Zj
No Variables are in Random Effects
o
WI
COLUHEAR VARIASUS: SX3. Wi and BX1, BX2. Zi
Figure 3.3 Condition Index for BX3 and Wi by Levels of Wi and Zj, NoRandom Effects Variables
magnitude (66 and 71) and the involvement of variables in the two
dependencies is somewhat obscured. However, it is possible to determine that
there are two dependencies present and that five variables are involved in
them. As the second dependency (BX1, BX2, Zj) becomes strong (at Z3)
relative to the first dependency (BX3, W2 ), the effects are again separable.
The X3{i,j} series can be examined from the other direction by holding
j constant and varying i. These results are shown in Appendix 2 in Sets 13,
18,23,28, and 33 for i=O, ..., 4 andj=2. These results can be seen also in
Figure 3.3 (plot of CI for BX3 and Wi) and in Figure 3.4 (plot of CI for BX1,
BX2, and Zj); look at the Z2 row in both. The dependency in 3.3.2 varies while
143
Value of Condition Index for 8X1 , 8X2 and Ziby Level of Wi and Zj
No Variables are in Random Effects
..
o
WI
COLLINEAR VARIAaL£S: aX3. Wi and BX1, BX2, Zj
Figure 3.4. Condition Index for BX1, BX2 and Zj by Levels of Wi and Zit NoRandom Effects Variables
the dependency in 3.3.3 is held constant. For ;=0, the dependency involving
BX3 and WO is relatively weak; the one high condition index and the variance
decomposition proportions indicate the strong dependency in BX1, BX2, and
Z2. When;= 1, the two dependencies are distinct. When;= 2, the two
dependencies are of similar strength and the variables involved in each are
somewhat obscured. However, when ;= 3, their separate identities emerge
again and remain distinct for ;=4. Again, the procedure can clearly detect two
dependencies and the variables involved in each.
Overall, the pattern for the dependency between BX3 and Wi is similar,
regardless of the level o'f Zj; the condition index increases from 9 at Wo to
144
nearly 700 at W•. Also as the Wi increases, the first dependency dominates
the second at lower levels of Zj. Similarly, the pattern for the dependency
between BX1, BX2, and ~ is the same, regardless of the level of Wi; the
condition index increases from 8-9 at Zo to about 700 at Z.. Also as the Zj
increases, the second dependency dominates the first at lower levels of Wi'
Adding Variables to Random Effects
For the data series X3{i,j} • [X Wi ~], i,j =0, ..., 4, the 25 sets (Sets
11-35) of diagnostics resulting from adding variables to the random effects can
be seen in Appendix 3. In Figures 3.5-3.9, the largest condition index
attributable to the dependency between BX3 and Wi is plotted by level of Wi
and number of variables included in the random effects, separately for each
level of Zj' In Figures 3.10-3.14, the largest condition index attributable to the
dependency between BX1, BX2, and ~ is plotted by level of Zj and number of
variables included in the random effects, separately for each level of Wi'
145
o
Value of Condition Index for BX3 and Wiby Ley.. of wr and
Number of Variables in Random Effectsat ZO
WI
Figure 3.5 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Zo
Value of Condition Index for BX3 end Wiby Lev.1 of WI and
Number of Variables In Random Effectsot Zl
o
WI
Figure 3.6 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z,
146
o
Value of Condition Index for BX3 and Wiby Lev" of WI and
Number of Variables in Random Effectsat Z2
Figure 3.7 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z2
Value of Condition Index for 8X3 and Wiby Lev•• of WI and
Number of Variables In Random EffectsCIt Z3
o
WI
Figure 3.8 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z3
147
o
Value of Condition Index for BX3 and Wiby Level of WI and
Number of Variables in Random EffectsalU
WI
Figure 3.9 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z4
Value of Condition Index for BX1. BX2 and Zjby Leve' of II and
Number of Variables In Random EffectsafWO
o
cow~V_liS: 113. WI _ 1I1.1IlCZ, Zl
Figure 3.10 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at Wo
148
o
Value of Condition Index for ex 1. eX2 and Zjby Level of ZI and
Number Df Variables in RandDm Effects01 W1
..
Figure 3.11 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 1
Value of Condition Index for eX1, eX2 and Zjby Level of ZI and
Number of Variables In Random EffectscllW2
..
o
Figure 3.12 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 2
149
o
Value of Condition Index for ex 1, BX2 and Zjby Level of II and
Number of Variables in Random Eff.cts0lW3
zr
..
Figure 3.13 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 3
Value of Condition Index for BX1, BX2 and Zjby Level of II and
Number of Variables In Random Effects01 W4
o
a1W_"_10: 10, WI _II Dl,lX2. ZI
Figure 3.14 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W 4
150
"
The fact that the two dependencies were designed to be nonoverlapping
is again confirmed by examining these figures. In Figures 3.5-3.9, the pattern
and values of the condition indexes are nearly the same in each plot, i. e.,
regardless of the level of~. Further, they are very similar to the values
obtained when this dependency was considered independently in the Xl {i}
series; this can be seen in Figure 3.1. Again, the addition of variables to the
random effects, especially collinear ones, diminishes the impact of the
collinearity in X. Thus, the effect of adding variables to the random effects
when the dependencies are coexisting is the same as the effect when the
dependencies are considered separately.
Similarly, in Figures 3.10-3.14, the pattern and values of the condition
indexes are nearly the same in each plot, i. e., regardless of the level of Wi'
Further, they are very similar to the values obtained when this dependency was
considered independently in the X2{j} series; this can be seen in Figure 3.2.
And again, the addition of variables to the random effects, especially collinear
ones, diminishes the impact of the collinearity in X. Once more, we conclude
that the effect of adding variables to the random effects when the
dependencies are coexisting is the same as the effect when the dependencies
are considered separately.
For the case when variables are added to the model composed of
coexisting dependencies, an interesting result is found regarding the competing
dependencies. In the baseline runs, it was found that when the two
dependencies were of nearly equal strength, e.g., for X3{2,2} (W2 and Z2),
their effects became confounded. The condition indices for the two
151
dependencies with no variables in the random effects were 66 for the
dependency between BX3 and W 2 and 71 for the dependency among BX1,
BX2, and Z2' However, when variables were added to the model, the
competition disappeared. This can be seen in Appendix 3, Set 23 and in Figure
3.7 (W2 column) and Figure 3.12 (Z2 column). When only one variable was in
Z, the competition was still present; the Cis were 29 and 31. However, when
two or three variables were in Z, there was only one high CI (27-28) indicating
the presence of the dependency between BX3 and Wi' Thus, it appears that
increasing the tightness of the dependency in ~ or adding variables to the
model diminishes the competition between the two dependencies. However,
the fact that the variables added are those involved in the dependency among
BX1, BX2, and Zj may confound this conclusion. As has been established,
when variables are added, the condition indexes decrease. For these particular
dependencies for fixed Wi and Zit the second one disappears faster than the
first as variables are added.
These same competing dependencies can now be viewed from the other
direction. An examination of Sets 13,18,23,28, and 33 (i=0, ..., 4) andj=2)
for a given number of variables in the model reveals that increasing Wi also
diminishes the competition. However, when two or more variables are in Z, the
competition disappears anyway due to the additional variables. Then we can
surmise that any of the three actions affects the diagnostics.
152
3.4 Effect of Adding Variables to Random Effects
As noted repeatedly, the collinearity in the fixed effects (X), that was
present in baseline runs, disappeared as more variables were added to the
random effects. It appeared that this was especially true when the variables
added were those involved in one of the two dependencies. To explore further
whether the disappearance was due to the number of variables added or their
nature, two sets of models were rerun varying the set of variables added to the
random effects Z; all possible combinations of variables were examined. This
was carried out only for the X1 {4} series and the X2{4} series since each had
the tightest dependencies and effects of changing variables in the random
effects might be more easily seen for them. The results for X1 {4}are
summarized in Table 3.6; the results for X2{4} are summarized in Table 3.7.
153
Table 3.6 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for X 1{4}
Fixed Effects: Variebles in X Random Effect.: Varieble. in Z Highest CI
INT BX1 BX2 BX3 W4 833
INT BX1 BX2 BX3 W4 INT 298
INT BX1 BX2 BX3 W4 BX1 361
INT BX1 BX2 BX3 W4 BX2 388
INT BX1 BX2 BX3 W4 BX3 3
INT BX1 BX2 BX3 W4 W4 3
INT BX1 BX2 BX3 W4 INT BX1 290
INT BX1 BX2 BX3 W4 INT BX2 278
INT BX1 BX2 BX3 W4 INT BX3 3
INT BX1 BX2 BX3 W4 INTW4 3
INT BX1 BX2 BX3 W4 BX1 BX2 298
INT BX1 BX2 BX3 W4 BX1 BX3 3
INT BX1 BX2 BX3 W4 BX1 W4 3
INT BX1 BX2 BX3 W4 BX2 BX3 3
INT BX1 BX2 BX3 W4 BX2W4 3
INT BX1 BX2 BX3 W4 BX3W4 3
INT BX1 BX2 BX3 W4 INT BX1 BX2 271
INT ax 1 aX2 aX3 W4 INT aX1 aX3 3
INT BX1 BX2 BX3 W4 INT BX1 W4 3
INT BX 1 BX2 BX3 W4 INT BX2 BX3 3
INT aX1 aX2 BX3 W4 INT aX2 W4 3
INT BX1 BX2 BX3 W4 INT BX3 W4 2
INT BX1 BX2 BX3 W4 BX1 BX2 BX3 3
INT BX1 BX2 BX3 W4 BX1 BX2 W4 3
INT BX1 BX2 BX3 W4 BX1 BX3 W4 2
INT ax1 BX2 BX3 W4 BX2 BX3 W4 2
INT BX1 BX2 BX3 W4 INT BX1 BX2 BX3 3
INT BX 1 BX2 BX3 W4 INT BX1 BX2 W4 3
INT BX1 BX2 BX3 W4 INT BX1 BX3 W4 2
INT BX 1 BX2 BX3 W4 INT BX2 BX3 W4 2
INT BX 1 BX2 BX3 W4 BX 1 BX2 BX3 W4 2
INT BX 1 BX2 BX3 W4 INT BX1 BX2 BX3 W4 2
154
Table 3.7 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for X2{4}
Fixed Effects: Veriabln in X Random Effects: Variables in Z Highest CI
INT BX1 BX2 BX3 Z4 640
INT BX1 BX2 BX3 Z4 INT 271
INT BX1 BX2 BX3 Z4 BX1 84
INT BX1 BX2 BX3 Z4 BX2 315
INT BX1 BX2 BX3 Z4 BX3 327
INT BX1 BX2 BX3 Z4 Z4 68
INT BX1 BX2 BX3 Z4 INT BX1 83
INT BX1 BX2 BX3 Z4 INT BX2 228
INT BX1 BX2 BX3 Z4 INT BX3 243
INT BX1 BX2 BX3 Z4 INTZ4 59
INT BX1 BX2 BX3 Z4 BX1 BX2 24
INT BX1 BX2 BX3 Z4 BX1 BX3 72
INT BX1 BX2 BX3 Z4 BX1 Z4 34
tNT BX1 BX2 BX3 Z4 BX2 BX3 242
INT BX1 BX2 BX3 Z4 BX2 Z4 29
INT BX1 BX2 BX3 Z4 BX3 Z4 81
INT BX1 BX2 BX3 Z4 INT BX1 BX2 27
INT BX 1 BX2 BX3 Z4 INT BX1 BX3 60
INT BX 1 BX2 BX3 Z4 INT BX1 Z4 38
INT BX 1 BX2 BX3 Z4 INT BX2 BX3 212
tNT BX 1 BX2 BX3 Z4 INT BX2 Z4 30
INT BX1 BX2 BX3 Z4 INT BX3 Z4 59
INT BX1 BX2 BX3 Z4 BX1 BX2 BX3 25
INT BX 1 BX2 BX3 Z4 BX1 BX2 Z4 18
INT BX1 BX2 BX3 Z4 BX1 BX3 Z4 34
INT BX 1 BX2 BX3 Z4 BX2 BX3 Z4 31
INT BX1 BX2 BX3 Z4 INT BX1 BX2 BX3 27
INT BX1 BX2 BX3 Z4 INT BX1 BX2 Z4 20
INT BX1 BX2 BX3 Z4 INT BX1 BX3 Z4 37
INT BX1 BX2 BX3 Z4 INT BX2 BX3 Z4 31
INT BX1 BX2 BX3 Z4 BX1 BX2 BX3 Z4 19
INT BX1 BX2 BX3 Z4 INT BX1 BX2 BX3 Z4 21
155
An examination of the highest condition indexes resulting from these
additional models reveals several patterns. For the case of X1 {4}, there are
two variables involved in the dependency, BX3 and W4. As shown in Table
3.6, the highest condition index (633) occurred when there were no variables
in the random effects; the lowest condition indexes (2-3) occurred when either
or both of these two variables were in the random effects. When neither of
these variables was present in Z, the values were very high (271-351), but
were reduced from the model without any variables in the random effects.
Thus, we conclude that for this two-variable dependency, adding any variables
to the random effects reduces the impact of collinearity. However, the impact
is still enormous. In contrast, adding collinear variables to the random effects
causes a huge reduction in the impact of the collinearity in the fixed effects.
For the case of X2{4}, there are three variables involved in the
dependency, BX1, BX2 and Z4. As shown in Table 3.7, the highest condition
index (640) occurred when there were no variables in the random effects; the
lowest condition index (18) occurred when all three ofthe variables (BX1, BX2,
Z4) were in the random effects without any other variables. Similar low values
(19-21) were found when these three variables occurred along with the
intercept, BX3 or both. In addition, when two of the three collinear variables
were in Z, with or without other uninvolved variables, the values ranged from
24-38. When only one of the three collinear variables was present in Z along
with an uninvolved variable, the values ranged from 59-72 if the variable was
BX1 or Z4; they ranged from 212-228 if the variable was BX2. For each
collinear variable alone in Z, values were 84 for BX1, 66 for Z4, and 31 5 for
156
aX2.
The conclusion for the X{2} series is not as clear. It appears that for this
three-variable dependency, adding any variables to the random effects also
reduces the impact of collinearity. However, as for the two-variable
dependency, the impact is still enormous. In contrast, adding collinearvariables
to the random effects causes a great reduction in the impact of the collinearity
in the fixed effects. For this dependency, however, the reduction is greatest
when all collinear variables are present in the random effects.
3.5 Summary of Results
The results in this chapter confirm that collinearity diagnostics behave
similarly for the GLUM and for the mixed model. The important new discovery
is that adding variables, especially collinear variables, to the random effects of
the mixed model can greatly attenuate the impact of the collinearity in the fixed
effects.
157
CHAPTER IV
IMPACT OF RANDOM EFFECTS COVARIANCE ON COLLINEARITY
4. 1 Introduction
The exploratory analyses in Chapter 2 revealed that changing the
A-
structure of the covariance matrix of the random effects, 4, affected the
collinearity diagnostics. This was determined by deliberately constraining one
A-
of the elements of 4 in the estimation procedure, fitting the same model
(except for the constraint on 4) twice, and examining the results. In Chapter
3, the effect of a changing 4 also was seen. However, the only way in which
the matrix 4 changed was a result of a change in the number of variables used
in the model for a given set. Even then, the elements of 4 corresponding to
any specific pair of Z-variables used did not change. Over sets containing the
same variables in Z, but at different degrees of dependency (e. g., Wi' i =0,
...4), the same covariance matrix A was used. The was done in order to be able
to attribute changes in effects of collinearity to the tightening dependency
alone, rather to than a changing covariance as well. Thus, the only changes
in 4 in the experiment reported in Chapter 3 were essential changes that
resulted from changing the variables in the random effects; they were not
changes in the basic structure of 4.
In order to truly determine the effect of a different covariance structure
on the diagnostics, the entire experiment described in Chapter 3 was repeated
using a completely different 4; the same X and Z data were used in both
experiments. Then the effect of the change in the structure of 4 can be seen
by contrasting corresponding models in the two experiments.
In this chapter, the results of replicating the experiment with a different
4 are reported. Since all other aspects of the second experiment are the same,
they are not described again here. Only the results and a comparison to those
in Chapter 3 are provided.
4.2 The Mixed Model Experiment
In order to investigate the effect of a different covariance structure for
4 on the collinearity in the fixed effects of the mixed model, a different overall
4, analogous to 4(1) defined in (3.3.7), was specified and called 4 121• Then for
the second experiment, the same steps described in Chapter 3 in section 3.3.5
were repeated using the same values of rr =1 and V. = In. ' but with
1 0.8 0.7 0.6 0.5 0.40.8 1 0.7 0.6 0.5 0.40.7 0.7 1 0.7 0.6 0.50.6 0.6 0.7 1 0.7 0.60.5 0.5 0.6 0.7 1 0.70.4 0.4 0.5 0.6 0.7 1
(4.2.1 )
The comparison of corresponding models in the two experiments reveals the
behavior of the diagnostics with a different 4. 4 121 was chosen to be very
different from 4 111 so that if effects required a drastic change in order to be
159
seen, they would be. The difference between the two matrices is in the
off-diagonal elements; there was only one nonzero off-diagonal element in AI1I,
whereas all off-diagonal elements of A(2) are nonzero and are fairly large.
4.3 Results
The results of the second mixed model experiment using the new A (AI2»)
are shown in Appendix 4. They are directly comparable to the results in
Appendix 3 that were obtained using the original A (AI1I). (See Table 3.5 for the
menu of models.) This presentation of results follows the order of Chapter 3,
except that the baseline results are not re-presented per se since they are the
same for both experiments.
4.3.1 Simple Dependency: Two Variables
4.3.1.1 Adding Variables to Random Effects
For the data series X1{i} !5 [X Wi]' i =0, .u, 4, the results of adding
variables to the random effects can be seen in Appendix 4 (Sets 1-5) and
graphically, in Figure 4.1 (Compare to Appendix 3 (Sets 1-5) and Figure 3.1).
The baseline diagnostics are shown in row one; this row is the same in both
figures. The patterns found with A(2) are the same are those found previously
for A(1). As Wi increases, the condition index increases, reaching a value of
633 for W4 • The patterns in the !QM indicating one, two, or three variables
in Z are similar to that in the baseline row, but again, the values are smaller.
As before, when four or five variables are in the random effects, the condition
index remains about the same (3-5) as Wi increases.
160
..
Value of Condition Index for BX3 and Wiby Level of Wi and
Number of Variables in Random Effects
WI
COlUNEAR VAR....BLES: aX3, WI CII'Id BlCI. BX2. ZI
Figure 4.1 Condition Index for BX3 and Wi by Number of Random EffectsVariables
The lowest dependency is shown in column one for WOo As the number
of variables in Z increases, the condition index decreases, falling to a value of
3 when five variables are in Z. For each Wi (column), the pattern is similar;
there is a reduction in the largest condition index as variables are added to Z.
However, the reduction becomes more dramatic as the dependency increases.
The most dramatic change occurs for W4 ; the condition index is reduced from
277, when three variables are in Z, to 3 and 5, when four or five variables are
in Z.
Again, there appears to be a relationship between the number and the
161
nature of the variables in Z and the indication of collinearity by these
diagnostics. As found for 411), when variables involved in the constructed
dependency are included in the random effects, the impact of collinearity is
diminished.
4.3.1.2 Comparison to Experiment 1
A comparison of Figure 3.1 and Figure 4.1 reveals that for the X1 {i}
series the patterns found for 4 12) are the same as those found for 411). There
is a slight difference in the magnitude of the values; those in Figure 4.1 for
4 12) are somewhat higher. The higher values occur mainly after the introduction
of the collinear variables and at the tightest dependency, W 4 • However, the
differences are negligible given the magnitude of the condition indexes; i.e.,
there is very little actual difference between the value of 271 for three variables
in random effects in Figure 3.1 and the corresponding value of 277 in Figure
4.1.
4.3.2 Simple Dependency: Three Variables
4.3.2.1 Adding Variables to Random Effects
For the data series X2{i} !II! [X ~], j = 0, ..., 4, the results of adding
variables to the random effects can be seen in Appendix 4 (Sets 6-10) and
graphically, in Figure 4.2 (Compare to Appendix 3 (Sets 6-10) and Figure 3.2).
The baseline diagnostics are shown in row one. The patterns found with 4 12)
are the same found previously for 411). As Zj increases, the condition index
increases reaching a value of 640 for Z4' The patterns in the~ indicating
one to five variables in Z are similar to that in the baseline row, but the values
162
Value of Condition Index for BX 1, BX2 and Ziby Level of Zi and
Number of Voriables in Random Effects
o
Z\
COlUNEAR VARIABl£S: BX3. WI CNld BXl. BX2. ZI
Figure 4.2 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables
are smaller. When two or more variables are in the random effects, the
condition index values are substantially smaller as Zj increases.
The lowest dependency is shown in column one for Zo. As the number
of variables in Z increases, the condition index decreases, falling to a value of
3 when five variables are in Z. For each Zj (column), the pattern is similar;
there is a reduction in the largest condition index as variables are added to Z.
However, the reduction becomes more dramatic as the dependency increases.
The most dramatic change occurs for Z4; the condition index is reduced from
271, when one variable is in Z, to 66, when two variables are in Z. When
163
three or more variables are in Z, the condition indexes range from 30 to 37.
Again there appears to be a relationship between the number and the
nature of the variables in Z and the indication of collinearity by these
diagnostics. As was found for 4(1), when variables involved in the constructed
dependency are included in the random effects, the impact of collinearity is
attenuated.
4.3.2.2 Comparison to Experiment 1
A comparison of Figure 3.2 and Figure 4.2 reveals that for the X2{j}
series the patterns found for 4 121 are the same as those found for 4(1). There
is a slight difference in the magnitude of the condition indexes; those in Figure
4.2 for 4 121 are somewhat higher. The higher values occur mainly after the
introduction of the collinear variables and at the tightest dependencies. For Z4'
the condition indexes are larger for the second experiment when three or more
variables are in the random effects. For 4 121 values range from 30-36;
corresponding values for 4(1) range from 21-27. The differences are not great
enough to change any conclusions however.
4.3.3
4.3.3.1
Coexisting Dependency: Two Variables, Three Variables(Nonoverlapping)
Adding Variables to Random Effects
For the data series X3{i,j} iii [X Wi ~], i,j =0, ..., 4, the 25 sets (Sets
11-35) of diagnostics resulting from adding variables to the random effects can
be seen in Appendix 4. In Figures 4.3-4.7 (Compare to Figures 3.5-3.9.), the
largest condition index attributable to the dependency between BX3 and Wi is
plotted by level of Wi and number of variables included in the random effects,
164
..
..
separately for each level of~. In Figures 4.8-4.12 (Compare to Figures 3.10
3.14.), the largest condition index attributable to the dependency between
BX1, BX2, and ~ is plotted by level of Zj and number of variables included in
the random effects, separately for each level of Wi'
165
o
Value of Condition Index for BX3 and Wiby Lev" of WT and
Number of Variables in Random Effectsat ZO
WI
Figure 4.3 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Zo
Value of Condition Index for 8X3 and Wiby Lev.1 of WT Gnd
Number of Variables In Random Effects<It Zl
..
o
WI
..
Figure 4.4 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z,
166
o
Value of Condition Index for BX3 and Wiby Ley.. of WI and
Number of Variables in Random Effectsat Z2
Figure 4.5 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z2
Value of Condition Index for BX3 and Wiby Level of WT and
Number of Variables Tn Random Effectsat Z3
o
WI
Figure 4.6 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z3
167
o
Value of Condition Index for BX3 and WIby Level of WT and
Number of Variables in Random Effec:tsatU
Figure 4.7 Condition Index for BX3 and Wi by Number of Random EffectsVariables, at Z4
Value of Condition Index for BX1. 8X2 and Zjby Level of II and
Number of Vanables In Random Effecisat WO
o
Z1
CCWNrMl VAIIAlIl.IS: 113. WI _ 111. 112. ZI
Figure 4.8 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at Wo
168
o
Value of Condition Index for eX1, eX2 and Zjby Level of ZI and
Number of Variables in Random Effectsaf W1
ClllalI:AIl V.wAIIIS, 1ltJ. WI _ lin. IlC2, Zl
Figure 4.9 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at W 1
Velue of Condition Index for ex 1, eX2 end Zjby Level of ZI and
Number of Variables Tn Random EffectsatW2
o
Z1
COWIIL\Ilv-.s: 1ltJ. WI _ 1111. Ill2, Zl
Figure 4.10 Condition Index for aX1, aX2, and Zj by Number of RandomEffects Variables, at W 2
169
o
Value of Condition Index for 8X1, 8X2 and Zjby Leve' of ZI Clnd
Number of Variables in Random Effectsat W3
zr
CllllaIrAIl VAllAlllS, 113. WI _ IXl. IX2. Zl
Figure 4.1 1 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W3
Value of Condition Index for 8X1. 8X2 and Zjby Level of ZI Clnd
Number of Variables In Random EffectsCIt W4
o
Z1
Clll.lJIlfM V_LD, 113. WI _ IXI. ilia, ZI
Figure 4.12 Condition Index for BX1, BX2, and Zj by Number of RandomEffects Variables, at W4
170
The patterns in these plots are similar to those found previously for 4 111•
The nonoverlapping nature of the dependencies is again confirmed. In Figures
4.3-4.7, the pattern and values of the condition indexes are nearly the same in
each plot, i. e., regardless of the level of~. Further, they are very similar to
the values obtained when this dependency was considered independently in the
X1 {i} series; this can be seen in Figure 4.1. Again, the addition of variables to
the random effects, especially collinear ones, diminishes the impact of the
collinearity in X. Thus, the effect of adding variables to the random effects
when the dependencies are coexisting is the same as the effect when the
dependencies are considered separately.
Similarly, in Figures 4.8-4.12, the pattern and values of the condition
indexes are nearly the same in each plot, i. e., regardless of the level of Wi'
Further, they are very similar to the values obtained when this dependency was
considered independently in the X2{j} series; this can be seen in Figure 4.2.
And again, the addition of variables to the random effects, especially collinear
ones, diminishes the impact of the collinearity in X. Once more, we conclude
that the effect of adding variables to the random effects when the
dependencies are coexisting is the same as the effect when the dependencies
are considered separately.
4.3.3.2 Comparison to Experiment 1
A comparison of Figures 4.3-4.7 and Figures 3.5-3.9 reveals that for the
X3{i,j} series the patterns found for 4 (2) are again similar to those for 4 111 • For
both dependencies, the condition indexes for 4 (2) are slightly higher than for 4 111
171
when at least two variables are in the random effects. The discrepancies
between the series are greater for the tighter dependencies (W. and Z.).
Figures 3.9 and 4.7 give the condition indexes for the dependency
between BX3 and Wi at Z. for the two experiments. For Wo, there is very little
difference in the results; the CI decreases from 9 to 1 in Figure 3.9 and from
9 to 4 in Figure 4.7. For W., the CI decreases from 691 to 3 in Figure 3.9 and
from 691 to 4 in Figure 4.7. The main difference is for W3 and W. when two
or three variables are in the model; the Cis in Figure 4.7 are from 3 to 17 units
higher than those in Figure 3.9. However, none of these differences is large
enough to change any conclusions about the collinearity.
Figures 3.14 and 4.12 give the condition indexes for the dependency
between BX1, BX2, and ~ at W. for the two experiments. For lo, there is very
little difference in the results; the CI decreases from 9 to 1 in Figure 3.14 and
from 9 to 4 in Figure 4.12. For Z., the CI decreases from 703 to 21 in Figure
3.14 and from 703 to 34 in Figure 4.12. The main differences is for Z3 and Z.
when two or more variables are in the model; the Cis in Figure 4.12 are from
1 to 13 units higher than those in Figure 3.14. Again, none of these
differences is large enough to change any conclusions about the collinearity.
4.4 Summary of Results
The impetus for changing the covariance structure of A and repeating the
experiment came from the exploratory analyses in Chapter 2. Recall that actual
A
data were analyzed and that A was deliberately constrained. When either one
172
..
or all off-diagonal elements were set to zero, the condition index was reduced
dramatically. However, it was still high enough to indicate collinearity and the
variance deomposition proportions still pointed to the involved variables. InA
Chapter 2 with the actual data, 4 was estimated. In addition, all elements ofA
4 changed for any constraint imposed, but in unpredictable ways. Thus, when
the structure of 4 was changed and models were compared, it was not a pure
comparison. Because the same V was used, at least one of the models was
misspecified. However, the exploratory research reported in Chapter 2 did
determine that changing the covariance structure impacted the effect of
collinearity.
A major goal in this chapter was to compare "models" that were identical
except for their covariance structure 4, i.e., to compare models reported in this
chapter to those in Chapter 3. This was possible using the direct approach that
specified the required parameters with only 4 changed and did not involve a
dependent variable Y. In this case, the result was that the collinearity
diagnostics for the new covariance structure 4 121 are virtually identical to those
found for 4 111 •
173
CHAPTER V
DATA EXAMPLES OF THE IMPACT OF RANDOM EFFECTSON COLLINEARITV
5.1 Introduction
Analyses in Chapters 3 and 4 determined the behavior of the collinearity
diagnostics under controlled conditions. The advantage of this approach was
that effects of the contrived dependencies could be readily seen without their
being contaminated by changes in other parameters (A, I) of the mixed model,
parameters that would actually be expected to change when actual data are
analyzed. While this approach provided the information needed to clearly
assess the dependencies themselves, it nevertheless, by design, did not provide
a complete inquiry. The purpose of the analyses in this chapter is to combine
the knowledge gained from the previous inquires with an analysis of actual
data. In Chapter 2, components of the mixed model thought to impact
collinearity were explored using actual data. Now we revisit those same data
equipped with additional information.
The specific goal of the analyses is to investigate the impact of varying
the number and nature of the random effects in a model, while retaining the
same fixed effects. This was undertaken in two basic inquiries. First, using a
subset of the variables in the data set and a deliberately derived collinear
..
variable, a dependency was created in the spirit of the dependency of the data
series X1 of the experimental data, Le., one simple dependency involving two
variables. Second, by using another subset of the data and the deliberately
derived collinear variable, two overlapping dependencies were created. Recall
that the two dependencies of the X2 experimental series were non-overlapping.
The purpose of this was to determine whether the techniques used for simple
dependencies would hold in this new case; this is new territory that was not
explored previously in the experiment.
There are several advantages to fitting models and using actual data in
these analyses. Since the complete dynamics of the mixed model are
operating, the other components of the mixed model that may be impacted by
(or may impact) collinearity can be examined. These include changes in J1 and
A A
changes in IJ as well as the impact on V(JJ), as indicated by the diagnostics.
Further, the time to convergence of the model may be seen under different
conditions.
The only constraints remaining in the investigation as a whole are those
naturally associated with an examination of only one data set. 1) This data set
is unique with respect to the particular level of collinearity present. 2) Only
three basic continuous variables are in the model. 3) When the artificial
dependency is used, the data contain the basic underlying collinearity of the
three variables in addition to that of the created dependency. This is in
contrast to the experimental data of Chapter 3 and 4; since they were
generated randomly, there was no underlying collinearity. 4) Since J1 is
modelled from the data, it is an estimate and one that changes as variables in
175
the random effects are changed. This is in contrast to the fixed 4 used in the
experimental data. (However, both estimated and fixed 4's are explored in this
chapter.) 5) The number of subjects and the number of observations per
subject are specific to this data set. While these constraints are enumerated
as qualifications for the results, they are also reminders of the breadth of this
problem. No data set can serve all inquiries. Having previously examined the
behavior of this general type of data experimentally, we now determine
whether those findings apply to actual data of the same type.
5.2 The Data
The data for the examples in this chapter are the same as those used in
Chapter 2; see Section 2.7.1 for a full description. Recall that the dependent
variable was forced vital capacity (FVC) and that age, height, and weight were
independent variables. Because there is some question as to whether a straight
line is the best fit for these data, the square of height used in these analyses
as well. This increases the collinearity in the data.
5.3 Example 1: One Simple Dependency
In the experimental analyses of Chapters 3 and 4, two of the factors
determined to impact the diagnostics were 1) the number and nature of
variables in Z and 2) the structure of 4. It is of particular interest here to
determine whether, in actual data, the collinearity in the fixed effects
disappears as more variables are added to the random effects. The derived
176
variable, height2, was used in models along with the intercept and height.
Models were fit in which all three variables were retained as fixed effects and
all combinations of variables were used as random effects. In the estimation
process, the starting values for 4 for each pair of variables in Z were the same
for each run, but final values are not necessarily the same because they are
estimated. From model to model, 4 always changes as Z changes, but we are
not deliberately constraining 4.
For this experiment with the actual data, seven models were fit and are
summarized in Table 5.1. When only the intercept is in the random effects
component of the model, the impact of the collinearity in the fixed effects is
strong; the highest condition index is 164. When either collinear variable
(height or heighe) is in the random effects, the condition index is greatly
reduced. When they are included as either as single variables or as the only
variable along with the intercept, the highest condition index ranges from 47
52. When they are both included with or without the intercept, the condition
index is 78. Thus, we see that for a simple dependency involving two
variables, the pattern observed in the experimental data is only partially borne
out in the actual data. We would have expected the least collinearity when all
variables are in the random effects. The number of iterations and time to
convergence is also shown in Table 5.1; these appear to be positively
correlated with the degree of collinearity present in the models.
177
Table 5.1 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for Example 1
FIxed Effec:ta: v..... In X AMdom Effec:ta: v..... In Z High.. ItenltioneCI (Mlnutu)
INTERCEPT HEIGHT HEIGHT2 INTERCEPT 184 179(113)
INTERCEPT HEIGHT HEIGHT2 HEIGHT 47 18(18)
INTERCEPT HEIGHT HEIGHT2 HEIGHT2 61 18(12)
INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT 48 97(74)
INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT2 62 136(93)
INTERCEPT HEIGHT HEIGHT2 HEIGHT HEIGHT2 78 37, (29)
INTERCEPT HEIGHT HEIGHT2 INTERCEPT HEIGHT HEIGHT2 78 106(80)
While the objective of this analysis was to compare the behavior of the
diagnostics in actual data to those of the experiment, the results were
somewhat surprising. In order to put the actual data in the context of the
experiment, another approach was taken. Constraints were put on A and the
analyses just described was repeated; Le., the number of variables in Z was
varied. In this case the normal dynamics of the model are not allowed to
operate. Comparing the results of the two approaches permits one to see the
extent to which constraining A in actual data changes the impact of
collinearity.
For this approach, collinearity diagnostics were computed using the
procedures of Chapters 3 and 4, as described in Section 3.3.5. Recall that no
dependent variable was involved and that values for A, 02 and Vk were
A
specified and used to compute ~ = Zk A Zk' +o2vk. Then r', instead of r' was
178
K
used to compute (X'I -'X) = Ex: I;'X. + u2V.*-,
diagnostics were obtained from the matrix (X'F'X).
Finally, the collinearity
To implement the approach described above for this data example, the
values for 4 and 0'- used were the actual estimates obtained from the model in
which all three independent variables (intercept, height, and height2) were in the
random effects. When subsets of the variables were in Z, the corresponding
submatrices of the 4 were used. Thus, as with the experimental data, the
variation due to having a dependent variable and fitting a model is removed.
Using this approach, the collinearity diagnostics for the same seven
models were computed; results are summarized in Table 5.2 along with the
results produced by actual fitting of models. The estimated 4 and the
"constrained" 4 are necessarily the same for the case when all variables are in
the random effects. Thus, the cOllinearity for those models is the same
(CI = 78). Since the submatrices of 4 for the models with height and height2
are so similar to the 4 of the full model, the collinearity for those models also
is approximately the same (CI = 78). The condition index for the models with
only the intercept in the random effects is also the same for both the
constrained and estimated 4 and quite high (CI =164). However, when height
or height2 was included singly or with the intercept, the condition indexes for
the constrained models are much lower. These findings are similar to those
found for the experimental data for data series X1. Lack of reduction in all
models is probably due to the fact that the "constrained" matrix used is the
same as that for the actual 4 for the case when all variables are in the random
179
Table 5.2 Impact of Constraining Covariance of Random Effects 4 for Model with Intercept, Height, and Height2 inFixed Effects
Z Actual Coveri--=- CI. carwtr.ined Covariance CI.
INT 0.0000012154 164 0.000000715 164
HT 0.000001433 47 0.0000174 13
HT2 0.0000000000957 51 o.0000000013846 21
INT 0.00000074779 -0.0000002171 48 0.000000715 0.000000052049 13HT -0.0000002171 0.0000014259 0.000000052049 0.0000174
INT 0.00000083244 -0.000000003325 52 0.000000715 -0.000000000537 21HT 2 -0.000000003325 0.00000000009508 -0.000000000537 0.0000000013846
HT 0.0000172 -0.0000001478 78 0.0000174 -0.00000015 78HT 2 -0.0000001478 0.00000000136 -0.00000015 0.0000000013846
INT 0.000000715 0.000000052049 -0.000000000537 78 0.000000715 0.00000oo52049 -O. OOOOOOOOO537 78HT 0.000000052049 0.0000174 -0.00000015 0.000000052049 0.0000174 -0.00000015
HT 2 -0.000000000537 -0.00000015 0.0000000013846 -0.000000000537 -0.00000015 0.0000000013846
..
..
effects. In this case, it is not possible to show the expected reduction.
5.4 Example 2: Overlapping Dependencies
One type of dependency not explored in the experiments of Chapters 3
and 4 was that of overlapping dependencies. Recall that when two
dependencies occurred together in the experimental data, they were coexisting
(non-overlapping); this data set does not have non-overlapping dependencies.
Thus, the heretofore unexplored type of dependency in the mixed model was
examined in a manner similar to that for Analysis 1 above. In this analysis, the
new variable HEIGHT2 was used in models along with the intercept, age and
height. Models were fit in which all four variables were retained as fixed
effects and all combinations of variables were used as random effects. As for
Example 1, the starting values for Ii. for each pair of variables in Z were the
same for each run, but final values are not necessarily the same, due to the
estimation process.
For this second analysis, 15 models were fit and are summarized in Table
5.3. For each model, the degree of collinearity present is about the same.
There appear to be two overlapping dependencies, probably involving all the
variables. Thus, there was only slight or no reduction of collinearity when
subsets of variables were included in the random effects component of the
model. It is not known whether there would be reduction if additional
noncollinear variables were also in the model.
181
Table 5.3 Impact of Collinearity in Fixed Effects for Different Combinationsof Variables in Random Effects for Example 2
Fixed Effects: v...... In X ......... Effects: v...... InZ High.. h.......CI. (MInut..)
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT 18,66 136(82)
INTERCEPT AGE HEIGHT HEIGHT2 AGE 18,81 8(81
INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT 16,49 18(141
INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT2 16,64 8(81
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE 18,81 121(87)
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT 16,49 101(79)
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT2 16,63 118(88)
INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT 18, 71 18(14)
INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT2 20, 66 12(12)
INTERCEPT AGE HEIGHT HEIGHT2 HEIGHT HEIGHT2 16, 71 22(11 )
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT 16, 71 94(76)
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT2 20, 68 109(841
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT HEIGHT HEIGHT2 16, 71 86(71)
INTERCEPT AGE HEIGHT HEIGHT2 AGE HEIGHT HEIGHT2 18, 74 86(721
INTERCEPT AGE HEIGHT HEIGHT2 INTERCEPT AGE HEIGHT HEIGHT2 18, 76 78(68)
The procedure used in Example 1 to constrain 11 and determine its effect
was repeated for Example 2. For this example, values for 11 and if- were the
actual estimates obtained from the model in which all four independent
variables (intercept, age, height, and height2) were in the random effects.
When subsets of the variables were in Z, the corresponding submatrices of the
182
A were used. Thus, as with the experimental data, the variation due to having
a dependent variable and fitting a model is removed.
Using this approach, collinearity diagnostics for the same 15 models
were computed; results are summarized in Table 5.4 along with the results
produced by actual fitting of models. As before, the estimated A and the
"constrained" A are necessarily the same for the case when all variables are in
the random effects. Thus, the collinearity for those models is the same
(Cis = 18 and 75) (The computation for the constrained A produced Cis of 19
and 75). Also, since the submatrices for the models with age, height, and
heighe are so similar to that of the matrix for the full model, the collinearity for
those models also is similar. In contrast to the previous experiment, the Cis for
the model with only the intercept in the random effects, were much higher than
those based on the estimated A, 32 and 188 compared to 18 and 55. For all
other combinations of variables in the random effects, both Cis based on the
"constrained" A were greatly reduced compared to those based on the
estimated A. In fact, only one dependency is indicated. This finding is
intriguing since it is similar to the results found in the experiments for the non
overlapping dependencies. Perhaps further research will confirm that the
cancellation of collinearity found in the experimental data holds for overlapping
dependencies as well.
183
Table 5.4 Impact of Constraining Covariance of Random Effects 4 for Model with Intercept, Age, Height, and Height2 in Fixed Effects
Z Actuel COv8rtance Cia Conatr8tned COv8rtence Cia
INT 0.0235966 18 0.0000026843 3255 188
AGE 0.0005593 16 0.0046687 861 37
HT 0.0000016193 15 0.0000069018 949 22
HT2 0.0000000001091 15 0.0000000009589 654 24
INT 0.0000029204 -0.000008384 16 0.0000026843 0.0000105 8AGE -0.000008384 0.0005643 61 0.0000105 0.0046687 37
INT 0.0000029466 -0.0000006566 15 0.0000026843 -0.000000005879 9HT -0.0000006566 0.0000016382 49 -0.000000005879 0.0000069018 22
INT 0.000003105 -0.000000006503 15 0.0000026843 -0.000000005353 6Hy2 -0.000000006503 0.0000000001109 53 -0.000000005353 0.0000000009589 24
AGE 0.0025303 -0.000118 16 0.0046687 -0.000073 7HT -0.000118 0.0000071368 71 -0.000073 0.0000069018 34
AGE 0.0119744 -0.000004829 20 0.0046687 -0.000001416 8Hy2 -0.000004829 0.0000000020548 66 -0.000001416 0.0000000009589 35
HT 0.0000174 -0.0000001319 16 0.0000069018 -0.0000000272 6HT2 -0.0000001319 0.0000000011142 71 -0.0000000272 0.0000000009589 23
INT 0.0000026592 0.0000034256 -0.0000002325 16 0.0000026843 0.0000105 -0.000000005879 7AGE 0.0000034256 0.0025261 -0.000118 71 0.0000105 0.0046687 ·0.000073 34HT -0.0000002325 -0.000118 0.00000715 -0.000000005879 -0.000073 0.0000069018
INT 0.0000027295 -0.000014 0.0000000031676 20 0.0000026843 0.0000105 -0.000000005353 8AGE -0.000014 0.0119576 -0.000004825 66 0.0000105 0.0046687 -0.000001416 35Hy2 0.0000000031676 -0.000004825 0.0000000020545 -0.000000005353 -0.000001416 0.0000000009589
INT 0.0000026684 0.0000003939 -0.000000004052 16 0.0000026843 -0.000000005879 -0.000000005353 6HT 0.0000003939 0.0000175 -0.0000001329 71 -0.000000005879 0.0000069018 -0.0000000272 23Hy2 -0.000000004052 -0.0000001329 0.0000000011223 -0.000000005353 -0.0000000272 0.0000000009589
AGE 0.0046454 -0.000074 -0.000001394 18 0.0046687 -0.000073 -0.000001416 19HT -0.000074 0.000006831 -0.0000000261 74 -0.000073 0.0000069018 -0.0000000272 75HT2 -0.000001394 -0.0000000261 0.0000000009403 -0.000001416 -0.0000000272 0.0000000009589
INT 0.0000026843 0.0000105 -0.000000005879 -0.000000005353 18 0.0000026843 0.0000105 -0.000000005879 -0.000000005353 19AGE 0.0000105 0.0046687 -0.000073 -0.000001416 75 0.0000105 0.0046687 -0.000073 -0.000001416 75HY -0.000000005879 -0.000073 0.0000069018 -0.0000000272 -0.000000005879 -0.000073 0.0000069018 -0.0000000272HT2 -0.000000005353 -0.000001416 -0.0000000272 0.0000000009589 -0.000000005353 -0.000001416 -0.0000000272 0.0000000009589
,,.
5.5 Summary
In actual data models, the impact of collinearity in the fixed effects was
diminished when variables were added to the random effects. However the
patterns were not always the same as those seen previously in the
experimental data of Chapters 3 and 4. There are several reasons for the
departure of these findings from those found with the experimental data. The
first reason stems from the differing types of collinearity in the artificial and real
data. The basic variables in the experimental data set were free of underlying
collinearity; one dependency was created and five variables were in the fixed
effects of each model. Only two or three variables were collinear. In contrast,
in the actual data set, the basic variables appeared to be involved in one or
more dependencies; one additional dependency was created and three or four
variables were in the fixed effects for these models. So all variables in a model
were collinear. Thus, even though the dependencies in the actual data were
similar to those in the experimental data, the actual models were different with
respect to the numbers of variables included and the levels of collinearity
present.
Another difference between the experimental models and the actual
models was the structure of the covariance matrix A. The covariance in the
experimental models had equal diagonal elements and only two nonzero off
diagonal elements. In contrast, the covariance in actual data does not have
equal diagonal elements and the off-diagonal elements are not only not zero,
but are a mixture of positive and negative values.
185
A third reason for the differences found is related to the estimation
process. The experimental ·models· were not estimated, i.e., a dependent
variable was not involved and the same pair-wise elements of the covariance
were used for each model. In contrast, parameters for the actual data were
estimated; 4 was different for each model. The ·constrained· 4 subsets
examined the extent to which this was an issue. Thus, the dynamics of the
estimation process and the additional variability of the dependent variable
obscure the patterning in the actual data. However, it was determined that for
a dependency similar to that found in the experiment, adding collinear variables
to the random effects does reduce the impact the collinearity, but not as much
as was found in the experiments. Further research is needed to clarify the
findings for the overlapping dependencies, though the diagnostics tended to
behave in a manner similar to those for the simple dependencies examined.
186
CHAPTER VI
EXAMPLE MIXED MODEL DATA ANALYSISUSING COLLINEARITY DIAGNOSTICS
6.1 Introduction
Until now, this dissertation has focused on determining the behavior of
the diagnostics in the presence of several types of dependencies. Performance
of the diagnostics has been characterized extensively for two types of
dependencies (simple, two and three variables involved and coexisting, five
variables involved) in experimental data. Performance of the diagnostics in
actual data has been examined for a simple dependency (three variables
involved) similar to the simple dependency in the experimental data and for an
overlapping dependency (four variables involved). It has been demonstrated
that the diagnostics can be used confidently to detect the presence of
collinearity in the mixed model context. Even though the particular impact of
adding variables to the random effects may be specific to a given data set, the
general pattern of behavior has been established in both experimental and
actual data.
The purpose of this chapter to suggest and illustrate a strategy for
incorporating the diagnostics into an overall approach for model fitting. These
analyses go beyond the diagnostic behavior to demonstrate a realistic data
analysis in which the collinearity diagnostics are included in the process of
refining the model.
6.2 Model Fitting Strategy
A general strategy for mixed model fitting has been proposed previously
by Helms (1993). That strategy is expanded here by using the collinearity
diagnostics in decisions about variable inclusion and deletion. The steps
involved in the procedure, which are described below, begin with variable
selection for the fixed effects. This is because an error in specifying the
covariance structure has relatively little impact on the fixed effects while the
reverse is not true.
Step 1
a) Put all variables of interest in the fixed effects.
b) Assume a simplified covariance structure for 4, e.g., only have random
subject intercepts as a random effect.
c) Fine tune the fixed effects. Delete nonsignificant variables, keeping the
significance level liberal (0.10 or 0.15). Alternatively, delete highly
collinear variables. (Recall that collinearity shows up best when only the
intercept is in the random effects of the model.)
Step 2
a) Use the fixed effects decided on from Step 1.
b) Fine tune the random effects. Add variables one-by-one to the random
effects. Take the difference in -2 log likelihood between two models
188
with the same fixed effects; this indicates the significance of the
additional parameters in the covariance 4. The -2 log likelihood ratio has
approximately a Chi-square distribution with degrees of freedom equal
to the difference between the number of parameters in the two models.
(This is not formal hypothesis testing, but rather is used as a guide to
variable selection.)
Step 3
a) Use the random effects from Step 2.
b) Fine tune the fixed effects. Consider deleting variables that have
become nonsignificant in Step 2 or those that contribute greatly to
collinearity. Also consider inclusion of additional variables, as logic
suggests.
Step 4
a) Compare results from Steps 1 and 3.
b) If there is a big difference in conclusions, repeat Steps 2 and 3, with
obvious modifications.
6.3 Model Fitting Example
Once again, the data for this example are the same as those used in
Chapter 2; see Section 2.7.1 for a full description. Recall that the dependent
variable was forced vital capacity (FVC) and that age, height, and weight were
independent variables in both the fixed and random effects. The additional
variable height2 also was used for this example. The strategy described above
189
was applied to these data. Each step in the process is described in detail;
tables giving the details of· each model are provided. The results are
summarized in Table 6.10.
The first model considered in Step 1 included all variables (intercept, age,
height, weight, height2) in the fixed effects and only the intercept in the random
effects. Table 6.1 provides the model parameter estimates and collinearity
diagnostics. The objective of fitting this model was to determine important
fixed effects. The results show that all variables except weight were highly
significant (p <0.001); weight was nonsignificant (p =0.296). The two small
eigenvalues (0.010 and 0.0008) of the scaled version of the (>rr'X) and the
two high condition indexes (21 and 74) indicate the presence of collinearity in
these data. There are two dependencies with the intercept, age, height and
height2 involved in one or both. This is determined by summing over the
variance decomposition proportions for the two highest condition indexes.
The choice of a second model to fit could be based on significance levels
for fixed effects; weight would be deleted and the model refit. Alternatively,
the choice could be made with a view toward eliminating the collinearity; one
of the collinear variables would be deleted and the model refit. In this case, the
decision was based on the collinearity since it was considered more serious;A
the variance of the /l seemed quite unstable. Heighe was deleted since it is
certainly collinear with height.
The second model fit in Step 1 included the intercept, age, height, and
weight in the fixed effects and only the intercept in the random effects. Table
6.2 provides the model parameter estimates and collinearity diagnostics. The
190
..
Table 6.1 Results for Model 1 in Step 1 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept 2.586 0.401 6.46 <0.001Age 0.198 0.020 9.92 <0.001Height -0.047 0.006 -7.30 <0.001Weight -0.002 0.002 -1.05 0.296Height2 0.0002 0.00003 7.57 <0.001
A
4, Estimate of Covariance Matrix of the Random Effects
Estimate of Correlation Matrix of the Random EffectsI Intercept
...tr=0.025
h223 7256114722
1425481320527
19876447
31381 16977317555641 223457457
6153540 0.00000000283167543 . 0.0000000011
0.000000000045
Collinearity Diagnostics
Eigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT HT2
4.285 2.07 1 0.000 0.001 0.000 0.003 0.0000.630 0.79 3 0.004 0.003 0.000 0.025 0.0000.075 0.27 8 0.004 0.096 0.000 0.467 0.0020.010 0.10 21 0.047 0.821 0.008 0.113 0.1450.0008 0.03 74 0.944 0.079 0.992 0.393 0.853
significant fixed effects in this model are age and weight. Now, there is only
one small eigenvalue (0.003) and one high condition index (34). Thus, as
expected, one of the dependencies has been eliminated. It appears, from the
191
variance decomposition proportions, that the intercept, age and height are
involved in the remaining dependency. The choice of a third model to fit was
based on both the significance levels and the collinearity. Height was deleted•
from the fixed effects since it was both nonsignificant and collinear with age.
Table 6.2 Results for Model 2 in Step 1 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD T P-VAlUE
ERRIntercept 0.052 0.239 0.22 0.827Age 0.203 0.020 9.98 <0.001Height -0.00288 0.003 -0.95 0.344Weight 0.010 0.002 6.41 <0.001
A.
4, Estimate of Covariance Matrix of the Random Effects
Intercept 0.0195I Intercept
Estimate of Correlation Matrix of the Random EffectsI InterceptIntercept 1.00
A.
a2=0.0281533 9128
119312178830
1496219
23842583
39460570122
68801373164673
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.401 1.84 1 0.001 0.002 0.000 0.0090.534 0.73 3 0.010 0.005 0.001 0.0770.063 0.25 7 0.010 0.159 0.001 0.8760.003 0.05 34 0.979 0.834 0.998 0.038
192
..
The third model fit in Step 1 included the intercept, age, and weight in
the fixed effects and only the intercept in the random effects. Table 6.3
provides the model parameter estimates and collinearity diagnostics. For this
model, all fixed effects are significant. No collinearity is indicated since the
highest condition index is 6, though there are two high VDPs associated with
the condition index (for age and weight). Now, both criteria (significance level
Table 6.3 Results for Model 3 in Step 1 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD T P-
ERR VALUEIntercept -0.171 0.035 -4.88 <0.001Age 0.185 0.009 20.85 <0.001Weight 0.010 0.002 6.39 <0.001
A.
4, Estimate of Covariance Matrix of the Random EffectsI Intercept
Estimate of Correlation Matrix of the Random Effects
I Intercept
A.
02=0.0281601 9542
12176941252
580751
3211191
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE WT2.469 1.57 1 0.055 0.016 0.0190.471 0.69 2 0.764 0.014 0.0630.060 0.24 6 0.181 0.970 0.918
193
and lack of collinearity) are satisfied and we assume that the intercept, age and
weight are the best variables for the fixed effects of this model. This
concludes Step 1 of the model fitting.
For Step 2, the fixed effects from Step 1 are used and the random
effects are fine tuned. Any of the original variables could have been added to
the random effects component at this stage, however it seemed logical to
chose from those already in the fixed effects. Age was chosen first, though
weight could have been. Thus, the first model fit in Step 2 contained the
intercept, age, and weight in the fixed effects and the intercept and age in the
random effects. Table 6.4 provides the model parameter estimates and
collinearity diagnostics. All fixed effects are still significant and no collinearity
is present (the highest condition index is 7). The variance decomposition
proportion for weight associated with the highest condition index has
diminished, possibly indicating any impact of dependency between age and
weight has diminished. The difference between the -2 log likelihood statistics
for this and the previous model (123.6) indicates that age is an important
random effect ("p" <0.001). The statistic has two degrees of freedom due to
the additional covariance parameters for age and the covariance of the intercept
and age. The conclusion based on these results was to retain age and add
weight to the random effects.
194
..
Table 6.4 Results for Model 1 In Step 2 of Model Fitting
Parameter Estimates of Fixed Effects
• VARIABLE BETA STD ERR T PVALUE
InterceptAgeWeight
-0.1410.1740.011
0.0530.0140.002
-2.6512.79
4.51
0.008<0.001<0.001
A
/i, Estimate of Covariance Matrix of the Random EffectsIntercept Age
InterceptAge
0.0639 -0.00850.0016
Estimate of Correlation Matrix of the Random EffectsIntercept Age
InterceptAge
1.00 -0.851.00
A
a2=O.0201643 8682
5846134731
236933
1132662
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE WT2.742 1.66 1 0.025 0.012 0.0180.198 0.44 4 0.633 0.006 0.3090.060 0.25 7 0.342 0.982 0.673
The second model fit in Step 2 contained the intercept, age, and weight
in the fixed effects and the intercept, age, and weight in the random effects.
Table 6.5 provides the model parameter estimates and collinearity diagnostics.
All fixed effects are still significant and no collinearity is present (the highest
condition index is 9). The variance decomposition proportion for weight
195
Table 6.5 Results for Model 2 in Step 2 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p
VALUEInterceptAgeWeight
-0.1500.1590.015
0.0450.0140.004
-3.3511.494.32
0.001<0.001<0.001
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight
InterceptAgeWeight
0.0351 -0.0005 -0.00070.0015 -0.0003
0.0001
Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight
InterceptAgeWeight
1.00 -0.06 -0.361.00 -0.84
1.00
A.
a2=O.0201796 9640
7374839012
284365
1182490
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE WT2.766 1.66 1 0.031 0.009 0.0080.197 0.44 4 0.963 0.063 0.0470.037 0.19 9 0.006 0.928 0.944
associated with the highest condition index has increased, possibly indicating
an increase in the dependency between age and weight. The difference
between the -2 log likelihood statistics for this and the previous model (8.7)
indicates that weight is a moderately important random effect ("p" =0.034).
196
•
The statistic has three degrees of freedom due to the additional covariance
parameters for weight and the covariances of the intercept and weight and of
age and weight. The conclusion based on these results was to retain both age
and weight in the random effects. This concludes Step 2 of the model fitting.
For Step 3, the random effects from Step 2 are used and the fixed
effects are revisited. Since all fixed effects were significant in the previous
model (Table 6.5), none were deleted. Of the two original variables remaining
as candidates for the fixed effects, it seemed logical to add height before
height2 • Thus, the first model fit in Step 3 contained the intercept, age,
weight, and height in the fixed effects and the intercept, age, and weight in the
random effects. Table 6.6 provides the model parameter estimates and
collinearity diagnostics. In this model, all fixed effects except the intercept are
significant. (Recall that for Model 2 in Step 1 when these four variables were
included as fixed effects and only the intercept was a random effect, height
was not significant.) However, the addition of height has created collinearity.
There is one moderately high condition index (38) and one small condition index
(10), indicating two possible dependencies. The corresponding variance
decomposition proportions indicate that all variables are involved in one or both
dependencies. Based on these results, it is not logical to add heighe to the
fixed effects. Thus, Step 3 is of the model fitting is concluded.
197
Table 6.6 Results for Model 1 in Step 3 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p-
VALUEIntercept 0.370 0.257 1.44 0.153Age 0.198 0.022 8.88 <0.001Height -0.007 0.003 -2.05 0.043Weight 0.017 0.004 4.62 <0.001
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Weight
•
InterceptAgeWeight
0.0309 0.0021 -0.00110.0014 -0.0004
0.0001
Estimate of Correlation Matrix of the Random EffectsIntercept Age Weight
InterceptAgeWeight
1.00 0.31 -0.581.00 -0.89
1.00
A
02=0.0201781 9138
72561199416
1193333
23559335
39147289150
49987701245103
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.704 1.92 1 0.001 0.002 0.000 0.0040.255 0.51 4 0.016 0.030 0.001 0.0410.037 0.19 10 0.000 0.308 0.002 0.8720.003 0.05 38 0.984 0.660 0.997 0.084
For Step 4, previous results are re-examined and additional models fit as
believed to be necessary. The results in Step 3 indicate that height probably
198
should not have been added to the fixed effects. Thus, one option would be
to stop the process at this point and consider the Model 2 of Step 2 to be the
final model. However, we have learned from the results in Chapters 3 and 4
that adding a collinear variable to the random effects may alleviate the impact
of collinearity in the fixed effects. Because of this finding, height was added
to the random effects. Thus, the first model fit in Step 4 contained the
intercept, age, weight, and height in the fixed effects and the intercept, age,
weight, and height in the random effects. Table 6.7 provides the model
parameter estimates and collinearity diagnostics. In this model, height is no
longer a significant fixed effect. However, height is an important random
effect. The difference in -2 log likelihoods between this model and the previous
model is 34.5 with 4 degrees of freedom (due to additional covariance
parameters) ("p" <0.001). The conclusion based on these results was to delete
height as a fixed effect, but to retain it as a random effect.
199
Table 6.7 Results for Model 1 in Step 4 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T p-
VALUE
Intercept -0.127 0.359 -0.35 0.724Age 0.153 0.030 5.10 <0.001Height 0.00002 0.005 0.00 0.997Weight 0.016 0.004 4.00 <0.001
A.
A, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
..
InterceptAgeHeightWeight
2.3081 0.1801 -0.0313 0.00970.0154 -0.0024 0.0004
0.0004 -0.00010.0002
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.96 -0.99 0.491.00 -0.95 0.27
1.00 -0.541.00
•
A.
02=0.0193903 16799
96970408783
1928606
44029265
72718395852
82161441712904
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT WT3.795 1.95 1 0.000 0.001 0.000 0.0020.176 0.42 5 0.005 0.020 0.000 0.0340.027 0.17 12 0.000 0.169 0.000 0.7550.006 0.02 78 0.995 0.810 0.999 0.209
200
The second model fit in Step 4 contained the intercept, age, and weight in the
fixed effects and the intercept, age, height, and weight in the random effects.
Table 6.8 provides the model parameter estimates and collinearity diagnostics.
Table 6.8 Results for Model 2 in Step 4 of Model Fitting
Parameter Estimates of Fixed EffectsVARIABLE BETA STD ERR T P-VALUEIntercept -0.127 0.035 -3.59 <0.001Age 0.153 0.014 11.33 <0.001Weight 0.016 0.004 4.62 <0.001
A
4, Estimate of Covariance Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
2.3245 0.1817 -0.0315 0.00960.0155 -0.0024 0.0004
0.0004 -0.00010.0002
Estimate of Correlation Matrix of the Random EffectsIntercept Age Height Weight
InterceptAgeHeightWeight
1.00 0.96 -0.99 0.491.00 -0.95 0.27
1.00 -0.531.00
A
x'r'xA
02=0.0193756 16267
9519670128
386556
1665503
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE WT2.813 1.68 1 0.024 0.007 0.0060.160 0.40 4 0.899 0.082 0.0290.027 0.17 10 0.076 0.911 0.965
201
All fixed effects are still significant. Height is an important random variable.
The difference in -2 log likelihoods between this model and Model 2 of Step 2
(Table 6.5) is 37.0 with 4 degrees of freedom (due to additional covariance
parameters related to height) ("p" <0.001). The conclusion was to stop the
model fitting process; criteria based on both significance levels and collinearity
are met.
Two routes toward obtaining a final model seemed apparent after fitting
Model 1 in Step 1. Recall that the alternative chosen involved deleting a
collinear variable. The road not taken involved eliminating weight as a fixed
effect because it was not significant. To determine the results had that choice
been made, weight was deleted and the model refit. The reduced model
contained the intercept age, height, and height2 as fixed effects and only the
intercept as a random effect. Table 6.9 provides the model parameter
estimates and collinearity diagnostics. All fixed effects are significant, however
there are two collinearities present; the highest condition indexes are 18 and
54. Thus, had this path been taken, the next step would have been to delete
one of the collinear variable, height2 • This probably would have led us back to
the model ultimately chosen. A summary of the entire model fitting strategy
is presented in Table 6.10.
202
..
•
Table 6.9 Results for Alternative Model 2 in Step 1
Parameter Estimates of Fixed EffectsVARIABLE BETA STD T p-
ERR VALUEIntercept 2.364 0.346 6.82 <0.001Age 0.195 0.199 9.82 <0.001Height -0.042 0.005 -8.18 <0.001Height2 0.0002 0.00002 9.99 <0.001
A
4, Estimate of Covariance Matrix of the Random EffectsI Intercept
Estimate of Correlation Matrix ,of the Random EffectsI Intercept
A
X'I"'X
A
a2=0.0251294 7682
117093150849
1369265
20842101
17969880229132500
0.0000000030.00000000005
Collinearity DiagnosticsEigen- Singular Condition Variance Decompositionvalue Value Index Proportions
INT AGE HT HT2
3.542 1.88 1 0.000 0.001 0.000 0.0000.445 0.67 3 0.008 0.014 0.000 0.0010.012 0.11 18 0.028 0.876 0.003 0.2570.0012 0.03 54 0.964 0.108 0.996 0.741
203
Table 6.10 Summary of Model Fitting
-2 logFixed Random Uke-
Step Model Effects Effects lihood Results/Conclusions
1 1 INT- INT -364.5 ResultsAGE-HEIGHT- 1) WEIGHT is not significant.WEIGHTHEIGHT2
• 2) INT, AGE, HEIGHT and HEIGHT2 areinvolved in one or more dependencies.
Conclusions
1) Delete WEIGHT from fixed effects;retain INT in random effects (ignorescollinearity) .
2) Presume HEIGHT2 is culprit and delete itfrom fixed effects (ignores nonsignificanceof WEIGHT); retain INT in random effects.[This option was chosen.]
1 2 INT" INT -311.8 ResultsAGE-HEIGHT 1) HEIGHT is not significant.WEIGHT-
2) HEIGHT is collinear with AGE and INT.
Conclusions
Delete HEIGHT from fixed effects; retainINT in random effects.
1 3 INT- INT -310.9 ResultsAGE-WEIGHT" 1) All fixed effects are significant.
2) No collinearity is present. Even thoughtwo variables have high VDPs, there is nohigh CI.
Conclusions
Retain all fixed effects; add variables torandom effects. AGE is picked next,though WEIGHT could have been.
204
•
"
..
Table 6.10 Summary of Model Fitting
-2 logFixed Random Like-
Step Model Effects Effects lihood Results/Conclusions
2 1 INT· INT -434.5 ResultsAGE· AGEWEIGHT· 1) All fixed effects are still significant.
2) No collinearity is present. The VDP forweight has decreased, perhaps indicatingless collinearity between age and weightthan for previous model. Still there is nohigh CI.
3) AGE is an important random effect(difference in likelihoods between previousmodel and this one is 123.6 with 2 df,P<O.OOl) (df due to 4 parameters AGEand covarllNT, AGE)).
Conclusions
Retain AGE and add WEIGHT to randomeffects.
2 2 INT· INT -443.2 ResultsAGE· AGEWEIGHT" WEIGHT 1) All fixed effects are still significant.
2) No collinearity is present. The VDP forweight has increased, perhaps indicatingmore collinearity between age and weightthan for previous model. Still there is nohigh CI.
3) WEIGHT is a moderately importantrandom effect (difference in likelihoodsbetween previous model and this one is8.7 with 3 df, P=O.034) (df due to 4parameters WEIGHT, covarllNT, WEIGHT),and covar (AGE, WEIGHT)).
Conclusions
Retain all random effects; go back toexamination of fixed effects.
205
Table 6.10 Summary of Model Fitting
-2 logFixed Random Uke-
Step Model Effects Effects Iihood Results/Conclusions
3 1 INT- INT -446.1 ResultsAGE- AGEHEIGHT- WEIGHT 1) All fixed effects, except the intercept,WEIGHT- are significant.
2) INT, AGE and HEIGHT are involved inone dependency; high CI is 38. There ispossibly another dependency indicated bythe second CI (1 0). Both Cis are lowerthan found previously.
Conclusions
Retain all fixed effects. Fine tune randomeffects again. Add HEIGHT to the randomeffects.
4 1 INT- INT -480.6 ResultsAGE- AGEHEIGHT WEIGHT 1) INT and HEIGHT are not significantWEIGHT- HEIGHT fixed effects.
2) HEIGHT is an important random effect(difference in likelihoods between previousmodel and this one is 34.5 with 4 df,P<0.001) (df due to 4 parametersHEIGHT and covar(lNT, HEIGHT),covar(AGE, HEIGHT), and covar(WEIGHT,HEIGHT)).
Conclusions
Delete HEIGHT as a fixed effect; retainHEIGHT as a random effect.
206
,
..
Table 6.10 Summary of Model Fitting
-2 logFixed Random Like-
Step Model Effects Effects lihood Results/Conclusions
4 2 INT- INT -480.2 ResultsAGE- AGEWEIGHT- WEIGHT 1) All fixed effects are significant.
HEIGHT2) All random effects are important.HEIGHT is still important random effecteven when HEIGHT is not in the model asa fixed effect (difference in likelihoodsbetween Model 2 of Step 2 and this modelis 37 with 4 df, P<0.0011 (df due to 4parameters HEIGHT and covar(lNT,HEIGHT), covar(AGE, HEIGHT), andcovar(WEIGHT, HEIGHT)).
Conclusions
Stop the process. This is the final model.
• p<0.05
Applying the model fitting strategy to these data has resulted in the
selection of a final model, shown in Table 6.8, that includes the intercept, age
and weight in the fixed effects and the intercept, age, height and weight in the
random effects. All effects are important predictors of the variation in forced
vital capacity (FVC) in these black females aged 2-15 years and weighing 15-
120 kg over their entire time in the study. The slopes of both age (0.153) and
weight (0.016) are positive indicating that FVC increases with increasing age
(for fixed weight) and weight (for fixed age). These results are presented
graphically in Figures 6.1 and 6.2 for age and weight, respectively. Height is
not a fixed effect, thus its line has zero slope, as shown in Figure 6.3. The
lines in the graphs were produced using the following equations:
207
+ 11-zSge
A
For the graphs of ElY) versus age
Estimated population regression line:
Estimated individual regression line:
( 11, + 113weigh f)Yk (age,welght,helght)] =
+ (ak1 + ak4welght) + auhelght) + ak:Pge
A
For the graphs of ElY) versus weight
Estimated population regression line:
Estimated individual regression line:
Yk (age,weight,helght)] =(11, + 11~ge)
A
For the graphs of ElY) versus height
Estimated population regression line:
208
Estimated individual regression line:
'fir (age,welght,height)] =( ~, + ~~ge + ~3welght)
+ (air 1 + a/r2age + alr4welght) + a/r3height
Portions of the correlation matrix of the random effects, and the Figures,
aid in interpreting the results. The correlation between the random intercepts
and ages in this model is high and positive (0.96). This indicates that subjects
with positive intercept increments also have higher than average random slopes
for age and vice versa. In other words, subjects who start with high values of
FVC tend to have even higher values as they age. The correlation between the
intercept and weight is positive (0.49). This indicates that subjects with
greater than average intercept increments have higher than average random
slopes for weight. In other words, subjects who start with high values of FVC
tend to have even higher values as they gain weight. The correlation between
the intercept and height is very high, but negative (-0.99). This indicates that
subjects with negative intercept increments have higher than average random
slopes for height and vice versa. In other words, subjects who start with low
values of FVC tend to "catch up" as they grow taller.
209
Pulmonary Function StudyI~--------------------'
~l
j~
JiILl
2 I 4 • • 7 • • .. l' 12 II .4 II II
Age In Yearw
Figure 6.1 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Weight, PlottedAgainst Age
Pulmonary Function StudyI.r---------------------,
Ie • • • • • ~ • • _ ,.. I.
Figure 6.2 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Weight, PlottedAgainst Weight
210
Pulmonary Function Study.~------------------,
~I
J~
Jia.. 1
• • I. ,. •• t. '. 1. •• IJII
H"llhf 'n om
Figure 6.3 Values of FVC Predicted from Final Mixed Model withIntercept, Age and Weight in X and in Z at Mean Age, PlottedAgainst Height
The collinearity diagnostics shown in Table 6.8 indicate that this model
is free from serious effects of collinearity in the fixed effects. The highest
condition index is 10, indicating that only a weak dependency exists. The
variance decomposition proportions for both age (O.911) and weight (O.965)
are high. However, as noted repeatedly, Belsley (1991) states that a
"degrading" collinearity is indicated by both a high condition index and
corresponding high variance decomposition proportions for two or more
variables. High variance decomposition proportions in the absence of high
condition indexes may indicate that collinearity in the fixed effects exists, but
that it does not adversely impact the model.
As described in Chapter 2 (Section 2.7.1), the data set used in this
dissertation is a subset of a data from larger investigation that was analyzed
211
previously by Strope and Helms (1984) and Fairclough and Helms (1984). Both
of the earlier analyses were larger in scope and addressed different issues than
those focused on here, though both are directed at modeling lung function
using longitudinal parameters, primarily height. Since collinearity diagnostics
have only now been developed in this dissertation, they were not available as
tools in the earlier modeling. Clearly, they have been useful in the analysis
described in this chapter.
6.4 Summary
In this chapter, it has been demonstrated that collinearity diagnostics can
be incorporated reasonably into an existing mixed model fitting procedure. A
clearly collinear set of variables was used in the initial model. When collinear
variables were removed prior to assessing the significance of the fixed effects,
an interim model was found that was free of collinearity. When random effects
were fine tuned and the fixed effects revisited, it was found that adding more
fixed effects again produced collinearity. A final model was obtained that was
both free of collinearity and in which all fixed and random effects were
significant. Thus, use of the diagnostics in model fitting has improved the
process.
212
•
.'
CHAPTER VII
SUMMARY AND RECOMMENDATIONS FORFUTURE RESEARCH
7.1 Summary of Research
It is well known that collinearity in the independent variables of the
General Linear Univariate Model (GLUM) causes variances of the least squares
parameter estimates to be unduly large. This can affect estimation, hypothesis
testing, and prediction. Methods for detection of collinearity in the GLUM are
well established. However, no previously published research has been directed
toward extending these methods for use in the mixed model. In the mixed
A
model, collinearity in the fixed effects arises from the ill-conditioning of (r' /2X)
A A A A
and (X'r'x), leading to inflated elements of V(JJ) = (X'r'xr'.
The objective of this dissertation was to develop a method for assessing
collinearity in the fixed effects of the mixed model by expanding a strategy
currently used for the GLUM. In Chapter 1, the literature on both collinearity
and the mixed model was reviewed in order to provide a background for this
new research on the combination of these topics. In Chapter 2, mixed model
diagnostics were defined and a procedure for their use was specified. GLUM
and mixed model diagnostics were applied to a data set with three collinear
variables; they appeared to perform similarly. For the mixed model, initial
impressions were that variation in two factors, the number of variables in the
random effects and constraints on the covariance of the random effects,
produced different collinearity diagnostics for models with the same variables
in the fixed effects. Since these findings were specific to this data set, they
were used as a point of departure for designing the subsequent research.
In order to generalize the behavior of the diagnostics, they were applied
to experimental data with known predetermined collinearities with increasingly
tighter dependencies. In Chapter 3, two types of dependencies were created,
a simple dependency involving two variables and a coexisting dependency
involving three variables. For comparison, GLUM and mixed model diagnostics
were computed for the same dependencies, though the data for the two types
of models were necessarily different.
The focus of Chapter 3 was the behavior of the diagnostics when
different random effects were in models containing the same fixed effects, for
a given type and level of dependency and a specified covariance matrix A. In
practice, the assessment of collinearity would be made after model fitting since
.....it requires the estimated matrix J:. However, actual model fitting in this case
would have confounded the experiment. Thus a A and 02 were specified and
used to compute ~ which was used to compute r' and (X'r'x), from which
the diagnostics were computed directly. This enabled a pure assessment of the
impact of the varying the random effects, Le., A was always the same for each
pair of variables in the experiment. Eliminated was the "noise" that might have
been introduced due to the other component of the variance of V, crvk , and the
manner in which the estimation process would necessarily change the Ii. for
214
..
each model fit. The results indicated that adding variables, especially collinear
variables, to the random effects of the mixed model can greatly attenuate the
impact of the collinearity in the fixed effects.
The initial results of Chapter 2 also indicated that constraints on the
covariance of the random effects produced different collinearity diagnostics for
models with the same variables in the fixed effects. In order to determine the
effect of a different covariance structure on the diagnostics, the entire
experiment of Chapter 3 was repeated in Chapter 4 using a different covariance
matrix 4. The results indicated that the diagnostics, and therefore, the
conclusions, were virtually identical for these two covariance structures.
Several findings emerged from these experiments involving two types of
dependencies (simple, two variables involved and coexisting, three variables
involved). First, it was demonstrated that the diagnostics could be used with
confidence to pinpoint designed collinearities, Le., the procedure works in the
mixed model context for these dependencies. This was an important finding
because it is not general knowledge. Second, it was discovered that adding
variables to the random effects essentially cancelled the collinearity in the fixed
effects. This was unexpected and somewhat counterintuitive and also was not
previously known. Third, these results held for a different covariance structure
4. However, it is not yet known whether the pattern will hold for a variety of
covariance structures.
Analyses in Chapters 3 and 4 determined the behavior of the collinearity
diagnostics under controlled conditions. The advantage was that effects of the
contrived dependencies could be readily seen without being "contaminated" by
215
the estimation process. The disadvantage was that the full dynamics of the
model fitting process were not allowed to operate. Therefore, in Chapter 5, the
experimental results were explored in models fit to actual data. Using the data
originally analyzed in Chapter 2, one dependency similar to a dependency in the
experimental data was created. While retaining the same fixed effects, the
number of random effects was varied. The pattern of diminished collinearity
was seen when variables were added to the random effects, though not to the
same degree nor in exactly the same pattern as was seen for the experimental
data. The difference in findings was attributed to the difference in
dependencies in the two data sets, the effect of the estimation process (the l1.
changes for each model) and the structure of the covariance matrix in the
actual data, which was quite different than either of the matrices used in the
experiments. In Chapter 5, another dependency that was overlapping in nature
was also explored. The results suggested that the same cancellation of
collinearity as seen for simple dependencies, in both actual and experimental
data, might occur for this type as well. Overall, the results in Chapter 5
suggested that even though the behavior of the diagnostics is specific to an
actual data set, the general pattern will be similar to that seen in experimental
data; collinearity will be diminished when variables are added to the random
effects.
In Chapter 6, the focus of the dissertation shifted from the behavior of
the diagnostics to their practical use as a part of a model fitting strategy. Both
the strategy and the diagnostics were applied to a set of collinear independent
variables (age, height, weight and height2 ) that were candidates for a model to
216
•
..
..
assess the longitudinal variation of forced vital capacity (FVC) in children. A
final model was obtained that was both free of serious collinearity and in which
all fixed and random effects were significant. It was demonstrated that using
the diagnostics had improved the model fitting procedure.
7.2 Directions for Future Research
The experiments and the data for this research were necessarily limited
in scope. The investigation has focused on simple models with 3-5 continuous
time-dependent independent variables in both actual and experimental data.
The behavior of collinearity diagnostics was examined for two types of
dependencies that were created in the experimental data. One of the two
types was also examined in the actual data. The results are dependent on the
experimental data generated and the type of dependencies created and cannot
be generalized to other types with absolute certainty. Results also depend on
the sample manifestations for the series that were used to generate the
dependencies. And of course, the findings for the actual data may be specific
to the data set used. However, the weight of the evidence over all situations
examined supports the findings and provides an excellent basis for subsequent
research.
Future research can proceed in several directions, involving aspects of
both collinearity and of the mixed model. First, the viability of the findings of
this investigation might be examined under a variety of other conditions. For
example, using the same types of dependencies, several other factors that
217
might impact the results could be examined. For these same experiments, the
number of "subjects" and the number of observations per subject could be
varied. In these experiments, there were 30 subjects with 10 observations
each. Also, the degree to which "missingness" or incomplete data affects the
results could be studied; no observations were missing in these experiments.
These are all subject-related aspects.
Several variable-related aspects also could be pursued for these same
experiments. These experiments was comprised of three to five continuous
time-dependent covariates. When dichotomous, ordinal or time-invariant
covariates are included with these same independent variables, different
diagnostic behavior might result. For example, a continuous variable might be
collinear with a dichotomous term if most of the data falls into one of the two
categories. This would be similar to a dependency with the intercept, which
is another aspect that could be pursued. In these experiments, collinearities
with the intercept were noted, but not particularly dealt with. The number of
noncollinear variables in models also could be varied.
There are several avenues of specific collinearity-related research that
could be pursued, as suggested by the literature review. The effect of
centering the data could be studied since this is a controversial and seemingly
unsettled topic. Remedial measures, other than variable deletion, could be
examined, such as mixed model analogs of biased regression techniques used
in the GLUM. In actual data, the degree to which perturbation of the data
impacts conditioning and the related concept of collinearity-influential
observations could be examined. A systematic study of how collinearity
218
...
impacts mixed model parameter estimates of both collinear and noncollinear
variables could be undertaken.
Another obvious pursuit is the investigation of the behavior of the
collinearity diagnostics for more complicated, perhaps overlapping,
dependencies. With more complex dependencies, the condition indexes may
indicate the number of dependencies present. However, it may be difficult to
determine the particular variables involved from the variance decomposition
proportions alone. Thus, it would also be of interest to determine how to use
auxiliary regressions in the mixed model context. Whether they would be
comparable to the GLUM analogs is uncertain.
An investigation of certain model-related aspects would be crucial to
generalizing the characterization of these diagnostic measures. The effect of
the covariance matrix seems to be particularly important. In this research, fairly
simple covariance structures were used. One was essentially an identity
matrix, except for two non-zero off-diagonal elements; the other had one's on
the diagonal and all off-diagonal elements were positive and ranged from 0.4
to 0.8. It is especially important to look at other covariance structures,
particularly those with diagonal elements not equal to one. In addition, a range
of values might be chosen for u2 and a variety of structures, for Vk •
Finally, a theoretical aspect could be pursued. Although the experiments
showed repeatedly that collinearity in the fixed effects was virtually cancelled
when collinear variables were added to the random effects, the theoretical basis
for this cancellation has not yet been determined. If shown, this result would
provide a rationale for the empirical findings.
219
APPENDIX 1
GENERAL LINEAR UNIVARIATE MODEL (GLUM)COLLINEARITV DIAGNOSTICS
*** The W8 -*
Set 1: WO
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! wo
1 0.005 0.009 0.011 0.006 0.0074 0.005 0.133 0.1n 0.097 0.1475 0.021 0.356 0.735 0.003 0.0157 0.237 0.249 0.000 0.317 0.6658 0.732 0.253 0.083 0.577 0.166
Set 2: W1
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W1
1 0.005 0.009 0.011 0.001 0.0013 0.004 0.108 0.192 0.014 0.0145 0.016 0.424 0.717 0.000 0.0007 0.975 0.458 0.078 0.007 0.006
21 0.000 0.001 0.002 0.978 0.979
Set 3: W2
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! W2
1 0.005 0.009 0.011 0.000 0.0003 0.005 0.105 0.179 0.002 0.0025 0.016 0.417 0.716 0.000 0.0007 0.926 0.469 0.080 0.001 0.001
62 0.048 0.001 0.014 0.998 0.997
Set 4: W3
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W3
1 0.005 0.008 0.011 0.000 0.0003 0.004 0.095 0.186 0.000 0.0005 0.015 0.403 0.683 0.000 0.0007 0.957 0.425 0.082 0.000 0.000
204 0.018 0.070 0.039 1.000 1.000
Set 5: W4
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX! W4
..1 0.005 0.008 0.011 0.000 0.0003 0.004 0.100 0.191 0.000 0.0005 0.016 0.416 0.714 0.000 0.0007 0.970 0.445 0.083 0.000 0.000
731 0.004 0.030 0.001 1.000 1.000
..
-- The Zs --
set 6: ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.005 0.006 0.011 0.010 O.OOB4 0.009 0.066 0.001 0.426 0.1564 0.003 0.030 0.905 0.100 0.0417 0.159 0.445 0.019 0.151 0.744a 0.824 0.453 0.063 0.313 0.OS1
set 7: Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1
1 0.005 0.001 0.007 0.009 0.0014 0.012 0.015 0.026 0.521 0.0075 0.002 0.027 0.523 0.006 0.0037 0.958 O.OOB 0.024 0.418 0.012
21 0.023 0.949 0.421 0.046 0.977
Set 8: Z2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2
1 0.005 0.000 0.001 0.009 0.0004 0.012 0.001 0.003 0.512 0.0015 0.003 0.002 0.095 0.011 0.0007 0.977 0.001 0.005 0.457 0.001
69 0.003 0.995 0.896 0.010 0.998
Set 9: Z3
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z3
1 O.OOS 0.000 0.000 0.009 0.0004 0.012 0.000 0.000 0.503 0.0005 0.003 0.000 O.OOB 0.011 0.0007 0.978 0.000 0.000 0.443 0.000
251 0.003 1.000 0.991 0.035 1.000
Set 10: Z4
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.005 0.000 0.000 0.009 0.0004 0.012 0.000 0.000 0.499 0.0005 0.003 0.000 0.001 0.011 0.0007 0.980 0.000 0.000 0.444 0.000
723 0.000 1.000 0.999 0.037 1.000
221
Append i x 1: GLl.I4
Appendi x 1: GUM
... The Ws 8nd Zs ...
set 11: wa, ZO
CI V.rienee Proportions for Coefficients oflilT 1X1 1X2 IX3 wa ZO
1 0.004 0.004 0.008 0.004 0.005 0.0054 0.000 0.OS8 0.021 0.094 0.118 0.0845 0.000 0.043 0.866 0.009 0.006 0.0836 0.201 0.090 0.026 0.104 0.346 0.3238 0.209 0.7'90 0.005 0.OS8 0.221 0.4369 0.586 0.015 0.074 0.731 0.304 0.069
set 12: wa, Z1
CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z1
1 0.004 0.001 O.OOS 0.004 0.005 0.0014 0.000 0.012 0.024 0.089 0.138 0.0075 0.003 0.025 0.522 0.010 0.000 0.0037 0.384 0.011 0.000 0.184 0.598 0.0068 0.587 0.002 0.031 0.67'9 0.258 0.008
22 0.022 0.948 0.418 0.034 0.001 0.976
Set 13: wa, Z2
CI Veri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z2
1 0.004 0.000 0.001 0.004 0.005 0.0004 0.000 0.001 0.004 0.087 0.127 0.0015 0.004 0.002 0.094 0.010 0.000 0.0007 0.359 0.001 0.000 0.190 0.603 0.0008 0.632 0.001 0.006 0.671 0.232 0.000
76 0.002 0.995 0.895 0.038 0.033 0.998
Set 14: wa, Z3
CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 WO Z3
1 0.004 0.000 0.000 0.004 0.005 0.0004 0.000 0.000 0.000 0.085 0.131 0.0005 0.004 0.000 0.008 0.011 0.000 0.0007 0.365 0.000 0.000 0.188 0.613 0.0008 0.626 0.000 0.001 0.663 0.236 0.000
275 0.002 1.000 0.991 0.049 0.016 1.000
Set 15: wa, Z4
CI V.ri8nCe Proportions for Coefficients oflilT IX1 IX2 IX3 wa Z4
1 0.004 0.000 0.000 0.004 0.005 0.0004 0.000 0.000 0.000 0.089 0.129 0.0005 0.004 0.000 0.001 0.011 0.000 0.0007 0.365 0.000 0.000 0.197 0.608 0.0008 0.628 0.000 0.000 0.696 0.234 0.000
795 0.000 1.000 0.999 0.002 0.025 1.000
222
Appendix 1: GLlJ4
*** The \Is end Zs ***set 16: W1, ZO
CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W1 ZO
1 0.004 0.004 0.008 0.001 0.001 0.0053 0.000 0.043 0.029 0.012 0.012 0.0825 0.000 0.OS2 0.858 0.001 0.000 0.0887 0.242 0.346 0.035 0.005 0.002 0.7188 0.753 0.548 0.067 0.004 O.OOS 0.076
23 0.002 0.006 0.004 0.978 0.980 0.031
set 17: W1, Z1
CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 W1 Z1
1 0.003 0.001 0.004 0.001 0.001 0.0014 0.000 0.010 0.034 0.013 0.013 0.0065 0.001 0.029 0.507 0.000 0.000 0.0048 0.975 0.011 0.031 0.008 0.007 0.013
22 0.017 0.671 0.274 0.200 0.250 0.67524 0.003 0.277 0.150 0.778 0.730 0.302
Set 18: W1, Z2
CI Variance Proportions for Coefficients oflilT IX1 aX2 aX3 W1 Z2
1 0.003 0.000 0.001 0.001 0.001 0.0003 0.000 0.001 0.005 0.013 0.013 0.0015 0.001 0.003 0.092 0.000 0.000 0.0008 0.993 0.001 0.006 0.008 0.007 0.001
23 0.000 0.000 0.001 0.956 0.963 0.00076 0.002 0.995 0.896 0.023 0.016 0.998
Set 19: W1, Z3
CI Variance Proportions for Coefficients oflilT aX1 aX2 aX3 W1 Z3
1 0.003 0.000 0.000 0.001 0.001 0.0003 0.000 0.000 0.000 0.013 0.013 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.992 0.000 0.001 0.008 0.007 0.000
23 0.000 0.000 0.000 0.978 0.974 0.000275 0.003 1.000 0.991 0.000 0.005 1.000
Set 20: W1, Z4
CI Variance Proportions for Coefficients oflilT aX1 aX2 aX3 W1 Z4
1 0.003 0.000 0.000 0.001 0.001 0.0003 0.000 0.000 0.000 0.013 0.013 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.995 0.000 0.000 0.008 0.007 0.000
23 0.000 0.000 0.000 0.974 0.963 0.000795 0.000 1.000 0.999 O.OOS 0.017 1.000
223
Appendi x 1: GLlJ4
- The wa end Zs -
set 21: 112, ZO
CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 112 ZO
1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.031 0.001 0.002 0.0785 0.000 0.054 0.837 0.000 0.000 0.0977 0.282 0.285 0.047 0.000 0.000 0.7018 0.666 0.613 0.063 0.000 0.001 0.119
67 0.049 0.002 0.013 0.998 0.997 0.001
Set 22: 112, Z1
CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 112 Z1
1 0.003 0.001 0.004 0.000 0.000 0.0011 0.000 0.009 0.034 0.001 0.002 0.0055 0.001 0.028 0.506 0.000 0.000 0.0048 0.926 0.010 0.033 0.001 0.001 0.013
22 0.025 0.891 0.417 0.001 0.000 0.91470 0.045 0.061 0.006 0.997 0.997 0.064
Set 23: 112, Z2
C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 112 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.001 0.002 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.939 0.001 0.006 0.001 0.001 0.001
65 0.030 0.152 0.166 0.715 0.723 0.15479 0.026 0.843 0.731 0.283 0.274 0.844
Set 24: 112, 23
C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 112 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.002 0.002 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.945 0.000 0.001 0.001 0.001 0.000
68 0.048 0.000 0.000 0.996 0.997 0.000214 0.003 1.000 0.991 0.002 0.001 1.000
Set 25: 112, Z4
C[ Variance Proportions for Coefficients of[liT IX1 IX2 IX3 W2 Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.001 0.002 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.947 0.000 0.000 0.001 0.001 0.000
68 0.048 0.000 0.000 0.986 0.980 0.000795 0.000 1.000 0.999 0.012 0.017 1.000
224
Appendi x 1: GlLM
*** The lis end Zs --
set 26: 113, ZO
CI V.riance Proportions for Coefficients oflIT &X1 &X2 IlG 113 ZO
1 0.003 0.004 0.008 0.000 0.000 0.0053 0.000 0.040 0.033 0.000 0.000 0.0795 0.000 0.052 0.811 0.000 0.000 0.0997 0.273 0.292 0.045 0.000 0.000 0.7108 0.706 0.560 0.066 0.000 0.000 0.107
222 0.017 0.051 0.037 1.000 1.000 0.000
Set 27: 113, Z1
CI V.riance Proportions for Coefficients oflIT IX1 &X2 IlG 113 Z1
1 0.003 0.001 0.004 0.000 0.000 0.0014 0.000 0.009 0.036 0.000 0.000 0.0055 0.001 0.029 0.499 0.000 0.000 0.0048 0.955 0.010 0.035 0.000 0.000 0.013
22 0.021 0.917 0.418 0.000 0.000 0.969225 0.019 0.033 0.009 1.000 1.000 0.008
set 28: 113, Z2
CI V.riance Proportions for Coefficients oflIT IX1 IX2 IlG 113 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.090 0.000 0.000 0.0008 0.978 0.001 0.007 0.000 0.000 0.001
75 0.003 0.939 0.879 0.001 0.001 0.954229 0.015 0.056 0.018 0.999 0.999 0.044
Set 29: 113, Z3
CI V.riance Proportions for Coefficients oflIT IX1 IX2 IX3 113 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.008 0.000 0.000 0.0008 0.974 0.000 0.001 0.000 0.000 0.000
223 0.015 0.021 0.030 0.946 0.948 0.024276 0.006 0.978 0.961 0.054 0.051 0.976
Set 30: 113, Z4
CI V.riance Proportions for Coefficients oflIT IX1 IX2 IlG 113 Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.9n 0.000 0.000 0.000 0.000 0.000
224 0.018 0.000 0.000 0.991 0.992 0.000792 0.000 1.000 0.999 0.009 0.008 1.000
225
Appendi x 1: GLIJII
*** The wa end Zs *-Set 31: Wit, ZO
CI Varience Proportions for Coefficients ofJIlT IX1 8X2 8X3 Wit ZO
1 0.004 0.004 0.008 0.000 0.000 0.0053 0.000 0.042 0.034 0.000 0.000 0.0795 0.000 0.054 0.845 0.000 0.000 0.0977 0.274 0.306 0.045 0.000 0.000 0.7058 0.716 0.586 0.067 0.000 0.000 0.105
799 0.006 0.007 0.001 1.000 1.000 0.009
Set 32: Wit. Z1
CI Varience Proportions for Coefficients ofINT IX1 IX2 8X3 Wit Z1
1 0.003 0.001 0.004 0.000 0.000 0.0014 0.000 0.010 0.035 0.000 0.000 0.0055 0.001 0.030 0.501 0.000 0.000 0.0048 0.970 0.011 0.033 0.000 0.000 0.013
22 0.022 0.947 0.418 0.000 0.000 0.964809 0.004 0.002 0.009 1.000 1.000 0.013
Set 33: Wit. Z2
CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.000 0.001 0.005 0.000 0.000 0.0015 0.001 0.003 0.091 0.000 0.000 0.0008 0.989 0.001 0.006 0.000 0.000 0.001
76 0.003 0.982 0.890 0.000 0.000 0.989 •807 0.003 0.013 0.007 1.000 1.000 0.009
Set 34: Wit. Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 O.ooa 0.000 0.000 0.0008 0.989 0.000 0.001 0.000 0.000 0.000
274 0.003 0.979 0.968 0.000 0.000 0.977812 0.004 0.021 0.023 1.000 1.000 0.023
Set 35: Wit, Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Wit Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.000 0.000 0.000 0.000 0.000 0.0005 0.001 0.000 0.001 0.000 0.000 0.0008 0.991 0.000 0.000 0.000 0.000 0.000
764 0.002 0.565 0.567 0.353 0.351 0.567832 0.002 0.435 0.432 0.647 0.649 0.433
226
APPENDIX 2
MIXED MODEL BASELINECOLLINEARITY DIAGNOSTICS
- The wa-
Set 1: WO
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO
1 0.005 0.011 0.011 0.006 0.0083 0.015 0.080 0.105 0.040 0.2014 0.000 0.484 0.478 0.001 0.0046 0.215 0.309 0.188 0.319 0.4458 0.104 0.111 0.218 0.634 0.336
Set 2: '11
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 '11
1 0.005 0.010 0.010 0.001 0.0013 0.013 0.069 0.185 0.013 0.0174 0.000 0.515 0.392 0.001 0.0001 0.915 0.345 0.405 0.002 0.004
20 0.007 0.000 0.008 0.983 0.977
Set 3: W2
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2
1 0.005 0.010 0.010 0.000 0.0003 0.013 0.078 0.184 0.002 0.0024 0.000 0.551 0.409 0.000 0.0001 0.980 0.345 0.391 0.000 0.000
62 0.001 0.010 0.000 0.998 0.998
Set 4: W3
CI Variance Proportions for Coefficients ofINT aX1 BX2 BX3 W3
1 0.005 0.010 0.010 0.000 0.0003 0.013 0.016 0.191 0.000 0.0004 0.000 0.568 0.402 0.000 0.0001 0.980 0.345 0.391 0.000 0.000
183 0.001 0.000 0.000 1.000 1.000
Set 5: W4
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4
1 0.005 0.010 0.010 0.000 0.0003 0.013 0.076 0.191 0.000 0.0004 0.000 0.569 0.400 0.000 0.0008 0.919 0.345 0.396 0.000 0.000
633 0.002 0.000 0.003 1.000 1.000
Appendix 2: MIxeD Besel ine
-- The Z.·- ..set 6: ZO
CI Verience Proportions for Coefficients oflilT IX1 BX2 BX3 ZO
1 0.005 0.006 0.010 0.011 0.0074 0.010 0.096 0.064 0.278 0.1404 0.000 0.035 0.514 0.335 0.0017 0.392 0.199 0.029 0.316 0.5048 0.592 0.664 0.383 0.060 0.348
Set 7: 21
CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 Z1
1 0.005 0.001 0.007 0.011 0.0014 0.008 0.030 0.140 0.161 0.0114 0.000 0.000 0.314 0.548 0.0017 0.987 0.006 0.233 0.280 0.007
20 0.000 0.963 0.305 0.000 0.980
Set 8: Z2
CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 22
1 0.005 0.000 0.001 0.011 0.0004 0.008 0.004 0.026 0.184 0.0014 0.000 0.000 0.067 0.520 0.0007 0.985 0.001 0.045 0.285 0.001
63 0.002 0.996 0.861 0.000 0.998
Set 9: Z3
CI Verience Proportions for Coefficients oflilT BX1 BX2 BX3 Z3
1 0.005 0.000 0.000 0.011 0.0004 0.008 0.000 0.003 0.184 0.0004 0.000 0.000 0.007 0.516 0.0007 0.986 0.000 0.005 0.284 0.000
210 0.001 1.000 0.985 0.006 1.000
Set 10: 24
CI Veriance Proportions for Coefficients oflilT BX1 BX2 BX3 24
1 0.005 0.000 0.000 0.010 0.0004 O.OOS 0.000 0.000 0.183 0.0004 0.000 0.000 0.001 0.511 0.0007 0.985 0.000 0.001 0.281 0.000
640 0.002 1.000 0.998 0.014 1.000
228
Appendix 2: MIXED Baseline
-- The wa end Zs *-Set 11: WO end ZO
CI Variance Proportiona for Coefficients oflilT BX1 BX2 BX3 WO ZO
1 0.004 0.004 0.007 0.004 0.006 0.0053 0.002 0.031 0.016 0.044 0.168 0.0544 0.012 0.102 0.507 0.002 0.012 0.0466 0.263 0.040 0.098 0.102 0.283 0.2948 0.002 0.493 0.051 0.483 0.356 0.4799 0.717 0.330 0.320 0.365 0.176 0.122
set 12: WO end Z1
CI Variance Proportiona for Coefficients oflilT BX1 BX2 BX3 WO Z1
1 0.004 0.001 0.005 0.004 0.005 0.0013 0.003 0.008 0.013 0.044 0.182 0.0054 0.010 0.021 0.399 0.003 0.009 0.0057 0.341 0.007 0.152 0.253 0.427 0.0069 0.643 0.001 0.124 0.696 0.376 0.003
22 0.000 0.963 0.307 0.000 0.002 0.980
Set 13: WO end Z2
CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z2
1 0.004 0.000 0.001 0.004 0.005 0.0003 0.003 0.001 0.003 0.044 0.182 0.0004 0.009 0.003 0.082 0.002 0.007 0.0007 0.348 0.001 0.030 0.242 0.415 0.0019 0.635 0.000 0.024 0.706 0.385 0.000
68 0.001 0.996 0.860 0.002 0.006 0.998
Set 14: WO end Z3
CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z3
1 0.004 0.000 0.000 0.004 0.005 0.0003 0.003 0.000 0.000 0.044 0.183 0.0004 0.009 0.000 0.009 0.002 0.007 0.0007 0.347 0.000 0.003 0.243 0.420 0.0009 0.636 0.000 0.003 0.704 0.385 0.000
226 0.001 1.000 0.985 0.003 0.000 1.000
Set 15: WO Z4
CI Variance Proportions for Coefficients oflilT BX1 BX2 BX3 WO Z4
1 0.004 0.000 0.000 0.004 0.005 0.0003 0.003 0.000 0.000 0.044 0.183 0.0004 0.009 0.000 0.001 0.002 0.007 0.0007 0.347 0.000 0.000 0.241 0.419 0.0009 0.635 0.000 0.000 0.699 0.385 0.000
689 0.002 1.000 0.998 0.010 0.001 1.000
229
Appendix 2: MIXED Baseline
.- The Ws n Zs'-
Set 16: W1 n ZO
CI Variance Proportions for Coefficients ofliT BX1 BX2 IX3 W1 ZO
1 0.004 0.004 0.007 0.001 0.001 0.0053 0.001 0.033 0.023 0.012 0.015 0.0714 0.010 0.102 0.513 0.001 0.001 0.0427 0.435 0.162 0.064 0.004 0.005 0.5039 0.544 0.699 0.387 0.000 0.001 0.379
22 0.007 0.000 0.006 0.983 0.977 0.001
set 17: W1 n Z1
CI Variance Proportions for Coefficients oflifT IX1 IX2 IX3 W1 Z1
1 0.004 0.001 0.005 0.001 0.001 0.0013 0.002 0.009 0.021 0.013 0.017 0.0064 0.009 0.021 0.398 0.000 0.001 0.0058 0.980 0.006 0.268 0.003 0.005 0.008
22 0.002 0.598 0.230 0.349 0.341 0.60822 0.004 0.365 0.078 0.634 0.636 0.3n
Set 18: W1 n Z2
CI Variance Proportions for Coefficients ofliT IX1 IX2 IX3 W1 12
1 0.004 0.000 0.001 0.001 0.001 0.0003 0.002 0.001 0.005 0.013 0.017 0.0014 0.009 0.003 0.081 0.000 0.001 0.0008 0.978 0.001 0.052 0.003 0.005 0.001
22 0.006 0.000 0.001 0.981 0.976 0.00069 0.002 0.996 0.860 0.002 0.001 0.998
Set 19: W1 and Z3
CI Variance Proportions for Coefficients oflifT IX1 IX2 BX3 W1 Z3
1 0.004 0.000 0.000 0.001 0.001 0.0003 0.002 0.000 0.001 0.013 0.017 0.0004 0.009 0.000 0.009 0.000 0.001 0.0008 0.979 0.000 0.006 0.003 0.005 0.000
22 0.007 0.000 0.000 0.982 0.977 0.000229 0.001 1.000 0.985 0.001 0.000 1.000
Set 20: W1 n Z4
CI Variance Proportions for Coefficients ofliT IX1 IX2 BX3 W1 Z4
1 0.004 0.000 0.000 0.001 0.001 0.0003 0.002 0.000 0.000 0.013 0.016 0.0004 0.009 0.000 0.001 0.000 0.001 0.0008 0.977 0.000 0.001 0.003 0.005 0.000
22 0.007 0.000 0.000 0.949 0.951 0.000T06 0.002 1.000 0.998 0.035 0.026 1.000
230
ApPendix 2: MIXED lasel ine
-* The Ws and Zs -
set 21: W2 and ZO
CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W2 ZO
1 0.004 0.004 0.007 0.000 0.000 0.0053 0.001 0.037 0.019 0.001 0.001 0.0754 0.010 0.098 0.531 0.000 0.000 0.0377 0.433 0.160 0.058 0.000 0.001 0.5069 0.551 0.694 0.385 0.000 0.000 0.376
67 0.002 0.007 0.000 0.998 0.998 0.000
set 22: W2 and Z1
CI Variance Proportions for Coefficients oflilT IX1 IX2 III W2 Z1
1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.017 0.002 0.002 0.0074 0.009 0.020 0.413 0.000 0.000 0.0048 0.984 0.006 0.260 0.000 0.000 0.008
22 0.000 0.959 0.305 0.000 0.000 0.97968 0.002 0.004 0.000 0.998 0.998 0.001
Set 23: W2 and Z2
CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W2 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.002 0.002 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.983 0.001 0.050 0.000 0.000 0.001
66 0.003 0.346 0.311 0.586 0.584 0.35771 0.000 0.650 0.551 0.411 0.414 0.641
Set 24: W2 and Z3
CI Variance Proportions for Coefficients oflilT IX1 IX2 III W2 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.002 0.002 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000
68 0.002 0.000 0.000 0.998 0.998 0.000229 0.001 1.000 0.985 0.000 0.000 1.000
Set 25: W2 and Z4
CI Variance Proportions for Coefficients oflilT IX1 aX2 BX3 W2 Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.002 0.002 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.983 0.000 0.001 0.000 0.000 0.000
68 0.002 0.000 0.000 0.998 0.998 0.000698 0.002 1.000 0.998 0.000 0.000 1.000
231
Appendix 2: MIXED lasel ine
-- The wa rd zs *-set 26: W3 rd ZO
CI Veri~e Proportions for Coefficients oflilT 1X1 IX2 BX3 W3 ZO
1 0.004 0.004 0.001 0.000 0.000 0.0053 0.001 0.031 0.020 0.000 0.000 0.0114 0.010 0.099 0.530 0.000 0.000 0.0311 0.435 0.163 0.058 0.000 0.000 0.5059 0.550 0.698 0.385 0.000 0.000 0.311
199 0.001 0.000 0.000 1.000 1.000 0.000
Set 21: W3 rd Z1
CI Veriance Proportions for Coefficients oflilT IX1 IX2 IX3 W3 Z1
1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.011 0.000 0.000 0.0014 0.009 0.020 0.412 0.000 0.000 0.0048 0.984 0.006 0.260 0.000 0.000 0.008
22 0.000 0.961 0.305 0.000 0.000 0.918201 0.002 0.002 0.000 1.000 1.000 0.002
Set 28: W3 end Z2
CI Variance Proportions for Coefficients oflilT IX1 IX2 IX3 W3 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.000 0.000 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.983 0.001 0.050 0.000 0.000 0.001
69 0.002 0.996 0.861 0.000 0.000 0.998201 0.001 0.000 0.000 1.000 1.000 0.000
Set 29: W3 end Z3
CI Variance Proportions for Coefficients oflilT IX1 aX2 aX3 W3 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000
201 0.001 0.001 0.001 0.999 0.999 0.001230 0.001 0.999 0.984 0.001 0.001 0.999
Set 30: W3 end Z4
CI Veriance Proportions for Coefficients ofINT IX1 aX2 IX3 W3 Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.983 0.000 0.001 0.000 0.000 0.000
201 0.001 0.000 0.000 0.995 0.995 0.000100 0.002 1.000 0.998 0.005 0.004 1.000
232
Appendix 2: MIXED Basel ine
*** The Ws and Zs -*Set 31: W4 and ZO
CI Variance Proportions for Coefficients ofINT 1X1 IX2 BX3 W4 ZO
1 0.004 0.004 0.007 0.000 0.000 0.0053 0.001 0.037 0.020 0.000 0.000 0.0774 0.010 0.099 0.530 0.000 0.000 0.0377 0.434 0.163 0.058 0.000 0.000 0.5059 0.550 0.697 0.384 0.000 0.000 0.376
688 0.002 0.000 0.002 1.000 1.000 0.001
Set 32: W4 and Z1
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z1
1 0.003 0.001 0.005 0.000 0.000 0.0013 0.001 0.010 0.017 0.000 0.000 0.0074 0.009 0.020 0.410 0.000 0.000 0.0048 0.984 0.006 0.259 0.000 0.000 0.008
22 0.000 0.961 0.304 0.000 0.000 0.978695 0.002 0.002 0.005 1.000 1.000 0.002
Set 33: W4 end Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2
1 0.003 0.000 0.001 0.000 0.000 0.0003 0.001 0.001 0.004 0.000 0.000 0.0014 0.009 0.002 0.083 0.000 0.000 0.0008 0.982 0.001 0.050 0.000 0.000 0.001
69 0.002 0.990 0.854 0.000 0.000 0.993697 0.002 0.005 0.008 1.000 1.000 0.005
Set 34: W4 and Z3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z3
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.009 0.000 0.000 0.0008 0.984 0.000 0.005 0.000 0.000 0.000
230 0.001 0.993 0.978 0.000 0.000 0.993697 0.002 0.007 0.008 1.000 1.000 0.006
Set 35: W4 Z4
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4
1 0.003 0.000 0.000 0.000 0.000 0.0003 0.001 0.000 0.000 0.000 0.000 0.0004 0.009 0.000 0.001 0.000 0.000 0.0008 0.982 0.000 0.001 0.000 0.000 0.000
691 0.000 0.353 0.354 0.631 0.630 0.353703 0.004 0.647 0.644 0.369 0.370 0.647
233
APPENDIX 3
MIXED MODEL EXPERIMENT 1COLLINEARITV DIAGNOSTICS
- The Ws-
Set 1: WO
Veriebl.. in Z: INT
CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 WO
1 0.052 0.039 0.039 0.037 0.0352 0.059 0.143 0.132 0.063 0.1112 0.000 0.461 0.505 0.001 0.0003 0.781 0.357 0.303 0.003 0.0534 0.109 0.000 0.021 0.897 0.802
Veriebles in Z: INT IX1
CI Veriance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.057 0.035 0.044 0.040 0.0381 0.051 0.175 0.110 0.061 0.0972 0.003 0.546 0.446 0.002 0.0012 0.775 0.239 0.381 0.002 0.0653 0.114 0.004 0.019 0.894 0.798
Veriebles in Z: INT IX1 IX2
CI Verience Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.077 0.034 0.002 0.066 0.0641 0.116 0.396 0.059 0.031 0.0591 0.005 0.054 0.931 0.000 0.0012 0.730 0.514 0.008 0.003 0.0453 o.on 0.001 0.000 0.900 0.830
Veriebles in Z: INT IX1 IX2 IX3
CI Verienee Proportions for Coefficients ofINT IX1 IX2 IX3 we
0.324 0.309 0.025 0.000 0.0130.000 0.001 0.081 0.465 0.4380.001 0.013 0.614 0.314 0.0590.004 0.040 0.265 0.221 0.4840.671 0.637 0.015 0.000 0.006
Veriebles in Z: INT IX1 IX2 IX3 weCI Verienee Proportions for Coefficients of
INT IX1 IX2 IX3 we0.337 0.322 0.033 0.000 0.0000.000 0.000 0.001 0.474 0.5250.000 0.000 0.001 0.524 0.4750.004 0.050 0.945 0.002 0.0000.659 0.628 0.020 0.000 0.000
Appendix 3: MIXED Experiment 1
. - The wa-
set 2: \11
Vari8bl.. in Z: INT
Cl Variance Proportions for Coefficients ofINT IX1 IX2 BX3 WO
1 0.044 0.033 0.029 0.006 0.0062 0.062 0.119 0.197 0.011 0.0132 0.000 0.532 0.434 0.000 0.0003 0.889 0.315 0.330 0.002 0.0029 0.004 0.000 0.010 0.981 0.979
Vari8blea in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO
1 0.048 0.028 0.032 0.007 0.0071 0.056 0.169 0.143 0.010 0.0122 0.003 0.576 0.418 0.000 0.0003 0.889 0.228 0.396 0.002 0.0029 0.003 0.000 0.011 0.981 0.979
Vari8bles in Z: INT IX1 IX2
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO
1 0.062 0.024 0.001 0.011 0.0111 0.118 0.424 0.080 0.006 0.0062 0.007 0.070 0.911 0.000 0.0002 0.810 0.482 0.008 0.003 0.0048 0.004 0.000 0.000 0.980 0.979
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO
0.325 0.312 0.026 0.000 0.0080.001 0.005 0.002 0.490 0.4570.004 0.038 0.948 0.010 0.0000.000 0.003 0.010 0.500 0.5330.669 0.641 0.014 0.000 0.003
Variables in Z: INT IX1 IX2 IX3 \11
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO
0.333 0.319 0.031 0.000 0.0000.000 0.000 0.000 0.491 0.5060.000 0.001 0.003 0.507 0.4920.004 0.046 0.947 0.001 0.0030.663 0.635 0.019 0.000 0.000
235
Appendix 3: MIXED Experilllent 1
.- The wa'-
set 3: W2
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
1 0.043 0.031 0.029 0.001 0.0012 0.063 0.138 0.183 0.001 0.0012 0.000 0.500 0.465 0.000 0.0003 0.893 0.318 0.322 0.000 0.000
28 0.001 0.013 0.001 0.998 0.998
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
1 0.047 0.026 0.032 0.001 0.0012 0.058 0.185 0.140 0.001 0.0012 0.003 0.559 0.437 0.000 0.0003 0.892 0.230 0.389 0.000 0.000
27 0.000 0.000 0.001 0.998 0.998
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
1 0.060 0.023 0.001 0.001 0.0011 0.119 0.429 0.075 0.001 0.0012 0.007 0.067 0.915 0.000 0.0002 0.812 0.481 0.008 0.000 0.000
26 0.003 0.000 0.000 0.998 0.998
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients of(NT IX1 IX2 IX3 W2
0.328 0.318 0.027 0.000 0.0000.001 0.000 0.001 0.427 0.4300.004 0.037 0.957 0.002 0.0000.002 0.003 0.001 0.568 0.5640.665 0.642 0.015 0.002 0.006
Variables in Z: INT IX1 IX2 IX3 W2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
0.335 0.321 0.032 0.000 0.0000.000 0.000 0.000 0.486 0.4870.003 0.045 0.934 0.008 0.0090.000 0.004 0.014 0.504 0.5030.661 0.630 0.020 0.001 0.000
236
..
Appendix 3: MIXED Experiment
*** The ... ***set 4: W3
V.riabl.. in Z: INT
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.043 0.032 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.456 0.000 0.0003 0.892 0.319 0.325 0.000 0.000
84 0.002 0.000 0.000 1.000 1.000
V.riables in Z: INT IX1
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.046 0.026 0.032 0.000 0.0002 0.058 0.183 0.140 0.000 0.0002 0.003 0.560 0.436 0.000 0.0003 0.890 0.230 0.391 0.000 0.000
82 0.003 0.000 0.000 1.000 1.000
V.riables in z: INT IX1 IX2
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.059 0.023 0.001 0.000 0.0001 0.119 0.430 0.076 0.000 0.0002 0.007 0.067 0.915 0.000 0.0002 0.809 0.481 0.008 0.000 0.000
78 0.006 0.000 0.000 1.000 1.000
V.riables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.051 0.039 0.003 0.243 0.2621 0.275 0.278 0.024 0.059 0.0351 0.004 0.037 0.959 0.000 0.0001 0.601 0.617 0.014 0.046 0.0322 0.069 0.029 0.001 0.652 0.671
V.riables in Z: INT IX1 IX2 IX3 W3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
0.333 0.316 0.037 0.002 0.0000.000 0.000 0.000 0.412 0.4150.004 0.055 0.941 0.000 0.0000.003 0.016 0.000 0.575 0.5760.660 0.614 0.022 0.011 0.009
237
Appendix 3: MIXED ExperiMnt
-- The wa--
Set 5: W4
Variebles in Z: INT
CI Variance Proportions for Coefficient. ofINT IX1 IX2 BX3 W4
1 0.043 0.031 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.455 0.000 0.0003 0.892 0.318 0.324 0.000 0.000
298 0.002 0.001 0.002 1.000 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4
1 0.046 0.026 0.032 0.000 0.0002 0.058 0.183 0.140 0.000 0.0002 0.003 0.560 0.436 0.000 0.0003 0.891 0.230 0.390 0.000 0.000
290 0.002 0.000 0.002 1.000 1.000
Variebles in Z: INT IX1 IX2
CI
1122
271
Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4
0.059 0.023 0.001 0.000 0.0000.119 0.430 0.077 0.000 0.0000.007 0.068 0.914 0.000 0.0000.815 0.479 O.ooa 0.000 0.0000.000 0.000 0.000 1.000 1.000
Variebles in Z: INT IX1 BX2 BX3
•
CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W4
1 0.000 0.000 0.000 0.086 0.0861 0.328 0.318 0.027 0.000 0.0001 0.004 0.038 0.959 0.000 0.0002 0.667 0.644 0.015 0.000 0.0003 0.001 0.000 0.000 0.914 0.914
Variables in Z: INT BX1 BX2 BX3 W4
CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W4
1 0.000 0.000 0.000 0.152 0.1521 0.329 0.318 0.027 0.000 0.0001 0.004 0.039 0.957 0.000 0.0002 0.665 0.643 0.015 0.000 0.0002 0.002 0.000 0.000 0.848 0.848
238
Appendix 3: MIXED Experillll!nt
••• The Zs -
Set 6: ZO
Veriables in Z: INT
CI VerilnCe Proportions for Coefficients ofINT IX1 IX2 8X3 ZO
1 0.050 0.036 0.038 0.037 0.0382 0.041 0.111 0.104 0.249 0.0912 0.000 0.023 0.509 0.384 0.0043 0.806 0.002 0.133 0.330 0.0983 0.102 0.828 0.216 0.000 0.768
Veriables in Z: INT IX1
CI VerilnCe Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.090 0.060 0.083 0.069 0.0241 0.010 0.157 0.064 0.027 0.4892 0.000 0.344 0.028 0.612 0.0072 0.027 0.276 0.451 0.085 0.4632 0.873 0.163 0.375 0.207 0.018
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.194 0.141 0.007 0.147 0.0051 0.005 0.061 0.236 0.032 0.5811 0.005 0.020 0.742 0.068 0.1681 0.002 0.455 0.012 0.368 0.2332 0.795 0.323 0.004 0.385 0.012
Variables in Z: INT IX1 IX2 aX3
CI Variance Proportions for Coefficients ofINT aX1 ax2 aX3 zo
0.297 0.317 0.018 0.000 0.0310.029 0.012 0.421 0.000 0.4760.000 0.000 0.001 0.999 0.0000.061 0.010 0.554 0.000 0.4150.613 0.662 0.006 0.000 0.078
Variables in Z: INT aX1 IX2 IX3 ZO
CI Variance Proportions for Coefficients ofINT IX1 aX2 IX3 ZO
0.337 0.325 0.037 0.000 0.0030.000 0.014 0.273 0.000 0.7010.000 0.000 0.001 0.999 0.0000.011 0.036 0.673 0.001 0.2920.651 0.626 0.016 0.000 0.004
239
Apeendix 3: MIXED Experiment 1
... The Zs ...
Set 7: Z1
Variables in Z: INT
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX! Z1
1 0.043 0.007 0.023 0.031 0.D072 0.045 0.022 0.131 0.193 0.0112 0.000 0.001 0.353 0.504 O.ODO3 0.912 0.002 0.176 0.272 0.0059 0.000 0.968 0.317 0.000 0.976
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z1
1 0.080 0.035 0.072 0.060 0.0361 0.016 0.215 0.038 0.028 0.2352 0.000 0.215 0.060 0.675 0.0022 0.769 0.445 0.002 0.214 0.1632 0.134 0.089 0.828 0.023 0.564
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z1
1 0.157 0.146 0.003 0.107 0.0501 0.050 0.070 0.113 0.143 0.3541 0.007 0.022 0.848 0.086 0.0232 0.130 0.225 0.037 0.454 0.3872 0.656 0.538 0.000 0.210 0.186
Variables in Z: INT IX1 IX2 IX!
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1
1 0.142 0.239 0.000 0.000 0.1631 0.142 0.000 0.610 0.001 0.1371 0.000 0.000 0.001 0.998 0.0001 0.410 0.001 0.384 0.001 0.2562 0.307 0.760 0.004 0.000 0.443
Variables in Z: INT IX1 IX2 IX! Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 8X! . Z1
0.237 0.291 0.021 O.ODO 0.1010.054 0.004 0.539 0.001 0.3360.000 O.DOO 0.002 0.998 0.0000.207 0.010 0.439 0.001 0.4010.502 0.695 0.000 0.000 0.162
240
" Appendix 3: MIXED Expe~;.ent 1
*** The Zs ***set 8: Z2
V.~i~l .. in Z: INT
CI V.~ianc. P~opo~t;ona fo~ Coefficients ofINT IX1 IX2 IX3 Z2
1 0.042 0.001 O.OOS 0.030 0.0012 0.047 0.002 0.023 0.212 0.0012 0.000 0.000 0.071 0.476 0.0003 0.910 0.000 0.032 0.282 0.001
28 0.001 0.997 0.869 0.000 0.998
V.~i~l.. in Z: INT IX1
Cl V.~;ance P~opo~tions fo~ Coefficients oflNT IX1 IX2 IX3 Z2
1 0.060 0.Q08 0.015 0.041 0.0111 0.039 0.107 0.005 0.056 0.0252 0.000 0.124 0.009 0.653 0.0002 0.899 0.146 0.018 0.249 0.0027 0.002 0.616 0.953 0.001 0.962
V.~i~les in Z: lNT IX1 IX2
Cl V.~iance P~opo~tiona fo~ Coefficients ofINT IX1 IX2 IX3 Z2
1 0.074 0.062 0.001 0.037 0.0591 0.114 0.015 0.212 0.245 0.0451 0.015 0.030 0.646 0.183 0.0052 0.776 0.009 0.031 0.535 0.0223 0.021 0.884 0.111 0.001 0.869
V.~iables in Z: lNT IX1 IX2 IX3
CI V.~iance P~opo~tions fo~ Coefficients ofINT IX1 IX2 IX3 Z2
1 0.065 0.076 0.002 0.000 0.0751 0.148 0.001 0.661 0.001 0.0121 0.000 0.000 0.001 0.999 0.0002 0.769 0.033 0.214 0.000 0.0283 0.019 0.890 0.122 0.000 0.885
V.~iabl .. in Z: INT IX1 IX2 IX3 Z2
CI V.~iance P~opo~tions fo~ Coefficients ofINT IX1 IX2 IX3 Z2
1 0.071 0.127 0.001 0.000 0.1191 0.132 0.000 0.627 0.001 0.0471 0.000 0.000 0.001 0.999 0.0001 0.770 0.032 0.224 0.001 0.0282 0.028 0.841 0.147 0.000 0.807
241
Appendix 3: MIXED Experilllent 1
*** The Zs ***Set 9: 13
Variebl.. in Z: lIT
CI Variance Proportions for Coefficients oflIT IX1 IX2 8X3 13
1 0.041 0.000 0.001 0.030 0.0002 0.041 0.000 0.003 0.211 0.0002 0.000 0.000 0.008 0.413 0.0003 0.911 0.000 0.004 0.280 0.000
91 0.000 1.000 0.986 0.001 1.000
Variebl.. in Z: lIT IX1
CI Variance Proportions for Coefficients oflIT IX1 IX2 IX3 13
1 0.058 0.001 0.002 0.039 0.0011 0.041 0.016 0.001 0.058 0.0022 0.000 0.019 0.001 0.649 0.0002 0.901 0.022 0.002 0.245 0.000
21 0.000 0.943 0.995 0.009 0.996
Variables in Z: lIT IX1 IX2
CI Variance Proportions for Coefficients oflIT IX1 IX2 IX3 13
1 0.058 0.008 0.001 0.021 0.0081 0.128 0.002 0.115 0.254 0.0042 0.016 0.004 0.341 0.184 0.0002 0.793 0.002 0.015 0.525 0.002
10 0.005 0.984 0.522 0.010 0.986
Variables in Z: lIT IX1 IX2 IX3
CI Variance Proportions for Coefficients oflIT IX1 IX2 BX3 13
1 0.052 0.009 0.002 0.000 0.0081 0.150 0.000 0.329 0.001 0.0011 0.000 0.000 0.000 0.999 0.0002 0.796 0.005 0.099 0.000 0.002
10 0.002 0.981 0.570 0.000 0.988
Variables in Z: INT BX1 BX2 BX3 13
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13
1 0.045 0.016 0.000 0.000 0.0151 0.140 0.000 0.319 0.000 0.0041 0.000 0.000 0.000 0.999 0.0002 0.812 0.005 0.098 0.001 0.0028 0.002 0.979 0.583 0.000 0.979
242
Appendix 3: MIXED Experiment 1
*** The Zs ***set 10: Z4
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 &X2 IX3 Z4
1 0.041 0.000 0.000 0.030 0.0002 0.047 0.000 0.000 0.210 0.0002 0.000 0.000 0.001 0.472 0.0003 0.911 0.000 0.000 0.278 0.000
271 0.000 1.000 0.998 0.010 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.057 0.000 0.000 0.039 0.0001 0.042 0.002 0.000 0.058 0.0002 0.000 0.002 0.000 0.651 0.0002 0.901 0.003 0.000 0.245 0.000
63 0.000 0.993 0.999 0.008 1.000
Variables in z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.057 0.001 0.000 0.025 0.0011 0.130 0.000 0.024 0.259 0.0012 0.015 0.001 0.075 0.181 0.0002 0.798 0.000 0.003 0.525 0.000
27 0.001 0.998 0.897 0.010 0.998
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.051 0.001 0.000 0.000 0.0011 0.149 0.000 0.071 0.001 0.0001 0.000 0.000 0.000 0.999 0.0002 0.799 0.001 0.021 0.000 0.000
27 0.000 0.998 0.907 0.000 0.998
Variables in Z: INT IX1 IX2 IX3 Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.044 0.002 0.000 0.000 0.0021 0.140 0.000 0.069 0.000 0.0011 0.000 0.000 0.000 0.999 0.0002 0.816 0.001 0.021 0.001 0.000
21 0.000 0.997 0.910 0.000 0.997
243
Appendix 3: MIXED Experiment 1
-- The Ws lIIld Zs --
set 11: WO lIIld ZO •Vari8bles in Z: INT
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO
1 0.038 0.024 0.028 0.021 0.019 0.0251 0.001 0.054 0.007 0.070 0.098 0.0632 0.035 0.092 0.550 0.009 0.012 0.0192 0.716 0.001 0.187 0.000 0.068 0.1224 0.153 0.760 0.228 0.045 0.036 0.6904 0.056 0.071 0.000 0.855 0.766 0.081
Vari8bles in Z: INT IX1
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO
1 0.055 0.031 0.043 0.037 0.035 0.0111 0.016 0.012 0.138 0.053 o.on 0.2162 0.035 0.363 0.002 0.012 0.027 0.2792 0.051 0.419 0.339 0.001 0.007 0.4582 0.729 0.170 0.463 0.003 0.061 0.0363 0.113 0.005 0.015 0.895 0.798 0.000
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO zo
1 0.077 0.034 0.002 0.066 0.064 0.0001 0.067 0.326 0.001 0.022 0.042 0.2251 0.020 0.009 0.701 0.005 0.008 0.1972 0.060 0.092 0.291 0.006 0.014 0.5502 0.704 0.539 0.005 0.002 0.043 0.0283 o.on 0.002 0.000 0.899 0.830 0.001
Variables in z: INT IX1 IX2 IX3
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO ZO
0.294 0.309 0.017 0.000 0.011 0.0300.028 0.013 0.412 0.000 0.003 0.4820.000 0.003 0.017 0.510 0.450 0.0040.001 0.003 0.052 0.482 0.454 0.0190.060 0.017 0.495 0.007 0.078 0.3870.617 0.655 0.007 0.000 0.005 0.077
Variables in Z: INT IX1 IX2 IX3 WO ZO
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO zo
0.345 0.329 0.046 0.000 0.000 0.0050.002 0.019 0.283 0.001 0.000 0.6810.000 0.000 0.001 0.4n 0.527 0.0000.000 0.000 0.003 0.524 0.473 0.0000.015 0.043 0.643 0.002 0.000 0.3060.638 0.604 0.023 0.000 0.000 0.007
244
Appendix 3: MIXED Experiment 1
-- The Ws end Zs --
Set 12: YO end Z1
Veriables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1
1 D.035 0.005 0.018 0.018 0.015 0.0052 0.000 0.011 0.000 0.069 0.101 0.0092 0.043 0.013 0.430 0.013 0.017 0.0033 0.822 0.003 0.219 0.001 0.062 0.0074 0.101 0.000 0.012 0.897 0.799 0.0009 0.000 0.968 0.320 0.002 0.005 0.976
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1
1 0.053 0.022 0.039 0.034 0.031 0.0171 0.001 0.031 0.095 0.032 0.042 0.2122 0.047 0.347 0.009 0.036 0.061 0.0412 0.783 0.390 0.089 0.001 0.062 0.0463 0.008 0.210 0.736 0.009 0.007 0.6783 0.108 0.001 0.032 0.889 0.797 0.007
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z1
1 0.076 0.036 0.001 0.063 0.061 0.0061 0.014 0.215 0.006 0.022 0.029 0.273
• 1 0.033 0.014 0.827 0.005 0.012 0.0312 0.325 0.040 0.165 0.009 0.047 0.3842 0.486 0.695 0.001 0.001 0.019 0.3043 0.067 0.000 0.000 0.900 0.832 0.003
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1
1 0.143 0.237 0.000 0.000 0.002 0.1611 0.143 0.000 0.524 0.000 0.068 0.1481 0.000 0.001 0.065 0.4n 0.445 0.0001 0.001 0.001 0.068 0.525 0.413 0.0021 0.406 0.000 0.339 0.002 0.071 0.2472 0.307 0.760 0.004 0.000 0.000 0.442
Variables in Z: INT IX1 IX2 IX3 YO Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 YO Z1
0.216 0.290 0.021 0.000 0.000 0.1320.074 0.005 0.558 0.001 0.000 0.2860.000 0.000 0.001 0.483 0.515 0.0000.000 0.000 0.001 0.514 0.485 0.0000.302 0.005 0.418 0.001 0.000 0.3480.408 0.701 0.001 0.000 0.000 0.234
245
246
Appendix 3: MIXED Experflllent 1
Appendix 3: MIXED Experiment 1
*** The wa end Zs *-set 14: WO end 13
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 &X2 BX3 WO 13
1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.072 0.102 0.0002 0.042 0.000 0.009 0.012 0.015 0.0003 0.819 0.000 0.004 0.001 0.064 0.0004 0.105 0.000 0.000 0.896 0.803 0.000
95 0.000 1.000 0.986 0.003 0.000 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13
1 0.045 0.001 0.001 0.026 0.024 0.0011 0.003 0.005 0.001 0.034 0.037 0.0032 0.055 0.024 0.000 0.042 0.073 0.0002 0.787 0.027 0.002 0.001 0.066 0.0004 0.110 0.001 0.000 0.893 0.800 0.000
22 0.000 0.943 0.995 0.004 0.000 0.996
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13
1 0.055 O.OOS 0.000 0.026 0.023 0.0051 0.004 0.006 0.009 0.062 0.074 0.0062 0.031 0.001 0.428 O.OOS 0.010 0.0002 0.836 0.004 0.040 0.007 0.061 0.0033 0.069 0.000 0.000 0.896 0.832 0.000
10 0.005 0.984 0.522 0.004 0.000 0.986
Vari ables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13
1 0.052 0.009 0.002 0.000 0.001 O.ooa1 0.153 0.000 0.311 0.000 0.034 0.0011 0.000 0.000 0.010 0.504 0.456 0.0001 0.001 0.000 0.012 0.496 0.485 0.0002 0.7'92 0.005 0.095 0.001 0.023 0.002
10 0.003 0.987 0.570 0.000 0.001 0.988
Variables in Z: INT IX1 IX2 IX3 WO 13
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO 13
1 0.041 0.015 0.000 0.000 0.000 0.0141 0.161 0.000 0.292 0.001 0.000 0.0031 0.000 0.000 0.000 0.535 0.464 0.0001 0.000 0.000 0.000 0.463 0.536 0.0002 0.7'96 0.005 0.103 0.001 0.000 0.0018 0.002 0.981 0.604 0.000 0.000 0.981
247
Appendix 3: MIXED Experilller'lt 1
*** The ... end Zs ***
set 15: WI) end Z4
Vari~les in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 WI) Z4
1 0.034 0.000 0.000 0.011 0.015 0.0002 0.000 0.000 0.000 0.011 0.102 0.0002 0.042 0.000 0.001 0.012 0.015 0.0003 0.819 0.000 0.000 0.001 0.064 0.0004 0.105 0.000 0.000 0.891 0.802 0.000
284 0.000 1.000 0.998 0.009 0.001 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4
1 0.045 0.000 0.000 0.025 0.024 0.0001 0.003 0.001 0.000 0.035 0.031 0.0002 0.055 0.003 0.000 0.041 0.013 0.0002 0.781 0.003 0.000 0.001 0.065 0.0004 0.110 0.000 0.000 0.891 0.7'99 0.000
61 0.000 0.993 0.999 0.001 0.001 1.000
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4
1 0.055 0.001 0.000 0.025 0.022 0.0011 0.004 0.001 0.002 0.063 0.014 0.0012 0.031 0.000 0.092 0.005 0.010 0.0002 0.839 0.001 O.ooa 0.001 0.062 0.0003 0.069 0.000 0.000 0.891 0.830 0.000
28 0.001 0.998 0.891 0.010 0.002 0.998
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4
1 0.051 0.001 0.000 0.000 0.001 0.0011 0.153 0.000 0.061 0.000 0.036 0.0001 0.000 0.000 0.002 0.502 0.456 0.0001 0.001 0.000 0.003 0.498 0.482 0.0002 0.795 0.001 0.020 0.001 0.024 0.000
21 0.000 0.998 0.901 0.000 0.002 0.998
Variables in Z: INT IX1 IX2 IX3 WI) Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z4
1 0.040 0.002 0.000 0.000 0.000 0.0021 0.161 0.000 0.058 0.001 0.000 0.0001 0.000 0.000 0.000 0.536 0.463 0.0001 0.000 0.000 0.000 0.462 0.531 0.0002 0.7'99 0.001 0.020 0.001 0.000 0.000
22 0.000 0.991 0.922 0.000 0.000 0.998
248
Appendix 3: MIXED Experiment 1
- The wa ..:I Zs *-set 16: W1 ..:I ZO
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 W1 ZO
1 0.035 0.021 0.023 0.004 0.004 0.0211 0.003 0.050 0.020 0.012 0.013 0.0692 0.035 0.099 0.535 0.001 0.001 0.0233 0.822 0.002 0.200 0.003 0.003 0.1194 0.100 0.828 0.215 0.000 0.000 0.769
10 0.004 0.001 0.008 0.981 0.979 0.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO
1 0.048 0.025 0.032 0.007 0.007 0.0071 0.022 0.019 0.151 0.008 0.009 0.2022 0.032 0.380 0.002 0.003 0.003 0.2912 0.046 0.401 0.350 0.000 0.001 0.4753 0.849 0.175 0.455 0.002 0.002 0.0269 0.003 0.000 0.009 0.981 0.979 0.000
VariabLes in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO
1 0.062 0.024 0.001 0.011 0.011 0.0001 0.067 0.350 0.002 0.004 0.004 0.2361 0.022 0.011 0.699 0.001 0.001 0.1992 0.061 0.109 0.292 0.001 0.001 0.5382 0.785 0.506 0.006 0.003 0.003 0.0278 0.004 0.000 0.000 0.980 0.979 0.000
VariabLes in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO
0.295 0.312 0.018 0.000 0.007 0.0300.016 0.015 0.297 0.099 0.153 0.3560.012 0.000 0.119 0.402 0.304 0.1190.001 0.001 0.056 0.473 0.505 0.0100.061 0.013 0.504 0.026 0.030 0.4070.615 0.659 0.006 0.000 0.002 0.078
VariabLes in Z: INT IX1 IX2 IX3 W1 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 ZO
0.342 0.324 0.051 0.000 0.000 0.0080.001 0.016 0.312 0.002 0.000 0.6540.000 0.000 0.001 0.487 0.509 0.0000.000 0.000 0.001 0.510 0.491 0.0010.017 0.058 0.611 0.000 0.000 0.3340.640 0.602 0.024 0.000 0.000 0.003
249
Appendix 3: MIXED Experiment 1
*** The Ws 8nd zs ***set 17: W1 8nd Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z1
1 0.032 0.004 0.015 0.003 0.003 0.0042 0.001 0.011 0.006 0.012 0.013 0.0102 0.044 0.014 0.426 0.001 0.002 0.0033 0.919 0.003 0.234 0.002 0.003 0.0069 0.000 0.916 0.318 0.035 0.032 0.926
10 0.004 0.051 0.002 0.947 0.947 0.050
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.046 0.019 0.030 0.006 0.006 0.0121 0.005 0.011 0.112 0.005 0.007 0.1932 0.046 0.380 0.007 0.005 0.006 0.0613 0.823 0.464 0.028 0.002 0.002 0.1163 0.077 0.127 0.821 0.000 0.000 0.6149 0.003 0.000 0.004 0.981 0.979 0.003
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.061 0.024 0.001 0.010 0.011 0.0031 0.018 0.228 0.004 0.003 0.003 0.2822 0.029 0.014 0.852 0.001 0.001 0.0252 0.338 0.060 0.142 0.004 0.005 0.4122 0.551 0.673 0.000 0.001 0.001 0.2758 0.003 0.000 0.000 0.980 0.979 0.004
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.142 0.239 0.000 0.000 0.000 0.1631 0.122 0.000 0.339 0.047 0.244 0.1211 0.019 0.000 0.219 0.481 0.221 0.0181 0.006 0.000 0.155 0.455 0.417 0.0051 0.412 0.002 0.282 0.016 0.114 0.2442 0.299 0.759 0.004 0.000 0.004 0.449
Variables in Z: INT IX1 IX2 IX3 W1 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
0.212 0.284 0.021 0.000 0.000 0.1330.073 0.005 0.564 0.004 0.000 0.2790.000 0.000 0.001 0.488 0.507 0.0000.000 0.000 0.005 0.506 0.492 0.0000.310 0.005 0.409 0.002 0.000 0.3490.406 0.707 0.001 0.000 0.000 0.239
250
Appendix 3: MIXED Experiment'
.- The Ws and Zs --
set 18: W1 and Z2
Variebles in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z2
1 0.031 0.000 0.003 0.003 0.003 0.0002 0.001 0.001 0.001 0.012 0.013 0.0012 0.043 0.002 0.083 0.001 0.002 0.0003 0.920 0.000 0.043 0.002 0.003 0.001
10 0.004 0.000 0.002 0.980 0.978 0.00030 0.001 0.997 0.868 0.001 0.001 0.998
Variebles in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z2
1 0.043 0.007 0.007 0.005 0.005 0.0041 0.000 0.016 0.017 0.005 0.006 0.0272 0.056 0.186 0.002 0.007 0.007 0.0043 0.896 0.176 0.021 0.002 0.003 0.0037 0.002 0.615 0.953 0.000 0.000 0.9619 0.003 0.001 0.000 0.980 0.979 0.001
Variebles in Z: INT BX1 BX2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2
1 0.057 0.022 0.000 0.007 0.007 0.0201 0.000 0.064 0.014 0.007 0.007 0.0752 0.028 0.009 0.803 0.001 0.001 0.0022 0.889 0.020 0.072 0.004 0.005 0.0344 0.021 0.883 0.111 0.000 0.000 0.8679 0.004 0.002 0.000 0.980 0.979 0.002
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2
1 0.065 0.076 0.002 0.000 0.000 0.0751 0.131 0.000 0.500 0.036 0.153 0.0111 0.017 0.000 0.135 0.474 0.315 0.0011 0.000 0.000 0.040 0.488 0.511 0.0002 0.769 0.032 0.201 0.002 0.018 0.0273 0.018 0.890 0.122 0.000 0.003 0.885
Variables in Z: INT BX1 BX2 BX3 W1 Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W1 Z2
1 0.064 0.117 0.001 0.000 0.000 0.1101 0.147 0.000 0.602 0.001 0.000 0.041, 0.000 0.000 0.000 0.495 0.502 0.000, 0.000 0.000 0.001 0.504 0.498 0.000, 0.764 0.029 0.236 0.001 0.000 0.0203 0.024 0.854 0.160 0.000 0.000 0.828
251
Appendix 3: MIXED Expe,.illlent 1
.- The Ws n Zs'-
set 19: '11 n Z3
Va,.iebl.. in Z: INT
CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 BX3 '11 Z3
1 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.009 0.001 0.002 0.0003 0.920 0.000 0.005 0.002 0.003 0.000
10 0.004 0.000 0.000 0.980 0.979 0.00097 0.000 1.000 0.986 0.001 0.000 1.000
Va,.iebl.. in Z: INT IX1
CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3
1 0.042 0.001 0.001 0.005 0.005 0.0001 0.000 0.003 0.002 0.005 0.006 0.0032 0.056 0.028 0.000 0.007 0.008 0.0003 0.898 0.026 0.002 0.002 0.003 0.0009 0.003 0.000 0.000 0.975 0.976 0.000
23 0.000 0.942 0.995 0.006 0.003 0.996
Va,.iebl.. in Z: INT IX1 IX2
CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3
1 0.051 0.004 0.000 0.006 0.006 0.0041 0.000 0.007 0.008 0.009 0.009 0.0082 0.029 0.001 0.433 0.001 0.001 0.0002 0.910 0.004 0.036 0.004 0.005 0.0039 0.003 0.019 0.008 0.935 0.944 0.019
11 0.006 0.965 0.515 0.045 0.035 0.967
Va,.iables in Z: INT IX1 IX2 IX3
CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3
1 0.052 0.009 0.001 0.000 0.001 0.0081 0.134 0.000 0.280 0.025 0.096 0.0011 0.014 0.000 0.041 0.479 0.366 0.0001 0.000 0.000 0.009 0.495 0.525 0.0002 0.797 0.005 0.096 0.001 0.006 0.002
10 0.003 0.987 0.5n 0.000 0.006 0.989
Va,.iabl.. in Z: INT IX1 IX2 IX3 W1 Z3
CI Va,.iance P,.opo,.tions fo" Coefficients ofINT IX1 IX2 IX3 W1 Z3
1 0.043 0.014 0.000 0.000 0.000 0.0131 0.154 0.000 0.286 0.001 0.000 0.0031 0.000 0.000 0.000 0.497 0.500 0.0001 0.000 0.000 0.000 0.502 0.500 0.0002 0.799 0.004 0.098 0.001 0.000 0.0018 0.004 0.982 0.616 0.000 0.000 0.982
252
Appendix 3: MIXED Experiment 1
- The WII lind Zs -
set 20: "1 lind Z4
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4
1 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.001 0.001 0.002 0.0003 0.920 0.000 0.001 0.002 0.003 0.000
10 0.004 0.000 0.000 0.947 0.953 0.000294 0.000 1.000 0.998 0.034 0.027 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4
1 0.042 0.000 0.000 0.005 0.005 0.0001 0.000 0.000 0.000 0.005 0.006 0.0002 0.057 0.003 0.000 0.006 0.007 0.0003 0.898 0.003 0.000 0.002 0.003 0.0009 0.003 0.000 0.000 0.946 0.950 0.000
69 0.000 0.993 0.999 0.035 0.029 1.000
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4
1 0.051 0.000 0.000 0.005 0.005 0.0001 0.001 0.001 0.002 0.008 0.009 0.0012 0.029 0.000 0.090 0.001 0.001 0.0002 0.915 0.001 0.007 0.004 0.005 0.0009 0.004 0.000 0.000 0.926 0.934 0.000
30 0.001 0.998 0.901 0.055 0.046 0.998
Variables in Z: INT IX1 IX2 IlG
CI Variance Proportions for Coefficients ofINT Ix1 IX2 IlG 111 Z4
1 0.051 0.001 0.000 0.000 0.000 0.0011 0.134 0.000 0.059 0.025 0.094 0.0001 0.014 0.000 0.009 0.479 0.354 0.0002 0.000 0.000 0.002 0.495 0.508 0.0002 0.800 0.001 0.020 0.001 0.006 0.000
28 0.000 0.998 0.910 0.000 0.038 0.998
Variables in Z: INT IX1 IX2 IlG 111 Z4
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IlG 111 Z4
1 0.042 0.002 0.000 0.000 0.000 0.0021 0.153 0.000 0.055 0.001 0.000 0.0001 0.000 0.000 0.000 0.497 0.500 0.0001 0.000 0.000 0.000 0.502 0.500 0.0002 0.805 0.001 0.018 0.001 0.000 0.000
23 0.000 0.998 0.926 0.000 0.000 0.998
253
Appendix 3: MIXED ExperiMent 1
-* The wa 8nd Zs -*set 21: W2 8nd ZD
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO
1 0.035 0.019 0.023 0.000 0.000 0.0202 0.003 0.055 0.016 0.001 0.001 0.0112 0.033 0.094 0.554 0.000 0.000 0.0203 0.826 0.001 0.190 0.000 0.000 0.1214 0.103 0.820 0.216 0.000 0.000 0.161
30 0.001 0.010 0.001 0.998 0.998 0.001
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 \,/2 ZO
1 0.046 0.024 0.032 0.001 0.001 0.0011 0.021 0.017 0.155 0.001 0.001 0.2192 0.035 0.388 0.001 0.000 0.000 0.2152 0.046 0.395 0.361 0.000 0.000 0.4n3 0.852 0.176 0.450 0.000 0.000 0.026
27 0.000 0.000 0.001 0.998 0.998 0.001
Variables in Z: INT IX1 Ix2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO
1 0.059 0.023 0.001 0.001 0.001 0.0001 0.067 0.349 0.002 0.000 0.000 0.2412 0.022 0.012 0.710 0.000 0.000 0.1882 0.061 0.111 0.282 0.000 0.000 0.5432 0.787 0.505 0.005 0.000 0.000 0.026
26 0.003 0.000 0.000 0.998 0.998 0.002
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO
0.296 0.316 0.018 0.000 0.000 0.0320.003 0.000 0.015 0.352 0.413 0.0660.025 0.012 0.407 0.092 0.010 0.3950.053 0.012 0.550 0.049 0.006 0.3660.021 0.007 0.004 0.503 0.555 0.0530.601 0.653 0.006 0.004 0.016 0.088
Variables in Z: INT IX1 IX2 IX3 W2 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO
0.344 0.325 0.047 0.001 0.000 0.0020.000 0.000 0.000 0.483 0.485 0.0000.001 0.021 0.256 0.003 0.003 0.7020.013 0.044 0.671 0.000 0.001 0.2860.000 0.007 0.002 0.509 0.510 0.0030.642 0.603 0.024 0.003 0.002 O.OOS
254
Appendix 3: MIXED Experiment 1
*** The Ws end Zs ***-.set 22: W2 end Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.001 0.001 0.0102 0.042 0.013 0.438 0.000 0.000 0.0033 0.925 0.003 0.226 0.000 0.000 0.0079 0.000 0.965 0.316 0.000 0.000 0.976
31 0.001 0.003 0.000 0.998 0.998 0.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.044 0.018 0.029 0.001 0.001 0.0121 0.004 0.017 0.112 0.001 0.001 0.2072 0.049 0.375 0.010 0.001 0.001 0.0503 0.826 0.462 0.027 0.000 0.000 0.1133 0.076 0.128 0.821 0.000 0.000 0.618
28 0.000 0.001 0.000 0.998 0.998 0.001
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.059 0.023 0.001 0.001 0.001 0.0021 0.019 0.228 0.004 0.000 0.000 0.2832 0.027 0.014 0.859 0.000 0.000 0.0242 0.342 0.061 0.135 0.000 0.000 0.4172 0.550 0.674 0.000 0.000 0.000 0.274
26 0.002 0.000 0.000 0.998 0.998 0.001
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.140 0.237 0.000 0.000 0.002 0.1641 0.012 0.001 0.002 0.411 0.425 0.0031 0.136 0.000 0.605 0.015 0.002 0.1341 0.225 0.000 0.257 0.267 0.177 0.1351 0.183 0.002 0.131 0.307 0.392 0.1192 0.304 0.759 0.004 0.000 0.001 0.445
Variables in Z: INT IX1 IX2 IX3 W2 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
0.215 0.287 0.019 0.000 0.000 0.1290.073 0.004 0.555 0.005 0.001 0.2860.000 0.000 0.004 0.481 0.483 0.0010.002 0.000 0.006 0.508 0.513 0.0010.290 0.005 0.415 0.006 0.003 0.3530.420 0.703 0.001 0.001 0.000 0.231
255
Appendix 3: MIXED Expe";.."t 1
-ThewanZ.*- ..set 23: W2 lind Z2
Variables in Z: INT
CI Variance Proportions for Coefficient. ofINT IX1 IX2 IX3 W2 Z2
1 D.D31 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.001 0.001 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.926 0.000 0.042 0.000 0.000 0.001
29 0.001 0.816 0.729 0.142 0.142 0.82631 0.000 0.180 0.140 0.856 0.856 0.1n
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.041 0.006 0.007 0.001 0.001 0.0041 0.000 0.018 0.017 0.001 0.001 0.0282 0.057 0.184 0.002 0.001 0.001 0.0033 0.899 '0.176 0.021 0.000 0.000 0.0037 0.002 0.615 0.953 0.000 0.000 0.960
28 0.000 0.002 0.000 0.998 0.998 0.002
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.056 0.021 0.000 0.001 0.001 0.0181 0.001 0.065 0.014 0.001 0.001 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.894 0.020 0.070 0.000 0.000 0.0344 0.020 0.884 0.111 0.000 0.000 0.869
27 0.003 0.001 0.000 0.998 0.998 0.000
Variables in z: INT IX1 Ix2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.065 0.076 0.002 0.000 0.000 0.0751 0.000 0.000 0.003 0.426 0.428 0.0001 0.148 0.001 0.658 0.002 0.001 0.0122 0.020 0.001 0.004 0.559 0.547 0.0012 0.748 0.031 0.210 0.013 0.022 0.0283 0.019 0.890 0.121 0.000 0.001 0.884
Variables in Z: INT IX1 IX2 IX3 W2 Z2
CI Variance Proportions for Coefficients ofINT IX1 BX2 IX3 W2 Z2
1 0.060 0.110 0.001 0.000 0.000 0.1041 0.155 0.000 0.590 0.002 0.000 0.0371 0.000 0.000 0.001 0.486 0.485 0.0001 0.003 0.000 0.005 0.506 0.512 0.0001 0.757 0.026 0.235 0.006 0.003 0.0173 0.024 0.863 0.169 0.000 0.000 0.842
256
Appendix 3: MIXED Experiment 1
.- The wa lind Z• .-10.
set 24: W2 rd 13
Variebles in z: INT
CI Variance Proportions for Coefficient. ofINT IX1 IX2 BX3 W2 13
1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.927 0.000 0.005 0.000 0.000 0.000
31 0.001 0.000 0.000 0.996 0.997 0.00098 0.000 1.000 0.986 0.002 0.001 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 13
1 0.040 0.001 0.001 0.001 0.001 0.0001 0.000 0.003 0.002 0.001 0.001 0.0032 0.057 0.028 0.000 0.001 0.001 0.0003 0.901 0.026 0.002 0.000 0.000 0.000
23 0.000 0.936 0.990 0.002 0.003 0.99028 0.000 0.006 0.005 0.996 0.995 0.006
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 13
1 0.050 0.003 0.000 0.001 0.001 0.0031 0.000 0.007 0.008 0.001 0.001 0.0082 0.028 0.001 0.434 0.000 0.000 0.0002 0.913 0.004 0.035 0.000 0.000 0.003
10 0.005 0.981 0.521 0.000 0.000 0.98328 0.003 0.003 0.001 0.998 0.998 0.003
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2 13
1 0.052 0.009 0.002 0.000 0.000 0.0081 0.000 0.000 0.002 0.426 0.428 0.0001 0.150 0.000 0.327 0.002 0.001 0.0012 0.019 0.000 0.002 0.560 0.547 0.0002 0.776 0.005 0.097 0.012 0.021 0.002
10 0.002 0.987 0.571 0.000 0.002 0.988
Variables in Z: INT BX1 BX2 BX3 W2 13
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 13
1 0.042 0.014 0.000 0.000 0.000 0.0141 0.159 0.000 0.291 0.001 0.000 0.0031 0.000 0.000 0.000 0.487 0.485 0.0001 0.003 0.000 0.002 0.505 0.512 0.0002 0.794 0.004 0.100 0.006 0.003 0.0018 0.002 0.981 0.607 0.000 0.000 0.982
257
Appendix 3: MIXED Experi.ent 1
*- The W8 8nd Zs *-set 25: W2 8nd Z4
Variables in Z: INT
CI VarilnCe Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.927 0.000 0.001 0.000 0.000 0.000
31 0.001 0.000 0.000 0.998 0.998 0.000291 0.000 1.000 0.998 0.000 0.000 1.000
Variables in Z: INT IX1
CI VarilnCe Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.040 0.000 0.000 0.001 0.001 0.0001 0.000 0.000 0.000 0.001 0.001 0.0002 0.058 0.003 0.000 0.001 0.001 0.0003 0.901 0.003 0.000 0.000 0.000 0.000
28 0.000 0.000 0.000 0.998 0.998 0.00069 0.000 0.993 0.999 0.000 0.000 1.000
Variables in Z: INT IX1 IX2
CI Variance Proportiona for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.050 0.000 0.000 0.001 0.001 0.0001 0.000 0.001 0.002 0.001 0.001 0.0012 0.028 0.000 0.094 0.000 0.000 0.0002 0.918 0.001 0.007 0.000 0.000 0.000
28 0.003 0.048 0.042 0.940 0.936 0.04829 0.000 0.950 0.856 0.058 0.062 0.950
Variables in Z: INT IX1 BX2 BX3
CI VarilnCe Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z4
1 0.051 0.001 0.000 0.000 0.000 0.0011 0.000 0.000 0.000 0.427 0.429 0.0001 0.150 0.000 0.071 0.001 0.001 0.0002 0.023 0.000 0.001 0.558 0.543 0.0002 0.776 0.001 0.021 0.014 0.026 0.000
27 0.000 0.998 0.907 0.000 0.001 0.998
Variables in Z: INT IX1 IX2 IX3 W2 24
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W2 Z4
1 0.041 0.002 0.000 0.000 0.000 0.002 ..1 0.158 0.000 0.061 0.001 0.000 0.0001 0.000 0.000 0.000 0.487 0.485 0.0001 0.004 0.000 0.000 0.505 0.512 0.0002 0.797 0.001 0.021 0.007 0.003 0.000
22 0.000 0.997 0.918 0.000 0.000 0.997
258
Appendix 3: MIXED Experiment 1
- The Ws and Z. -
set 26: W3 and ZO
V.riables in Z: INT
CI V.riance Proportions for Coefficient. ofINT IX1 IX2 1lC3 W3 zo
1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.826 0.001 0.192 0.000 0.000 0.1194 0.101 0.828 0.216 0.000 0.000 0.767
89 0.001 0.001 0.000 1.000 1.000 0.002
V.riables in Z: INT IX1
CI V.riance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 ZO
1 0.045 0.024 0.032 0.000 0.000 0.0071 0.022 0.018 0.154 0.000 0.000 0.2152 0.034 0.387 0.001 0.000 0.000 0.2772 0.046 0.394 0.362 0.000 0.000 0.4713 0.850 0.177 0.450 0.000 0.000 0.025
82 0.003 0.000 0.001 1.000 1.000 0.004
Variables in Z: INT IX1 IX2
CI V.riance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 zo
1 0.059 0.022 0.001 0.000 0.000 0.0001 0.067 0.352 0.002 0.000 0.000 0.2382 0.022 0.012 0.704 0.000 0.000 0.1942 0.061 0.110 0.288 0.000 0.000 0.5412 0.785 0.504 0.005 0.000 0.000 0.026
78 0.006 0.000 0.000 1.000 1.000 0.001
V.riables in Z: INT IX1 IX2 1lC3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO
1 0.057 0.046 0.003 0.235 0.255 0.0011 0.238 0.270 0.015 0.064 0.042 0.0321 0.029 0.011 0.421 0.002 0.000 0.4741 0.060 0.010 0.555 0.003 0.000 0.4101 0.543 0.630 0.005 0.047 0.036 0.0822 0.073 0.032 0.001 0.648 0.667 0.000
V.riables in Z: INT IX1 IX2 1lC3 W3 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 1lC3 W3 ZO
0.338 0.315 0.053 0.004 0.000 0.0020.000 0.001 0.000 0.401 0.406 0.0000.001 0.021 0.235 0.000 0.000 0.n90.014 0.053 0.686 0.000 0.000 0.2620.024 0.067 0.001 0.541 0.544 0.0010.623 0.543 0.024 0.053 0.049 0.006
259
Appendix 3: MIXED Experiment
- The wa end Zs -*...
Set 27: W3 end Z1
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.437 0.000 0.000 0.0033 0.923 0.003 0.227 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975
92 0.002 0.001 0.000 1.000 1.000 0.001
Variabl.. in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.044 0.018 0.029 0.000 0.000 0.0121 0.004 0.016 0.113 0.000 0.000 0.2052 0.049 0.377 0.010 0.000 0.000 0.0523 0.824 0.462 0.027 0.000 0.000 0.1133 0.075 0.128 0.821 0.000 0.000 0.618
83 0.003 0.000 0.000 1.000 1.000 0.001
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.059 0.022 0.001 0.000 0.000 0.0021 0.019 0.228 0.004 0.000 0.000 0.2832 0.027 0.013 0.859 0.000 0.000 0.0242 0.340 0.061 0.135 0.000 0.000 0.4182 0.549 0.674 0.000 0.000 0.000 0.273
78 0.006 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.143 0.233 0.001 0.004 0.007 0.1551 0.000 0.005 0.000 0.291 0.291 0.0101 0.140 0.000 0.607 0.006 0.000 0.1361 0.394 0.001 0.387 0.014 0.002 0.2482 0.014 0.001 0.001 0.685 0.700 0.0102 0.309 0.759 0.004 0.001 0.001 0.440
Variables in Z: INT IX1 IX2 IX3 W3 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
0.217 0.282 0.025 0.001 0.000 0.1310.000 0.000 0.000 0.404 0.405 0.0000.069 0.006 0.558 0.000 0.001 0.2890.282 0.004 0.399 0.036 0.040 0.3180.018 0.014 0.019 0.552 0.548 0.0460.414 0.693 0.000 0.007 0.006 0.217
260
Af?pendix 3: MIXED Experilllent
-* The \Is end Zs *-• set 28: W3 end Z2
Variables in Z: INT
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2
1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.925 0.000 0.042 0.000 0.000 0.001
30 0.001 0.995 0.868 0.000 0.000 0.99692 0.001 0.002 0.002 1.000 1.000 0.002
Variables in Z: INT IX1
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2
1 0.041 0.006 0.007 0.000 0.000 0.0041 0.000 0.017 0.017 0.000 0.000 0.0282 0.057 0.184 0.002 0.000 0.000 0.0033 0.897 0.176 0.021 0.000 0.000 0.0037 0.002 0.614 0.950 0.000 0.000 0.957
85 0.003 0.003 0.003 1.000 1.000 0.004
Variables in Z: INT BX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 W3 Z2
1 0.055 0.021 0.000 0.000 0.000 0.0181 0.001 0.065 0.014 0.000 0.000 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.890 0.020 0.070 0.000 0.000 0.0344 0.020 0.881 0.111 0.000 0.000 0.864
81 0.006 0.003 0.000 1.000 1.000 0.006
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2
1 0.065 0.075 0.002 0.000 0.002 0.0741 0.000 0.001 0.000 0.299 0.295 0.0011 0.147 0.001 0.660 0.002 0.000 0.0122 0.746 0.033 0.214 0.012 0.004 0.0272 0.022 0.000 0.002 0.686 0.693 0.0013 0.020 0.890 0.122 0.001 0.005 0.884
Variables in Z: INT BX1 BX2 BX3 W3 Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z2
1 0.065 0.123 0.001 0.000 0.000 0.1151 0.000 0.000 0.000 0.412 0.410 0.0001 0.155 0.000 0.591 0.000 0.001 0.0441 0.484 0.024 0.178 0.182 0.190 0.0111 0.274 0.006 0.073 0.404 0.396 0.0093 0.022 0.847 0.157 0.002 0.002 0.822
261
Appendix 3: MIXED Experiment 1
*** The wa Md Zs ***Set 29: W3 Md Z3 •
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.042 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000
92 0.002 0.000 0.000 1.000 1.000 0.00098 0.000 1.000 0.986 0.000 0.000 1.000
Variabl.. in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3
1 0.040 0.001 0.001 0.000 0.000 0.0001 0.000 0.003 0.002 0.000 0.000 0.0032 0.057 0.028 0.000 0.000 0.000 0.0003 0.899 0.026 0.002 0.000 0.000 0.000
23 0.000 0.942 0.994 0.000 0.000 0.99685 0.003 0.000 0.001 1.000 1.000 0.000
Variabl.. in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3
1 0.050 0.003 0.000 0.000 0.000 0.0031 0.000 0.007 0.008 0.000 0.000 0.0082 0.028 0.001 0.434 0.000 0.000 0.0002 0.910 0.004 0.035 0.000 0.000 0.003
10 0.005 0.983 0.522 0.000 0.000 0.98583 0.006 0.001 0.000 1.000 1.000 0.001
Variabl.. in Z: INT IX1 IX2 8X3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3
1 0.052 0.009 0.002 0.000 0.001 0.0081 0.001 0.000 0.000 0.297 0.298 0.0001 0.148 0.000 0.328 0.003 0.000 0.0012 0.761 O.OOS 0.099 0.020 0.007 0.0022 0.035 0.000 0.001 0.680 0.694 0.000
10 0.002 0.987 0.570 0.000 0.001 0.988
Variables in Z: INT IX1 IX2 IX3 W3 Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z3
1 0.042 0.015 0.000 0.000 0.000 0.0141 0.000 0.000 0.000 0.414 0.412 0.0001 0.164 0.000 0.290 0.000 0.002 0.0042 0.503 0.003 0.081 0.17'9 0.188 0.0012 0.290 0.001 0.027 0.407 0.398 0.0008 0.001 0.981 0.602 0.000 0.000 0.981
262
Appendix 3: MIXED Experiment 1
*** The ... lind Zs ***
set 30: W3 lind Z4
Variebl.. in Z: INT
CI Variance Proportions for Coefficients oflilT 1X1 1X2 IX3 W3 Z4
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000
92 0.002 0.000 0.000 0.996 0.997 0.000292 0.000 1.000 0.998 0.003 0.003 1.000
Variables in Z: lilT IX1
CI Variance Proportions for Coefficients oflilT IX1 IX2 BX3 W3 Z4
1 0.040 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.058 0.003 0.000 0.000 0.000 0.0003 0.899 0.003 0.000 0.000 0.000 0.000
69· 0.000 0.971 0.978 0.009 0.010 0.97885 0.003 0.022 0.022 0.991 0.990 0.022
Variables in Z: lilT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4
1 0.050 0.000 0.000 0.000 0.000 0.0001 0.000 0.001 0.002 0.000 0.000 0.0012 0.028 0.000 0.093 0.000 0.000 0.0002 0.914 0.001 0.007 0.000 0.000 0.000
29 0.001 0.995 0.895 0.000 0.000 0.99583 0.007 0.003 0.003 1.000 1.000 0.003
Variables in Z: INT 1X1 IX2 BX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4
1 0.051 0.001 0.000 0.000 0.001 0.0011 0.001 0.000 0.000 0.297 0.297 0.0001 0.148 0.000 0.071 0.003 0.000 0.0002 0.766 0.001 0.021 0.019 0.007 0.0002 0.033 0.000 0.000 0.681 0.694 0.000
27 0.000 0.998 0.907 0.000 0.001 0.998
Variables in Z: INT IX1 IX2 IX3 W3 Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z4
1 0.040 0.002 0.000 0.000 0.000 0.0021 0.000 0.000 0.000 0.414 0.412 0.0001 0.165 0.000 0.062 0.000 0.002 0.0012 0.507 0.000 0.017 0.177 0.185 0.0002 0.286 0.000 0.006 0.409 0.401 0.000
21 0.001 0.997 0.915 0.000 0.000 0.997
263
Appendix 3: MIXED Experilllent 1
*** The Ws end Zs ***set 31: W4 end ZO
Vari~les in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 &X2 8X3 W4 ZO
1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.825 0.001 0.192 0.000 0.000 0.1194 0.102 0.828 0.216 0.000 0.000 0.765
318 0.002 0.001 0.001 1.000 1.000 0.005
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 8X3 W4 ZO
1 0.045 0.024 0.032 0.000 0.000 0.0071 0.021 0.018 0.155 0.000 0.000 0.2162 0.034 0.387 0.001 0.000 0.000 0.2772 0.046 0.394 0.362 0.000 0.000 0.4n3 0.852 0.177 0.450 0.000 0.000 0.025
293 0.002 0.000 0.001 1.000 1.000 0.004
Variables in Z: INT IX1 IX2
Cl Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 ZO
1 0.059 0.022 0.001 0.000 0.000 0.0001 0.067 0.351 0.002 0.000 0.000 0.2392 0.022 0.012 0.706 0.000 0.000 0.1902 0.061 O. "1 0.286 0.000 0.000 0.5372 0.790 0.503 0.005 0.000 0.000 0.026
2n 0.000 0.000 0.000 1.000 1.000 0.008
Variables in Z: INT IX1 BX2 IX3
Cl Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 ZO
1 0.001 0.000 0.000 0.085 0.085 0.0011 0.296 0.316 0.018 0.000 0.000 0.0301 0.029 0.012 0.421 0.000 0.000 0.4741 0.061 0.010 0.555 0.000 0.000 0.4122 0.612 0.661 0.006 0.000 0.000 0.0783 0.002 0.000 0.000 0.914 0.915 0.005
Variables in Z: INT IX1 IX2 IX3 W4 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 ZO
1 0.000 0.000 0.000 0.135 0.135 0.0001 0.337 0.325 0.037 0.000 0.000 0.0021 0.000 0.014 0.2n 0.000 0.000 0.7011 0.011 0.036 0.674 0.000 0.000 0.2922 0.648 0.625 0.016 0.000 0.000 0.0043 0.003 0.000 0.000 0.865 0.865 0.000
264
Appendix 3: MIXED Experiment
*** The Ws end Zs ***•
set 32: W4 end Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.436 0.000 0.000 0.0033 0.923 0.003 0.226 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975
325 0.002 0.001 0.003 1.000 1.000 0.001
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.044 0.018 0.029 0.000 0.000 0.0121 0.004 0.016 0.113 0.000 0.000 0.2052 0.049 0.377 0.010 0.000 0.000 0.0523 0.826 0.462 0.027 0.000 0.000 0.1133 0.076 0.127 0.819 0.000 0.000 0.618
295 0.001 0.000 0.002 1.000 1.000 0.000
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.059 0.022 0.001 0.000 0.000 0.0021 0.019 0.228 0.004 0.000 0.000 0.283
" 2 0.027 0.014 0.860 0.000 0.000 0.0242 0.342 0.062 0.134 0.000 0.000 0.4192 0.553 0.673 0.000 0.000 0.000 0.2n
271 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.001 0.000 0.000 0.086 0.086 0.0001 0.141 0.238 0.000 0.000 0.000 0.1631 0.141 0.000 0.611 0.000 0.000 0.1371 0.410 0.001 0.384 0.000 0.000 0.2562 0.306 0.760 0.004 0.000 0.000 0.4433 0.001 0.000 0.000 0.914 0.914 0.000
Variables in Z: INT IX1 IX2 IX3 W4 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.000 0.000 0.000 0.144 0.144 0.0001 0.234 0.291 0.021 0.000 0.000 0.1041 0.056 0.004 0.541 0.000 0.000 0.3301 0.215 0.009 0.438 0.000 0.000 0.3952 0.492 0.695 0.000 0.000 0.000 0.1702 0.002 0.001 0.000 0.855 0.855 0.001
265
Appendix 3: MIXED Experiment 1
- The Ws end Zs --
set 33: W4 end Z2 •Varieblea in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2 •
1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.084 0.000 0.000 0.0003 0.925 0.000 0.041 0.000 0.000 0.001
30 0.001 0.986 0.858 0.000 0.000 0.987328 0.002 0.011 0.013 1.000 1.000 0.011
Variebles in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2
1 0.041 0.006 0.007 0.000 0.000 0.0041 0.000 0.017 0.017 0.000 0.000 0.0282 0.057 0.184 0.002 0.000 0.000 0.0033 0.898 0.175 0.021 0.000 0.000 0.0037 0.002 0.613 0.944 0.000 0.000 0.954
301 0.001 0.005 0.009 1.000 1.000 0.008
Varieblea in Z: INT IX1 IX2
CI Variance Proportions for Coefficients of •INT IX1 IX2 IIG W4 Z2
1 0.055 0.021 0.000 0.000 0.000 0.0181 0.001 0.065 0.014 0.000 0.000 0.0762 0.027 0.009 0.805 0.000 0.000 0.0022 0.896 0.020 0.070 0.000 0.000 0.0344 0.020 0.883 0.111 0.000 0.000 0.867
281 0.000 0.002 0.000 1.000 1.000 0.003
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2
1 0.065 0.076 0.002 0.000 0.000 0.0741 0.000 0.000 0.000 0.085 0.085 0.0001 0.147 0.001 0.662 0.000 0.000 0.0122 0.768 0.032 0.214 0.000 0.000 0.0283 0.020 0.764 0.102 0.108 0.103 0.7513 0.000 0.127 0.020 0.806 0.812 0.134
Variables in Z: INT IX1 IX2 IIG W4 Z2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IIG W4 Z2
1 0.070 0.125 0.001 0.000 0.000 0.1171 0.000 0.000 0.000 0.149 0.149 0.0001 0.134 0.000 0.624 0.000 0.000 0.0451 0.769 0.032 0.225 0.000 0.000 0.0262 0.014 0.220 0.037 0.601 0.602 0.2083 0.013 0.623 0.113 0.249 0.249 0.603 •
266
Appendix 3: MIXED Experilllent 1
.. *** The WI and Zs ***set 34: W4 and Z3
Vari8bl.. in Z: INT
CI Varience Proportions for Coefficients ofINT IX1 8X2 IX3 W4 Z3
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000
98 0.000 0.993 0.979 0.000 0.000 0.993321 0.002 0.001 0.001 1.000 1.000 0.006
Vari8bles in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3
1 0.040 0.001 0.001 0.000 0.000 0.0001 0.000 0.003 0.002 0.000 0.000 0.0032 0.051 0.028 0.000 0.000 0.000 0.0003 0.900 0.026 0.002 0.000 0.000 0.000
23 0.000 0.939 0.990 0.000 0.000 0.993301 0.001 0.004 0.004 1.000 1.000 0.004
.: Vari8bles in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z3
1 0.050 0.003 0.000 0.000 0.000 0.0031 0.000 0.001 0.008 0.000 0.000 0.0082 0.028 0.001 0.435 0.000 0.000 0.0002 0.916 0.004 0.035 0.000 0.000 0.003
11 0.005 0.984 0.522 0.000 0.000 0.986286 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3
1 0.052 0.009 0.002 0.000 0.000 0.0081 0.000 0.000 0.000 0.086 0.085 0.0001 0.149 0.000 0.329 0.000 0.000 0.0012 0.795 0.005 0.099 0.000 0.000 0.0024 0.001 0.000 0.000 0.913 0.912 0.000
10 0.003 0.981 0.510 0.002 0.002 0.988
Variables in Z: INT IX1 IX2 IX3 W4 Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3
1 0.045 0.015 0.000 0.000 0.000 0.0151 0.000 0.000 0.000 0.151 0.151 0.0001 0.142 0.000 0.314 0.000 0.000 0.0042 0.809 0.005 0.098 0.000 0.000 0.0013 0.002 0.000 0.000 0.841 0.841 0.0008 0.002 0.980 0.588 0.001 0.001 0.980
267
Apeendix 3: MIXED Experilllllnt 1
*** The wa end Z. --
Set 35: W4 end Z4
Variabl.. in Z: INT
CI Varience Proportions for Coefficient. ofINT IX1 IX2 BX3 W4 Z4
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000
291 0.000 0.997 0.995 0.002 0.002 0.997327 0.002 0.003 0.003 0.998 0.998 0.003
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4
1 0.040 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.058 0.003 0.000 0.000 0.000 0.0003 0.900 0.003 0.000 0.000 0.000 0.000
69 0.000 0.992 0.998 0.000 0.000 0.998301 0.002 0.001 0.001 1.000 1.000 0.001
Variables in Z: INT IX1 IX2
'.CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4
1 0.050 0.000 0.000 0.000 0.000 0.0001 0.000 0.001 0.002 0.000 0.000 0.0012 0.028 0.000 0.094 0.000 0.000 0.0002 0.920 0.001 0.007 0.000 0.000 0.000
29 0.001 0.997 0.897 0.000 0.000 0.998286 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4
1 0.051 0.001 0.000 0.000 0.000 0.0011 0.000 0.000 0.000 0.086 0.085 0.0001 0.149 0.000 0.071 0.000 0.000 0.0002 0.798 0.001 0.021 0.000 0.000 0.0004 0.001 0.000 0.000 0.913 0.912 0.000
27 0.000 0.998 0.907 0.001 0.002 0.998
Variables in Z: INT IX1 IX2 aX3 W4 Z4
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W4 Z4
1 0.043 0.002 0.000 0.000 0.000 0.0021 0.000 0.000 0.000 0.151 0.151 0.0001 0.143 0.000 0.068 0.000 0.000 0.0012 0.812 0.001 0.021 0.000 0.000 0.0003 0.002 0.000 0.000 0.847 0.847 0.000
21 0.000 0.997 0.911 0.001 0.001 0.997
268
APPENDIX 4
MIXED MODEL EXPERIMENT 2COLLINEARITY DIAGNOSTICS
- The WI!-
set 1: WO
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.052 0.039 0.039 0.037 0.0352 0.059 0.143 0.132 0.063 0.1112 0.000 0.461 0.505 0.001 0.0003 0.781 0.357 0.303 0.003 0.0534 0.109 0.000 0.021 0.897 0.802
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.042 0.038 0.039 0.031 0.0292 0.037 0.111 0.095 0.069 0.1152 0.002 0.396 0.603 0.001 0.0003 0.743 0.434 0.232 0.020 0.1054 0.175 0.021 0.031 0.880 0.751
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.065 0.016 0.000 0.064 0.0631 0.000 0.130 0.164 0.006 0.0072 0.427 0.025 0.073 0.027 0.0943 0.340 0.769 0.713 0.052 0.0873 0.169 0.060 0.049 0.851 0.748
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.030 0.069 0.123 0.063 0.0011 0.105 0.149 0.009 0.160 0.0031 0.191 0.022 0.019 0.026 0.5681 0.363 0.017 0.024 0.034 0.4283 0.312 0.743 0.825 0.718 0.000
Variables in Z: INT IX1 IX2 IX3 WO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we
1 0.027 0.062 0.090 0.072 0.0291 0.020 0.081 0.026 0.066 0.1191 0.526 0.012 0.085 0.003 0.0251 0.180 0.191 0.020 0.068 0.1893 0.247 0.654 0.780 0.792 0.639
ApPendix 4: MIXED ExperiMent 2
-- The In--
Set 2: 111
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111
1 0.044 0.033 0.029 0.006 0.0062 0.062 0.119 0.197 0.011 0.0132 0.000 0.532 0.434 0.000 0.0003 0.889 0.315 0.330 0.002 0.0029 0.004 0.000 0.010 0.981 0.979
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111
1 0.037 0.032 0.031 0.005 0.0052 0.040 0.111 0.125 0.012 0.0142 0.002 0.429 0.565 0.000 0.0003 0.918 0.428 0.268 0.002 0.0029 0.003 0.000 0.011 0.981 0.979
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111
1 0.053 0.010 0.000 0.010 0.0101 0.000 0.136 0.163 0.001 0.0012 0.469 0.030 0.082 0.008 0.0093 0.474 0.824 0.755 0.000 0.0008 0.004 0.000 0.000 0.981 0.979
Variables in Z: INT aX1 aX2 aX3
CI Variance Proportions for Coefficients ofINT aX1 BX2 aX3 111
1 0.029 0.065 0.121 0.064 0.0091 0.093 0.158 0.006 0.155 0.0151 0.266 0.011 0.037 0.006 0.5021 0.300 0.023 0.012 0.057 0.4743 0.312 0.743 0.825 0.718 0.000
Variables in Z: INT aX1 BX2 aX3 111
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 111
1 0.034 0.075 0.097 0.061 0.0191 0.016 0.067 0.017 0.079 0.1301 0.498 0.013 0.090 0.004 0.0291 0.194 0.188 0.016 0.069 0.1893 0.258 0.657 0.780 0.787 0.634
270
•
Appendix 4: MIXED Experilllent 2
- The Ws-
set 3: W2
Variables in z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2
1 0.043 0.031 0.029 0.001 0.0012 0.063 0.138 0.183 0.001 0.0012 0.000 0.500 0.465 0.000 0.0003 0.893 0.318 0.322 0.000 0.000
28 0.001 0.013 0.001 0.998 0.998
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
1 0.035 0.031 0.031 0.001 0.0012 0.042 0.121 0.124 0.001 0.0012 0.002 0.418 0.582 0.000 0.0003 0.920 0.430 0.262 0.000 0.000
29 0.000 0.000 0.002 0.998 0.998
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2
1 0.052 0.010 0.000 0.001 0.0011 0.000 0.136 0.164 0.000 0.0002 0.411 0.031 0.081 0.001 0.0013 0.415 0.824 0.155 0.000 0.000
21 0.002 0.000 0.000 0.998 0.998
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2
1 0.024 0.049 0.111 0.016 0.0251 0.055 0.158 0.000 0.100 0.1361 0.509 0.002 0.060 0.001 0.1601 0.101 0.052 0.000 0.101 0.6163 0.310 0.139 0.824 0.716 0.003
Variables in Z: INT IX1 IX2 IX3 W2
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W2
1 0.036 0.091 0.112 0.043 0.0041 0.016 0.042 0.004 0.118 0.1601 0.451 0.020 0.081 0.004 0.0551 0.230 0.110 0.006 0.071 0.1193 0.260 0.670 0.191 0.751 0.603
271
Appendix 4: MIXED Experiment 2
- The wa-
set 3: W3
Vari8bl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W3
1 0.043 0.032 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.456 0.000 0.0003 0.892 0.319 0.325 0.000 0.000
84 0.002 0.000 0.000 1.000 1.000
Vari8bl.. in Z: INT IX1
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX! W3
1 0.035 0.031 0.031 0.000 0.0002 0.042 0.120 0.124 0.000 0.0002 0.002 0.419 0.582 0.000 0.0003 0.919 0.430 0.263 0.000 0.000
87 0.002 0.000 0.000 1.000 1.000
Vari8bles in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W3
1 0.051 0.010 0.000 0.000 0.0001 0.000 0.136 0.164 0.000 0.0002 0.469 0.030 0.081 0.000 0.0003 0.473 0.824 0.755 0.000 0.000
79 0.006 0.000 0.000 1.000 1.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.012 0.006 0.064 0.080 0.1061 0.052 0.210 0.029 0.028 0.0471 0.565 0.002 0.079 0.000 0.0262 0.088 0.072 0.033 0.176 0.7883 0.283 0.710 0.794 0.716 0.033
Variables in Z: INT IX1 IX2 IX! W3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3
1 0.035 0.082 0.122 0.032 0.0291 0.125 0.105 0.020 0.224 0.0011 0.003 0.038 0.001 0.056 0.5091 0.551 0.048 0.032 0.086 0.0083 0.286 0.727 0.826 0.602 0.453
272
Appendix 4: MIXED ExperiJRent 2
*** The wa ***set 5: W4
Vari8bles in Z: INT
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4
1 0.043 0.031 0.029 0.000 0.0002 0.063 0.134 0.190 0.000 0.0002 0.000 0.515 0.455 0.000 0.0003 0.892 0.318 0.324 0.000 0.000
298 0.002 0.001 0.002 1.000 1.000
Vari8bl.. in Z: INT IX1
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4
1 0.035 0.031 0.030 0.000 0.0002 0.042 0.120 0.124 0.000 0.0002 0.002 0.419 0.581 0.000 0.0003 0.919 0.430 0.263 0.000 0.000
310 0.002 0.000 0.002 1.000 1.000
Variables in Z: INT IX1 IX2
..
•
CI
1123
277
Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4
0.051 0.010 0.000 0.000 0.0000.000 0.136 0.164 0.000 0.0000.472 0.031 0.081 0.000 0.0000.476 0.824 0.755 0.000 0.0000.000 0.000 0.000 1.000 1.000
Variables in Z: INT IX1 IX2 IX3
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 W4
1 0.003 0.000 0.037 0.027 0.0311 0.066 0.208 0.049 0.003 0.0051 0.602 0.007 0.073 0.001 0.0013 0.275 0.688 0.724 0.005 0.1465 0.054 0.097 0.117 0.963 0.817
Variables in Z: INT IX1 IX2 IX3 W4
Cl Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4
1 0.005 0.001 0.050 0.073 0.0761 0.069 0.216 0.035 0.016 0.0181 0.599 0.010 0.071 0.007 0.0033 0.000 0.005 0.007 0.644 0.8063 0.327 0.768 0.837 0.261 0.097
273
Appendix 4: MIXED Experilllent 2
*** The Zs ***set 6: ZD
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.050 0.036 0.038 0.037 0.0382 0.041 0.1" 0.104 0.249 0.0912 0.000 0.023 0.509 0.384 0.0043 0.806 0.002 0.133 0.330 0.0983 0.102 0.828 0.216 0.000 0.768
Variabl.. in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.062 0.059 0.064 0.057 0.0141 0.007 0.070 0.064 0.028 0.5792 0.003 0.202 0.037 0.743 0.0062 0.001 0.292 0.598 0.007 0.4003 0.926 0.377 0.238 0.165 0.002
Variables in Z: INT IX1 IX2
CI
11123
Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
0.065 0.124 0.065 0.054 0.0180.133 0.015 0.113 0.178 0.0060.000 0.004 0.018 0.002 0.9600.314 0.038 0.060 0.742 0.0060.488 0.819 0.744 0.024 0.010
Variables in z: INT IX1 IX2 IX3
..
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.029 0.074 0.120 0.055 0.0071 0.046 0.115 0.010 0.155 0.1211 0.499 0.000 0.027 0.004 0.2521 0.112 0.065 0.019 0.072 0.6153 0.314 0.745 0.824 0.713 0.005
Variables in Z: INT IX1 IX2 IX3 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 ZO
1 0.030 0.082 0.121 0.050 0.0051 0.007 0.070 0.003 0.107 0.1681 0.583 O.OOS 0.064 0.001 0.0351 0.128 0.172 0.011 0.067 0.3613 0.251 0.668 0.801 0.776 0.432
274
Appendix 4: MIXED Experiment 2
*** The Zs ***
set 7: Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1
1 0.043 0.007 0.023 0.031 0.0072 0.045 0.022 0.131 0.193 0.0112 0.000 0.001 0.353 0.504 0.0003 0.912 0.002 0.176 0.272 0.0059 0.000 0.968 0.317 0.000 0.976
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1
1 0.058 0.044 0.055 0.053 0.0211 0.010 0.113 0.053 0.023 0.3002 0.003 0.155 0.046 0.759 0.0013 0.023 0.197 0.789 0.005 0.6423 0.906 0.490 0.057 0.161 0.037
Variables in Z: INT IX1 IX2
CI
11223
Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z1
0.030 0.095 0.063 0.019 0.0850.166 0.002 0.066 0.214 0.0110.002 0.012 0.151 0.049 0.7140.346 0.041 0.017 0.701 0.0950.457 0.851 0.702 0.017 0.095
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 Z1
1 0.028 0.097 0.064 0.004 0.0991 0.001 0.017 0.063 0.192 0.0801 0.659 0.000 0.033 0.010 0.0262 0.000 0.126 0.035 0.121 0.7463 0.312 0.760 0.805 0.673 0.049
Variables in Z: INT IX1 IX2 IX3 Z1
CI Variance Proportions for Coefficients ofINT BX1 BX2 Ix3 Z1
1 0.020 0.119 0.037 0.020 0.1151 0.009 0.007 0.097 0.140 0.0411 0.646 0.003 0.047 0.001 0.0162 0.044 0.256 0.005 0.065 0.5393 0.281 0.616 0.813 0.774 0.289
275
Appendix 4: MIXED Experiment 2
*** The Zs ***set 8: Z2
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2
1 0.042 0.001 0.005 0.030 0.0012 0.047 0.002 0.023 0.212 0.0012 0.000 0.000 0.071 0.476 0.0003 0.910 0.000 0.032 0.282 0.001
28 0.001 0.997 0.869 0.000 0.998
Variabl.. in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2
1 0.048 0.015 0.012 0.042 0.0071 0.023 0.061 0.009 0.040 0.0352 0.003 0.090 0.006 0.748 0.0003 0.925 0.206 0.022 0.169 0.0017 0.001 0.628 0.952 0.000 0.958
Variables in Z: INT IX1 IX2
CI
11224
Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2
0.028 0.032 0.029 0.015 0.0430.150 0.004 0.088 0.198 0.0000.094 0.002 0.178 0.627 0.0500.636 0.001 0.248 0.159 0.2070.093 0.962 0.457 0.001 0.700
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! Z2
1 0.025 0.037 0.015 0.003 0.0481 0.000 0.000 0.101 0.176 0.0061 0.651 0.003 0.036 0.024 0.0013 0.223 0.005 0.308 0.592 0.2935 0.100 0.955 0.539 0.204 0.652
Variables in Z: INT IX1 IX2 IX! Z2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z2
1 0.012 0.050 0.015 0.017 0.0661 0.003 0.004 0.116 0.137 0.0071 0.707 0.001 0.024 0.012 0.0003 0.096 0.103 0.095 0.427 0.8204 0.183 0.843 0.750 0.407 0.108
276
Appendix 4: MIXED Experiment 2
-TheZs-
Set 9: 13
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 BX2 BX3 13
1 0.041 0.000 0.001 0.030 0.0002 0.047 0.000 0.003 0.211 0.0002 0.000 0.000 0.008 0.473 0.0003 0.911 0.000 0.004 0.280 0.000
91 0.000 1.000 0.986 0.007 1.000
Variables in Z: INT BX1
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13
1 0.047 0.002 0.001 0.040 0.0011 0.024 0.010 0.001 0.041 0.0032 0.003 0.014 0.001 0.742 0.0003 0.926 0.031 0.003 0.168 0.000
22 0.001 0.944 0.995 0.009 0.996
Variables in Z: INT BX1 BX2
•
CI
1123
13
Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13
0.024 0.004 0.014 0.012 0.0050.152 0.000 0.047 0.198 0.0000.128 0.000 0.091 0.677 0.0030.683 0.003 0.231 0.104 0.0170.014 0.992 0.617 0.010 0.975
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13
1 0.023 0.004 0.006 0.004 0.0051 0.001 0.000 0.052 0.174 0.0002 0.651 0.000 0.018 0.025 0.0003 0.318 0.002 0.265 0.787 0.016
14 0.008 0.993 0.659 0.009 0.978
Variables in Z: INT BX1 BX2 BX3 13
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 13
1 0.011 0.006 0.006 0.015 0.0081 0.001 0.000 0.060 0.139 0.0012 0.705 0.000 0.011 0.015 0.0003 0.276 0.004 0.273 0.826 0.047
11 0.006 0.989 0.650 0.005 0.944
277
Appendix 4: MIXED Experiment 2
... The Zs *-Set 10: Z4
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 &X2 IX3 Z4
1 0.041 0.000 0.000 0.030 0.0002 0.047 0.000 0.000 0.210 0.0002 0.000 0.000 0.001 0.4n 0.0003 0.911 0.000 0.000 0.278 0.000
211 0.000 1.000 0.998 0.010 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Z4
1 0.041 0.000 0.000 0.040 0.0001 0.025 0.001 0.000 0.041 0.0002 0.003 0.002 0.000 0.143 0.0003 0.926 0.004 0.000 0.161 0.000
66 0.000 0.993 0.999 0.008 1.000
Variables in Z: INT BX1 BX2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 Z4
1 0.024 0.001 0.003 0.012 0.0011 0.152 0.000 0.010 0.198 0.0002 0.124 0.000 0.020 0.610 0.0003 0.699 0.000 0.051 0.110 0.002
36 0.002 0.999 0.916 0.010 0.991
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 Z4
1 0.023 0.001 0.001 0.004 0.0011 0.001 0.000 0.011 0.174 0.0002 0.650 0.000 0.004 0.025 0.0003 0.326 0.000 0.059 0.796 0.002
31 0.001 0.999 0.925 0.001 0.991
Variables in Z: INT BX1 BX2 aX3 Z4
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 Z4
1 0.010 0.001 0.001 0.014 0.0011 0.001 0.000 0.013 0.138 0.0002 0.706 0.000 0.002 0.015 0.0003 0.282 0.001 0.061 0.831 0.006
30 0.001 0.998 0.922 0.001 0.993
278
279
Appendix 4: MIXED Experiment 2
Appendix 4: M(XED Experiment 2
-* The Ws end Zs .-
set 12: WO end Z1
Vari~l .. in Z: INT
CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z1
1 0.035 0.005 0.018 0.018 0.015 0.0052 0.000 0.011 0.000 0.069 0.101 0.0092 0.043 0.013 0.430 0.013 0.017 0.0033 0.822 0.003 0.219 0.001 0.062 0.0074 0.101 0.000 0.012 0.897 0.799 0.0009 0.000 0.968 0.320 0.002 0.005 0.976
Variables in Z: INT aX1
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO Z1
1 0.039 0.028 0.033 0'.027 0.025 0.0111 0.001 0.013 0.094 0.029 0.041 0.2402 0.041 0.228 0.004 0.042 0.077 0.0693 0.162 0.024 0.821 0.016 0.038 0.4533 0.593 0.698 0.008 0.006 0.064 0.2214 0.164 0.009 0.040 0.879 0.755 0.006
Variables in Z: INT aX1 aX2
C( Variance Proportions for Coefficients ofINT aX1 aX2 aX3 WO Z1
1 0.059 0.027 0.004 0.053 0.053 0.0171 0.004 0.069 0.101 0.017 0.018 0.0812 0.252 0.001 0.161 0.021 0.060 0.1832 0.200 0.042 0.024 0.005 0.037 0.6153 0.297 0.787 0.645 0.076 0.109 0.1033 0.188 0.075 0.065 0.828 0.723 0.001
Variables in Z: (NT aX1 aX2 aX3
CI Variance Proportions for Coefficients of(NT aX1 aX2 aX3 WO Z1
1 0.028 0.097 0.064 0.004 0.000 0.0991 0.001 0.017 0.063 0.192 0.001 0.0801 0.343 0.000 0.021 0.003 0.423 0.0141 0.317 0.000 0.013 0.007 0.576 0.0112 0.000 0.126 0.035 0.121 0.000 0.7463 0.311 0.760 0.805 0.673 0.000 0.049
Variables in Z: (NT aX1 aX2 aX3 WO Z1
C( Variance Proportions for Coefficients of(NT aX1 aX2 ax3 WO Z1
1 0.017 0.093 0.037 0.002 0.037 0.0871 0.000 0.010 0.043 0.182 0.007 0.0541 0.013 0.045 0.084 0.009 0.149 0.0341 0.705 0.000 0.037 0.003 0.000 0.0082 0.049 0.351 0.064 0.005 0.092 0.4404 0.215 0.500 0.736 0.800 0.716 0.377
280
Appendix 4: MIXED ExperiMent 2
- The \Is 8nd Zs -
Set 13: WO 8nd 12
Variebl.. in Z: INT
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO 12
1 0.034 0.001 0.004 0.011 0.015 0.0012 0.000 0.001 0.000 0.012 0.102 0.0012 0.042 0.002 0.084 0.011 0.015 0.0003 0.811 0.000 0.041 0.001 0.064 0.0014 0.101 0.000 0.003 0.895 0.796 0.000
29 0.000 0.991 0.869 0.004 0.008 0.998
variebles in Z: INT IX1
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2
1 0.036 0.011 0.001 0.023 0.022 0.0041 0.002 0.020 0.015 0.026 0.026 0.0342 0.041 0.104 0.002 0.050 0.095 0.0043 0.138 0.226 0.020 0.018 0.103 0.0014 0.116 0.008 0.005 0.812 0.142 0.0008 0.000 0.631 0.950 0.011 0.012 0.958
Variebles in Z: INT IX1 IX2
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2
1 0.038 0.022 0.014 0.020 0.011 0.031• 1 0.022 0.013 0.053 0.050 0.055 0.013
2 0.255 0.000 0.166 0.024 0.011 0.0222 0.543 0.002 0.311 0.001 0.020 0.2293 0.031 0.015 0.011 0.868 0.796 0.0085 0.105 0.948 0.444 0.035 0.034 0.698
Variables in Z: INT IX1 IX2 IX3
CI Varience Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z2
1 0.026 0.031 0.015 0.003 0.001 0.0481 0.000 0.000 0.101 0.116 0.000 0.0061 0.268 0.002 0.019 0.011 0.511 0.0001 0.385 0.001 0.011 0.012 0.474 0.0013 0.220 0.004 0.311 0.596 0.004 0.2905 0.101 0.955 0.536 0.201 0.005 0.655
Variables in Z: INT IX1 IX2 IX3 WO Z2
CI Varience Proportions for Coefficients ofINT IX1 BX2 BX3 WO Z2
1 0.012 0.042 0.014 0.001 0.018 0.0501 0.001 0.001 0.043 0.174 0.032 0.0111 0.023 0.006 0.138 0.002 0.164 0.0041 0.746 0.000 0.001 0.002 0.036 0.0024 0.052 0.181 0.034 0.331 0.368 0.8584 0.161 0.110 0.111 0.484 0.382 0.076
281
Appendix 4: MIXED Experiment 2
*** The Ws lind Zs ***Set 14: WO end Z3
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z3
1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.072 0.102 0.0002 0.042 0.000 0.009 0.012 0.015 0.0003 0.819 0.000 0.004 0.001 0.064 0.0004 0.105 0.000 0.000 0.896 0.803 0.000
95 0.000 1.000 0.986 0.003 0.000 1.000
V.riables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 WO Z3
1 0.035 0.002 0.001 0.023 0.022 0.0001 0.003 0.003 0.002 0.025 0.027 0.0032 0.047 0.016 0.000 0.051 0.095 0.0003 0.742 0.033 0.003 0.018 0.104 0.0004 0.173 0.002 0.000 0.879 0.753 0.000
24 0.001 0.944 0.995 0.004 0.000 0.996
Variables in Z: INT IX1 IX2
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3
1 0.031 0.003 0.008 0.015 0.013 0.0041 0.027 0.001 0.027 0.056 0.059 0.0012 0.282 0.000 0.087 0.028 0.082 0.0023 0.558 0.003 0.260 0.001 0.030 0.0183 0.089 0.000 0.001 0.897 0.815 0.000
13 0.014 0.992 0.617 0.004 0.000 0.975
V.riables in Z: INT IX1 IX2 IX3
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3
1 0.023 0.004 0.006 0.004 0.000 0.0051 0.001 0.000 0.052 0.175 0.001 0.0011 0.276 0.000 0.009 0.009 0.517 0.0002 0.376 0.000 0.009 0.016 0.481 0.0003 0.317 0.002 0.265 0.788 0.000 0.016
14 0.008 0.993 0.660 0.009 0.001 0.978
V.riables in Z: INT IX1 IX2 IX3 WO Z3
CI V.riance Proportions for Coefficients ofINT IX1 IX2 IX3 WO Z3
1 0.011 0.005 0.006 0.001 0.014 0.0061 0.001 0.000 0.020 0.171 0.039 0.0011 0.045 0.001 0.070 0.002 0.155 0.0002 0.725 0.000 0.000 0.003 0.052 0.0004 0.213 0.005 0.245 0.820 0.738 0.040
12 0.005 0.989 0.659 0.004 0.002 0.952
282
Appendix 4: MIXED ExperiMnt 2
*** The WS end Zs -*set 15: WO end Z4
Variables in Z: INT
CI VarilnCe Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4
1 0.034 0.000 0.000 0.017 0.015 0.0002 0.000 0.000 0.000 0.071 0.102 0.0002 0.042 0.000 0.001 0.012 0.015 0.0003 0.819 0.000 0.000 0.001 0.064 0.0004 0.105 0.000 0.000 0.891 0.802 0.000
284 0.000 1.000 0.998 0.009 0.001 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 we Z4
1 0.035 0.000 0.000 0.023 0.021 0.0001 0.003 0.000 0.000 0.025 0.027 0.0002 0.047 0.002 0.000 0.051 0.095 0.0003 0.743 0.004 0.000 0.018 0.103 0.0004 0.172 0.000 0.000 0.877 0.752 0.000
71 0.000 0.993 0.999 0.007 0.001 1.000
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4
1 0.030 0.000 0.002 0.014 0.012 0.001• 1 0.028 0.000 0.006 0.056 0.060 0.000
2 0.277 0.000 0.019 0.027 0.081 0.0003 0.571 0.000 0.057 0.001 0.033 0.0023 0.092 0.000 0.000 0.891 0.812 0.000
37 0.002 0.999 0.916 0.010 0.002 0.997
Variables in Z: INT IX1 IX2 IlG
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG we Z4
1 0.023 0.001 0.001 0.004 0.000 0.0011 0.001 0.000 0.011 0.174 0.001 0.0001 0.275 0.000 0.002 0.009 0.519 0.0002 0.376 0.000 0.002 0.016 0.478 0.0003 0.325 0.000 0.059 0.796 0.000 0.002
37 0.001 0.999 0.925 0.001 0.002 0.997
Variables in Z: INT IX1 IX2 IlG WO Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG WO Z4
1 0.011 0.001 0.001 0.001 0.014 0.0011 0.001 0.000 0.004 0.170 0.039 0.0001 0.048 0.000 0.014 0.002 0.153 0.0002 0.723 0.000 0.000 0.003 0.054 0.0004 0.217 0.001 0.050 0.824 0.739 0.005
33 0.000 0.998 0.931 0.000 0.000 0.994
283
Appendix 4: MIXED Experiment 2
.- The \Is 8nd Zs .-
set 16: 111 8nd ZO
Vari~l .. in Z: INT
CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 111 ZO
1 0.035 0.021 0.023 0.004 0.004 0.0211 0.003 0.050 0.020 0.012 0.013 0.0692 0.035 0.099 0.535 0.001 0.001 0.0233 0.822 0.002 0.200 0.003 0.003 0.1194 0.100 0.828 0.215 0.000 0.000 0.769
10 0.004 0.001 0.008 0.981 0.979 0.000
Vari~les in Z: INT IX1
CI Variance Proportions for Coefficients ofINT 1X1 IX2 IX3 111 ZO
1 0.036 0.030 0.030 0.005 0.005 0.0052 0.014 0.017 0.128 0.008 0.010 0.2132 0.032 0.223 0.001 0.004 0.004 0.3822 0.001 0.324 0.565 0.000 0.000 0.3973 0.914 0.406 0.268 0.002 0.002 0.002
10 0.003 0.000 0.009 0.981 0.979 0.000
Vari~l .. in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 111 ZO
1 0.053 0.010 0.000 0.010 0.010 0.0011 0.000 0.128 0.152 0.001 0.001 0.0242 0.002 0.003 0.025 0.000 0.000 0.9432 0.470 0.031 0.071 0.008 0.009 0.0223 0.470 0.828 0.752 0.000 0.000 0.0108 0.004 0.000 0.000 0.981 0.979 0.000
Variables in Z: INT IX1 IX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 111 ZO
1 0.029 0.070 0.118 0.057 0.008 0.0061 0.042 0.120 0.007 0.147 0.014 0.1201 0.397 0.000 0.036 0.001 0.249 0.1051 0.082 0.003 0.000 0.002 0.640 0.2601 0.137 0.062 0.016 0.081 0.089 0.5033 0.313 0.745 0.824 0.713 0.000 0.005
Variables in Z: INT BX1 BX2 IX3 111 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 111 ZO
1 0.021 0.083 0.069 0.022 0.045 0.0481 0.014 0.040 0.058 0.000 0.098 0.0921 0.007 0.065 0.004 0.185 0.008 0.0541 0.694 0.000 0.049 0.004 0.001 0.0062 0.070 0.251 0.097 0.001 0.124 0.2644 0.194 0.561 0.n2 0.787 0.n4 0.536
284
Appendix 4: MIXED Experiment 2
-- The Ws 8nd Za --
Set 17: W1 8nd Z1
Var;8blea ;n Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W1 Z1
1 0.032 0.004 0.015 0.003 0.003 0.0042 0.001 0.011 0.006 0.012 0.013 0.0102 0.044 0.014 0.426 0.001 0.002 0.0033 0.919 0.003 0.234 0.002 0.003 0.0069 0.000 0.916 0.318 0.035 0.032 0.926
10 0.004 0.051 0.002 0.947 0.947 0.050
Vari8bles ;n Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.035 0.024 0.026 0.005 0.005 0.0081 0.003 0.003 0.107 0.005 0.007 0.2132 0.041 0.250 0.003 0.007 0.008 0.0983 0.035 0.168 0.810 . 0.000 0.000 0.6253 0.883 0.554 0.051 0.002 0.002 0.052
10 0.003 0.000 0.004 0.981 0.979 0.003
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.052 0.013 0.000 0.010 0.010 0.006• 1 0.000 0.082 0.102 0.001 0.001 0.092
2 0.190 0.000 0.186 0.004 0.004 0.3102 0.310 0.047 0.005 0.004 0.005 0.4933 0.444 0.857 0.707 0.000 0.000 0.0968 0.003 0.000 0.000 0.981 0.979 0.003
Vari8bles in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.028 0.096 0.064 0.005 0.000 0.0971 0.001 0.017 0.057 0.182 0.027 0.0811 0.418 0.000 0.035 0.000 0.311 0.0131 0.242 0.002 0.004 0.024 0.654 0.0102 0.000 0.125 0.036 0.117 0.006 0.7493 0.310 0.760 0.804 0.673 0.001 0.049
Variables in Z: INT IX1 IX2 IX3 W1 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W1 Z1
1 0.018 0.093 0.034 0.001 0.036 0.0891 0.000 0.009 0.047 0.185 0.007 0.0481 0.012 0.044 0.084 0.009 0.152 0.0321 0.697 0.000 0.038 0.003 0.000 0.0092 0.050 0.357 0.060 0.004 0.093 0.4474 0.222 0.497 0.736 0.796 0.713 0.374
285
Appendix 4: MIXED Experiment 2
-- The \Is lind Zs --
Sat 18: .,1 lind Z2
Variables in Z: INT
CI Variance Proportiona for Coefficients of •INT IX1 IX2 BX3 .,1 Z2
1 0.031 0.000 0.003 0.003 0.003 0.0002 0.001 0.001 0.001 0.012 0.013 0.0012 0.043 0.002 0.083 0.001 0.002 0.0003 0.920 0.000 0.043 0.002 0.003 0.001
10 0.004 0.000 0.002 0.980 0.978 0.00030 0.001 0.997 0.868 0.001 0.001 0.998
Variables in Z: INT IX1
CI Variance Proportiona for Coefficients ofINT IX1 IX2 IX3 .,1 Z2
1 0.033 0.010 0.006 0.004 0.004 0.0031 0.000 0.010 0.017 0.004 0.005 0.0322 0.049 0.125 0.001 0.009 0.010 0.0063 0.913 0.227 0.024 0.002 0.002 0.0018 0.001 0.628 0.952 0.000 0.000 0.957
10 0.003 0.000 0.000 0.981 0.979 0.001
Variables in Z: INT BX1 BX2
CI Variance Proportiona for Coefficients ofINT BX1 BX2 BX3 .,1 Z2
1 0.040 0.015 0.006 0.005 0.005 0.0231 0.009 0.021 0.061 0.006 0.006 0.0212 0.277 0.000 0.181 0.007 O.ooa 0.0252 0.576 0.001 0.295 0.001 0.002 0.2305 0.094 0.961 0.457 0.000 0.000 0.6989 0.004 0.002 0.000 0.980 0.979 0.003
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 .,1 Z2
1 0.025 0.037 0.015 0.003 0.000 0.0481 0.001 0.000 0.096 0.170 0.019 0.0061 0.332 0.001 0.034 0.002 0.435 0.0001 0.319 0.002 0.007 0.029 0.544 0.0013 0.224 0.005 0.309 0.590 0.000 0.2925 0.099 0.955 0.538 0.205 0.002 0.653
Variables in Z: INT BX1 BX2 BX3 .,1 Z2
CI Variance Proportiona for Coefficients ofINT BX1 BX2 BX3 .,1 Z2
1 0.014 0.043 0.014 0.001 0.018 0.0511 0.001 0.000 0.045 0.176 0.031 0.0101 0.020 0.006 0.136 0.003 0.167 0.0041 0.737 0.000 0.001 0.002 0.037 0.0023 0.046 0.200 0.025 0.307 0.346 0.8764 0.183 0.750 0.779 0.510 0.402 0.057
286
Appendix 4: MIXED"Experiment 2
-* The Ws end Zs -*set 19: "1 8nd Z3
Variables in Z: INT,CI Variance Proportions for Coefficients of
INT IX1 IX2 IlG "1 Z31 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.009 0.001 0.002 0.0003 0.920 0.000 0.005 0.002 0.003 0.000
10 0.004 0.000 0.000 0.980 0.979 0.00091 0.000 1.000 0.986 0.001 0.000 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3
1 0.033 0.001 0.001 0.004 0.004 0.0001 0.000 0.002 0.002 0.004 0.005 0.0032 0.049 0.019 0.000 0.009 0.010 0.0013 0.914 0.034 0.003 0.002 0.002 0.000
10 0.003 0.000 0.000 0.916 0.916 0.00024 0.001 0.944 0.995 0.005 0.002 0.995
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3
1 0.034 0.002 0.005 0.004 0.004 0.003• 1 0.014 0.002 0.029 0.001 O.ooa 0.002
2 0.298 0.000 0.091 0.001 O.ooa 0.0023 0.631 0.003 0.252 0.001 0.001 0.0189 0.003 0.003 0.001 0.966 0.910 0.002
13 0.014 0.990 0.616 0.015 0.009 0.9n
Variables in Z: INT IX1 IX2 IlG
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3
1 0.023 0.004 0.006 0.004 0.000 0.0051 0.001 0.000 0.049 0.110 0.016 0.0001 0.319 0.000 0.011 0.003 0.446 0.0002 0.333 0.000 0.004 0.026 0.531 0.0003 0.316 0.002 0.264 0.788 0.001 0.016
14 O.ooa 0.993 0.661 0.009 0.006 0.979
Variables in Z: INT IX1 IX2 IlG "1 Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG "1 Z3
1 0.012 0.005 0.005 0.001 0.014 0.0061 0.001 0.000 0.021 0.113 0.038 0.0011 0.042 0.001 0.067 0.002 0.158 0.0002 0.117 0.000 0.000 0.003 0.053 0.0004 0.221 O.OOS 0.238 0.818 0.135 0.038
12 0.007 0.989 0.668 0.003 0.002 0.955
287
Appendix 4: "MIXED Experiment 2
-- The lis 8nd Zs --•
set 20: 111 8nd Z4
Vari~les in Z: INT
CI Variance Proportions for Coefficients of •INT IX1 IX2 IX3 111 Z41 0.031 0.000 0.000 0.003 0.003 0.0002 0.001 0.000 0.000 0.012 0.013 0.0002 0.043 0.000 0.001 0.001 0.002 0.0003 0.920 0.000 0.001 0.002 0.003 0.000
10 0.004 0.000 0.000 0.947 0.953 0.000294 0.000 1.000 0.998 0.034 0.027 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 111 Z4
1 0.033 0.000 0.000 0.004 0.004 0.0001 0.000 0.000 0.000 0.004 0.005 0.0002 0.050 0.002 0.000 0.009 0.010 0.0003 0.914 0.004 0.000 0.002 0.002 0.000
10 0.003 0.000 0.000 0.945 0.950 0.00073 0.000 0.993 0.999 0.036 0.030 1.000
Variables in Z: INT aX1 aX2
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 111 Z4
1 0.034 0.000 0.001 0.003 0.003 0.0001 0.015 0.000 0.006 0.007 0.007 0.0002 0.293 0.000 0.021 0.007 0.008 0.0003 0.653 0.000 0.053 0.001 0.001 0.0029 0.003 0.000 0.000 0.930 0.937 0.000
38 0.002 0.999 0.919 0.051 0.043 0.997
Vari~les in Z: INT aX1 aX2 aX3
CI Variance Proportions for Coefficients ofINT aX1 IX2 aX3 111 Z4
1 0.023 0.001 0.001 0.004 0.000 0.0011 0.001 0.000 0.010 0.169 0.016 0.0001 0.317 0.000 0.004 0.003 0.435 0.0002 0.334 0.000 0.001 0.026 0.512 0.0003 0.324 0.000 0.057 0.7'96 0.001 0.002
37 0.001 0.999 0.927 0.001 0.037 0.997
Variables in Z: INT aX1 aX2 aX3 111 Z4
CI Variance Proportions for Coefficients ofINT aX1 IX2 IX3 111 Z4
1 0.012 0.001 0.001 0.001 0.014 0.0011 0.001 0.000 0.004 0.172 0.038 0.0001 0.046 0.000 0.013 0.002 0.156 0.0002 0.714 0.000 0.000 0.003 0.056 0.0004 0.227 0.001 0.047 0.821 0.735 0.005
34 0.000 0.998 0.935 0.000 0.001 0.994
288
Appendix 4: MIXED Experillllnt 2
- The wa .-.:I Za -
set 21: W2 .-.:I ZO
Vari~les in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W2 ZO
1 0.035 0.019 0.023 0.000 0.000 0.0202 0.003 0.055 0.016 0.001 0.001 0.0712 0.033 0.094 0.554 0.000 0.000 0.0203 0.826 0.001 0.190 0.000 0.000 0.1214 0.103 0.820 0.216 0.000 0.000 0.767
30 0.001 0.010 0.001 0.998 0.998 0.001
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 ZO
1 0.035 0.029 0.029 0.001 0.001 0.0052 0.013 0.015 0.131 0.001 0.001 0.2362 0.034 0.230 0.000 0.000 0.000 0.3592 0.001 0.318 0.576 0.000 0.000 0.3973 0.917 0.408 0.262 0.000 0.000 0.002
29 0.000 0.000 0.001 0.998 0.998 0.001
Variables in Z: INT aX1 aX2
CI Variance Proportions for Coefficients ofINT aX1 aX2 BX3 W2 ZO
1 0.051 0.010 0.000 0.001 0.001 0.0001 0.000 0.128 0.152 0.000 0.000 0.0242 0.004 0.003 0.026 0.000 0.000 0.9372 0.471 0.032 0.070 0.001 0.001 0.0263 0.471 0.827 0.752 0.000 0.000 0.010
27 0.002 0.000 0.000 0.998 0.998 0.002
Variables in Z: INT aX1 aX2 aX3
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2 ZO
1 0.025 0.054 0.115 0.069 0.023 0.0051 0.036 0.135 0.000 0.107 0.100 0.0681 0.562 0.001 0.052 0.002 0.076 0.0371 0.004 0.003 0.009 0.002 0.317 0.6011 0.061 0.067 0.001 0.110 0.481 0.2843 0.312 0.740 0.823 0.710 0.002 0.005
Variables in Z: INT aX1 aX2 aX3 W2 ZO
CI Variance Proportions for Coefficients ofINT aX1 aX2 aX3 W2 ZO
1 0.029 0.119 0.101 0.017 0.014 0.0181 0.009 0.001 0.023 0.012 0.112 0.1531 0.035 0.068 0.003 0.198 0.045 0.0071 0.645 0.001 0.057 0.015 0.010 0.0042 0.085 0.236 0.084 0.001 0.126 0.2843 0.196 0.575 0.732 0.757 0.692 0.535
289
Appendix 4: MIXED Experillll!nt 2
-- The ... n Zs"-
set 22: W2 n Z1
Variables in Z: INT
CI Variance Proportions for Coefficients of •INT IX1 IX2 IX3 W2 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.001 0.001 0.0102 0.042 0.013 0.438 0.000 0.000 0.0033 0.925 0.003 0.226 0.000 0.000 0.0079 0.000 0.965 0.316 0.000 0.000 0.976
31 0.001 0.003 0.000 0.998 0.998 0.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.034 0.023 0.026 0.001 0.001 0.0081 0.002 0.006 0.108 0.001 0.001 0.2312 0.044 0.249 0.005 0.001 0.001 0.0813 0.036 0.167 0.813 0.000 0.000 0.6273 0.884 0.554 0.048 0.000 0.000 0.051
29 0.001 0.001 0.000 0.998 0.998 0.001
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.051 0.013 0.000 0.001 0.001 0.0051 0.000 0.083 0.102 0.000 0.000 0.0932 0.190 0.000 0.186 0.000 0.000 0.3112 0.313 0.048 0.005 0.000 0.000 0.4943 0.444 0.857 0.707 0.000 0.000 0.096
27 0.002 0.000 0.000 0.998 0.998 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.028 0.098 0.063 0.004 0.000 0.1001 0.000 0.013 0.047 0.155 0.098 0.0601 0.576 0.000 0.050 0.000 0.089 0.0061 0.086 0.002 0.005 0.037 0.793 0.0702 0.000 0.131 0.031 0.131 0.017 0.7143 0.310 0.756 0.803 0.6n 0.003 0.050
Variables in Z: INT IX1 IX2 IX3 W2 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z1
1 0.016 0.098 0.029 0.000 0.038 0.0971 0.004 0.002 0.108 0.128 0.009 0.0561 0.030 0.047 0.017 0.090 0.175 0.0051 0.673 0.000 0.044 0.010 0.002 0.0072 0.049 0.339 0.053 0.006 0.095 0.465 ..4 0.229 0.514 0.748 0.766 0.681 0.370
290
Appendix 4: MIXED Experiment 2
-- The WS 8nd Zs --
Set 23: W2 8nd Z2
Vari8bles in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX! W2 Z2
1 0.031 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.001 0.001 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.926 0.000 0.042 0.000 0.000 0.001
29 0.001 0.816 0.729 0.142 0.142 0.82631 0.000 0.180 0.140 0.856 0.856 0.1n
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0342 0.051 0.125 0.001 0.001 0.001 0.0053 0.916 0.226 0.023 0.000 0.000 0.0018 0.001 0.627 0.952 0.000 0.000 0.956
30 0.000 0.002 0.001 0.998 0.998 0.002
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 22.. 1 0.039 0.015 0.006 0.001 0.001 0.021
1 0.008 0.021 0.061 0.001 0.001 0.0222 0.276 0.000 0.182 0.001 0.001 0.0262 0.582 0.001 0.294 0.000 0.000 0.2305 0.092 0.962 0.457 0.000 0.000 0.700
28 0.002 0.001 0.000 0.998 0.998 0.001
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.025 0.037 0.014 0.004 0.003 0.0491 0.002 0.000 0.085 0.149 0.068 0.0041 0.469 0.002 0.057 0.002 0.196 0.0002 0.187 0.002 0.000 0.045 0.n1 0.0063 0.217 0.005 0.305 0.601 0.012 0.2905 0.100 0.955 0.539 0.199 0.000 0.652
Variables in Z: INT IX1 IX2 IX3 W2 Z2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z2
1 0.012 0.040 0.012 0.002 0.020 0.0491 0.000 0.000 0.083 0.176 0.007 0.0091 0.005 0.006 0.089 0.027 0.229 0.0011 0.752 0.000 0.005 0.004 0.024 0.0014 0.077 0.103 0.084 0.415 0.444 0.7874 0.153 0.850 0.n8 0.377 0.277 0.152
291
Appendix 4: MIXED Experiment 2
- The wa end Zs -
Set 24: W2 rd Z3
Variebl.. in Z: INT
CI Variance Proportions for Coefficients of "INT IX1 IX2 1X3 W2 Z31 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.927 0.000 0.005 0.000 0.000 0.000
31 0.001 0.000 0.000 0.996 0.997 0.00098 0.000 1.000 0.986 0.002 0.001 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3
1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.001 0.001 0.0003 0.916 0.034 0.003 0.000 0.000 0.000
24 0.000 0.939 0.991 0.001 0.002 0.99130 0.001 0.005 0.004 0.996 0.996 0.004
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3
1 0.034 0.002 0.005 0.000 0.000 0.0031 0.013 0.002 0.029 0.001 0.001 0.0022 0.299 0.000 0.097 0.001 0.001 0.0023 0.638 0.003 0.252 0.000 0.000 0.018
13 0.013 0.988 0.615 0.000 0.000 0.97129 0.003 0.004 0.001 0.998 0.998 0.004
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3
1 0.022 0.004 0.005 0.005 0.003 0.0051 0.003 0.000 0.044 0.147 0.067 0.0001 0.469 0.000 0.028 0.002 0.195 0.0002 0.185 0.000 0.000 0.046 0.n5 0.0003 0.313 0.002 0.263 0.792 0.008 0.016
14 0.008 0.993 0.660 0.008 0.002 0.979
Variables in Z: INT IX1 IX2 IX3 W2 Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z3
1 0.011 0.005 0.005 0.002 0.016 0.0061 0.000 0.000 0.038 0.178 0.012 0.0011 0.021 0.001 0.049 0.020 0.222 0.0002 0.738 0.000 0.000 0.004 0.040 0.0004 0.224 0.006 0.249 0.794 0.706 0.039 ..
12 0.006 0.989 0.658 0.002 0.003 0.953
292
Appendix 4: MIXED Experiment 2
- The wa 8nd Zs -*Set 25: W2 8nd Z4
Variebles in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.031 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.001 0.001 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.927 0.000 0.001 0.000 0.000 0.000
31 0.001 0.000 0.000 0.998 0.998 0.000291 0.000 1.000 0.998 0.000 0.000 1.000
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.031 0.000 0.000 0.000 0.000 0.0001 0.001 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.001 0.001 0.0003 0.917 0.004 0.000 0.000 0.000 0.000
30 0.000 0.000 0.000 0.998 0.998 0.00073 0.000 0.993 0.999 0.000 0.000 1.000
... Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.034 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.001 0.001 0.0002 0.295 0.000 0.021 0.001 0.001 0.0003 0.654 0.000 0.055 0.000 0.000 0.002
29 0.002 0.001 0.001 0.994 0.993 0.00138 0.002 0.997 0.915 0.004 0.005 0.995
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.022 0.001 0.001 0.005 0.003 0.0011 0.003 0.000 0.010 0.146 0.067 0.0001 0.470 0.000 0.006 0.002 0.194 0.0002 0.183 0.000 0.000 0.048 0.n9 0.0003 0.321 0.000 0.059 0.798 0.007 0.002
31 0.001 0.999 0.925 0.001 0.001 0.997
Variables in Z: INT IX1 IX2 IX3 W2 Z4
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W2 Z4
1 0.011 0.001 0.001 0.002 0.016 0.0011 0.000 0.000 0.008 0.177 0.012 0.0001 0.022 0.000 0.010 0.020 0.221 0.0002 0.738 0.000 0.000 0.004 0.042 0.0004 0.229 0.001 0.053 0.196 0.109 0.005
32 0.000 0.998 0.921 0.000 0.000 0.994
293
Appendix 4: MIXED ExperiMent 2
*** The Ws IInCI Zs -*Set 26: W3 IInCI ZO
Variabl.. in Z: INT
CI Variance Proportions for Coefficients ofINT 8X1 8X2 8X3 W3 ZO
1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.826 0.001 0.192 0.000 0.000 0.1194 0.101 0.828 0.216 0.000 0.000 0.767
89 0.001 0.001 0.000 1.000 1.000 0.002
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 ZO
1 0.035 0.029 0.029 0.000 0.000 0.0052 0.014 0.016 0.131 0.000 0.000 0.2312 0.034 0.229 0.001 0.000 0.000 0.3622 0.001 0.317 0.576 0.000 0.000 0.3963 0.915 0.409 0.262 0.000 0.000 0.002
aa 0.002 0.000 0.001 1.000 1.000 0.004
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO
1 0.051 0.010 0.000 0.000 0.000 0.0011 0.000 0.128 0.152 0.000 0.000 0.0242 0.003 0.003 0.026 0.000 0.000 0.9412 0.470 0.032 0.071 0.000 0.000 0.0243 0.470 0.827 0.752 0.000 0.000 0.010
79 0.006 0.000 0.000 1.000 1.000 0.001
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO
1 0.012 0.005 0.063 0.081 0.107 0.0001 0.039 0.183 0.027 0.023 0.037 0.0731 0.564 0.000 0.060 0.000 0.013 0.0661 0.015 0.025 0.026 0.004 0.034 0.8472 0.086 0.075 0.030 0.181 o.m 0.0103 0.285 0.712 0.794 0.711 0.033 0.005
Variables in Z: INT IX1 IX2 IX3 W3 ZO
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 ZO
1 0.005 0.000 0.027 0.072 O.OU 0.0891 0.035 0.161 0.090 0.000 0.004 0.0251 0.461 0.009 0.057 0.062 0.032 0.0151 0.232 0.027 0.004 0.257 0.139 0.0152 0.049 0.181 0.055 0.004 0.192 0.333 ..3 0.218 0.621 0.769 0.605 0.546 0.523
294
Appendix 4: MIXED Experilnent 2
-* The wa 8M Zs .-
set 27: W3 8M Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT ax1 IX2 IX3 W3 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.437 0.000 0.000 0.0033 0.923 0.003 0.227 0.000 0.000 0.0069 0.000 0.967 0.316 0.000 0.000 0.975
92 0.002 0.001 0.000 1.000 1.000 0.001
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.034 0.023 0.026 0.000 0.000 O.OOS1 0.002 0.006 0.1OS 0.000 0.000 0.2282 0.043 0.250 0.005 0.000 0.000 0.0843 0.036 0.167 0.813 0.000 0.000 0.6273 0.882 0.555 0.048 0.000 0.000 0.051
88 0.003 0.000 0.000 1.000 1.000 0.001
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 Z1
1 0.050 0.012 0.000 0.000 0.000 0.005.: 1 0.000 0.083 0.102 0.000 0.000 0.094
2 0.189 0.000 0.186 0.000 0.000 0.3122 0.313 0.048 0.005 0.000 0.000 0.4933 0.442 0.857 0.707 0.000 0.000 0.096
80 0.006 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 BX2 IX3 W3 Z1
1 0.012 0.006 0.065 0.079 0.104 0.0001 0.017 0.106 0.014 0.017 0.023 0.1471 0.598 0.000 0.065 0.000 0.016 0.0042 0.036 0.044 0.071 0.001 0.214 0.6382 0.052 0.114 0.006 0.230 0.611 0.1633 0.285 0.730 0.780 0.673 0.031 0.047
Variables in Z: INT IX1 IX2 IX3 W3 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z1
1 0.004 0.057 0.000 0.044 0.065 0.1091 0.026 0.069 0.137 0.045 0.021 0.0001 0.567 0.005 0.036 0.028 0.042 0.0081 0.124 0.017 O.OOS 0.265 0.212 0.0042 0.028 0.286 0.030 0.002 0.127 0.5203 0.250 0.566 0.788 0.615 0.534 0.359
295
Appendix 4: MIXED ExperiMent 2
- The Ws and Zs -*Set 28: Y3 and Z2
Verfebl.. in Z: INT
CI Veriance Proportions for Coefficients ofINT 1X1 IX2 BX3 Y3 Z2
1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.085 0.000 0.000 0.0003 0.925 0.000 0.042 0.000 0.000 0.001
30 0.001 0.995 0.868 0.000 0.000 0.99692 0.001 0.002 0.002 1.000 1.000 0.002
Variebl.. in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Y3 Z2
1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0332 0.051 0.125 0.001 0.000 0.000 0.0053 0.914 0.226 0.023 0.000 0.000 0.0018 0.001 0.627 0.949 0.000 0.000 0.954
90 0.002 0.002 0.003 1.000 1.000 0.004
Variables in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 Y3 Z2
1 0.039 0.014 0.006 0.000 0.000 0.0211 0.008 0.021 0.061 0.000 0.000 0.0222 0.275 0.000 0.182 0.000 0.000 0.026 '/2 0.57'9 0.001 0.294 0.000 0.000 0.2295 0.092 0.960 0.457 0.000 0.000 0.697
84 0.006 0.003 0.000 1.000 1.000 0.005
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT 1X1 IX2 BX3 Y3 Z2
1 0.020 0.036 O.ooa 0.010 O.ooa 0.0491 0.010 0.001 0.056 0.083 0.109 0.0001 0.538 0.002 0.07'9 0.000 0.032 0.0002 0.174 0.001 0.064 0.059 o.no 0.0353 0.157 0.006 0.254 0.676 0.131 0.2655 0.100 0.954 0.539 0.171 0.000 0.650
Variables in Z: INT IX1 IX2 IX3 Y3 Z2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 Z2
1 O.ooa 0.038 0.004 0.019 0.030 0.0531 0.006 0.011 0.136 0.089 0.038 0.0011 0.698 0.000 0.017 0.001 0.041 0.0002 0.035 0.002 O.ooa 0.237 0.315 0.0003 0.101 0.063 0.130 0.390 0.361 0.7874 0.151 0.886 0.706 0.263 0.214 0.159
296
Appendix 4: MIXED Experiment 2
.- The Ws 8nd Zs *-Set 29: W3 8nd 13
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.042 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 0.005 0.000 0.000 0.000
92 0.002 0.000 0.000 1.000 1.000 0.00098 0.000 1.000 0.986 0.000 0.000 1.000
Variebles in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13
1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.000 0.000 0.0003 0.915 0.034 0.003 0.000 0.000 0.000
24 0.001 0.944 0.994 0.000 0.000 0.99590 0.002 0.000 0.000 1.000 1.000 0.000
Variebles in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13
1 0.034 0.002 0.004 0.000 0.000 0.0031 0.013 0.002 0.029 0.000 0.000 0.0022 0.298 0.000 0.098 0.000 0.000 0.0023 0.636 0.003 0.252 0.000 0.000 0.018
13 0.014 0.992 0.616 0.000 0.000 0.97485 0.006 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13
1 0.018 0.004 0.003 0.010 0.009 0.0051 0.011 0.000 0.029 0.081 0.107 0.0001 0.542 0.000 0.039 0.000 0.031 0.0002 0.162 0.000 0.030 0.079 0.775 0.0023 0.260 0.002 0.239 0.819 0.078 0.015
14 0.007 0.993 0.660 0.009 0.000 0.978
Variables in Z: INT IX1 IX2 IX3 W3 13
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W3 13
1 0.008 0.005 0.002 0.015 0.024 0.0061 0.003 0.001 0.068 0.098 0.041 0.0002 0.705 0.000 0.007 0.000 0.043 0.0002 0.030 0.000 0.004 0.232 0.321 0.0004 0.250 0.006 0.271 0.651 0.571 0.042
12 0.004 0.988 0.649 0.004 0.001 0.951
297
Appendix 4: MIXED Experiment 2
*** The \Is and Zs ***set 30: W3 end Z4
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 BX3 W3 Z4
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000
92 0.002 0.000 0.000 0.996 0.997 0.000292 0.000 1.000 0.998 0.003 0.003 1.000
Variables in Z: INT BX1
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4
1 0.031 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.000 0.000 0.0003 0.915 0.004 0.000 0.000 0.000 0.000
n 0.000 0.973 0.980 0.008 0.009 0.97990 0.002 0.020 0.020 0.992 0.991 0.020
Variables in Z: INT BX1 BX2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4
1 0.033 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.000 0.000 0.0002 0.294 0.000 0.021 0.000 0.000 0.0003 0.652 0.000 0.055 0.000 0.000 0.002
37 0.002 0.995 0.913 0.000 0.000 0.99386 0.007 0.003 0.003 1.000 1.000 0.003
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4
1 0.018 0.001 0.001 0.010 0.009 0.0011 0.011 0.000 0.006 0.081 0.107 0.0001 0.539 0.000 0.009 0.000 0.032 0.0002 0.165 0.000 0.007 0.079 o.m 0.0003 0.266 0.000 0.053 0.829 0.079 0.002
37 0.001 0.999 0.925 0.000 0.001 0.997
Variables in Z: INT BX1 BX2 BX3 W3 Z4
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W3 Z4
1 0.008 0.001 0.000 0.015 0.024 0.0011 0.003 0.000 0.015 0.098 0.041 0.0002 0.705 0.000 0.001 0.000 0.043 0.0002 0.030 0.000 0.001 0.231 0.323 0.0004 0.253 0.001 0.060 0.656 0.569 0.006
32 0.001 0.998 0.923 0.000 0.001 0.993
298
Appendix 4: MIXED Experiment 2
*** The wa end Zs ***Set 31: W4 end ZO
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT 8X1 IX2 BX3 W4 ZO
1 0.034 0.020 0.023 0.000 0.000 0.0202 0.003 0.055 0.017 0.000 0.000 0.0712 0.034 0.096 0.552 0.000 0.000 0.0203 0.825 0.001 0.192 0.000 0.000 0.1194 0.102 0.828 0.216 0.000 0.000 0.765
318 0.002 0.001 0.001 1.000 1.000 0.005
Variables in Z: INT 8X1
CI Varience Proportions for Coefficients ofINT IX1 8X2 8X3 W4 ZO
1 0.035 0.029 0.029 0.000 0.000 0.0052 0.014 0.016 0.131 0.000 0.000 0.2322 0.034 0.230 0.001 0.000 0.000 0.3612 0.001 0.318 0.576 0.000 0.000 0.3963 0.916 0.408 0.262 0.000 0.000 0.002
312 0.002 0.000 0.001 1.000 1.000 0.003
Variables in Z: INT BX1 8X2..CI Variance Proportions for Coefficients of
INT BX1 8X2 8X3 W4 ZO1 0.051 0.010 0.000 0.000 0.000 0.0011 0.000 0.128 0.152 0.000 0.000 0.0242 0.003 0.003 0.026 0.000 0.000 0.9342 0.4n 0.032 0.071 0.000 0.000 0.0233 0.473 0.827 0.752 0.000 0.000 0.010
278 0.001 0.000 0.000 1.000 1.000 0.008
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT 8X1 BX2 BX3 W4 ZO
1 0.003 0.000 0.036 0.027 0.031 0.0021 0.051 0.184 0.047 0.002 0.003 0.0591 0.548 0.000 0.042 0.000 0.000 0.1562 0.068 0.029 0.036 0.003 0.003 0.7753 0.275 0.686 0.719 0.004 0.146 0.0035 0.056 0.101 0.119 0.963 0.816 0.005
Variables in z: INT BX1 BX2 BX3 W4 ZO
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 ZO
1 0.001 0.001 0.012 0.041 0.042 0.0431 0.039 0.164 0.096 0.000 0.000 0.0152 0.625 0.001 0.066 0.000 0.000 0.0112 0.080 0.1n 0.029 0.021 0.010 0.4224 0.000 0.010 0.016 0.612 0.876 0.0314 0.255 0.652 0.781 0.326 0.073 0.478
299
Appendix 4: MIXED Experilllent 2
*** The WI and Zs ***set 32: W4 and Z1
Variables in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 Z1
1 0.031 0.004 0.015 0.000 0.000 0.0042 0.001 0.012 0.004 0.000 0.000 0.0102 0.043 0.013 0.436 0.000 0.000 0.0033 0.923 0.003 0.226 0.000 0.000 0.0069 0.000 0.961 0.316 0.000 0.000 0.915
325 0.002 0.001 0.003 1.000 1.000 0.001
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 Ix2 IX3 W4 Z1
1 0.034 0.023 0.025 0.000 0.000 0.0081 0.002 0.006 0.108 0.000 0.000 0.2292 0.043 0.250 0.005 0.000 0.000 0.0843 0.036 0.161 0.811 0.000 0.000 0.6213 0.884 0.554 0.049 0.000 0.000 0.051
314 0.002 0.000 0.002 1.000 1.000 0.001
Variables in Z: INT IX1 IX2»
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.051 0.012 0.000 0.000 0.000 0.0051 0.000 0.083 0.102 0.000 0.000 0.0942 0.181 0.000 0.186 0.000 0.000 0.3162 0.311 0.048 0.004 0.000 0.000 0.4893 0.445 0.857 0.701 0.000 0.000 0.096
218 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IX3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.002 0.000 0.032 0.028 0.032 0.0051 0.027 0.111 0.038 0.000 0.001 0.1311 0.645 0.000 0.050 0.001 0.000 0.0112 0.005 0.014 0.080 0.006 0.012 0.n43 0.266 0.722 0.681 0.003 0.142 0.0185 0.056 0.092 0.120 0.962 0.813 0.001
Variables in Z: INT IX1 IX2 IX3 W4 Z1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z1
1 0.000 0.012 0.005 0.043 0.044 0.0501 0.034 0.120 0.110 0.004 0.002 0.0192 0.644 0.001 0.056 0.000 0.000 0.0062 0.034 0.248 0.010 0.022 0.014 0.5883 0.000 0.008 0.010 0.627 0.839 0.0114 0.288 0.611 0.809 0.304 0.101 0.326
300
Appendix 4: MIXED Experiment 2
• .- The wa end Zs *-set 33: W4 end Z2
Vari~les in Z: INT
CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z2
1 0.030 0.000 0.003 0.000 0.000 0.0002 0.001 0.001 0.001 0.000 0.000 0.0012 0.041 0.002 0.084 0.000 0.000 0.0003 0.925 0.000 0.041 0.000 0.000 0.001
30 0.001 0.986 0.858 0.000 0.000 0.987328 0.002 0.011 0.013 1.000 1.000 0.011
Vari~les in Z: INT IX1
CI Vari~e Proportions for Coefficients ofINT IX1 IX2 IX3 .W4 Z2
1 0.032 0.010 0.006 0.000 0.000 0.0031 0.000 0.011 0.017 0.000 0.000 0.0332 0.051 0.125 0.001 0.000 0.000 0.0053 0.915 0.225 0.023 0.000 0.000 0.0018 0.001 0.625 0.943 0.000 0.000 0.950
319 0.002 0.005 0.010 1.000 1.000 0.008
Vari~les in Z: INT IX1 IX2
CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2
1 0.040 0.014 0.006 0.000 0.000 0.021, 1 0.008 0.022 0.061 0.000 0.000 0.0222 0.276 0.000 0.182 0.000 0.000 0.0262 0.583 0.001 0.294 0.000 0.000 0.2295 0.092 0.961 0.457 0.000 0.000 0.698
292 0.000 0.002 0.000 1.000 1.000 0.003
Variables in Z: INT IX1 IX2 IX3
CI Vari.nce Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z2
1 0.001 0.009 0.008 0.022 0.025 0.0191 0.028 0.028 0.048 O.OOS 0.005 0.0272 0.606 0.002 0.065 0.001 0.001 0.0003 0.254 0.000 0.355 0.017 0.083 0.2325 0.036 0.779 0.311 0.064 0.204 0.6685 0.075 0.182 0.213 0.892 0.682 0.054
Variables in Z: INT BX1 BX2 BX3 W4 Z2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z2
1 0.002 0.018 0.000 0.030 0.032 0.0331 0.019 0.033 0.104 0.020 0.018 0.0102 0.688 0.000 0.036 0.003 0.001 0.000
.~ 3 0.134 0.014 0.210 0.023 0.358 0.6603 0.027 0.001 0.021 0.870 0.525 0.0534 0.131 0.933 0.628 0.053 0.065 0.244
301
Appendix 4: MIXED Experiment 2
-- The wa 8nd Zs --
set 34: W4 8nd Z3
Variebl.. in Z: INT
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 D.OOO 0.000 0.0002 0.041 0.000 0.009 0.000 0.000 0.0003 0.925 0.000 O.OOS 0.000 0.000 0.000
98 0.000 0.993 0.979 0.000 0.000 0.993321 0.002 0.001 0.001 1.000 1.000 0.006
Varieblea in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3
1 0.031 0.001 0.001 0.000 0.000 0.0001 0.000 0.002 0.002 0.000 0.000 0.0032 0.051 0.019 0.000 0.000 0.000 0.0003 0.915 0.034 0.003 0.000 0.000 0.000
24 0.001 0.941 0.991 0.000 0.000 0.992318 0.002 0.003 0.004 1.000 1.000 0.004
Varieblea in Z: INT IX1 IX2
CI Variance Proportions for Coefficients ofINT IX1 IX2 IlG W4 Z3
1 0.034 0.002 0.004 0.000 0.000 0.0031 0.013 0.002 0.029 0.000 0.000 0.0022 0.300 0.000 0.098 0.000 0.000 0.0023 0.640 0.003 0.252 0.000 0.000 0.018
13 0.014 0.992 0.611 0.000 0.000 0.914298 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT IX1 IX2 IlG
CI Variance Proportions for coefficients ofINT IX1 IX2 IlG W4 Z3
1 0.004 0.002 0.001 0.016 0.019 0.0031 0.024 0.003 0.026 0.010 0.011 0.0022 0.606 0.000 0.032 0.001 0.001 0.0003 0.321 0.002 0.251 0.020 0.111 0.0155 0.038 0.000 0.023 0.944 0.855 0.002
14 O.ooa 0.993 0.661 0.009 0.002 0.918
Variables in Z: INT IX1 IX2 IlG W4 Z3
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z3
1 0.002 0.003 0.000 0.026 0.021 0.004 ..1 0.016 0.003 0.050 0.026 0.024 0.0012 0.684 0.000 0.011 0.004 0.002 0.0004 0.026 0.001 0.044 0.351 0.941 0.0014 0.266 0.005 0.246 0.582 0.006 0.038
13 0.006 0.981 0.641 0.005 0.000 0.950
302
Appendix 4: MIXED Experiment 2
... The lis end Z. *-Set 35: W4 end Z4
V.riables in Z: INT
CI Vari8nCe Proportions for Coefficients ofINT IX1 IX2 BX3 W4 Z4
1 0.030 0.000 0.000 0.000 0.000 0.0002 0.001 0.000 0.000 0.000 0.000 0.0002 0.041 0.000 0.001 0.000 0.000 0.0003 0.925 0.000 0.001 0.000 0.000 0.000
291 0.000 0.997 0.995 0.002 0.002 0.997327 0.002 0.003 0.003 0.998 0.998 0.003
Variables in Z: INT IX1
CI Variance Proportions for Coefficients ofINT IX1 IX2 IX3 W4 Z4
1 0.031 0.000 0.000 0.000 0.000 0.0001 0.000 0.000 0.000 0.000 0.000 0.0002 0.051 0.002 0.000 0.000 0.000 0.0003 0.916 0.004 0.000 0.000 0.000 0.000
73 0.000 0.992 0.999 0.000 0.000 0.999318 0.002 0.001 0.001 1.000 1.000 0.001
Variables in Z: INT IX1 BX2
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4
1 0.034 0.000 0.001 0.000 0.000 0.0001 0.013 0.000 0.006 0.000 0.000 0.0002 0.295 0.000 0.021 0.000 0.000 0.0003 0.656 0.000 0.055 0.000 0.000 0.002
38 0.002 0.999 0.916 0.000 0.000 0.996298 0.000 0.000 0.000 1.000 1.000 0.000
Variables in Z: INT BX1 BX2 BX3
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4
1 0.004 0.000 0.000 0.016 0.018 0.0001 0.024 0.000 0.006 0.010 0.012 0.0002 0.604 0.000 0.007 0.001 0.001 0.0003 0.327 0.000 0.056 0.019 0.114 0.0025 0.041 0.000 0.006 0.950 0.853 0.000
38 0.001 0.999 0.925 0.003 0.002 0.997
Variablell in Z: INT BX1 BX2 BX3 W4 Z4
CI Variance Proportions for Coefficients ofINT BX1 BX2 BX3 W4 Z4
1 0.003 0.000 0.000 0.025 0.027 0.0011 0.016 0.000 0.011 0.027 0.025 0.0002 0.684 0.000 0.004 0.004 0.002 0.0004 0.032 0.000 0.011 0.332 0.943 0.0014 0.265 0.001 0.053 0.609 0.002 0.005
34 0.001 0.998 0.921 0.002 0.001 0.993
303
REFERENCES
Beckman, R. J., Nachtsheim, C. J., and Cook, D. R. (1987), "Diagnostics forMixed-Model Analysis of Variance," Technometrics, 29 (4), 413-426.
Belsley, D. A., Kuh, E., and Welsch, R. E. (1980), Regression Diagnostics:Identifying Influential Data and Sources of Collinearity, New York: JohnWiley.
Belsley, D. A. (1984), "Demeaning Conditioning Diagnostics ThroughCentering," The American Statistician, 38, 73-77.
Belsley, D. A. (1987), "Comment: Well-conditioned Collinearity Indices?"Statistical Science, 2, 86-91.
Belsley, D. A. (1991), Conditioning Diagnostics: Collinearity and Weak Datain Regression, New York: John Wiley.
Callanan, T. P. and Harville, D. A. (1991), "Some New Algorithms forComputing Restricted Maximum Likelihood Estimates of VarianceComponents," Journal of Statistical Computation and Simulation, 38,239-259.
Chatterjee, S. and Hadi, A. S. (1988), Sensitivity Analysis in Linear Regression,New York: John Wiley.
Chatterjee, S. and Price, B. (1991), Regression Analysis by Example, NewYork: John Wiley.
Christensen, R., Pearson, L. M. and Johnson, W. (1992), "Case-DeletionDiagnostics for Mixed Models," Technometrics, 34 (1), 38-45.
Cook, R. D. (1984), Comment on "Demeaning Conditioning DiagnosticsThrough Centering," The American Statistician, 38, 78-79.
Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), "Maximum Likelihoodfrom Incomplete Data via the EM algorithm (with Discussion)," Journalof the Royal Statistical Society (8),39,1-38.
.\<
Diggle, P. J. 1988), "An Approach to the Analysis of Repeated Measures,"Biometrics, 44, 959-971.
Fairclough, D. L. and Helms, R. W. (1984), "Mixed Effects Model Analyses ofIncomplete Longitudinal Pulmonary Function Measurements in Children, "Institute of Statistics Mimeo Series, University of North Carolina atChapel Hill.
Fairclough, D. L. and Helms, R. W. (1986), "A Mixed Model with LinearCovariance Structure: A Sensitivity Analysis of the Maximum LikelihoodEstimators, " JournalofStatistical Computation andSimulation, 25, 205236.
Grady, J. J. and Helms, R. W. (1992), "Structured Covariance Matrices forIncomplete Longitudinal Data," Institute of Statistics Mimeo Series,University of North Carolina at Chapel Hill.
Gunst, R. A. (1984), "Toward a Balanced Assessment of CollinearityDiagnostics (Comment)," The American Statistician, 38, 79-82.
Hadi, A. S. and Velleman, P. F. (1987), "Comment: Diagnosing NearCollinearities in Least Squares Regression," Statistical Science, 2, 93-98.
Hamilton, D. (1987), "Sometimes R2 >r~, + r~.: Correlated Variables Are Not
Always Redundant," The American Statistician, 41, 129-132.
Harville, D. A. (1977), "Maximum Likelihood Approaches to VarianceComponent Estimation and to Related Problems," Journal of theAmerican Statistical Association, 72, 320-340.
Helms, R. W. (1992), The Joy of Modeling.
Helms, R. W. (1993), Personal Communication.
Holditch-Davis, D., Helms, R. W., and Edwards, L. J. (1992), "A Model ofDevelopment of Sleeping and Waking in Preterm Infants,"
Jennrich, R. I. and Schluchter, M. D. (1986), "Unbalanced Repeated-MeasuresModels with Structured Covariance Matrices," Biometrics, 42, 805-820.
Kleinbaum, D. G., Kupper, L. L., and Muller, K. E. (1988), Applied RegressionAnalysis and Other Multivariable Methods, Boston, MA: PWS-KentPublishing Company.
Laird, N. M. (1988), "Missing Data in Longitudinal Studies," Statistics inMedicine, 7, 305-315.
305
Laird, N. M. and Ware, J. H. (1982), "Random-Effects Models for LongitudinalData," Biometrics, 38, 963-974.
Laird, N. M., Lange, N, and Stram, D. (1987), "Maximum LikelihoodComputations With Repeated Measures: Application of the EMAlgorithm, " Journalof the American Statistical Association, 82, 97-105.
Lindstrom, M. J. and Bates, D. M. (1988), "Newton-Raphson and EMAlgorithms for Linear Mixed-Effects Models for Repeated MeasuresData," Journal of the American Statistical Association, 83, 1014-1022.
Louis, T. A. (1988), "General Methods for Analysing Repeated Measures,"Statistics in Medicine, 7, 29-45.
Mandel, J. (1982), "Use of the Singular Value Decomposition in RegressionAnalysis," The American Statistician, 36, 15-24.
Mansfield, E.R. and Helms, B. P. (1982), "Detecting Multicol/inearities," TheAmerican Statistician, 36, 158-60.
Marquardt, D. W. (1980), "You Should Standardize the Predictor Variables inYour Regression Models, " Journalof the American StatisticalAssociation75, 74-103.
Marquardt, D. W. (1987), "Comment on Collinearity and Least SquaresRegression," Statistical Science, 1, 84-85.
Mason, R. L. and Gunst, R. F. (1985), "Outlier-induced Col/inearities,"Technometrics 27, 401-407.
Myers, R. H. (1986), Classical and Modern Regression with Applications,Boston: Duxbury Press.
Neter, J., Wasserman, W., and Kutner, M. H. (1985), Applied Linear StatisticalModels, Homewood, 11/: Richard D. Irwin, Inc.
Oman, S.D. (1982), "Shrinking Towards Subspaces in Multiple LinearRegression," Technometrics, 24, 307-311.
Rawlings, J. O. (1988), Applied Regression Analysis: A Research Tool,Belmont, CA: Wadsworth, Inc.
SAS Institute Inc (1988), SASIIML™ User's Guide, Release 6.03 Edition, Cary,NC: SAS Institute Inc.
Searle, S. R. (1988), "Mixed Models and Unbalanced Data: Wherefrom,Whereat, Whereto?, " Communications in Statistics-Theory andMethods,
306
)
J
•
17, 935-968.
Schindler, J. S. (1986), "Regression Diagnostics: Mechanical and StructuralAspects of Collinearity," Institute of Statistics Mimeo Series, Universityof North Carolina at Chapel Hill.
Smith, G. and Campbell, F., (1980), "A Critique of Some Ridge RegressionMethods," Journal of the American Statistical Association, 75, 74-81.
Snee, R. D. and Marquardt, D. W. (1984), "Collinearity Diagnostics Depend onthe Domain of Prediction, the Model, and the Data (Comment)," TheAmerican Statistician, 38, 83-87.
Stewart, G. W. (1987), "Collinearity and Least Squares Regression, " StatisticalScience 1, 68-100.
Strang G. (1988), Linear Algebra and Its Applications, New York: AcademicPress.
Strope, G. L. and Helms, R. W. (1984) A Longitudinal Study of Spirometry inYoung Black and White Children. manuscript
Swamy, P. A. V. B, Mehta, J. S., Thurman, S. S., and Iyengar, N. S. (1985),"A Generalized Multicollinearity index for Regression Analysis, " Sankhya:The Indian Journal of Statistics, 47, 401-431.
Thisted, R. A. (1987), "Comment on Collinearity and Least SquaresRegression," Statistical Science, 1, 91-93.
Velleman, P. F. and Welsch, R. E. (1981), "Efficient Computing of RegressionDiagnostics," The American Statistician, 35, 234-242.
Ware, J. H. (1985), "Linear Models for the Analysis of Longitudinal Studies,"The American Statistician, 39, 95-101.
Willan, A. R. and Watts, D. C., (1978), "Meaningful MulticollinearityMeasures," Technometrics, 20, 407-412.
Wood, F. S. (1984), "Effect of Centering on Collinearity and Interpretation ofthe Constant (Comment)," The American Statistician, 38, 88-90.
307
Top Related