UsingMultivariate

StatisticsThird Edition

Barbara G TabachnickLinda S Fidell

California State University Northndge

HarperCollmsCollegePubhshers

Contents

Preface xxix

CHAPTER 1 Introduction 1

1 1 MULTIVARIATE STATISTICS W H Y ' 11 1 1 The Domain of Multivanate Statistics Number of IVs and DVs

1 1 2 Expenmental and Nonexpenmental Research 21 1 2 1 Multivanate Statistics in Nonexpenmental Research 31 1 2 2 Multivanate Statistics in Expenmental Research 3

1 1 3 Computers and Multivanate Statistics 41 1 3 1 Program Updates 61 1 3 2 Garbage In Roses Out? 6

1 1 4 Why Not? 6

1 2 SOME USEFUL DEFINITIONS 712 1 Continuous Discrete and Dichotomous Data 712 2 Samples and Populations 8

12 3 Descnptive and Inferential Statistics 91 2 4 Orthogonality 912 5 Standard and Sequential Analyses 10

1 3 COMBINING VARIABLES 11

1 4 NUMBER AND NATURE OF VARIABLES TO INCLUDE 12

1 5 DATA APPROPRIATE FOR MULTIVARIATE STATISTICS 1 31 5 1 The Data Matnx 1315 2 The Correlation Matnx 1415 3 The Vanance Covanance Matnx 14

VII

VIII CONTEI

15 4 The Sum of Squares and Cross Products Matnx 151 5 5 Residuals 16

1 6 ORGANIZATION OF THE BOOK 17

CHAPTER 2 A Guide to Statistical TechniquesUsing the Book 19

2 1 RESEARCH QUESTIONS AND ASSOCIATED TECHNIQUES 192 11 Degree of Relationship among Vanables 19

2 1 1 1 Bivanate r 20

2 1 1 2 Multiple R 202 1 1 3 Sequential R 202 1 1 4 Canonical R 202 1 1 5 Multiway Frequency Analysis 21

2 12 Significance of Group Differences 212 12 1 One Way ANOVA and t Test 212 12 2 One Way ANCOVA 212 1 2 3 FactonalANOVA 22

2 1 2 4 Factonal ANCOVA 222 1 2 5 HotelhngsT2 22

2 12 6 One Way MANOVA 23

2 12 7 One Way MANCOVA 232 1 2 8 Factonal MANOVA 242 12 9 Factonal MANCOVA 242 1 2 10 Profile Analysis 24

2 13 Prediction of Group Membership 252 13 1 One Way Discnminant Function 25

2 13 2 Sequential One Way Discnminant Function 252 13 3 Multiway Frequency Analysis (Logit) 26

2 13 4 Logistic Regression 262 13 5 Sequential Logistic Regression 262 13 6 Factonal Discnminant Function 272 13 7 Sequential Factonal Discnminant Function 27

2 1 4 Structure 272 14 1 Pnncipal Components 27

2 14 2 Factor Analysis 272 14 3 Structural Equation Modeling 28

CONTENTS tx

2 2 A DECISION TREE 28

2 3 TECHNIQUE CHAPTERS 28

2 4 PRELIMINARY CHECK OF THE DATA 29

CHAPTER 3 Review of Univanate and Bivanate Statistics 33

3 1 HYPOTHESIS TESTING 333 1 1 One Sample z Test 333 12 Power 363 13 Extensions of the Model 37

3 2 ANALYSIS OF VARIANCE 373 2 1 One Way Between Subjects ANOVA 383 2 2 Factonal Between Subjects ANOVA 403 2 3 Within Subjects ANOVA 433 2 4 Mixed Between Within Subjects ANOVA 453 2 5 Design Complexity 46

3 2 5 1 Nesting 463 2 5 2 Latin Square Designs 47

3 2 5 3 Unequal n and Nonorthogonality 483 2 5 4 Fixed and Random Effects 49

3 2 6 Specific Compansons 49

3 2 6 1 Weighting Coefficients for Compansons 493 2 6 2 Orthogonality of Weighting Coefficients 503 2 6 3 Obtained F for Compansons 503 2 6 4 Cntical F for Planned Compansons 513 2 6 5 Cntical F for Post Hoc Compansons 52

3 3 PARAMETER ESTIMATION 52

3 4 STRENGTH OF ASSOCIATION 53

3 5 BIVARIATE STATISTICS CORRELATION AND REGRESSION 543 5 1 Correlation 543 5 2 Regression 55

3 6 CHI-SQUARE ANALYSIS 56

x CONTENTS

CHAPTER 4 Cleaning Up Your Act Screening Data Priorto Analysis 57

4 1 IMPORTANT ISSUES IN DATA SCREENING 584 11 Accuracy of Data Files 584 12 Honest Correlations 58

4 12 1 Inflated Correlation 584 12 2 Deflated Correlation 58

4 13 Missing Data 604 13 1 Deleting Cases or Vanables 624 13 2 Estimating Missing Data 634 13 3 Using a Missing Data Con-elation Matnx 64

4 13 4 Treating Missing Data as Data 654 13 5 Repeating Analyses with and without Missing Data 65

4 1 4 Outliers 654 14 1 Detecting Univanate and Multivanate Outliers 66

4 14 2 Descnbmg Outliers 68

4 14 3 Reducing the Influence of Outliers 694 1 4 4 Outliers in a Solution 69

4 1 5 Normality Lineanty and Homoscedasticity 704 15 1 Normality 714 15 2 Lineanty 784 15 3 Homoscedasticity Homogeneity of Vanance Homogeneity

of Vanance Covanance Matnces 804 1 6 Common Data Transformations 814 17 Multicolhneanty and Singulanty 844 1 8 A Checklist and Some Practical Recommendations 87

4 2 COMPLETE EXAMPLES OF DATA SCREENING 8 84 2 1 Screening Ungrouped Data 88

4 2 1 1 Accuracy of Input Missing Data Distnbutions and Univanate Outliers 88

All 2 Lineanty and Homoscedasticity 944 2 13 Transformation 94

4 2 14 Detecting Multivanate Outliers 944 2 1 5 Vanables Causing Cases to be Outliers 964 2 16 Multicolhneanty 104

4 2 2 Screening Grouped Data 104

4 2 2 1 Accuracy of Input Missing Data Distnbutions Homogeneity of Vanance andUnivanate Outliers 106

4 2 2 2 Lineanty 1074 2 2 3 Multivanate Outliers 109

CONTENTS xi

4 2 2 4 Vanables Causing Cases to be Outliers 1114 2 2 5 Multicolhneanty 113

CHAPTER 5 Multiple Regression 127

5 1 GENERAL PURPOSE AND DESCRIPTION 127

5 2 KINDS OF RESEARCH QUESTIONS 12852 1 Degree of Relationship 1295 22 Importance of IVs 1295 23 Adding IVs 129

5 24 Changing IVs 1295 2 5 Contingencies among IVs 130

5 26 Companng Sets of IVs 1305 27 Predicting DV Scores for Members of a New Sample 1305 28 Parameter Estimates 130

5 3 LIMITATIONS TO REGRESSION ANALYSES 13153 1 Theoretical Issues 1315 3 2 Practical Issues 132

5 3 2 1 Ratio of Cases to IVs 1325 3 2 2 Outliers Among the IVs and on the DV 133

5 3 2 3 Multicolhneanty and Singulanty 1345 3 2 4 Normality Lineanty Homoscedasticity and Independence of Residuals 136

5 3 2 5 Outliers in the Solution 139

5 4 FUNDAMENTAL EQUATIONS FOR MULTIPLE REGRESSION 1 3 95 4 1 General Linear Equation 1405 4 2 Matnx Equations 1415 4 3 Computer Analyses of Small Sample Example 144

5 5 MAJOR TYPES OF MULTIPLE REGRESSION 1 4 65 5 1 Standard Multiple Regression 1495 5 2 Sequential Multiple Regression 1495 5 3 Statistical (Stepwise) and Setwise Regression 1505 5 4 Choosing among Regression Strategies 153

5 6 SOME IMPORTANT ISSUES 1 5 65 6 1 Importance of IVs 156

5 6 11 Standard Multiple and Setwise Regression 1585 6 12 Sequential or Statistical Regression 159

5 6 2 Statistical Inference 159

5 6 2 1 Test for Multiple R 159

XII CONTENTS

5 6 2 2 Test of Regression Components 1615 6 2 3 Test of Added Subset of IVs 161

5 6 2 4 Confidence Limits around B 162

5 6 2 5 Companng Two Sets of Predictors 1635 6 3 Adjustments of R2 1645 6 4 Suppressor Vanables 165

5 7 COMPARISON OF PROGRAMS 1 6 55 7 1 SPSS Package 1665 72 BMDPSenes 1725 7 3 SAS System 1735 74 SYSTAT System 173

5 8 COMPLETE EXAMPLES OF REGRESSION ANALYSIS 1745 8 1 Evaluation of Assumptions 174

5 8 11 Ratio of Cases to IVs 17458 12 Normality Lineanty Homoscedasticity and Independence of Residuals 1745 8 13 Outliers 1785 8 14 Multicollmeanty and Singulanty 179

5 82 Standard Multiple Regression 182

5 8 3 Sequential Regression 185

5 9 SOME EXAMPLES FOR THE LITERATURE 192

CHAPTER 6 Canonical Correlation 195

6 1 GENERAL PURPOSE AND DESCRIPTION 195

6 2 KINDS OF RESEARCH QUESTIONS 1966 2 1 Number of Canonical Vanate Pairs 196

6 2 2 Interpretations of Canonical Vanates 1966 2 3 Importance of Canonical Vanates 1976 2 4 Canonical Vanate Scores 197

6 3 LIMITATIONS 1976 3 1 Theoretical Limitations 1976 3 2 Practical Issues 198

6 3 2 1 Ratio of Cases to IVs 1986 3 2 2 Normality Lineanty and Homoscedasticity 1986 3 2 3 Missing Data 1996 3 24 Outliers 199

6 3 2 5 Multicolhneanty and Singulanty 199

CONTENTS XIII

6 4 FUNDAMENTAL EQUATIONS FOR CANONICAL CORRELATION 1 9 96 4 1 Eigenvalues and Eigenvectors 2006 4 2 Matnx Equations 2036 4 3 Proportions of Vanance Extracted 20564 4 Computer Analyses of Small Sample Example 207

6 5 SOME IMPORTANT ISSUES 2206 5 1 Importance of Canonical Vanates 2206 5 2 Interpretation of Canonical Vanates 221

6 6 COMPARISON OF PROGRAMS 2226 6 1 SPSS Package 2226 6 2 BMDPSenes 222

6 6 3 SAS System 2246 64 SYSTATSystem 224

6 7 COMPLETE EXAMPLE OF CANONICAL CORRELATION 2 2 467 1 Evaluation of Assumptions 227

6 7 11 Missing Data 2276 7 12 Normality Lineanty and Homoscedasticity 2276 7 13 Outliers 230

6 7 14 Multicolhneanty and Singulanty 2306 7 2 Canonical Con-elation 230

6 8 SOME EXAMPLES FROM THE LITERATURE 237

CHAPTER 7 Multiway Frequency Analysis 239

7 1 GENERAL PURPOSE AND DESCRIPTION 239

7 2 KINDS OF RESEARCH QUESTIONS 2407 2 1 Associations among Vanables 2407 2 2 Effect on a Dependent Vanable 2407 2 3 Parameter Estimates 241

7 2 4 Importance of Effects 2417 25 Strength of Association 2417 2 6 Specific Compansons and Trend Analysis 242

7 3 LIMITATIONS TO MULTIWAY FREQUENCY ANALYSIS 2 4 27 3 1 Theoretical Issues 2427 3 2 Practical Issues 242

7 3 2 1 Independence 242

xiv CONTENTS

7 3 2 2 Ratio of Cases to Vanables 2437 3 2 3 Adequacy of Expected Frequencies 2437 3 2 4 Outliers in the Solution 244

7 4 FUNDAMENTAL EQUATIONS FOR MULTIWAY FREQUENCY ANALYSIS 2447 4 1 Screening for Effects 245

7 4 11 Total Effects 2467 4 12 First Order Effects 2477 4 13 Second Order Effects 2487 4 14 Third Order Effects 253

7 4 2 Modeling 2537 4 3 Evaluation and Interpretation 256

7 4 3 1 Residuals 256

7 4 3 2 Parameter Estimates 2567 4 4 Computer Analyses of Small Sample Example 261

7 5 SOME IMPORTANT ISSUES 2797 5 1 Hierarchical and Nonheirarchical Models 2797 5 2 Statistical Cntena 279

7 5 2 1 Tests of Models 2797 5 2 2 Tests for Individual Effects 280

7 5 3 One Vanable as DV (Logit Analysis) 281

7 5 3 1 Program for Logit Analysis 2817 5 3 2 Odds Ratios 282

7 5 4 Strategies for Choosing a Model 2827 5 4 1 BMDP4F (Hierarchical) 2837 5 4 2 SPSS HILOGLINEAR (Hierarchical) 2847 5 4 3 SPSS LOGLINEAR and GENLOG (General Loglinear) 284

7 5 4 4 SAS CATMOD and SYSTAT LOGLIN (General Loglinear) 284

7 5 5 Contrasts 284

7 6 COMPARISON OF PROGRAMS 2 8 57 6 1 SPSS Package 2907 6 2 BMDPSenes 2907 6 3 SAS System 2917 6 4 SYSTAT System 291

7 7 COMPLETE EXAMPLE OF MULTIWAY FREQUENCY ANALYSIS 2 9 17 7 1 Evaluation of Assumptions Adequacy of Expected Frequencies 2917 7 2 Hierarchical Loglinear analysis 292

7 7 2 1 Preliminary Model Screening 292

CONTENTS xv

7 7 2 2 Stepwise Model Selection 2957 7 2 3 Adequacy of Fit 3027 7 2 4 Interpretation of the Selected Model 302

7 8 SOME EXAMPLES FROM THE LITERATURE 317

CHAPTER S Analysis of Covanance 3 2 1

8 1 GENERAL PURPOSE AND DESCRIPTION 321

8 2 KINDS OF RESEARCH QUESTIONS 324

82 1 Main Effects of IVs 3248 2 2 Interactions among IVs 3258 2 3 Specific Compansons and Trend Analysis 3258 24 Effects of Covanates 3258 25 Strength of Association 3258 2 6 Adjusted Marginal and Cell Means 326

8 3 LIMITATIONS TO ANALYSIS OF COVARIANCE 3268 3 1 Theoretical Issues 3268 3 2 Practical Issues 327

8 3 2 1 Unequal Sample Sizes and Missing Data 3278 3 22 Outliers 3278 3 2 3 Multicolhneanty and Singulanty 3288 3 24 Normality 3288 3 2 5 Homogeneity of Vanance 3288 3 2 6 Lineanty 3298 3 2 7 Homogeneity of Regression 329

8 3 2 8 Reliability of Covanates 330

8 4 FUNDAMENTAL EQUATIONS FOR ANALYSIS OF COVARIANCE 33084 1 Sums of Squares and Cross Products 331

8 4 2 Significance Test and Strength of Association 3358 4 3 Computer Analyses of Small Sample Example 336

8 5 SOME IMPORTANT ISSUES 3388 5 1 Test̂ for Homogeneity of Regression 33885 2 Design Complexity 342

8 5 2 1 Within Subjects and Mixed Within Between Designs 3438 5 2 2 Unequal Sample Sizes 3448 5 2 3 Specific Compansons and Trend Analysis 3468 5 2 4 Strength of Association 348

xvi CONTENTS

8 5 3 Evaluation of Covanates 3498 5 4 Choosing Covanates 3508 5 5 Alternatives to ANCOVA 350

8 6 COMPARISON OF PROGRAMS 35286 1 BMDPSenes 3528 6 2 SPSS Package 3568 6 3 SYSTAT System 3568 6 4 SAS System 356

8 7 COMPLETE EXAMPLE OF ANALYSIS OF COVARIANCE 3578 7 1 Evaluation of Assumptions 357

8 7 11 Unequal n and Missing Data 3578 7 12 Normality 3578 7 13 Lineanty 3608 7 14 Outliers 360

8 7 15 Multicolhneanty and Singulanty 3608 7 16 Homogeneity of Vanance 3648 7 17 Homogeneity of Regression 3648 7 18 Reliability of Covanates 364

8 7 2 Analysis of Covanance 364

8 8 SOME EXAMPLES FROM THE LITERATURE 373

CHAPTER 9 Multivanate Analysis of Variance andCovanance 375

9 1 GENERAL PURPOSE AND DESCRIPTION 375

9 2 KINDS OF RESEARCH QUESTIONS 3779 2 1 Main Effects of IVs 378

9 2 2 Interactions among IVs 3789 2 3 Importance of DVs 3789 2 4 Adjusted Marginal and Cell Means 3789 2 5 Specific Compansons and Trend Analysis 379

9 2 6 Strength of Association 3799 27 Effects of Covanates 3799 2 8 Repeated Measures Analysis of Vanance 379

9 3 LIMITATIONS TO MULTIVARIATE ANALYSIS OF VARIANCE ANDCOVARIANCE 3 8 09 3 1 Theoretical Issues 3809 3 2 Practical Issues 380

CONTENTS XVII

9 3 2 1 Unequal Sample Sizes and Missing Data 3819 3 2 2 Multivanate Normality 381

9 3 2 3 Outliers 3819 3 2 4 Homogeneity of Vanance Covanance Matnces 382

9 3 2 5 Lineanty 382

9 3 2 6 Homogeneity of Regression 3839 3 27 Reliability of Covanates 3839 3 2 8 Multicolhneanty and Singulanty 383

9 4 FUNDAMENTAL EQUATIONS FOR MULTIVARIATE ANALYSIS OF VARIANCEAND COVARIANCE 3 8 49 4 1 Multivanate Analysis of Vanance 3849 4 2 Computer Analyses of Small Sample Example 3919 4 3 Multivanate Analysis of Covanance 394

9 5 SOME IMPORTANT ISSUES 4 0 09 5 1 Cntena for Statistical Inference 4009 5 2 Assessing DVs 402

9 5 2 1 Univanate F 402

9 5 2 2 Roy Bargmann Stepdown Analysis 4039 5 2 3 Choosing among Strategies for Assessing DVs 404

9 5 3 Specific Compansons and Trend Analysis 4059 5 4 Design Complexity 405

9 5 4 1 Within Subjects and Between Within Designs 405

9 5 4 2 Unequal Sample Sizes 406

9 5 5 MANOVA vs ANOVAs 406

9 6 COMPARISON OF PROGRAMS 4 0 79 6 1 SPSS Package 4079 6 2 BMDPSenes 4099 6 3 SYSTAT System 4109 6 4 SAS System 410

9 7 COMPLETE EXAMPLES OF MULTIVARIATE ANALYSIS OF VARIANCE ANDCOVARIANCE 4 1 197 1 Evaluation of Assumptions 412

9 7 11 Unequal Sample Sizes and Missing Data 4129 7 12 Multivanate Normality 4139 7 13 Lineanty 4139 7 14 Outliers 41397 15 Homogeneity of Vanance Covanance Matnces 41397 16 Homogeneity of Regression 413

XVIII CONTENTS

9 7 17 Reliability of Covanates 4159 7 18 Multicolhneanty and Singulanty 415

9 7 2 Multivanate Analysis of Vanance 4159 7 3 Multivanate Analysis of Covanance 428

9 7 3 1 Assessing Covanates 42897 3 2 Assessing DVs 431

9 8 SOME EXAMPLES FROM THE LITERATURE 4389 8 1 Examples of MANOVA 4389 8 2 Examples of MANCOVA 439

CHAPTER 10 Profile Analysis of Repeated Measures 441

10 1 GENERAL PURPOSE AND DESCRIPTION 441

10 2 KINDS OF RESEARCH QUESTIONS 442

10 2 1 Parallelism of Profiles 44210 2 2 Overall Difference among Groups 44210 2 3 Flatness of Profiles 44210 2 4 Contrasts Following Profile Analysis 44310 2 5 Marginal/Cell Means and Plots 44310 2 6 Strength of Association 44310 2 7 Treatment Effects in Multiple Time Senes Designs 443

1 0 3 LIMITATIONS TO PROFILE ANALYSIS 4 4 310 3 1 Theoretical Issues 44310 3 2 Practical Issues 444

10 3 2 1 Sample Size and Missing Data 44410 3 2 2 Multivanate Normality 444

103 2 3 Outliers 44510 3 2 4 Homogeneity of Vanance Covanance Matnces 445

10 3 2 5 Lineanty 44510 3 2 6 Multicolhneanty and Singulanty 445

10 4 FUNDAMENTAL EQUATIONS FOR PROFILE ANALYSIS 44510 4 1 Differences in Levels 44710 4 2 Parallelism 44810 4 3 Flatness 45010 4 4 Computer Analyses of Small Sample Example 451

10 5 SOME IMPORTANT ISSUES 45910 5 1 Contrasts in Profile Analysis 459

CONTENTS xix

10 5 1 1 Parallelism and Flatness Significant Levels Not Significant (Simple EffectsAnalysis) 462

10 5 1 2 Parallelism and Levels Significant Flatness Not Significant (Simple EffectsAnalysis) 466

10 5 13 Parallelism Levels and Flatness Significant (Interaction Contrast) 47010 5 1 4 Only Parallelism Significant 474

10 5 2 Multivanate Approach to Repeated Measures 47410 5 3 Doubly Multivanate Designs 476

10 5 3 1 Kinds of Doubly Multivanate Analysis 47610 5 3 2 Example of Doubly Multivanate Analysis of Vanance 477

10 5 4 Classifying Profiles 483

10 6 COMPARISON OF PROGRAMS 48810 6 1 SPSS Package 48810 6 2 BMDPSenes 48810 6 3 SAS System 49010 6 4 SYSTAT System 490

1 0 7 COMPLETE EXAMPLE OF PROFILE ANALYSIS 4 9 010 7 1 Evaluation of Assumptions 491

10 7 1 1 Unequal Sample Sizes and Missing Data 49110 7 1 2 Multivanate Normality 49110 7 13 Lineanty 495

10 7 1 4 Outliers 49510 7 1 5 Homogeneity of Vanance Covanance Matnces 49510 7 16 Multicolhneanty and Singulanty 495

10 7 2 Profile Analysis 495

10 8 SOME EXAMPLES FROM THE LITERATURE 504

CHAPTER 1 1 Discriminant Function Analysis 5 0 7

11 1 GENERAL PURPOSE AND DESCRIPTION 507

11 2 KINDS OF RESEARCH QUESTIONS 509

112 1 Significance of Prediction 509112 2 Number of Significant Discnminant Functions 509112 3 Dimensions of Discnmination 509112 4 Classification of Functions 510112 5 Adequacy of Classification 510112 6 Strength of Association 510

xx CONTENTS

112 7 Importance of Predictor Vanables 510112 8 Significance of Prediction with Covanates 511

112 9 Estimation of Group Means 511

11 3 LIMITATIONS TO DISCRIMINANT FUNCTION ANALYSIS 511113 1 Theoretical Issues 51111 3 2 Practical Issues 512

113 21 Unequal Sample Sizes and Missing Data 512

113 2 2 Multivanate Normality 512113 23 Outliers 513113 2 4 Homogeneity of Vanance Covanance Matnces 513

113 25 Lineanty 51411326 Multicolhneanty and Singulanty 514

11 4 FUNDAMENTAL EQUATIONS FOR DISCRIMINANT FUNCTIONANALYSIS 5 1 4114 1 Denvation and Test of Discnminant Functions 514114 2 Classification 517

114 3 Computer Analyses of Small Sample Example 520

11 5 TYPES OF DISCRIMINANT FUNCTION ANALYSIS 528115 1 Direct Discnminant Function Analysis 528115 2 Sequential Discnminant Function Analysis 529

115 3 Stepwise (Statistical) Discnminant Function Analysis 532

11 6 SOME IMPORTANT ISSUES 533116 1 Statistical Inference 533

116 11 Cntena for Overall Statistical Significance 533

116 12 Stepping Methods 533116 2 Number of Discnminant Functions 536116 3 Interpreting Discnminant Functions 536

116 3 1 Discnminant Function Plots 5381163 2 Loading Matnces 539

116 4 Evaluating Predictor Vanables 540116 5 Design Complexity Factonal Designs 542116 6 Use of Classification Procedures 543

116 61 Cross Validation and New Cases 5441166 2 JackknifedClassification 545116 6 3 Evaluating Improvement in Classification 545

CONTENTS xxi

11 7 COMPARISON OF PROGRAMS 546117 1 SPSS Package 553117 2 BMDPSenes 553117 3 SYSTAT System 554

117 4 SAS System 554

11 8 COMPLETE EXAMPLE OF DISCRIMINANT FUNCTION ANALYSIS 554118 1 Evaluation of Assumptions 555

118 11 Unequal Sample Sizes and Missing Data 555118 12 Multivanate Normality 555118 13 Lineanty 555118 14 Outliers 556118 15 Homogeneity of Variance Covanance Matnces 556118 16 Multicolhneanty and Singulanty 556

118 2 Direct Discriminant Function Analysis 556

11 9 SOME EXAMPLES FROM THE LITERATURE 573

CHAPTER 12 Logistic Regression 575

12 1 GENERAL PURPOSE AND DESCRIPTION 575

12 2 KINDS OF RESEARCH QUESTIONS 576

12 2 1 Prediction of Group Membership of Outcome 57612 2 2 Importance of Predictors 57712 2 3 Interactions among Predictors 57712 2 4 Parameter Estimates 57712 2 5 Classification of Cases 57712 2 6 Significance of Prediction with Covanartes 57812 2 7 Strength of Association 578

12 3 LIMITATIONS TO LOGISTIC REGRESSION ANALYSIS 57812 3 1 Theoretical Issues 57812 3 2 Practical Issues 579

12 3 2 1 Ratio of Cases to Vanables 57912 3 2 2 Adequacy of Expected Frequencies 579123 2 3 Multicolhneanty 58012 3 2 4 Outliers in the Solution 580

xxii CONTENTS

12 4 FUNDAMENTAL EQUATIONS FOR LOGISTIC REGRESSION 5 8 012 4 1 Testing and Interpreting Coefficients 58112 4 2 Goodness of fit 58212 4 3 Companng Models 58312 4 4 Interpretation and Analysis of Residuals 58412 4 5 Computer Analyses of Small Sample Example 585

1 2 5 TYPES OF LOGISTIC REGRESSION 5 8 712 5 1 Direct Logistic Regression 58712 5 2 Sequential Logistic Regression 59112 5 3 Stepwise (Statistical) Logistic Regression 592

12 6 SOME IMPORTANT ISSUES 59412 6 1 Statistical Inference 594

12 6 11 Assessing Goodness of Fit of Models 59412 6 12 Tests of Individual Vanables 598

12 6 2 Number and Type of Outcome Categones 59912 6 2 1 Ordered Response Categones with BMDPPR 60012 6 2 2 Ordered Response Categones with SAS LOGISTIC 60212 6 2 3 Coding Outcome and Predictor Categones 605

12 6 3 Classification of Cases 60612 6 4 Hierarchical and Nonhierarchical Analysis 60712 6 5 Dosage Response Relationships 60712 6 6 Interpretation of Coefficients Using Odds 60712 6 7 Logistic Regression for Matched Groups 608

12 7 COMPARISON OF PROGRAMS 6 0 912 7 1 SPSS Package 60912 7 2 BMDPSenes 61312 7 3 SAS System 61412 7 4 SYSTAT System 614

12 8 COMPLETE EXAMPLES OF LOGISTIC REGRESSION 61512 8 1 Evaluation of Limitations 615

12 8 11 Adequacy of Expected Frequencies 615

12 8 1 2 Ratio of Cases to Vanables 61812 8 1 3 Multicolhneanty 618

12 8 14 Outliers in the Solution 61812 8 2 Direct Logistic Regression with Two Category Outcome 61812 8 3 Sequential Logistic Regression with Three Categones of Outcome 624

12 9 SOME EXAMPLES FROM THE LITERATURE 632

CONTENTS XXIII

CHAPTER 13 Principal Components and Factor Analysis 635

13 1 GENERAL PURPOSE AND DESCRIPTION 635

13 2 KINDS OF RESEARCH QUESTIONS 63713 2 1 Number of Factors 63813 2 2 Nature of Factors 63813 2 3 Importance of Solutions and Factors 63813 2 4 Testing theory in FA 63813 2 5 Estimating Scores on Factors 638

13 3 LIMITATIONS 63913 3 1 Theoretical Issues 63913 3 2 Practical Issues 640

13 3 2 1 Sample Size and Missing Data 64013 3 2 2 Normality 64013 3 2 3 Lineanty 64113 3 2 4 Outliers among Cases 64113 3 2 5 Multicolhneanty and Singulanty 64113 3 2 6 Factorabihty of R 64113 3 2 7 Outliers among Vanables 642

13 3 2 8 Outlying Cases among the Factors 642

13 4 FUNDAMENTAL EQUATIONS FOR FACTOR ANALYSIS 64213 4 1 Extraction 64413 4 2 Orthogonal Rotation 64713 4 3 Communahties Vanance and Covanance 64813 4 4 Factor Scores 649

13 4 5 Oblique Rotation 65113 4 6 Computer Analyses of Small Sample Example 653

13 5 MAJOR TYPES OF FACTOR ANALYSIS 66013 5 1 Factor Extraction Techniques 660

135 1 1 PCAvs FA 66213 5 12 Pnncipal Components 66413 5 13 ^Pnncipal Factors 66413 5 14 Image Factor Extraction 66413 5 15 Maximum Likelihood Factor Extraction 66513 5 16 Unweighted Least Squares Factonng 66513 5 17 Generalized (Weighted) Least Factonng 66513 5 18 Alpha Factonng 666

xxiv CONTENTS

13 5 2 Rotation 66613 5 2 1 Orthogonal Rotation 666

13 5 2 2 Oblique Rotation 668

13 5 2 3 Geometnc Interpretation 66913 5 3 Some Practical Recommendations 670

13 6 SOME IMPORTANT ISSUES 67113 6 1 Estimates of Communahties 671

13 6 2 Adequacy of Extraction and Number of Factors 67213 6 3 Adequacy of Rotation and Simple Structure 674

13 6 4 Importance and Internal Consistency of Factors 67513 6 5 Interpretation of Factors 67713 6 6 Factor Scores 37813 6 7 Compansons among Solutions and Groups 679

1 3 7 COMPARISON OF PROGRAMS 6 7 913 7 1 SPSS Package 67913 7 2 BMDPSenes 68313 7 3 SAS System 68313 7 4 SYSTAT System 683

13 8 COMPLETE EXAMPLE OF FA 68413 8 1 Evaluation of Limitations 684

13 8 11 Sample Size and Missing Data 684

13 8 1 2 Normality 68413 8 1 3 Lineanty 68513 8 14 Outliers among Cases 68513 8 15 Multicolhneanty and Singulanty 68513 8 16 Factorabihty of R 68513 8 1 7 Outliers among Vanables 685

13 8 1 8 Outlying Cases among the Factors 68913 8 2 Pnncipal Factors Extraction with Vanmax Rotation 689

13 9 SOME EXAMPLES FROM THE LITERATURE 706

CHAPTER 14 Structural Equation Modeling 709by Jodie B Ullman

14 1 GENERAL PURPOSE AND DESCRIPTION 709

CONTENTS xxv

14 2 KINDS OF RESEARCH QUESTIONS 71214 2 1 Adequacy of the Model 71314 2 2 Testing Theory 713

14 2 3 Amount of Vanance a Vanable Accounted for by a Factor 71314 2 4 Reliability of the Indicators 713

14 2 5 Parameter Estimated 71314 2 6 Mediation 71414 2 7 Group Differences 714

14 2 8 Longitudinal Differences 714

14 3 LIMITATIONS TO STRUCTURAL EQUATION MODELING 71414 3 1 Theoretical Issues 71414 3 2 Practical Issues 715

14 3 2 1 Sample Size and Missing Data 71514 3 2 2 Multivanate Normality and Outliers 715143 2 3 Lineanty 71614 3 2 4 Multicolhneanty and Singulanty 71614 3 2 5 Analyzabihty of Covanances 716143 2 6 Residuals 716

14 4 FUNDAMENTAL EQUATIONS FOR STRUCTURAL EQUATIONSMODELING 71714 4 1 Covanance Algebra 71714 4 2 Model Hypotheses 71914 4 3 Model Specifications 72014 4 4 Model Estimation 72314 4 5 Model Evaluation 72814 4 6 Computer Analysis of Small Sample Example 729

14 5 SOME IMPORTANT ISSUES 74314 5 1 Model Identification 74314 5 2 Estimation Techniques 746

14 5 2 1 Estimation Methods and Sample Size 747

14 5 2 2 Estimation Methods and Nonnormahty 74714 5 2 3 Estimation Methods and Dependence 748

14 5 2 4 v Some Recommendations for Estimation Method 74814 5 3 Assessing the Fit of the Model 748

14 5 3 1 Comparative Fit Indices 749145 3 2 Absolute Fit Index 750

xxvi CONTENTS

14 5 3 3 Indices of Proportion of Vanance Accounted For 75014 5 3 4 Degree of Parsimony Fit Indices 751

14 5 3 5 Residual Based Fit Indices 752

14 5 3 6 Choosing among Fit Indices 75214 5 4 Model Modification 752

14 5 4 1 Chi Square Difference Test 75214 5 4 2 Lagrange Multiplier Test (LM) 753145 4 3 WaldTest 75814 5 4 4 Some Caveats and Hints on Model Modification 758

14 5 5 Reliability and Proportion of Vanance 76314 5 6 Discrete and Ordinal Data 76414 5 7 Multiple Group Models 76514 5 8 Mean and Covanance Structure Models 766

14 6 COMPARISON OF PROGRAMS 76714 6 1 EQS 76714 6 2 LISREL 76714 6 3 SAS 77214 6 4 SYSTAT 772

14 7 COMPLETE EXAMPLES OF STRUCTURAL EQUATION MODELINGANALYSIS 77214 7 1 Model Specification for CFA 77214 7 2 Evaluation of Assumptions for CFA 774

14 7 2 1 Sample Size and Missing Data 77414 7 2 2 Normality and Lineanty 774147 2 3 Outliers 77414 7 2 4 Multicolhneanty and Singulanty 774147 2 5 Residuals 774

14 7 3 CFA Model Estimation and Preliminary Evaluation 77414 7 4 Model Modification 78314 7 5 SEM Model Specification 78914 7 6 Evaluation of Assumptions for SEM 789

14 7 6 1 Sample Size and Missing Data 78914 7 6 2 Normality and Lineanty 789147 6 3 Outliers 79314 7 6 4 Multicolhneanty and Singulanty 79314 7 6 5 Adequacy of Covanances 793147 66 Residuals 794

14 7 7 SEM Model Estimation and Preliminary Evaluation 79414 7 8 Model Modification 796

CONTENTS XXVII

14 8 SOME EXAMPLES FROM THE LITERATURE 810

CHAPTER 15 An Overview of the General Linear Model 813

15 1 LINEARITY AND THE GENERAL LINEAR MODEL 813

1 5 2 BIVARIATE TO MULTIVARIATE STATISTICS AND OVERVIEW OFTECHNIQUES 8 1 415 2 1 BivanateForm 81415 2 2 Simple Multivanate Form 81415 2 3 Full Multivanate Form 816

15 3 ALTERNATIVE RESEARCH STRATEGIES 818

Appendix A A Skimpy Introduction to Matrix Algebra 8 2 1

A 1 THE TRACE OF A MATRIX 822

A 2 ADDITION OF SUBTRACTION OF A CONSTANT TO A MATRIX 822

A 3 MULTIPLICATION OR DIVISION OF A MATRIX BY A CONSTANT 822

A 4 ADDITION AND SUBTRACTION OF TWO MATRICES 823

A 5 MULTIPLICATION TRANSPOSES AND SQUARE ROOTS OF MATRICES 824

A 6 MATRIX DIVISION (INVERSES AND DETERMINANTS] 826

A 7 EIGENVALUES AND EIGENVECTORS PROCEDURES FOR CONSOLIDATINGVARIANCE FROM A MATRIX 828

Appendix B Research Designs for Complete Examples 833

B 1 WOMEN S HEALTH AND DRUG STUDY 833

B 2 LEARNING DISABILITIES DATA BANK 834

B 3 SEXUAL ATTRACTION STUDY 837

Appendix C Statistical Tables 839

References 851

Index 861