RESEARCH METHODS A Process of Inquiry

ANTHONY M. GRAZIANO MICHAEL L RAULIN STA TE UNIVERSITY OF NEW YORK A T BUFFALO

tm HarperCollinsPublishers

C O N T E N T S

Instructor's Preface xv Student's Preface xix

1 CURIOSITY, CREATIVITY, AJVD COMMITMENT 1 Science Is a Way of Thinking 2 Asking Questions 2 Science and Art 5 Common Methods of Acquiring Knowledge 6

Tenacity 6 Intuition 6 Authority 6 Rationalism 7 Empiricism 8 Science 9

Emerging Modern Science 9 Early Civilization 10 Greek Science 11 Medieval Science 14 The Scientific Revolution 15

The Science of Psychology 18 Summary 18 Review Exercises 20

2 RESEARCH IS A PROCESS OF INQUIRY 22 Basic Assumptions of Science 23 Observation and Inference: Facts and Constructs 23 Conceptual Models in Science 26 Inductive and Deductive Thinking 29 Models and Theories in Science 31

v

A Model of the Research Process 32 Phases of Research 32

Idea-generating Phase 32 / Problem-Definition Phase 34 / Procedures Design Phase 34 / Observation Phase 35 / Data-Analysis Phase 35 / Interpretation Phase 35 / Communication Phase 36

Levels of Constraint 37 Naturalistic Observation 40 / Case-Study Method of Observation 41 / Correlational Research 41 / Differential Research 41 / Experimental Research 41

Summary 42 Review Exercises 43

THE STARTING POINT: ASKING QUESTIONS 45 Asking Questions 46

Sources of Questions 46 Researcher's Personal Interests and Observations 46 / Other Investigators' Theories and Research 46 / Seeking Solutions to Practical Problems 47

Refining Questions for Research 48 Types of Variables in Research 51

Behavioral Variables 51 Stimulus Variables 51 Organismic or Subject Variables 52 Independent and Dependent Variables 52

Validity and the Control of Extraneous Variables 54 Research Ethics 55

Ethical Guides for Human Research 56 Ethical Checks 60 / Institutional Review Boards (IRBs) 61

Ethical Principles in Research with Animals 61 Summary 63 Review Exercises 64

DATA AND THE NATURE OF MEASUREMENT 67 Research Variables 68 Measurement 68 Scales of Measurement 71

Nominal Scales 71 Ordinal Scales 72 Interval Scales 72 Ratio Scales 73

Types of Data 73 Nominal Data 74 Ordered Data 74 Score Data 74

CONTENTS VII

Measuring and Controlling Variables 75 Measurement Error 75 Operational Definitions 75 Reliability, Effective Range, and Validity 78 Scale Attenuation Effects 80

The Need for Objective Measurement 80 Statistical Analyses 81

Summary 81 Review Exercises 82

STATISTICAL ANALYSIS OF DATA 85 Individual Differences and Statistical Procedures 86 Descriptive Statistics 87

Frequency Counts and Distributions 89 Nominal and Ordinal Data 89 / Score Data 89

Graphical Representation of Data 90 Summary Statistics 93

Measures of Central Tendency: Mode, Median, and Mean 93 / Measures of Variability: Range, Variance, and Standard Deviation 95 / Measures of Relationship (Correlations) 99

Summary of Descriptive Statistics 102 Inferential Statistics 102

Populations and Samples 102 The Null Hypothesis 103 Statistical Decisions and Alpha Levels 104 Type I and Type II Errors 104 Testing for Mean Differences 104

Simple Mest 105 / Correlated Mest 105 / Analysis of Variance (ANOVA) 105 Summary 106 Review Exercises 106

NATÜRALISTIC AND CASE-STÜDY RESEARCH 109 Examples of Naturalistic Research 111 Examples of Case-Study Research 114 The Value of Low-Constraint Methods 116

Conditions for Using Low-Constraint Research 116 Information Gained from Low-Constraint Research 117

Problem Statements and Hypotheses in Naturalistic and Case-Study Research 120

Using Naturalistic and Case-Study Methods 121 Making Observations 121 Ethical Issues 123 Sampling of Subjects 124

VÜi CONTENTS

Sampling of Situations 125 Sampling of Behaviors 125

Evaluating and Interpreting Data 125 Limitations of Naturalistic and Case-Study Methods 126

Poor Representativeness 126 Poor Replicability 127 Causal Inferences and Low-Constraint Research: The Ex Post Facto

Fallacy 127 Limitations of the Observer 129

Summary 130 Review Exercises 130

7 CORRELATIONAL AND DIFFERENTIAL METHODS OF RESEARCH 133

Correlational Research Methods 134 Differential Research Methods 135

Artifacts and Confounding Variables 136 What Makes Differential Research Higher Constraint than Correlational

Research? 138 When to Use Correlational and Differential Research 139 Conducting Correlational and Differential Research 140

Conducting Correlational Research 140 Measurement 140 / Sampling 141 / Analysis of Data 141 / Interpreting the Correlation 142

Conducting Differential Research 143 Measurement 143 / Selecting Appropriate Control Groups 143 / Sampling 146 / Analysis of Data 148 / Interpretation of Data 148

Limitations of Correlational and Differential Research 148 Problems in Determining Causation 148 Confounding Variables 150

Summary 150 Review Exercises 151

8 HYPOTHESIS TESTING, VALIDITY, AND THREATS TOVALIDITY 153

Hypothesis Testing 154 Statement of the Problem 155 Operational Definitions 157 Research Hypothesis 158 Testing the Research Hypothesis 159

Null Hypothesis 160 / Confounding Variable Hypothesis 160 / Causal Hypothesis 161

Summary 163

CONTENTS

Validity and Threats to Validity 163 Types of Validity 164

Statistical Validity 164 / Construct Validity 164 / External Validity 165 / Internal Validity and Confounding Variables 166

Some Major Confounding Factors 168 Maturation 168 History 168 Testing 169 Instrumentation 169 Regression to the Mean 170 Selection 170 Attrition 170 Diffusion of Treatment 171 Sequencing Effects 171 Summary 172

Threats to Validity from Subject and Experimenter Effects 172 Subject Effects 172 Experimenter Effects 173

Statistical Versus Practical Significance 174 Validity, Control, and Constraint 175 Summary 175 Review Exercises 176

CONTROLS TO REDUCE THREATS TO VALIDITY IN EXPERIMENTAL RESEARCH 178

Threats to Validity 179 General Control Procedures 180

Preparation of Setting 180 Response Measurement 181 Replication 181

Control Over Subject and Experimenter Effects 182 Single- and Double-Blind Procedures 182 Automation 184 Using Objective Measures 185 Multiple Observers 185 Using Deception 185

Control Through Subject Selection and Assignment 187 Subject Selection 187

Random Sampling 188 / Stratified Random Sampling 189 / Ad hoc Samples 189

Subject Assignment 190 Free Random Assignment 191 / Matched Random Assignment 191 / Other Matching Procedures 193

Control Achieved Through Specific Experimental Design 194 The True Experiment 195

Summary 197 Review Exercises 198

CONTROL OF VARIANCE THROUGH EXPERIMENTAL DESIGN: SINGLE-VARIABLE, INDEPENDENT-GROUPS DESIGN 200

Experimental Design 201 Variance 201

Forms of Variance 202 Systematic Between-Groups Variance 202 / Nonsystematic Within-Groups Variance 203

Controlling Variance in Research 205 Maximizing Experimental Variance 205 / Controlling Extraneous Variance 205 / Minimizing Error Variance 206

Nonexperimental Research Designs 207 Ex Post Facto Design 207 Single-Group, Posttest-Only Design 208 Single-Group, Pretest-Posttest Design 209 Pretest-Posttest, Natural Control-Group Design 210

Experimental Research Designs 211 Designs for Testing One Independent Variable 211

Randomized, Posttest-Only, Control-Group Design 211 / Randomized, Pretest-Posttest, Control-Group Design 212 / Multilevel, Completely Randomized, Between-Subjects Design 213 / Solomon Four-Group Design 213

Summary 215 Statistical Analyses of Completely Randomized Designs 215

Mest 215 / Analysis of Variance (ANOVA) 215 / Specific Means Comparisons in ANOVA 219

Other Experimental Designs 220 Independent- Versus Correlated-Groups Designs 220 / Single-Variable Versus Multivariable or Factorial Designs 220

Summary 221 Review Exercises 221

CONTROL OF VARIANCE THROUGH EXPERIMENTAL DESIGN: SINGLE-VARIABLE, CORRELATED-GROUPS DESIGN 224

Correlated-Groups Design 225 Within-Subjects Design 226

Using Within-Subjects Design 226 / Analyzing Within-Subjects Design 228 / Strengths and Weaknesses of Within-Subjects Design 229 / Summary 233

CONTENTS Xi

Matched-Subjects Design 233 Using Matched-Subjects Design 233 / Analyzing Matched-Subjects Design 236 / Strengths and Weaknesses of Matched-Subjects Design 237 / Summary 237

Single-Subject Design 238 Summary 238 Review Exercises 238

CONTROL OF VARIANCE THROUGH EXPERIMENTAE DESIGN: FACTORIAL DESIGN 240

Factorial Design 241 Main Effects and Interactions 242 Possible Outcomes of Factorial Design 246 An Example: Children's Dark-Fears Study 250 Analysis of Variance in Factorial Design 254

ANOVA Summary Table for Children's Dark-Fears Study 255 Variations of Basic Factorial Design 256

Within-Subjects (Repeated Measures) Factorial 256 Mixed Designs 258

Between-Subjects and Within-Subjects Variables 258 / Manipulated and Nonmanipulated Variables 260 / Summary 261

ANOVA: A Postscript 261 Summary 264 Review Exercises 264

FIELD RESEARCH: A SECOND LOOK AT RESEARCH IN NATURAL SETTINGS 267

Conducting Field Research 268 Reasons for Doing Field Research 268 Difficulties in Field Research 270

Quasi-experimental Design 271 Nonequivalent Control-Group Design 272 Interrupted Time-Series Design 276

Single-Subject Design 281 Reversal Design (ABA Design) 285 Multiple Baseline Design 287 Single-Subject, Randomized, Time-Series Design 288

Program Evaluation 291 Practical Problems in Program Evaluation Research 291 Issues of Control 292

Selecting Appropriate Dependent Measures 293 / Minimizing Bias in Dependent Measures 293 / Control Through Research Design in Program Evaluation 293

XÜ CONTENTS

Typical Program Evaluation Designs 293 Randomized, Control-Group Design 293 / Single-Group, Time-Series Design 294 / Pretest-Posttest Design 294

Summary of Program Evaluation 294

Survey Research 294 Types of Surveys 294

Status Survey 294 / Survey Research 295 Steps in Survey Research 295

Developing the Instrument 295 / Sampling Considerations 295 / Sampling Procedures 296 / Sample Size and Confidence Intervals 297

Research Design of Surveys 297 Cross-Sectional Design 297 / Longitudinal Design 298

Summary of Survey Research 298 Longitudinal and Cross-Sectional Research 298 Summary 298 Review Exercises 299

14 SELECTING APPROPRIATE STATISTICAL PROCEDURES 302

An Initial Example 303 A Decision-Tree Model 306

Decision-Tree Flowcharts 306 Identifying Research Variables 307 Identifying Major Characteristics of the Study 312 Selecting Appropriate Descriptive and Inferential Statistics 313

Statistical Procedures for the First Hypothesis 313 / Statistical Procedures for Other Hypotheses 314

Secondary Analyses 314 Post hoc Analyses 315 / Secondary Analyses to Help Explain Results 315 / Data Snooping 315

Caveats and Disclaimers 315 Summary 317 Review Exercises 317

15 RESEARCH METHODOLOGY: AN EVOLVEVG DISCIPLINE 320

New Directions in Research Methodology 322 New Questions and New Methods 322

The Evolution of Research Questions and Methods 322 / The Impact of Other Disciplines 323 / Multidisciplinary Research 324 / Moving Research Out of the Laboratory 324 / The Impact of Computers 324

Science: An Interaction Between Empiricism and Rationalism 327 Review Exercise 328

CONTENTS XÜi

APPENDIX A Writing a Research Report: APA Publication Style 329

Structure of a Research Article 330 Writing the Research Report 330

Using Levels of Headings to Organize 330 Sections of a Research Report 331

Title Page 331 / Abstract 331 / Introduction 332 / Method 332 / Results 333 / Discussion 335 / References 335

Writing Style 336 Summary 341

APPENDIX B Statistical Tables 342 B.1 Area Under the Standard Normal Curve 343 B.2 Critical Values of Student's t 345 B.3 Critical Values of the F Distribution 346 B.4 Critical Values of the Pearson Product-Moment

Correlation 350 B.5 Critical Values of the Spearman Rank-Order

Correlation 351 B.6 Probability Values of the Chi-Square Distribution 351 B.7 Random Numbers 352

APPENDIX C Statistical Computation Procedures 357 Descriptive Statistics 358

Measures of Central Tendency 358 Mean 358 / Median 358 / Mode 358

Measures of Variability 358 Variance 358 / Standard Deviation 359

Measures of Relationship 359 Pearson Product-Moment Correlation 359 / Spearman Rank-Order Correlation 360

Inferential Statistics 360 Parametric Statistics 360

Independent Samples /-Test 361 / Correlated /-Test 362 / Analysis of Variance (ANOVA) 363

Nonparametric Statistics 371 Chi-Square Goodness-of-Fit Test 371 / Chi-Square Test for Independence 371 / Cautions Regarding Chi-Square 372 / Mann-Whitney U-Jesi 373 / Other Nonparametric Statistics 374

Summary 374

XiV CONTENTS

APPENDIX D Selected Answers to Section III Exercises 375

Glossary 383 References 399 Name Index 405 Subject Index 407