SINGLE CASE EXPERIMENTAL DESIGNS (INTRA-SUBJECT REPLICATION DESIGNS)
III. Research Design Part I: Experimental Designs
description
Transcript of III. Research Design Part I: Experimental Designs
![Page 1: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/1.jpg)
III. Research Design Part I: Experimental Designs
![Page 2: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/2.jpg)
Hypotheses A hypothesis is a testable statement of causal
relationship between two variables, derived from theory
Must specify a relationship between an independent and dependent variable
Clear, specific, amenable to empirical testing, value-free (F-N & N)
![Page 3: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/3.jpg)
Research Design “The program that guides the investigator in
the process of collecting, analyzing, and interpreting observations. It is a logical model of proof that allows the researcher to draw inferences concerning causal relations among the variables under investigation” (F-N and N).
![Page 4: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/4.jpg)
Relationships (Covariation) between Variables
Relationships between variables: Two variables are related to one another (i.e. are correlated) if one or more values of one variable tend to be associated with one or more values of the other variable.
![Page 5: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/5.jpg)
Directional Relationships Apply to cases where the values of the IV and
DV are orderable (directional) variables Positive relationship:
As one’s education level increases, the frequency of voting increases
There is a positive relationship between one’s education level and voting frequency
![Page 6: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/6.jpg)
Directional Hypotheses (cont’d) Negative relationship:
As the number of hours of negative ads watched increases, the frequency of voting decreases
There is a negative relationship between exposure to negative advertising and one’s voting frequency
![Page 7: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/7.jpg)
Alachua
BakerBay
Bradford
Brevard
Broward
Calhoun
Charlotte
Citrus
ClayCollier
Columbia
Dade
DeSoto
Dixie
Duval
EscambiaFlagler
Franklin
Gadsden
GilchristGulf
Hamilton
Hardee
HendryHernando
Highland
Hillsborough
Holmes
Indian RiverJackson
Jefferson
Lafayette
LakeLee
LeonLevy
Liberty
Madison
Manatee
MarionMartin
Monroe
NassauOkaloosa
Okeechobee
OrangeOsceolaPalm Beach
Pasco
Pinellas
Polk
Putnam Santa RosaSarasota
Seminole
St. Johns
St. Lucie
Sumter
Suwannee
Taylor
Union
Volusia
Wakulla
Walton
Washington
.05
.1.1
5.2
Ave
rage
Mon
thly
San
ctio
n R
ate
-3 -2 -1 0 1 2Local Conservatism (Initiative-Based)
Average Monthly Sanction Rate Fitted values
![Page 8: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/8.jpg)
Non-Directional Hypotheses Appropriate for variables that are not orderable Hypothesis describes comparison among categories Examples
Men have greater levels of support for President Bush than do women
Whites are most likely to be Republican, while African-Americans are most likely to be Democrat
![Page 9: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/9.jpg)
Research Design and Causality Relationships between variables: Two
variables are related to one another (i.e. are correlated) if one or more values of one variable tend to be associated with one or more values of the other variable.
Causal relationship: A relationship in which one variable directly causes/explains the other variable.
![Page 10: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/10.jpg)
Establishing Causality (F-N&N) 3 Criteria (Evidence Needed) for Establishing
Causality Covariation (X is correlated with Y) Time Order (X precedes Y in time) Nonspuriousness (The observed relationship
between X and Y is not spurious)
![Page 11: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/11.jpg)
Spurious Relationship A relationship between two variables that is
presumed to be causal, when in fact it is not
An observed relationship between X and Y is said to be spurious (or partly spurious) if there exists a third variable Z (a “control variable”) which is both a cause of Y AND is correlated with X.
![Page 12: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/12.jpg)
Spurious Relationship
YX
Z
(Presumed Causal Relationship)
(True Causal Relationship)
![Page 13: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/13.jpg)
Example of Spuriousness
![Page 14: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/14.jpg)
Gender and Corruption – Causal or Spurious?
http://www.iq.harvard.edu/blog/pb/2005/10/sex_and_corruption.html
![Page 15: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/15.jpg)
The Democratic Peace: Causal or Spurious?
WarDemocracy
Z(???)
![Page 16: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/16.jpg)
Experimental Designs1. Select a sample2. Randomly assign subjects into 2 or more groups. The number of groups is equivalent to the number of values of the independent variable(s).3. Observe (measure) DV for all groups (if design includes pretest)4. Introduce the stimulus (IV)5. Observe (measure) DV for each group6. If the change in the value of the dependent variable varies significantly across groups, then we conclude that X Y
Key distinguishing features of an experimental design: Randomization and Manipulation of IV by the researcher (when introduced and to whom)
![Page 17: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/17.jpg)
Experimental Designs
A bunch of people
TreatmentGroup
ControlGroup
RandomAssignment
RandomAssignment
Measurethe DV
MeasureThe DV
(PRE-TEST)
“Stimulus”
“Placebo”
Introducethe IV
TreatmentGroup
ControlGroup
Measurethe DV
(POST-TEST)
Measurethe DV
![Page 18: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/18.jpg)
Simple Experimental Designs2-Group Pretest - Posttest Design (Classical or
“Simple” Experiment) R O1 X O2 R O3 O4
![Page 19: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/19.jpg)
Simple Experimental Designs2-Group Pretest - Posttest Design (Classical or
“Simple” Experiment) R O1 X O2 R O3 O4 OR R O1 X1 O2 R O3 X2 O4
![Page 20: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/20.jpg)
Experiments and Causality Correlation?
![Page 21: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/21.jpg)
Experiments and Causality Covariation? Comparison of two or more groups (on
dependent variable) experiencing different levels of exposure to the causal (explanatory) variable (X). This establishes covariation.
![Page 22: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/22.jpg)
Experiments and Causality Time Order?
![Page 23: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/23.jpg)
Experiments and Causality Time Order? The introduction of the independent variable
(“stimulus”) is manipulated by the researcher to insure that changes in IV precede changes in DV.
![Page 24: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/24.jpg)
Experiments and Causality Spuriousness?
![Page 25: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/25.jpg)
Experiments and Causality Spuriousness? Random assignment insures that rival
hypotheses are ruled out, thus eliminating the threat of spuriousness. (How?)
![Page 26: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/26.jpg)
Experiments and Causality Spuriousness? Random assignment insures that rival
hypotheses are ruled out, thus eliminating the threat of spuriousness. (How?)
Use of “matching” as a strategy to control for rival explanations
![Page 27: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/27.jpg)
Simple Experimental Designs 2-Group Posttest Only Design R X O1 R O2
![Page 28: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/28.jpg)
Simple Experimental Designs 2-Group Posttest Only Design R X O1 R O2 OR R X1 O1 R X2 O2
![Page 29: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/29.jpg)
Other Types of Experimental Designs Multiple Group Pretest - Posttest Design
R O1 X1 O2R O3 X2 O4R O5 X3 O6R O7 X4 O8
Multiple Group Posttest Only DesignR X1 O1R X2 O2R X3 O3R X4 O4
![Page 30: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/30.jpg)
Other Types of Experimental Designs Solomon 4-Group Design
R O1 X O2R O3 O4R X O5
R O6
![Page 31: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/31.jpg)
Extensions of the Classical Experiment Multiple observations over time R O1 X1 O2 O3……
R O4 X2 O5 O6……
![Page 32: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/32.jpg)
Extensions of the Classical Experiment Factorial designs - each group represents a unique combination of
values on two (or more) different variables.
For two independent variables X and Z (where X and Z each take on two possible values):R O1 X1, Z1 O2R O3 X2, Z1 O4R O5 X1, Z2 O6R O7 X2, Z2 O8
Factorial designs allow the researcher to test for an interaction effect: Two independent variables interact to affect a dependent variable if the effect of one variable depends on the value of the second variable.
![Page 33: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/33.jpg)
![Page 34: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/34.jpg)
Zilber and Niven (SSQ) Table 1: 2-Group Posttest Only
R X1(black) O1R X2(A-A) O2
![Page 35: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/35.jpg)
![Page 36: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/36.jpg)
Zilber and Niven (SSQ) Table 3: 2X2 Factorial Design
R (black/liberal) O1 R (A-A/liberal) O2 R (black/conserv) O3 R (A-A/conserv) O4
To see how the effect of racial label varies as a function of ideology, we compare
O1-O2 to O3-O4
![Page 37: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/37.jpg)
![Page 38: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/38.jpg)
Zilber and Niven (SSQ) Table 3: 2X2 Factorial Design
R (black/liberal) O1 R (A-A/liberal) O2 R (black/conserv) O3 R (A-A/conserv) O4
Conclusion: The choice of racial labels does affect white attitudes toward blacks, but only among liberals.
![Page 39: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/39.jpg)
1.What effect do A & B have on O?2. Is there an interactioneffect? (Explain)
![Page 40: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/40.jpg)
Evaluating Research Designs:Internal Validity Internal Validity - the degree to which we can be sure that the
independent variable caused the dependent variable within the current sample
“Extrinsic Factors”: Selection effects with respect to recruitment/assignment of subjects (units) to treatment and control groups
“Intrinsic Factors”: Factors threatening validity that… occur outside the “laboratory” during the period of the study result from changes in, reactions to (or general ineffectiveness of) the
measuring instrument, or involve some type of reactive effect of observation
![Page 41: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/41.jpg)
Evaluating Research Designs:Internal Validity Extrinsic factors:
Selection Important intrinsic factors include:
History Maturation Experimental mortality Instrumentation Testing Regression artifact Interactions with selection - e.g. “selection history” and
“selection maturation”
![Page 42: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/42.jpg)
For each of the following intrinsic threats to internal validity, explain: What the specific threat means Whether or not (and why or why not) an
experimental design is protected from this threat
![Page 43: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/43.jpg)
History? R O1 X1 O2 R O3 X2 O4
![Page 44: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/44.jpg)
Maturation? R O1 X1 O2 R O3 X2 O4
![Page 45: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/45.jpg)
Experimental Mortality? R O1 X1 O2 R O3 X2 O4
![Page 46: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/46.jpg)
Instrumentation? R O1 X1 O2 R O3 X2 O4
![Page 47: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/47.jpg)
Testing? R O1 X1 O2 R O3 X2 O4
![Page 48: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/48.jpg)
Regression Artifact? R O1 X1 O2 R O3 X2 O4
![Page 49: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/49.jpg)
“Selection History” and “Selection Maturation” R O1 X1 O2 R O3 X2 O4
![Page 50: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/50.jpg)
Evaluating Research Designs:External Validity External Validity - the degree to which the
results of the analysis can be generalized beyond the current sample/study. Can be maximized by: Using subjects (units) that are representative of
the population to which one’s theory applies Using a “laboratory” that is as close to “real life”
conditions as possible Field experiments
![Page 51: III. Research Design Part I: Experimental Designs](https://reader035.fdocuments.us/reader035/viewer/2022062323/568161e5550346895dd20767/html5/thumbnails/51.jpg)
Applications1. Iyengar, Shanto, Mark D. Peters, and Donald R. Kinder. 1982. “
Experimental Demonstrations of the ‘Not-So-Minimal’ Consequences of Television News Programs.” American Political Science Review 76: 848-58.
2. Schram, Sanford F., Joe Soss, Richard C. Fording and Linda Houser. 2009. “Deciding to Discipline: A Multi-Method Study of Race, Choice, and Punishment at the Frontlines of Welfare Reform .” American Sociological Review, 74(3), 398-422.
3. Gerber, Alan S., Donald P. Green, and Christopher W. Larimer. 2008. “Social pressure and voter turnout: evidence from a large-scale field experiment.” American Political Science Review 102:33–48.
Assignment #5(Due October 5): In approximately 2 single-spaced pages, answer the following questions.
1. For each of the three application readings, identify and diagram (using the notation in Frankfort-Nachmias and Nachmias) the specific type of experimental design employed by the authors.
2. Choose one of the three studies to focus on for this question. Evaluate the internal validity and external validity of the study you have chosen.