Experimental Research

Experimental Research

What is experimental research? Research investigation in which conditions are

controlled so that hypotheses can be tested and alternative explanations can be ruled out

Research used to make “cause-and-effect” statements (X causes Y) X is the independent (or manipulated or causal) variable Y is the dependent variable

Experimental Research

Evidence of causality (i.e., X causes Y) Evidence of concomitant variation (X and Y co-

vary) The more of X, the better chance that we will get Y

Time order sequence of variables If X causes Y, X must occur before Y

Elimination of other possible explanations of why Y occurred

Key is to keep all experimental conditions equal

Experimental Research

Must try to eliminate all possible extraneous influences

Possible extraneous influences History - event occurring during the course of an

experiment (but not really part of the experimental manipulation) that influences the results

Maturation - changes which occur in the experimental unit which occur during the course of an experiment that are due to the passage of time

Experimental Research

Possible extraneous influences (continued) Testing Effects - changes in experimental unit

due to the experiment itself (but not related to the key experimental manipulation Measurement effect -- prior measurements effect the

measure of the dependent variable Interaction effects -- subject pays greater attention to

stimuli than they normally would because its and experiment

Experimental Research

Possible extraneous influences (continued) Instrument variation -- changes in the measuring

instrument cause changes in Y (i.e, interviewer changes the way in which they ask questions)

Selection Bias -- if two groups are compared, this notion suggests that the groups may not have been equivalent to begin with; i.e., a manipulation did not cause Y; group dissimilarities did

Experimental Research

Possible extraneous influences (continued) Experimental mortality -- respondents in

experimental groups were lost during the duration of the experiment

Statistical Regression -- Extreme responses move closer to the midpoint during the course of an experiment; subjects after questionning do not want to appear to be extreme

Experimental Research

Experimental Symbols X = Experimental Treatment (e.g., ad exposure) O = Observation R = Random Assignment of Subjects

Pre - Experimental Designs One-Shot Case Study

X O Problems -- Selection Bias, Control Group, Mortality,

History, etc.

Experimental Research

Pre-Experimental Designs (Cont’d) One Group Pre-Test/Post-Test Design

O1 X O2

Analysis -- O2 - O1

Problems -- History, Maturation, interactive Testing Effect, etc. Static Comparison

X O1

O2

Analysis -- O2 - O1

Problems -- Selection Bias, Mortality, etc.

Experimental Research

True Experimental Designs Before/After with Control Group

(R) O1 X O2

(R) O3 O4

Analysis (O4 – O3) – (O2 – O1)

Lessens History and Maturation Problems However, interactive testing effect still an issue

O2 and O4 may a function of O1 and O3

Experimental Research

Experimental Designs (True Experimental Designs) Six Group - Four Study Design

(R) O1 X O2

(R) O3 O4

(R) X O5

(R) O6

O2 – O1 compared to O4 – O3 tells if X “worked” (manipulation check) – If yes, go on

If O5 > O6 – we have true effects

Experimental Research

True experiments are very complex Experimental designs that are used are often

flawed (or not “True”) Random assignment helps

Experimental Research

Analyzing Data Typically uses Analysis of Variance (ANOVA)

Tests (determines if at least one treatment group mean is different from the others) Ho: Μ1 = M2 = M3 = … = Mk Ha: M1 ≠ M2 ≠ M3 ≠ … ≠ Mk

Why not use t-tests? Probability

One comparison at 95% confidence – 95% of rejecting null when it should be rejected

Two comparisons -- .95 x .95 = 90.25% Three comparisons -- .95 x .95 x .95 = 86%

Like regression Dependent Variable (interval or ratio scaled) Independent Variable (nominal or ordinal scaled)

Often group membership

Experimental Research (Responses on 1 = Bad; 7 = Outstanding)Treat/Resp Ad 1 Ad 2 Ad 3

1 1 2 3

2 5 2 6

3 5 5 6

4 6 2 6

5 3 2 7

6 6 2 5

Mean 4.33 2.50 5.50

Experimental Research

One-Way ANOVA Attempts to partition variance

Total Variance (Within Group + Between Group) Within treatment groups (Observation – Group Mean)

SSw = Σ Σ (x ij – Meani)2

MSw = SSw / df w

Between treatment groups (Group Mean – Grand Mean) SSb = Σ nj (Meanj – Meani) 2

MSb = SSb / dfb

F-statistic MSb / MSw

With dfb,; df w

Follow-up test Tukey (most common in MR)

Experimental Research

Additional Issues with ANOVA

Real World Often used with survey data

Compare means of several groups DV – is interval (or ratio) scaled IV -- categorical

Two-Way ANOVA One dependent variable

Two or more independent variables Simultaneous effects

MANOVA Multiple dependent variables

ANCOVA Investigate effects after “controlling” for another variable (that is interval or ratio scaled)

Experimental Research

Compare means of several groups EXAMPLE (Survey Data)

Cool Warm

Permissive NeglectingIndulgent

Restrictive Authoritarian Authoritative

Experimental Research

DV = Parent Responsibility to Restrict TV Choices for Kids F= 4.42; p = .005

Neglecting (M = 29.5) Indulgent (M = 31.26) Authoritarian (M = 30.26) Authoritative (M = 32.27)

Follow-up Tests Authoritative > Authoritarian & Neglecting Indulgent > Neglecting Authoritarian = Neglecting

Experimental Research

Two-Way ANOVA – Example Main Effects

Similar to one-way ANOVA effect (for 2 variables) Interactive Effects

Main Effects look null Differences observed only when looking at both factors

simultaneously

Experimental Research

Two-Way ANOVA One dependent variable – Main Effects

P

N

Named Not-Named

Experimental Research

Two-Way ANOVA One dependent variable – Interaction Effects

P

N

Named Not-Named