Quiz 1 next Monday & Tuesday - University of Queenslanduqwloui1/stats/3010 for post... ·...
Transcript of Quiz 1 next Monday & Tuesday - University of Queenslanduqwloui1/stats/3010 for post... ·...
1
11
psyc3010 lecture 4psyc3010 lecture 4
factorial betweenfactorial between--Ps ANOVA III:Ps ANOVA III:higherhigher--order ANOVAorder ANOVA
last lecture: factorial between-Ps ANOVA II(following up significant effects)
next lecture: power analyses and blocking designs
22
Quiz 1 next Monday & Tuesdaypreparation• review Lectures 1 4• organize your notes so you know where to find information• look at quiz tips + complete the online practice questions • complete the online Practice Quiz on Blackboard• ?s welcome before the quiz. During the quiz, I will not be
available.
taking the quiz• opens @ 9am Sunday (27/3), closes @ 9pm Monday (28/3)• no time restrictions + can return to active quiz• can submit quiz only once, and must do so by closing date • work alone
2
33
last lecture this lecturelast lecture: – following up significant main effects and
interactions in 2-way factorial ANOVA
this lecture:– higher-order factorial designs
(“complex ANOVA”)
Next lectures:– March 28th guest lecture by Joanne Brown on
power analyses & Blocking designs.– I return in Week 6.
44
topics for this week topics for this week introduction to higherintroduction to higher--order designsorder designs
omnibus tests in 3omnibus tests in 3--way factorial ANOVAway factorial ANOVAmain effects, 2main effects, 2--way interactions, 3way interactions, 3--way interactionsway interactions
overview of followoverview of follow--up tests in 3up tests in 3--way ANOVAway ANOVAfollowing up main effects, 2following up main effects, 2--way interactions, way interactions, and 3and 3--way interactionsway interactions
following up 3following up 3--way interactions way interactions -- detailsdetailssimple interaction effectssimple interaction effectssimple simple simplesimple effectseffectssimple simple simplesimple comparisonscomparisons
3
55
higher-order factorial designs:an introduction
66
higherhigher--order factorial designsorder factorial designsmore than 2 independent variables (factors)allow for designs with higher external validity
world is often more complicated than a 2 x 3
EXAMPLE EXAMPLE –– predicting driving performancepredicting driving performance
•• a number of variables could have an important influence:a number of variables could have an important influence:* age (young, old)* age (young, old)* amount of alcohol drunk (0, 1, or 5 drinks)* amount of alcohol drunk (0, 1, or 5 drinks)* gender (men, women)* gender (men, women)
how do these factors influence driving performance,independently and/or interactively?
4
77
•• 2 (age)2 (age) x x 3 (alcohol)3 (alcohol) x x 2 (gender)2 (gender)between-subjects design
• 12 cells [calculated as for 2-way factorial design]
menmenno alc 1 drink 5 drinksno alc 1 drink 5 drinks
womenwomenno alc 1 drink 5 drinksno alc 1 drink 5 drinks
oldold
youngyoung
notation for highernotation for higher--order designsorder designs
88
main effects:main effects:–– differences between marginal means of one differences between marginal means of one
factor factor (averaging over levels of other factors)(averaging over levels of other factors)
effects in highereffects in higher--order designsorder designs
5
99
main effect of gendermain effect of gender
menmen
(averaged across (averaged across alcohol and age)alcohol and age)
womenwomen
(averaged across (averaged across alcohol and age)alcohol and age)
main effects in highermain effects in higher--order designsorder designs
1010
main effects:main effects:–– differences between marginal means of one differences between marginal means of one
factor factor (averaging over levels of other factors)(averaging over levels of other factors)
twotwo--way interactions:way interactions:–– whether the effect of one factor is the whether the effect of one factor is the
same at every level of another factor same at every level of another factor (averaging over levels of a third factor)(averaging over levels of a third factor)
effects in highereffects in higher--order designsorder designs
6
1111
age by alcohol interactionage by alcohol interaction
no alc 1 drink 5 drinksno alc 1 drink 5 drinks((averaged across averaged across men & women)men & women)
oldold
youngyoung
22--way intxs in higherway intxs in higher--order designsorder designs
1212
0100200300400500600700800
no alcohol 1 drink 5 drinks
Alcohol consumption
Driv
ing
perf
orm
ance
old
young
1. compare line slopes – are lines are (not) parallel? (to check for interaction)
2. compare averages of red line and blue line (to check for main effect of factor represented by lines)
3. compare average for each column (to check for main effect of factor represented on X axis)
graph for 2-way interaction
7
1313
1414
main effects:main effects:–– differences between marginal means of one differences between marginal means of one
factor factor (averaging over levels of other factors)(averaging over levels of other factors)
twotwo--way interactions:way interactions:–– examines whether the effect of one factor is examines whether the effect of one factor is
the same at every level of another factor the same at every level of another factor (averaging over levels of a third factor)(averaging over levels of a third factor)
threethree--way interaction:way interaction:–– examines whether the twoexamines whether the two--way interaction way interaction
between two factors is the same at every between two factors is the same at every level of the third factorlevel of the third factor
effects in highereffects in higher--order designsorder designs
8
1515
age x alcohol x gender age x alcohol x gender (no averaging)menmen
no alc 1 drink 5 drinksno alc 1 drink 5 drinkswomenwomen
no no alcalc 1 drink 5 drinks1 drink 5 drinks
oldold
youngyoung
3-way intx in higher-order designs
does a 2-way interaction hold at each level of the 3rd factor?
1616
Females
0
200
400
600
800
1000
No drink 1 drink 5 drinks
Alcohol consumption
Driv
ing
perf
orm
ance
old
young
Males
0
200
400
600
800
1000
no alcohol 1 drink 5 drinks
Alcohol consumption
Driv
ing
perf
orm
ance
old
young
1. plot 2-way interactions withineach level of the third factor
2. check if pattern for 1st graph (simple interaction of AB at C1) is different from 2nd graph (simple interaction of AB at C2) -- If graphs are
not same pattern, there is a 3-way interaction
difficult to interpret other 2-way interactions from graphs, let alone main effects
The more complex design, the more rely on statistical tests not eyeballing
graphs for 3-way interactions
9
1717
partitioning variance in partitioning variance in a 3a 3--way ANOVAway ANOVA
e
main effectsvariance due to αα
variance due to ββ
variance due to γγ
2-way interactionsvariance due to αβαβ
variance due to βγβγ
variance due to αγαγ
3-way interactionvariance due to αβγαβγ
error/residual
variance due to ee
1818
structural models in factorial ANOVA
22--way factorial design:way factorial design:
XXijkijk = = μμ. + . + ααj j + + ββk k + + αβαβjkjk + + eeijkijk
33--way factorial design:way factorial design:
XXijklijkl ==
μμ. + . + ααjj + + ββk k + + γγll + + αβαβjkjk + + βγβγklkl + + αγαγjljl + + αβγαβγjkljkl + + eeijklijkl
10
1919
degrees of freedomdegrees of freedomdfdftotal total = N = N -- 1 1
dfdfJ J = j = j -- 11dfdfK K = k = k -- 11dfdfL L = l = l -- 11
dfdfJK JK = (j = (j -- 1)(k 1)(k -- 1)1)dfdfJL JL = (j = (j -- 1)(l 1)(l -- 1)1)dfdfKL KL = (k = (k -- 1)(l 1)(l -- 1)1)
dfdfJKLJKL = (j = (j -- 1)(k 1)(k -- 1)(l 1)(l -- 1)1)
dfdferror error = N = N -- jkljkl
regardless of # of factors in between-groups design, df for a factor always = # of levels - 1
df for an interaction = product of df for factors involved
df for error =(N - # cells) or (n -1) x (# cells)
2020
33--way factorial designs:way factorial designs:omnibus tests omnibus tests
11
2121
yet another (quasi) experimentyet another (quasi) experiment
Reinforcement Sensitivity TheoryReinforcement Sensitivity Theory
do people learn better from a particular type of do people learn better from a particular type of reinforcement: reward versus punishment?reinforcement: reward versus punishment?
impulsive personality impulsive personality learn well from reward learn well from reward but not punishmentbut not punishment
anxious personality anxious personality learn well from punishment learn well from punishment but not rewardbut not reward
gender differences are also possible gender differences are also possible
2222
yet another (quasi) experimentyet another (quasi) experimentDV:DV: speed of responses in reaction time (RT) tasktime (RT) task
3 independent factors:3 independent factors:•• reinforcement typereinforcement type
-- reward for fast responses reward for fast responses -- punishment for slow responses, punishment for slow responses, -- control condition (no reward or punishment) control condition (no reward or punishment)
•• personality typepersonality type-- anxious personalityanxious personality-- impulsive personalityimpulsive personality
•• gendergender-- malemale-- femalefemale
12
2323
7 omnibus tests baby!7 omnibus tests baby!
–– 3 main effects3 main effects-- reinforcement (reward, punishment, none) reinforcement (reward, punishment, none) -- personality (impulsive, anxious)personality (impulsive, anxious)-- gender (male, female)gender (male, female)
–– 3 two3 two--way interactionsway interactions (a.k.a. first(a.k.a. first--order interactions)order interactions)-- reinforcement x personalityreinforcement x personality-- reinforcement x genderreinforcement x gender-- personality x genderpersonality x gender
–– 1 three1 three--way interactionway interaction (a.k.a. second(a.k.a. second--order interaction)order interaction)-- reinforcement x personality x genderreinforcement x personality x gender
yet another (quasi) experimentyet another (quasi) experiment
2424
data and cell totals / meansdata and cell totals / means(full layout)(full layout)
Males
Personality Rew None Pun
Impulsive 310 355 490320 350 495330 360 485
Total 960 1065 1470Mean 320 355 490
Anxious 485 450 310490 455 320495 445 330
Total 1470 1350 960Mean 490 450 320
Reinforcement
Females
Personality Rew None Pun
Impulsive 310 450 490320 455 486330 445 480
Total 960 1350 1456Mean 320 450 485
Anxious 485 345 310480 350 320490 355 330
Total 1455 1050 960Mean 485 350 320
Reinforcement
13
2525
dftotal = N - 1 = 36 - 1 = 35
dfP = p - 1 = 2 - 1 = 1dfG = g - 1 = 2 - 1 = 1dfR = r - 1 = 3 - 1 = 2
dfPG = (p - 1)(g - 1) = 1 x 1 = 1dfRG = (r - 1)(g - 1) = 2 x 1 = 2 dfPR = (p - 1)(r - 1) = 1 x 2 = 2dfPRG = (p - 1)(g - 1)(r - 1) = 1 x 1 x 2 = 2
dferror = N - prg = 36 - 2 x 3 x 2 = 36 – 12 = 24
degrees of freedomdegrees of freedomregardless of # of factors in between-groups design, df for a factor = # levels - 1
df for an interaction = product of df for factors involved
df for error =(N - # cells) or (n -1) x (# cells)
2626
Mathematics is the language with which Mathematics is the language with which God has written the universe.God has written the universe.
-- Galileo Galilei, physicist and Galileo Galilei, physicist and astronomer (1564astronomer (1564--1642)1642)
14
2727
summary table (from SPSS)summary table (from SPSS)Tests of Between-Subjects Effects
Dependent Variable: RT
51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709
53.778 1 53.778 1.075 .310
168516.722 2 84258.361 1684.232 .000
.056 2 .028 .001 .999
9538.778 1 9538.778 190.670 .000
19015.056 2 9507.528 190.045 .000
1200.667 24 50.028198383.889 35
Source
ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal
Type III Sumof Squares df Mean Square F Sig.
2828
no significant main effectsno significant main effectsTests of Between-Subjects Effects
Dependent Variable: RT
51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709
53.778 1 53.778 1.075 .310
168516.722 2 84258.361 1684.232 .000
.056 2 .028 .001 .999
9538.778 1 9538.778 190.670 .000
19015.056 2 9507.528 190.045 .000
1200.667 24 50.028198383.889 35
Source
ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal
Type III Sumof Squares df Mean Square F Sig.
15
2929
significant 2significant 2--way interaction: way interaction: personality x reinforcementpersonality x reinforcement
Tests of Between-Subjects Effects
Dependent Variable: RT
51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709
53.778 1 53.778 1.075 .310
168516.722 2 84258.361 1684.232 .000
.056 2 .028 .001 .999
9538.778 1 9538.778 190.670 .000
19015.056 2 9507.528 190.045 .000
1200.667 24 50.028198383.889 35
Source
ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal
Type III Sumof Squares df Mean Square F Sig.
3030
0100200300400500600
Rew Neut Pun
Task condition
Mea
n re
actio
n tim
e
Imp
Anx
significant 2significant 2--way interaction: way interaction: personality x reinforcementpersonality x reinforcement
effect of at least one of the two factors is different at different levels of the other factorignoring (averaging across) the third factor
16
3131
significant 2significant 2--way interaction:way interaction:personality x genderpersonality x gender
Tests of Between-Subjects Effects
Dependent Variable: RT
51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709
53.778 1 53.778 1.075 .310
168516.722 2 84258.361 1684.232 .000
.056 2 .028 .001 .999
9538.778 1 9538.778 190.670 .000
19015.056 2 9507.528 190.045 .000
1200.667 24 50.028198383.889 35
Source
ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal
Type III Sumof Squares df Mean Square F Sig.
3232
360370380390400410420430
Imp Anx
Personality
Mea
n re
actio
n tim
e
Men
Women
significant 2significant 2--way interaction:way interaction:personality x genderpersonality x gender
effect of at least one factor is different at different levels of the other factorignoring (averaging across) the third factor
17
3333
significant 3significant 3--way interaction:way interaction:reinforcement x personality x genderreinforcement x personality x gender
Tests of Between-Subjects Effects
Dependent Variable: RT
51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709
53.778 1 53.778 1.075 .310
168516.722 2 84258.361 1684.232 .000
.056 2 .028 .001 .999
9538.778 1 9538.778 190.670 .000
19015.056 2 9507.528 190.045 .000
1200.667 24 50.028198383.889 35
Source
ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal
Type III Sumof Squares df Mean Square F Sig.
3434
so there’s this 3so there’s this 3--way interactionway interactionmeans that a two-way interaction is different at different levels of the third factor
Simple interaction of AxB is for C1 than simple interaction for C2, etc..
But which simple interaction to look at ? Could mean one or more of the following:
personality X genderpersonality X gender 2-way interaction is different across levels of reinforcement
reinforcement X personalityreinforcement X personality 2-way interaction is different across levels of gender
reinforcement X genderreinforcement X gender 2-way interaction is different across levels of personality
18
3535
following up following up significant omnibus effects significant omnibus effects
in a 3in a 3--way ANOVA: way ANOVA: overview of all testsoverview of all tests
3636
following up main effectsfollowing up main effectsin 3in 3--way factorial ANOVAway factorial ANOVA
just as in a 2-way ANOVA, a significant omnibus main effect must be interpreted– e.g. what is the effect of Factor A, ignoring
(averaging over) Factor B and Factor C?
if the factor has > 2 levels, we don’t know exactly where the differences are
we then use main effect comparisons(with t-tests or linear contrasts), exactly as we did in 2-way ANOVA
19
3737
following up omnibus 2following up omnibus 2--way interactionsway interactionsin 3in 3--way factorial ANOVAway factorial ANOVA
just as in a 2-way ANOVA, a significant omnibus 2-way interaction must be interpreted– e.g., is the effect of Factor A different at different
levels of Factor B, and vice-versa? (ignoring Factor C)
we then test simple effects (with the F test), exactly as we did in 2-way ANOVA
if you find a significant simple effect for a factor with > 2 levels, you follow it up with simple comparisons (with t-tests or linear contrasts), exactly as we did in 2-way ANOVA
3838
following up a 3following up a 3--way interaction way interaction in 3in 3--way factorial ANOVA:way factorial ANOVA:
overview of the stepsoverview of the stepssimple interaction effects– F tests
if simple interaction effects are significant, follow up with simple simple effects– F tests
if simple simple effects are significant with > 2 levels, follow up with simple simple comparisons– t-tests and linear contrasts
20
3939
following up a following up a 33--way interaction, part I:way interaction, part I:simple interaction effectssimple interaction effects
4040
so there’s this 3so there’s this 3--way interactionway interaction
need to focus your investigation1) go back to theory and hypotheses2) conduct follow-up analyses to test predictions
this could mean one or more of the following:personality X genderpersonality X gender 22--way interaction way interaction is different across levels of is different across levels of reinforcementreinforcement
reinforcement X personalityreinforcement X personality 22--way interaction way interaction is different across levels of is different across levels of gendergender
reinforcement X genderreinforcement X gender 22--way interaction way interaction is different across levels of is different across levels of personalitypersonality
21
4141
simple interaction effectssimple interaction effects• simple interaction effects break down the 3-way
interaction into a series of 2-way interactions at each level of the third factor
WHY WOULD WE DO THIS?• this gives a first close-up look at where the differences
between cell means might be• once we know this, we can follow up these simple 2-way
interactions further to figure out where the differences are(simple simple effects & simple simple comparisons / contrasts)just as we follow up an interaction in a 2-way design
in a 3-way design there are three potential follow-up steps (compared to two in a 2-way design)
4242
the graphs depicting the 2 x 2 x 3 interaction between gender, personality, and reinforcement provide a visual representation of the simple interaction effects we would conduct: here, the simple personality x reinforcement interaction at each of the two levels of gender
GENDER: 2.00 Female
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
GENDER: 1.00 Male
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
Males Females
22
4343
GENDER: 2.00 Female
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
GENDER: 1.00 Male
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
omnibus 2-way interaction in a 3-way design: ignores levels of the third factor
tests the 2-way P x R interaction with data averaged across gender
(i.e., ignoring the third factor)
simple 2-way interactions in a 3-way design: test the 2-way interaction at each level of the third factor
tests the 2-way P x R interaction for each gender group separately(i.e., at each level of third factor)
4444
don’t mix up your designsdon’t mix up your designssimple interaction effects in a 3-way design look like a series of 2-way ANOVAs at each level of the third factor(e.g. a P x R ANOVA for men, then a P x R ANOVA for women)But the F ratios calculated for these tests are not the same– simple interaction (a) uses pooled error term; (b) is conducted after significant 3-way interaction is observed
omnibus P x R 2-way interaction in 2-way designs:• separate P x R studies for men and women (e.g., at different times, in different recruiting contexts – confounds?)• MSerror taken from each 2-way omnibus ANOVA table
(different tables for men and women, so different MSerror)
simple P x R 2-way interaction in a 3-way design:• is a follow-up test for 3-way interaction (P x R x G)• men and women were sampled as part of the same study• MSerror taken from 3-way omnibus ANOVA table
23
4545
Source SS df MS F p
PR at G1 95725.00 2 47862.50 956.72 0.000PR at G2 91806.78 2 45903.39 917.56 0.000
Error 1200.67 24 50.03
critical F at alpha=.05 (2,24) = 3.40
summary table for P x Rsimple interaction effects
includes F tests for all P x R simple interaction effects (i.e., the P x R simple interaction effect at each level of the gender factor–men and women)
4646
Source SS df MS F p
PR at G1 95725.00 2 47862.50 956.72 0.000PR at G2 91806.78 2 45903.39 917.56 0.000
Error 1200.67 24 50.03
critical F at alpha=.05 (2,24) = 3.40
these are the SS values for the effects
degrees of freedom for a simple interaction effect are the df for the
associated interaction df = dfPR (2 - 1)(3 - 1) = 2
SSerror term (and df) is taken from the omnibus 2 x 2 x 3 ANOVA
mean squares and F values calculated
as usual
indicates that the personality x reinforcement interaction is significant for males and females
follow-up tests will identify where the differences are
24
4747
following up a 3following up a 3--way interaction,way interaction,part II: simple part II: simple simplesimple effects &effects &simple simple simplesimple comparisonscomparisons
4848
simplesimple simple effectssimple effectssimple effects after an omnibus 2simple effects after an omnibus 2--way way interaction examine the effect of factor A interaction examine the effect of factor A at each level of factor Bat each level of factor B
simple simple simplesimple effects effects are like simple effects are like simple effects except that they examine except that they examine the effect of factor A the effect of factor A at each level of factor B, at each level of factor C at each level of factor B, at each level of factor C (i.e., within each combo of B & C)(i.e., within each combo of B & C)
Simple Simple simplesimple effects differ from oneeffects differ from one--way way ANOVAs because they use the ANOVAs because they use the MSMSerrorerror from the from the omnibus ANOVA table as the error term omnibus ANOVA table as the error term
25
4949
simple simple effect # 1:personality at each level of reinforcement (separately for men and women)
results: for both men and women, the effect of personality at each level of reinforcement was significant (although opposite under neutral reinforcement!)
GENDER: 2.00 Female
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
GENDER: 1.00 Male
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
Males Females
5050
Source SS df MS F p
P at R1 at G1 43350.00 1 43350.00 866.52 0.000P at R2 at G1 13537.50 1 13537.50 270.60 0.000P at R3 at G1 4330.50 1 4330.50 86.56 0.000
P at R1 at G2 40837.50 1 40837.50 816.30 0.000P at R2 at G2 15000.00 1 15000.00 299.83 0.000P at R3 at G2 41002.7 1 41002.66 819.60 0.000
Error 1200.67 24 50.03
critical F at alpha=.05 (1,24) = 4.26
R1 = reward, R2 = neutral, R3 = punishmentG1 = men, G2 = women
summary tablesummary tablesimple simple simplesimple effects of effects of personalitypersonality,,
at each level of reinforcement, at each level of reinforcement, at each level of genderat each level of gender (males and females)(males and females)
26
5151
simple simple effect # 2:reinforcement at each level of personality (separately for men and women)
results: for both men and women, the effect of reinforcement at each level of personality was significant
GENDER: 2.00 Female
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
GENDER: 1.00 Male
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
Males Females
5252
Source SS df MS F p
R at P1 at G1 48350.00 2 24175.00 483.23 0.000R at P2 at G1 47400.00 2 23700.00 473.74 0.000
R at P1 at G2 45483.56 2 22741.78 454.58 0.000R at P2 at G2 46350.00 2 23175.00 463.24 0.000
Error 1200.67 24 50.03
critical F at alpha=.05 (2,24) = 3.40BUT reinforcement factor has > 2 levels: how do we know which cell means are significantly different from each other?
P1 = impulsive, P2 = anxiousG1 = men, G2 = women
summary tablesummary tablesimple simple simplesimple effects of reinforcement, effects of reinforcement,
at each level of personality, at each level of personality, at each level of gender at each level of gender (males (males and and females)females)
27
5353
simplesimple simple comparisonssimple comparisons
Like simple comparisons (or contrasts) except we Like simple comparisons (or contrasts) except we compute for each level of a third factorcompute for each level of a third factor
formulae from Lecture 3 can be used:formulae from Lecture 3 can be used:
abdferror −=nMSaLt
errorj∑=
2
∑= jj XaL
5454
GENDER: 2.00 Female
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
GENDER: 1.00 Male
Reinforcement
PunNeutRew
500
400
300
Personality
Imp
Anx
Males Females
some possible comparisons…
R1 and R2 vs R3 at P1 for G1 R2 vs R3 at P2 for G2
28
5555
Males Rew None Pun
Impusivity 320 355 490
Contrast 1 1 -1 0Contrast 2 1 1 -2
Anxiety 490 450 320
Contrast 1 1 -1 0Contrast 2 1 1 -2
Reinforcement
simple simple comparisons simple simple comparisons for for reinforcementreinforcement at at
each level of each level of personalitypersonality (for (for malesmales) )
5656
calculations for impulsivity contrast 1calculations for impulsivity contrast 1
06.6
350.03)0)1(1(
00.35222
−=+−++
−=t
L = 2(66.88) – 1(66.88) – 1(35.63) = -35.63
t’α=.05 (24) = 2.39
(with Bonferroni adjustment for 2 comparisons)
L = 1(320) – 1(355) + 0(490) = -35.00
Males Rew None Pun
Impusivity 320 355 490
Contrast 1 1 -1 0Contrast 2 1 1 -2
Anxiety 490 450 320
Contrast 1 1 -1 0Contrast 2 1 1 -2
Reinforcement
29
5757
and so on for and so on for
–– impulsivity contrast 2impulsivity contrast 2–– anxiety contrast 1anxiety contrast 1–– anxiety contrast 2anxiety contrast 2–– then all four contrasts then all four contrasts
for femalesfor females
5858
oh, the followoh, the follow--up tests you’ll doup tests you’ll do
omnibus testsomnibus tests–– 7 (3 main effects, 3 two7 (3 main effects, 3 two--way interactions, 1 threeway interactions, 1 three--way interaction)way interaction)
simple interaction effectssimple interaction effects–– 2 (personality x reinforcement at each level of gender)2 (personality x reinforcement at each level of gender)
simple simple simplesimple effectseffects–– 10 (6 for personality at each level of reinforcement for men & women, 10 (6 for personality at each level of reinforcement for men & women,
4 for reinforcement at each level of personality for men & women)4 for reinforcement at each level of personality for men & women)
simple simple simplesimple comparisonscomparisons–– 8 (2 comparisons for each personality condition for men & women) 8 (2 comparisons for each personality condition for men & women)
total = 27 tests! (in this study)total = 27 tests! (in this study)
30
5959
but what if you don’t want to?but what if you don’t want to?conducting an exhaustive set of followconducting an exhaustive set of follow--up tests for higherup tests for higher--order factorial designs can inflate order factorial designs can inflate familywisefamilywise error rateerror rate27 tests from previous slide: each has Type27 tests from previous slide: each has Type--1 error of .051 error of .05
means a means a familywise alpha of 27 *.05 = 1.35 (!)
ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends entirely upon your research predictionsdepends entirely upon your research predictions
in our case we had (implicitly) predicted the personality x in our case we had (implicitly) predicted the personality x reinforcement interaction, and we were going to see if this reinforcement interaction, and we were going to see if this interaction was the same for males and females:interaction was the same for males and females:
•• impulsive impulsive reward not punishmentreward not punishment•• anxious anxious punishment but not rewardpunishment but not reward•• possible gender differencespossible gender differences
6060
steps for following up steps for following up a 3a 3--way interactionway interaction
is 3-way significant?
NO
calculate simple interaction effects at each level of least
important factor or according to hypotheses
YES
NOYES
does the factor have >2 levels?
NO YESconduct tests
for simple simple
comparisons
STOP
STOP
STOP
STOP
is A x B at C1 or C2
significant?
Conduct simple simple effects of key IV (A or
B). Are they significant?
NO
STOP
YES
31
6161
summaryin a 3-way ANOVA, you have main effects and 2-way interactions (like a 2-way ANOVA) plus a 3-way interaction
the follow-up tests for omnibus main effects and two-way interactions are like in a 2-way ANOVA –main effect comparisons, simple effects, simple comparisons
following up a 3-way interaction can be very complex and time-consuming
highlights the need for analyses to be driven by hypotheses
6262
Reporting Reporting some sources suggest that once you find a some sources suggest that once you find a
significant interaction you should ignore the significant interaction you should ignore the lower lower order effects order effects
––main main effects have been effects have been “qualified”“qualified” by the interaction by the interaction ––22--way interaction qualified by 3way interaction qualified by 3--wayway
this this is because the is because the higher order interactions higher order interactions maymayrequire you to change the interpretation given by the require you to change the interpretation given by the lower effect alone lower effect alone
––Partly depending on whether it’s a Partly depending on whether it’s a disordinaldisordinal interactioninteractionultimatelyultimately, there is no simple rule:, there is no simple rule:
––what what you report depends you report depends upon upon your research predictions your research predictions ––Usually, if Usually, if you predict you predict an effectan effect, then report that , then report that effect effect (and any follow(and any follow--up testsup tests))
32
6363
Reporting our 2 way Reporting our 2 way designdesign
“Results indicated a significant main effect of consumption, F(2,42) = 20.07, p<.001, ω2 = .34. Linear contrasts with a Bonferroni adjustment for 2 comparisons indicated that creativity ratings were significantly lower after 2 or 4 pints than after consuming no alcohol, t’(42) = 2.91, p<.05 (Ms = 63.75, 55.63), and were lower after 4 pints than after 2 pints, t’(42) = 5.63, p<.05 (Ms = 64.69, 46.56). There was no significant main effect for distraction, indicating that creativity ratings for distracted participants’ limericks (M = 56.46) were not significantly different from those for controls (M = 60.21), F(1,42) = 2.03, p = .16, ω2 = .01. There was, however, a significant interaction between consumption and distraction, indicating that the effect of consumption was different for distracted and control participants, F(1,42) =11.91, p<.001, ω2 = .20. The interaction is depicted in Figure 1.”NB Interaction needs following up in results section (simple effects + simple comparisons if nec.).Discuss: although the predicted main effect of alcohol consumption was significant, the direction of the effect was contrary to hypotheses: alcohol lowered creativity ratings. Also the predicted effect of distraction was not significant.
I haven’t put effect sizes in for the follow-up comparisons / contrasts; most do nowadays esp. if report Fs.
6464
reporting simple reporting simple effects effects it is preferable to not report all sets of simple effects, for 2 it is preferable to not report all sets of simple effects, for 2
reasons:reasons:a)a) the more simple effects we calculate, the greater our risk of making a the more simple effects we calculate, the greater our risk of making a
type 1 error (see Howell, p.436)type 1 error (see Howell, p.436)b)b) usually both sets of simple effects will communicate usually both sets of simple effects will communicate similarsimilar
information information -- redundancyredundancy–– so, in our case we would want to report so, in our case we would want to report eithereither the simple effects of the simple effects of
distraction (at each level of consumption) distraction (at each level of consumption) oror the simple effects of the simple effects of consumption (at each level of distraction)consumption (at each level of distraction)
ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends depends entirelyentirely upon your research predictions.upon your research predictions.
–– Let’s say we Let’s say we specifically predicted that “specifically predicted that “the effect of consumption on the effect of consumption on creativity ratings will be stronger for distracted participants than for creativity ratings will be stronger for distracted participants than for controls”. Therefore, we would want to report the simple effects for controls”. Therefore, we would want to report the simple effects for consumptionconsumption (and associated simple comparisons / contrasts)(and associated simple comparisons / contrasts)
33
6565
reportingreporting
“. . .To follow up the significant two-way interaction, the simple effects of consumption were analysed at each level of distraction. There was a significant simple effect of consumption for distracted participants, F(2,42) = 31.36, p<.001, ω2 = .56, but not for controls, F(2,42) = 0.61, p = .546, ω2 = .00. The significant simple effect of consumption for distracted participants was followed up with Linear contrasts using a Bonferroni adjustment for 2 comparisons. These indicated that, for distracted participants, creativity ratings were lower after 2 or 4 pints than after consuming no alcohol, t’(42) = 4.52, p<.001 (Ms = 66.88, 51.26), and also lower after 4 pints than after 2 pints, t’(42) = 6.86, p<.001 (Ms = 66.88, 35.63).”
I haven’t put effect sizes in for the follow-up comparisons / contrasts ; most do now . Effect sizes for simple effects are also required. NB if you calc w2 for controls it works out to -.01 – a meaningless % (% cannot be negative), so set to zero. Another reason some prefer to report eta2.
Discuss: the hypothesis was confirmed that the effect of consumption on hypothesis was confirmed that the effect of consumption on creativity will be stronger for distracted participants than for controls.creativity will be stronger for distracted participants than for controls.
6666
"Writing is easy; all you do is sit staring at "Writing is easy; all you do is sit staring at a blank sheet of paper until the drops of a blank sheet of paper until the drops of blood form on your forehead." blood form on your forehead." ---- Gene Gene FowlerFowler
"There's nothing to writing. All you do is "There's nothing to writing. All you do is sit down at a typewriter and open a vein." sit down at a typewriter and open a vein." ---- Red SmithRed Smith
34
6767
Reporting threeReporting three--way designsway designsconducting an exhaustive set of followconducting an exhaustive set of follow--up tests for up tests for higherhigher--order factorial designs can inflate familywise order factorial designs can inflate familywise alpha (and is very tedious!)alpha (and is very tedious!)
ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends depends entirelyentirely upon your research predictionsupon your research predictions–– in our case we had (implicitly) predicted the Personality x in our case we had (implicitly) predicted the Personality x
Reinforcement interaction, and we were going to see if this Reinforcement interaction, and we were going to see if this interaction was the same for males and femalesinteraction was the same for males and females
•• people with an impulsive personality learn well from reward people with an impulsive personality learn well from reward but not punishment, and people with an anxious personality but not punishment, and people with an anxious personality learn well from punishment but not reward.learn well from punishment but not reward.
•• Possible gender differences not well understood.Possible gender differences not well understood.–– hence our write up might have gone something like thishence our write up might have gone something like this
6868
reportingreporting“The predicted interaction was significant, “The predicted interaction was significant, FF(2, 24) = (2, 24) = 1684.23, 1684.23, pp<.001<.001, but this was qualified by 3, but this was qualified by 3--way interaction way interaction among personality, reinforcement, and gender among personality, reinforcement, and gender FF(2, 24) = (2, 24) = 190.67, 190.67, pp<.001<.001. Simple interaction analyses revealed the . Simple interaction analyses revealed the personality x reinforcement interaction was significant for personality x reinforcement interaction was significant for both males, both males, FF(2, 24) = 956.72, (2, 24) = 956.72, pp<.001<.001, and females, , and females, FF(2, (2, 24) = 917.56, 24) = 917.56, pp<.001<.001. The simple simple effects of . The simple simple effects of personality were then analysed for each level of gender and personality were then analysed for each level of gender and reinforcement, and Table 1 presents the relevant means. reinforcement, and Table 1 presents the relevant means. For both genders, as predicted, under punishment anxious For both genders, as predicted, under punishment anxious participants were faster than impulsive participants, participants were faster than impulsive participants, FFs > s > 819.58, 819.58, psps<.001,<.001, while under reward impulsive participants while under reward impulsive participants were faster than anxious participants, were faster than anxious participants, FFs > 816.29, s > 816.29, psps<.001<.001. . However, in the neutral reinforcement condition the gender However, in the neutral reinforcement condition the gender difference emerged: impulsive males performed difference emerged: impulsive males performed better better than than anxious males anxious males FF(1, 24) = 270.60, (1, 24) = 270.60, pp<.001 (Ms = 355, 450)<.001 (Ms = 355, 450), , while impulsive females performed while impulsive females performed worseworse than anxious than anxious females, females, FF(2, 24) = 299.83, (2, 24) = 299.83, pp<.001<.001 (Ms = 450, 350)(Ms = 450, 350).”.”
I haven’t put effect sizes. These would be required for all tests these days.
35
6969
Table 1. Mean reaction time as a function of Table 1. Mean reaction time as a function of personality, reinforcement, and gender. personality, reinforcement, and gender.
Personality TypePersonality TypeImpulsiveImpulsive AnxiousAnxious
Reinforcement:Reinforcement:PunishmentPunishment
WomenWomen 485.33485.33aa 320.00320.00aaMenMen 490.00490.00aa 320.00320.00aa
NoneNoneWomenWomen 450.00450.00aa 350.00350.00aaMenMen 355.00355.00aa 450.00450.00aa
RewardRewardWomenWomen 320.00320.00aa 485.00485.00aaMenMen 320.00320.00aa 490.00490.00aa
Note. Subscripts within the row indicate significant simple Note. Subscripts within the row indicate significant simple simple effects of personality.simple effects of personality.
Most experimental journals would also want to see standard deviations.
7070
Education: That which discloses to the wise Education: That which discloses to the wise and disguises from the foolish their lack of and disguises from the foolish their lack of understanding. understanding. --Ambrose Bierce, writer Ambrose Bierce, writer (1842(1842--1914)1914)
36
7171
readingsreadings
between-Ps ANOVA III: higher-order designs (this lecture)Field (3rd ed): review Chapter 12Field (2nd ed): review Chapter 10Howell (all eds): Chapter 13 (pp 446-453)
power analyses and blocking designs (next lecture)Field (2nd or 3rd ed): no new readingsHowell (all eds): Chapter 8