Post hoc tests

Placebo

Lucozade

Gatorade

Powerade

Post hoc tests (example)….

Random Assignment My Sample

Time to

Fatigue (min)

Dependent Variable

Independent Variable

ANOVA Hypotheses

The Null hypothesis for ANOVA is that the means for all groups are equal:

The Alternative hypothesis for ANOVA is that at least two of the means are not equal.

koH µµµµ ==== ....: 321

ANOVA

TimetoFatigue

3434.475 3 1144.825 6.364 .0016476.500 36 179.9039910.975 39

Between GroupsWithin GroupsTotal

Sum ofSquares df Mean Square F Sig.

Tim

e to

Fat

igue

(min

)

0

20

40

60

80

100

120 PlaceboLGIHGIGlucose

Thomas et al. (1991)

*

*P <0.05 vs. Placebo, HGI & Glucose

Placebo Lucozade Gatorade Powerade

Placebo Lucozade Gatorade Powerade

Why not multiple Anovas? i.e. Placebo vs Lucozade Placebo vs Gatorade Placebo vs Powerade Lucozade vs Gatorade Lucozade vs Powerade Gatorade vs Powerade

We accept ‘significance’ and reject the null hypothesis at P≤0.05 (i.e. a 5% chance that we are wrong)

Performing multiple tests therefore means that our overall chance of committing a type I error is >5%.

Inflated Type I Error and Pairwise Comparisons

# Tests Comparison α Familywise α

1 .05 .05

3 .05 .14

6 .05 .26

10 .05 .40

15 .05 .54

Determining Error Rates

Two Important Error Rates (Alpha): Per-Comparison: Probability of Type I Error for

each specific comparison Family-Wise: Overall Type I Error probability

Type I Error Rate for Multiple Tests:

where c = # of comparisons

cPCFW ) - (1 -1 αα =

Post Hoc Tests Bonferroni procedure Duncan Multiple range test Dunnett’s multiple comparison test Newman-Keuls test Scheffe’s test Tukey’s HSD (honestly significant difference) test Fisher’s LSD (Least Significant Difference)

Holm t-test Sidak

Tim

e to

Fat

igue

(min

)

0

20

40

60

80

100

120 PlaceboLGIHGIGlucose

Thomas et al. (1991)

*

*P <0.05 vs. Placebo, HGI & Glucose

Placebo Lucozade Gatorade Powerade

Placebo Lucozade Gatorade Powerade

A

Placebo Lucozade Gatorade Powerade

Placebo Lucozade Gatorade Powerade

B B

A, B

Adding post hoc test to ANOVA… SAS

proc anova; class VCAT; Model VCONT=VCAT; means VCAT /duncan; run;

ChiSquare …

White Black NAmer Asian Hispanic Nicotine

dependence

Bonferroni post hoc test … A conservative adjustment (Bonferroni adjustment) is to

divide 0.05/c (number of comparisons).

3 comparisons is 0.05/3, so that alpha for each test = 0.017.

ChiSquare …

White Black NAmer Asian Hispanic Nicotine

dependence

.05/10, so that alpha for each test = 0.005.

Subset data for paired comparisons… SAS

IF ETHRACE2A eq 1 or ETHRACE2A eq 3;

Rate of nicotine dependence by race/ethnicity….

0

5

10

15

20

25

30

35

40

White Black NativeAmerican

Asian Hispanic

% Nicotine

Dependent Lifetime

White Black Nicotine

dependence

White NAmer

Nicotine dependence

Rate of nicotine dependence by race/ethnicity….

0

5

10

15

20

25

30

35

40

White Black NativeAmerican

Asian Hispanic

% Nicotine

Dependent Lifetime

A,B

B

C

C

C