PY 427 Statistics 1 Fall 2006

Kin Ching Kong, Ph.D

Lecture 10

Chicago School of Professional Psychology

Agenda

Analysis of Variance (continue) Review Intro. to ANOVA Hypotheses for ANOVA The Test Statistic for ANOVA: F The Logic of Analysis of Variance ANOVA Notation & Formulas

The Distribution of F-Ratios

Hypothesis Testing

Measuring Effect Size

Assumptions for the independent-measure ANOVA

Post Hoc Tests

Analysis of Variance (ANOVA) ANOVA is used to compare two or more

means

The Question: Does the mean differences observed among the

samples reflect mean differences among the populations?

Two Possibilities: There really are no differences in the

population means, the observed differences are due to chance (sampling error).

The population means are truly different, and are partly responsible for differences in the sample means.

Figure 13.1

Hypotheses for ANOVA

The experiment: learning performance under three temperature: 50o, 70o, 90o

Design: single-factor, between-subject (or independent-measures.

The Hypotheses: H0: 1 = 2 = 3 (no differences in pop. means)

H1: At least one pop. mean is different from the

others, (or not all the pop. means are equal).

There are many possible specific alternative hypotheses (e.g. all 3 means are different, first two means are identical but the third is different, the first is different and last two identical, etc.)

The Test Statistic for ANOVA

The Test Statistic for t tests: t = obtained difference between sample means

difference expected by chance (error)

The Test Statistic for ANOVA, F-ratio: F = variance (differences) between sample means

variance (differences) expected by chance (error)

The F-ratio is based on variance rather than means Problem: how to define & calculate mean differences when

there are more than two means. Solution: use variance to define and measure the size of

differences among the sample means. E.g M1 = 20, M2 = 30, M3 =35 s2 = 58.33

M1 = 28, M2 = 30, M3 = 31 s2 = 2.33

The Logic of Analysis of Variance, The Data

The experiment: learning under 3 different temperature

The design: single factor between-subjects The data: DV = # of problems solved correctly.

Temp. 50O Temp. 70O Temp. 50O

0 4 1

1 3 2

3 6 2

1 3 0

0 4 0

M = 1 M = 4 M = 1

The Logic of Analysis of Variance

The Goal of ANOVA Measure the amount of variability in a data set and

explain where it comes from.

Step I: Measure Total Variance The variability in the whole data set (scores from all

samples combined)

Step II: Partition Total Variance into two components:

Between-Treatment Variance Differences between treatment conditions

Within-Treatment Variance: Differences within each treatment condition

The Between-Treatments Variance

Between-Treatments Variance Measures how much difference exists between the

treatment conditions (i.e. differences among treatment means).

Two Sources For The Differences Between Treatments:

Treatment effects Chance (unplanned & unpredictable difference)

Individual differences Experimental error (Measurement error)

To Demonstrate There Really Is A Treatment Effect: Show that differences between treatments are bigger

than would be expected by chance alone.

The Within-Treatment Variance

Within-Treatment Variance Measures differences due to chance,

or when there is no treatment effect, H0 is true.

Figure 13.2 Analysis of Variance

The F-Ratio: Test Statistic for ANOVA

The F-Ratio compares the two component variance: F = Variance between treatments

Variance within treatments

= actual differences between treatments

differences expected with no treatment effect

F = treatment effect + differences due to chance

differences due to chance (error)

When there is no treatment effect: F = 0 + differences due to chance F is close to 1

differences due to chance When there is a treatment effect:

The numerator significantly > the denominator F significantly > 1

ANOVA Notations

X2 = 106

G = 30

N = 15

k = 3

T1 = 5 T2 = 20 T3 = 5

SS1 = 6 SS2 = 6 SS3 = 4

n1 = 5 n2 = 5 n3 = 5

M1 = 1 M2 = 4 M3 = 1

Temp. 50O Temp. 70O Temp. 50O

0 4 1

1 3 2

3 6 2

1 3 0

0 4 0

M = 1 M = 4 M = 1

ANOVA Notations (Continue)

k = number of treatment conditions (or number of levels of a factor)

ni = number of scores in treatment i (i =

1 to k) N = total number of scores in the entire

study. Ti = The total (X) for treatment

condition I G = The Grand Total = X for all the

scores, or G = T

ANOVA Formulas

ANOVA Summary Table:

Source SS df MS

Between treatments F = MSbetween

Within treatments MSwithin

Total

F = Variance between treatments

Variance within treatments

Variance, s2 = SS/df =MS

F = MSbetween

MSwithin

To fill in the ANOVA summary table, need to calculate nine values: 3 SS, 3 df, 2 MS and F

ANOVA Formulas: SS

Total Sum of Squares: SStotal = X2 – (X)2 = X2 – G2

N N

For our example: SStotal = X2 – G2 = 106 -302/15 = 106 – 60 = 46

N

Within-Treatment Sum of Squares: SSwithin = SSwithin each treatment

For our example: SSwithin = 6 + 6 + 4 = 16

Between-Treatment Sum of Squares: SSbetween = T2 - G2

n N

For our example:

SSbetween = T2 - G2 = 52 + 202 + 52 – 302 = 5 + 80 +5 -60 =30

n N 5 5 5 15

ANOVA Formulas, df

Total Degrees of Freedom: dftotal = N - 1

For our example, dftotal = 15 – 1 = 14

Within-Treatment Degrees of Freedom: dfwithin = dfin each treatment = (n-1) = N - k

For our example, dfwithin = 15 – 3 = 12

Between-Treatment Degrees of Freedom: dfbetween = k – 1

For our example, dfbetween = 3 – 1 = 2

ANOVA Formulas, MS & F-Ratio

Variance Between-Treatment, MSbetween: s2 = MSbetween = SSbetween/dfbetween

For our example, MSbetween = 30/2 = 15

Variance Within-Treatment, MSwithin: s2 = MSwithin = SSwithin/dfwithin

For our example, MSwithin = 16/12 = 1.33

The F-Ratio: F = MSbetween

MSwithin

For our example: F = 15/1.33 = 11.28

ANOVA Formulas

ANOVA Summary Table:

Source SS df MS

Between treatments 30 2 15 F = 11.28

Within treatments 16 12 1.33

Total 46 14

The Distribution of F-Ratios

Two characteristics of F values: F values are always positive because variances are

always positive. When H0 is true, the numerator and denominator of

the F-ratio estimate the same variance, thus, the ratio should be near 1. In other words, the distribution of F-ratios should pile up around 1.00

The Distribution of F-ratios: Cut off at zero (all positive values) Piles up around 1.00 Tapers off to the right. The exact shape of the F distribution depends on the

df’s in the two variances. Figure 13.6 of your book

The F Distribution Table

Table B.4 Find df of the numerator in first row. Find df of the denominator in first column The intersection of these two df’s is a pair

of numbers: The smaller number is the critical value for =

.05 The larger number is the critical value for

= .01 Table 13.3

E.g. F = 4.18 with df = 2, 15. Is this value sufficient to reject H0 with = .05? =.01?

Hypothesis Testing (the experiment)

Research Goal: Evaluate the effectiveness of three pain relievers (A,

B, C) and a placebo.

Experiment: Participants:

four groups, n = 5 in each group

Treatment (IV): Drug A, B, C and placebo

Design: Single-factor, repeated-measures

Dependent Variable (DV): Amount of time participants can withstand a painfully hot

stimulus.

Hypothesis Testing (Data)

N = 20

G = 60

X2 = 262

T = 5 T = 10 T = 20 T = 25

SS = 8 SS = 8 SS = 6 SS = 10

Placebo Drug A Drug B Drug C

3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

ANOVA Summary Table

ANOVA Summary Table:

Source SS df MS

Between treatments F =

Within treatments

Total

ANOVA Calculations

dftotal = N – 1 = 20 – 1 = 19

dfbetween = k – 1 = 4 – 1 =3

dfwithin = N – k = 16

SStotal = X2 – G2/N = 262 – 602/20 = 82

SSwithin = SSinside each treatment= 8 + 8 + 6 + 10 = 32

SSbetween = T2/n – G2/N

= 52/5 + 102/5 + 202/5 +252/5 – 602/20

= 50

MSbetween = SSbetween/ dfbetween = 50/3 = 16.67

MSwithin = SSwithin/ dfwithin = 32/16 = 2.00

F = MSbetween/ MSwithin = 16.67/2.00 = 8.33

ANOVA Formulas

ANOVA Summary Table:

Source SS df MS

Between treatments 50 3 16.67 F = 8.33

Within treatments 32 16 2.00

Total 82 19

Hypothesis Testing with ANOVA

Step 1: State the Hypotheses: H0: 1 = 2 = 3 = 4 (there is no treatment effect)

H1: At least one of the treatment means are different. The level of significant is set at = .05

Step 2: Locate the Critical Region: df = 3, 16, Fcritical = 3.24 Figure 13.7

Step 3: Calculate the test statistic: F = MSbetween / MSwithin = 8.33

Step 4: Make a decision: Since the test statistic, F = 8.33 falls in the critical region,

reject H0 and concludes that there is a significant

treatment effect.

Measuring Effect Size for ANOVA

A significant difference Means that the difference observed in the samples is very

unlikely to have occurred just by chance. Statistical significant does not necessarily mean large

effect.

Measuring effect size for ANOVA: r2: the percentage of variance accounted for by treatment

r2 = SSbetween

SStotal

In published reports, the r2 value for ANOVA is usually call 2 (the Greek letter eta squared)

For our example, 2 = 50/82 = 0.61

Assumptions

Assumptions for the Independent-Measure ANOVA:

The observations within each sample must be independent.

The populations from which the samples are selected must be normal.

The populations from which the samples are selected must have equal variances (homogeneity of variance)

Post Hoc Tests, Intro

A significant F-ratio: Indicate that a significant difference exit, that not

all the means are equal.

Does not indicate which means are different and which are not.

Example: M1 = 3, M2 = 5, M3 = 10

M2 – M1 = 2, M3 – M2 = 5, M3 – M1 = 7 A significant F indicate that at least one of these

differences are significant, M3 = M1, but what about the other two?

Post hoc tests are used to find out which of these difference are significant.

Post Hoc Tests & Type I Error Post hoc tests:

are additional hypothesis test that are done after an analysis of variance revealed a significant difference

They are performed to determine exactly which mean differences are significant and which are not.

Post hoc test and Type I Error Post hoc tests compare two means at a time, i.e. pairwise

comparisons. The process involve performing a series of separate

hypothesis tests. Each of these tests includes the risk of a Type I error With more tests, the risk of a type I error accumulates

Experimentwise alpha level: the overall probability of a Type I Error that accumulates over a series of separate hypothesis tests.

Post Hoc Tests, Planned Comparisons

Controlling Type I Error (experimentwise alpha level)

Whenever more than one test is done, need to be concerned about the experimentwise Type I Error.

Planned Comparisons Planned Comparisons: specific mean differences

are predicted by specific hypotheses before the study is conducted.

Because a few specific comparisons were planned before the data were collected, many statisticians argue that planned comparisons can be conducted with a standard alpha, without concern about inflating the risk of a Type I error.

Dunn Test: It is often recommended that researchers protect against an inflated alpha level by dividing alpha equally among the planned comparisons.

Post Hoc Tests, Tukey’s HSD Unplanned Comparisons

sifting through the data by conducting a large number of comparisons.

Tukey’s Honestly Significant Difference (HSD) Compute a single value (HSD) that determines the

minimum difference that is necessary for significance. If a pairwise difference is > Tukey’s HSD, you conclude that

there is a significant difference between the two means.

HSD = q

n = number of scores in each treatment, Tukey’s HSD test requires equal n’s

Table B.5 to find value of q. (k = number of treatment conditions, df error term = df for the denominator of the F-ratio)

n

MSwithin

Post Hoc Tests, Tukey’s HSD, Example

M1 = 3.00 M2 = 5.44 M3 = 7.00

ANOVA Summary Table

Source SS df MS

Between 73.19 2 36.60 F (2, 24) = 9.15

Within 96.00 24 4.00

Total 169.19 26

HSD = q = 3.53 = 2.36

M2 – M1 = 2.44, significant

M3 – M1 = 4.00, significant

M3 – M2 = 1.56, nonsignificant

n

MSwithin

9

00.4