Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.

Analysis of Variance (ANOVA)

Quantitative Methods in HPELS

HPELS 6210

Agenda

Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions

Introduction Recall There are two possible

scenarios when obtaining two sets of data for comparison: Independent samples: The data in the first

sample is completely INDEPENDENT from the data in the second sample.

Dependent/Related samples: The two sets of data are DEPENDENT on one another. There is a relationship between the two sets of data.

Introduction Three or more data sets?

If the three or more sets of data are independent of one another Analysis of Variance (ANOVA)

If the three or more sets of data are dependent on one another Repeated Measures ANOVA

Introduction: Terminology

Factor: Synonym of independent variable Level: The treatment conditions that make

up the factor or independent variable Example: What is the effect of grade (1st,

2nd, 3rd) on IQ?Dependent variable: IQFactor: GradeLevels (3): 1st, 2nd and 3rd grades

Introduction: Terminology Between-Treatment Variance: Variance

between the treatments/levels As the between-treatment variance

increases: The statistic increases The p-value decreases

Greater chance of rejecting the H0

Introduction: Terminology Within-Treatment Variance: Variance

within the treatments/levels As the within-treatment variance

increases: The statistic decreases The p-value increases

Lesser chance of rejecting the H0

Recall the Independent-Measures t-Test

If there was a large difference between the means (between variance) t got bigger

Why? t = M1-M2 / s(M1-M2)

The t formula can be thought of as a ratio of: Between variance (M1-M2)

Within variance (s(M1-M2))

Several Scenarios can occur

-Small between variance

-Large within variance

-t = BV / WV = near zero value

Accept or reject the H0

-Large between variance

-Large within variance

-t = BV / WV = near value of 1.0


-Small between variance

-Small within variance

-t = BV / WV = near value of 1.0


-Large between variance

-Small within variance

-t = BV / WV = greater than 1.0


Introduction The F-Ratio

ANOVA is a ratio of between variance and within variance

Distinction: Three or more groups

The F Distribution Plot all possible F-ratios F distribution There is a family of F distributions As df increases, the distribution becomes more

narrow F-ratios are always positive in value

Computed with two variances Variances are always positive!

F distribution is skewed Most values cluster around 1.0 Figure 13.8 (p 413)

Agenda


ANOVA Statistical Notation:

k = number of treatment conditions (levels)nx = number of samples per treatment level

N = total number of samples N = kn if sample sizes are equal

Tx = X for any given treatment level

G = TMS = mean square = variance

ANOVA

Formula Considerations:SSbetween = T2/n – G2/N

SSwithin = SSinside each treatment

SStotal = SSwithin + SSbetween

SStotal = X2 – G2/N

ANOVA Formula Considerations:

dftotal = N – 1

dfbetween = k – 1

dfwithin = (n – 1) dfwithin = dfin each treatment

ANOVA Formula Considerations:

MSbetween = s2between = SSbetween / dfbetween

MSwithin = s2within = SSwithin / dfwithin

F = MSbetween / MSwithin

Independent-Measures Designs

Static-Group Comparison Design: Administer treatment to two or more groups

and perform posttest Perform posttest to control group Compare groups

X1 O

X2 O

O

Independent-Measures Designs Quasi-Experimental Pretest Posttest

Control Group Design: Perform pretest on three or more groups Administer treatments to treatment groups Perform posttests on all groups Compare delta (Δ) scores

O X1 O Δ

O X2 O Δ

O O Δ

Independent-Measures Designs Randomized Pretest Posttest Control Group

Design: Randomly select subjects from three or more

populations Perform pretest on all groups Administer treatments to treatment groups Perform posttests on all groups Compare delta (Δ) scores

R O X1 O Δ

R O X2 O Δ

R O O Δ

Agenda


Hypothesis Test: ANOVA Example 13.1 (p 415) Overview:

Researchers are interested in the effectiveness different pain relievers (A, B and C) compared placebo (D)

N = 20 randomly assigned to the four treatments (n = 5)

Amount of time (s) each subject could withstand a painfully hot stimulus was measured

Hypothesis Test: ANOVA

Questions:What is the experimental design?What is the independent variable/factor? How many levels are there?What is the dependent variable?

Step 1: State Hypotheses

Non-Directional

H0: µA = µB = µC = µD

H1: At least one mean is different than the others

Directional?

Too many too list

Step 2: Set Criteria

Alpha () = 0.05

Critical Value:

Use F Distribution Table

Appendix B.4 (p 693)

Information Needed:

dfbetween = k – 1

dfwithin = (n – 1)

Step 3: Collect Data and Calculate Statistic

Total Sum of Squares

SStotal = X2 – G2/N

SStotal = 262 – 602/20

SStotal = 262 - 180

SStotal = 82

Sum of Squares Between

SSbetween = T2/n – G2/N

SSbetween = 52/5+102/5+202/5+252/5 – 602/20

SSbetween = (5+20+80+125) - 180

SSbetween = 50

Sum of Squares Within

SSwithin = SSinside each treatment

SSwithin = 8+8+6+10

SSwithin = 32

Step 3: Collect Data and Calculate Statistic

Mean Square Between

MSbetween = SSbetween / dfbetween

MSbetween = 50 / 3

MSbetween = 16.67

Mean Square Within

MSwithin = SSwithin / dfwithin

MSwithin = 32/16

MSwithin = 2

F-Ratio


F = 16.67 / 2

F = 8.33

Step 4: Make Decision

Agenda


Post Hoc Analysis What ANOVA tells us:

Rejection of the H0 tells you that there is a

high PROBABILITY that AT LEAST ONE difference exists SOMEWHERE

What ANOVA doesn’t tell us:Where the differences lie

Post hoc analysis is needed to determine which mean(s) is(are) different

Post Hoc Analysis

Post Hoc Tests: Additional hypothesis tests performed after a significant ANOVA test to determine where the differences lie.

Post hoc analysis IS NOT PERFORMED unless the initial ANOVA H0 was rejected!

Post Hoc Analysis Type I Error Type I error: Rejection of a true H0

Pairwise comparisons: Multiple post hoc tests comparing the means of all “pairwise combinations”

Problem: Each post hoc hypothesis test has chance of type I error

As multiple tests are performed, the chance of type I error accumulates

Experimentwise alpha level: Overall probability of type I error that accumulates over a series of pairwise post hoc hypothesis tests

How is this accumulation of type I error controlled?

Two Methods Bonferonni or Dunn’s Method:

Perform multiple t-tests of desired comparisons or contrasts

Make decision relative to / # of testsThis reduction of alpha will control for the

inflation of type I error Specific post hoc tests:

Note: There are many different post hoc tests that can be used

Our book only covers two (Tukey and Scheffe)

Tukey’s Honestly Significant Difference (HSD) Test Overview:

Computes a single value that determines the minimum difference (HSD) between any two means necessary for rejection of the H0

Compares the HSD value to all of the contrast results

If the contrast result exceeds the HSD, the H0

of that particular contrast is rejected

Tukey’s HSD Calculation

Formulas: Equal sample sizes

HSD = q√MSwithin / n

Unequal sample sizesHSD = q√(MSwithin/2)(1/n1+1/n2)

Tukey’s HSD Calculations

Formula Considerations: q = value found in Table B.5 (p 696)

Left column: dfwithin

Top row: k treatments Body:

Regular font: = 0.05 Bold font: = 0.01

MSwithin = value from ANOVA calculation n = number of subjects in each treatment

Example 13.5 (p 427)


Null

H0: µA = µB

H0: µA = µC

H0: µB = µC

Alternative

H1: µA µB

H1: µA µC

H1: µB µC


Alpha () = 0.05

Step 3: Calculate Statistic

Get q from Table B.5

Information needed:

dfwithin = 24

k = 3

= 0.05

q = 3.53

Calculate Tukey’s HSD Value

HSD = qMSwithin / n

HSD = 3.53 4 / 9

HSD = 2.36

Step 4: Make Decision:

A significantly greater than B MA – MB = 2.44 > 2.36

A significantly greater than C MA – MC = 4.00 > 2.36

B not significantly different than C MB – MC = 1.56 < 2.36

Table 13.6

Scheffe Overview:

Most conservative/cautious of all post hoc tests Uses an F-ratio (like ANOVA) on only two treatments

Controls for type I error: Uses k value from the original ANOVA thus the numerator

of the F-ratio for the Scheffe test is k – 1 Uses same critical value used for the ANOVA

Calculation of Scheffe is identical to the ANOVA however: SSbetween uses the two means of interest

Example 13.6 (p 428)


Null

H0: µA = µB

H0: µA = µC

H0: µB = µC

Alternative

H1: µA µB

H1: µA µC

H1: µB µC


Alpha () = 0.05

Step 3: Calculate Statistic

Sum of squares between:

SSbetween = T2/n – G2/N

SSbetween = (272/9 + 492/9) – 762/18

SSbetween = (81+266.78) – 320.89

SSbetween = 26.89

SSwithin from original ANOVA = 96

Critical Value 3.40

dfbetween = 2

dfwithin = 24

= 0.05

Mean square between and within

MSbetween = SSbetween/dfbetween

MSbetween = 26.89 / 2 = 13.45

MSwithin from original ANOVA = 4


F = 13.45 / 4

F = 3.36


F = 13.45 / 4

F = 3.36

Step 4: Make Decision

F = 3.36 < 3.40 (critical value)

Accept or reject?

Repeat for the other two contrasts:

H0: µA = µC

H0: µB = µC

df = 2, 24

3.40

Agenda


Instat Type dependent variable data from the three or more

samples into one column: Label column appropriately

In a second column, type in the grouping variable (independent variable) next to each data point:

Label column appropriately Convert the grouping column into a “factor” column:

Highlight the grouping column. Choose “Manage” Choose “Column Properties” Choose “Factor” Select the appropriate column to be converted Indicate the number of levels in the factor Click OK

Instat Choose “Statistics”

Choose “Analysis of Variance” Choose “One-Way” Y-Variate:

Choose the dependent variable Factor:

Choose the factor column or grouping/independent variable Plots:

Not necessary to choose any Click OK. Interpret the p-value!!!

Post Hoc Analysis: Perform multiple Independent-Measures t-Tests with the

Bonferonni/Dunn correction method

Reporting ANOVA Results Information to include:

Value of the F statistic Degrees of freedom:

Between: k – 1 Within: (n – 1)

p-value Examples:

A significant treatment effect was observed (F(2, 24) = 8.33, p = 0.02)

Reporting ANOVA Results An ANOVA summary table is often

included

Source SS df MS

Between 50 3 16.67 F = 8.33

Within 32 16 2

Total 82 19

Agenda


Assumptions of ANOVA Independent Observations Normal Distribution Scale of Measurement

Interval or ratio Equal variances (homogeneity)

Violation of Assumptions Nonparametric Version Kruskall-Wallis

Test (Chapter 17) When to use the Kruskall-Wallis Test:

Independent-Measures design with three or more groups

Scale of measurement assumption violation: Ordinal data

Normality assumption violation: Regardless of scale of measurement

Textbook Assignment

Problems: 3, 5, 17a, 21

Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.

Documents

Transcript of Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.