1 of 47 Department of Cognitive Science Inferential Statistics II: Introduction to ANOVA Michael J....
-
Upload
laurence-marsh -
Category
Documents
-
view
212 -
download
0
Transcript of 1 of 47 Department of Cognitive Science Inferential Statistics II: Introduction to ANOVA Michael J....
1 of 47
Department of Cognitive Science
Inferential Statistics II: Introduction to ANOVA
Michael J. Kalsher
MGMT 6970 PSYCHOMETRICS © 2012, Michael Kalsher
2 of 47
ANOVA: A Framework• Understand the basic principles of ANOVA
– Why it is done?– What it tells us?
• Theory of one-way independent ANOVA
• Following up an ANOVA:– Planned Contrasts/Comparisons
• Choosing Contrasts• Coding Contrasts
– Post Hoc Tests
• Writing up Results
3 of 47
Why ANOVA?
• t tests are limited to situations in which there are only two levels of a single independent variable or two associated groups.
• There are many instances in which we’d like to compare more than two levels. But … performing multiple t tests can inflate the Type I error rate.
• The concept of “Omnibus test.”
• Familywise error: 1 - (.95)n
4 of 47
What is ANOVATests the null hypothesis that three or more means are roughly equal.
Produces a test statistic termed the F-ratio.
Systematic Variance (SSM)
Unsystematic Variance (SSR)
The F-ratio tells us only that the experimental manipulation has had an effect—not where the effect has occurred.
-- Planned comparisons (before you collect the data)
-- Post-hoc tests (after-the-fact “snooping”).
-- Both conducted to control the overall Type I error rate at 5%.
-- Guards against familywise error [1 - (.95)n ]
-- For a study with 3 conditions, Type 1 error rate = 14.3%!
5 of 47
ANOVA as RegressionSuppose we are interested in determining whether the drug Viagra is effective at making someone a better lover. A group of men are then randomly assigned to one of three groups: (1) Placebo (sugar pills); (2) Low-dose Viagra; or (3) High-dose Viagra. The dependent measure is an “objective” measure of libido. The data are below:
6 of 47
ANOVA as RegressionTo predict levels of libido from the different levels of
Viagra, we can use the general equation:
Outcomei = (model) + errori
7 of 47
Model for the Placebo group (base category)
We are predicting the level of libido when both doses of Viagra are ignored. Thus, the predicted value will be the mean of the placebo group.
The intercept, b0, is always equal to the mean of the base category
8 of 47
Model for the High-dose group
b2 represents the difference between the means of the High-dose group and the Placebo (base) group.
9 of 47
Model for the Low-dose group
b1 represents the difference between the means of the Low-dose group and the Placebo (base) group.
10 of 47
Regression Output
b1 = Low Dose
b2 = High Dose
Note:
b2 = the difference between the means of the high-dose group and the placebo group (5.0 - 2.2 = 2.8).
b1 = the difference between the means of the low-dose group and the placebo group (3.2 - 2.2 = 1)
Conclusion: Using group means to predict the outcome is significantly better than using the overall mean.
11 of 47
Theory of ANOVA: Partitioning Variance
• We compare the amount of variability explained by the Model (MSM), to the error in the model [individual differences] (MSR)
– This ratio is called the F-ratio.
• If the model explains a lot more variability than it can’t explain, then the experimental manipulation has had a significant effect on the outcome (DV).
12 of 47
Partitioning the Variance
Total sum of squares (SST)
Model sum of squares (SSM)
Residual sum of squares (SSR)
13 of 47
SST = based on
differences between each data point and
the grand mean
SSM = differences
between predicted values (group means) and the
grand mean
Grand Mean = 3.467
XPlacebo = 2.20
XLowDose = 3.20
XHighDose = 5.00
SSR = based on
differences between person’s score and their group mean.
Basis for SST
Basis for SSR
Basis for SSM
Par
titi
on
ing
th
e V
aria
nce
Base Category
14 of 47
Total sum of squares (SST)Compute the difference between each data point and the grand mean, square these differences, then add them together.
1
2
3
4
Grand Mean = 3.467
Grand SD = 1.767
Grand Variance = 3.124
15 of 47
Model sum of squares (SSM)1. Calculate the difference between the mean of each group and the
grand mean.
2. Square each of these differences
3. Multiply each result by the number of participants within that group.
4. Add the values for each group together.
16 of 47
Residual sum of squares (SSR)
SSR = SST - SSM … But, don’t rely on this relationship (e.g., might not work out if either SST or SSM is miscalculated)
SSR is the difference between what the model predicts and what was actually observed. Therefore it is calculated by by looking at the difference between the score obtained by a person and the mean of the group to which the person belongs.
17 of 47
Total sum of squares (SSR)
18 of 47
Mean Squares (MSM and MSR)
SSM = amount of variation explained by the model (exp. manipulation).
SSR = amount of variation due to extraneous factors.
These are “summed” scores and will therefore be influenced by the number of scores. To eliminate this bias we calculate the average sum of squares (mean squares) by dividing by the appropriate degrees of freedom.
Calculating Degrees of Freedom (for one-way independent groups ANOVA)
dftotal = N - 1 (number of all scores minus 1)
dfM / between = k - 1 (number of groups minus 1)
df R / within = N - k (number of all scores minus number of groups)
19 of 47
The F-ratio
F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).
Therefore, we again return to the basic conceptual model:
Test Statistic = systematic variance unsystematic variance
20 of 47
Assumptions
The one-way independent groups ANOVA test requires the following statistical assumptions:
1. Random and independent sampling.
2. Data are from normally distributed populations. Note: This test is robust against violation of this assumption if n > 30 for both groups.
3. Variances in these populations are roughly equal (Homogeneity of variance). Note: This test is robust against violation of this assumption if all group sizes are equal.
21 of 47
1-Way Independent-Groups ANOVA
Characteristics:• Used when testing 3 or more experimental groups.• Each participant contributes only one score to the data.• Used to test the (null) hypothesis that several means are equal.
Example: Testing the idea that exposure to mobile phones will lead to brain tumors.
IV: Hrs./day exposure
DV: Size of brain tumor
Mobile Phone Use
(hours per day)
22 of 47
-- Calculate the Model (or between-groups) sum of squares (SSM) and the Residual (or within-groups) sum of squares (SSR).
-- The number of scores used to calculate SSM and SSR are different, so to make them comparable, we convert them to the average sums of square or “mean squares” (MS) by dividing by their appropriate degrees of freedom.
-- F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).
Steps in the Analysis
23 of 47
Degrees of Freedom (df): One-way Independent-Groups ANOVA
Formulas for the number of degrees of freedom in one-way independent-groups ANOVA differ for each SS term. The degrees of freedom (df) reflect the number of values free to vary in any sum of squares term.
Calculating Degrees of Freedom
dftotal = N - 1 (number of all scores minus 1)
dfbetween = k - 1 (number of groups minus 1)
df within = N - k (number of all scores minus number of groups)
24 of 47
SPSS Summary Tables
N Mean
Std.
Deviation
Std. Error
Lower
Bound
Upper
Bound Min. Max.
0
1
2
3
4
5
Total
20
20
20
20
20
20
120
.0175
.5149
1.2614
3.0216
4.8878
4.7306
2.4056
.01213
.28419
.49218
.76556
.69625
.78163
2.02662
.00271
.06355
.11005
.17118
.15569
.17478
.18500
.0119
.3819
1.0310
2.6633
4.5619
4.3648
2.0393
.0232
.6479
1.4917
3.3799
5.2137
5.0964
2.7720
.00
.00
.48
1.77
3.04
2.70
.00
.04
.94
2.34
4.31
6.05
6.14
6.14
Sum of
Squares df
Mean
Square F Sig.
Between Groups
Within Groups
Total
450.664
38.094
488.758
5
114
119
90.133
.334
269.733 .000
95% Confidence IntervalHrs. of Exposure
Size of Tumor
25 of 47
Levene’s Test: Testing the Homogeneity of Variance Assumption
If Levene’s test is significant (p<.05), then the variances are significantly different.
What to do if Levene’s test is significant?1. Transform the data
2. Report the F value and the Levene’s test results and allow your audience to assess the accuracy of your results for themselves.
Levene
Statistic df1 df2 Sig.
10.245 5 114 .000
26 of 47
Computing Effect Size (2 - eta squared and 2 - omega squared)Analogous to r2, eta-squared reflects the proportion of DV variance explained by each IV in the sample data. It tends to be upwardly biased (i.e., an overestimate).
2 = SSModel / SStotal
• One 2 for each effect (i.e., 1 per IV plus 1 per interaction) and the 2s sum to 1.• SPSS reports partial eta squared (p
2 = SSEffect / SSEffect+ SSError)
• In 1-way ANOVA, eta-squared and partial eta-squared are the same.
An alternative, omega-squared (2), estimates the proportion of DV variance explained by the IVs in the population (see Field, 2005, p. 357-359).
27 of 47
2 = SSM = 450.66 = .922
SST 488.76
2 = SSM - (dfM)(MSR) = 450.664 - (5 x .334) SST + MSR 488.758 + .334
= 449.01 = .918 489.09
Computing Effect Size (2 vs. 2 omega squared)
(called eta squared, 2 )
(called omega squared, 2 )
28 of 47
Reporting the Results
Levene’s test indicated that the assumption of homogeneity of variance had been violated, F(5, 114) = 10.25, p<.001. Transforming the data did not rectify this problem and so F tests are reported nonetheless. The results show that using a mobile phone significantly affected the size of brain tumor found in participants, F(5, 114) = 269.73, p<.001, 2 = .92. Tumor size (measured in mm) increased with the duration of exposure. Specifically, tumor size for 0, 1, 2, 3, 4 or 5 hours/day of exposure was .02 mm, .51 mm, 1.26 mm, 3.02 mm, 4.89 mm, and 4.73 mm, respectively. The effect size indicated that the effect of phone use on tumor size was substantial. Games-Howell post hoc tests revealed significant differences between all groups (p<.001 for all tests) except between 4- and 5-hours (ns).
29 of 47
Computing a One-way Independent-groups
ANOVA by hand
30 of 47
31 of 47
32 of 47
Critical Values for F
33 of 47
2 = SSM = 93.17 = .69
SST 134.67
2 = SSM - (dfM x MSR) = 93.17 - (2 x 4.61) MSR + SST 4.61 + 134.67
= 83.95 = .60 139.28
Computing Effect Size (2 vs 2 Omega squared)
34 of 47
Computing the One-way Independent-groups ANOVA using SPSS
35 of 47
SPSS Variable View: Assign Value Labels
36 of 47
SPSS Data View: Type in the Data
37 of 47
SPSS Data Editor: Select Procedure
38 of 47
38
Specify IV and DV, then select “options”
Select “Descriptives” and “Homogeneity test”. If the variances are unequal, choose “Brown-Forsythe” or “Welch” tests.
39 of 47
39
Choosing Follow-up tests: Contrasts vs. Post-Hoc
Select “Contrasts” if you have apriori hypotheses.
Select “Post-Hoc” if you do not.
40 of 47
40
Planned Contrasts (if we have specific hypotheses)
The first contrast compares the combined “experimental drug groups” against the control (Saltwater) group.
The second contrast compares the the two experimental groups, Fenfluramine vs. Amphetamine.
41 of 47
41
Post-Hoc Tests (if we do NOT have specific hypotheses)
LSD (Least Significant Difference) does not control Type I error rate. Bonferroni tightly controls Type I error rate. Other post-hoc procedures have specific applications (see Field text).
42 of 47
SPSS ANOVA Output: Descriptives and ANOVA Summary Table
43 of 47
SPSS ANOVA Output: Planned Contrasts
Levene’s test was not significant (p>.05) and so we can assume equal variances.In the Contrast Tests summary, use the “Assume equal variances” row values.
Contrast 1: weight gain among rats receiving the experimental drugs is significantly different from rats receiving saltwater.
Contrast 2: weight gain among rats taking fenfluramine and amphetamine are not significantly different.
44 of 47
44
Post-hoc testsAfter the Fact “Snooping”: Some Considerations
LSD does not control Type I error rate, whereas the Bonferroni test tightly controls the Type I error rate.
Given equal sample sizes and equal variances, use REGWQ or Tukey HSD, as both have good power and tight control over Type I error rate.
If sample sizes across groups are slightly different, use Gabriel’s procedure because it has greater power, but if sample sizes are very different use Hochberg’s GT2.
If there is any doubt that the population variances are equal, use the Games-Howell procedure, in place of, or in addition to other tests.
45 of 47
Post-Hoc tests: LSD vs Bonferroni
46 of 47
Reporting the ResultsThe results show that the amount of weight gained by the rats was significantly affected by the type of drug they received, F(2,9) = 10.10, p < .01, 2 = .60. Specifically, weight gain was greatest for rats receiving Saltwater (M=10.0, SE=.82) and least for rats receiving Fenfluramine (M=3.25, SE=1.11). Weight gain for rats receiving Amphetamine was intermediate (M=5.75, SE=1.25).
If you used planned comparisons …Planned comparisons revealed a significant difference in weight gain between the combined experimental groups and the saltwater control group, t(9)=-4.18, p<.05 (1-tailed). The difference in weight gain between the two experimental groups receiving fenfluramine and amphetamine, respectively, was not significant, t(9)= -1.65, p>.05 (1-tailed).
If you used post-hoc tests …Bonferroni post hoc tests revealed a significant difference in weight gain between the Fenfluramine and Saltwater groups (p<.05). No other comparisons were significant (ps>.05).
47 of 47
Sample Problem: Your turn…