Expected Utility, Mean-Variance and Risk Aversion Lecture VII.
Section VII Comparing means & analysis of variance.
-
Upload
desiree-derbyshire -
Category
Documents
-
view
224 -
download
3
Transcript of Section VII Comparing means & analysis of variance.
Section VIIComparing means
& analysis of variance
How to display means-ok in simple situations
A B C D0
20
40
60
80
100
120
140
160
M
F
Presenting means - ANOVA data mean serum glucose (mg/dl) by drug and gender
0
20
40
60
80
100
120
140
160
A B C D
Drug
me
an
se
ru
m g
luc
os
e (
mg
/dl)
Males
Females
One can also add “error bars” to these means. In analysis of variance, these error bars are based on the sample size and the pooled standard deviation, SDe. This SDe is the same residual SDe as in regression.
4
Don’t use bar graphs in complex situations
5
Use line graph
Comparing MeansTwo groups – t test (review)
Mean differences are “statistically significant” (different beyond chance) relative to their standard error (SEd), a measure of mean variability noise”).
___ ____
t = (Y1 - Y2)= “signal” SEd “noise” _
Yi = mean of group i, SEd =standard error of mean difference
t is mean difference in SEd units. As |t| increases, p value gets smaller. Rule of thumb: p < 0.05 when |t| > 2
____ ___
| Y1 - Y2| > tcr SEd = 2 SEd =LSD
tcr SEd = 2 SEd is the critical or least significant difference (LSD). So, getting the correct SEd is crucial!!
SEd is the “yardstick” for significance
How to compute SEd?SEd depends on n, SD and study design. (example: factorial or repeated measures) For a single mean, if n=sample size _ _____ SEM = SD/n = SD2/n __ __ For a mean difference (Y1 - Y2) The SE of the mean difference, SEd is given by _________________ SEd = [ SD1
2/n1 + SD22/n2 ] or
________________ SEd = [SEM1
2 + SEM22]
If data is paired (before-after), first compute differences (di=Y2i-Y1i) for each person. For paired: SEd =SD(di)/√n
3 or more groups-analysis of variance (ANOVA) Pooled SDs
What if we have many treatment groups, each with its own mean and SD?
Group Mean SD sample size (n)
__
A Y1 SD1 n1
B Y2 SD2 n2
C Y3 SD3 n3
… __
k Yk SDk nk
Check variance homogeneity
mean vs SD
2
2.4
2.8
3.2
3.6
0 5 10 15 20 25 30
mean
SD
The Pooled SDe
SD2pooled error = SD2
e =
(n1-1) SD12 + (n2-1) SD2
2 + … (nk-1) SDk2
(n1-1) + (n2-1) + … (nk-1)
____
so, SDe = = SD2e
In ANOVA - we use pooled SDe to compute SEd and to compute “post hoc” (post pooling) t statistics and p values.
____________________
SEd = [ SD12/n1 + SD2
2/n2 ] ____________
= SDe (1/n1) + (1/n2)
SD1 and SD2 are replaced by SDp a “common yardstick”.
If n1=n2=n, then SEd = SDe2/n=constant
Transformations There are two requirements for the analysis of
variance (ANOVA) model.
1. Within any treatment group, the mean should be the middle value. That is, the mean should be about the same as the median. When this is true, the data can usually be reasonably modeled by a Gaussian (“normal”) distribution.
2. The SDs should be similar (variance homogeneity) from group to group.
Can plot mean vs median & residual errors to check #1 and mean versus SD to check #2.
What if its not true? Two options:
a. Find a transformed scale where it is true.
b. Don’t use the usual ANOVA model (use non constant variance ANOVA models or non parametric models).
Option “a” is better if possible - more power.
Most common transform is log transformationUsually works for: 1. Radioactive count data 2. Titration data (titers), serial dilution data 3. Cell, bacterial, viral growth, CFUs 4. Steroids & hormones (E2, Testos, …) 5. Power data (decibels, earthquakes) 6. Acidity data (pH), … 7. Cytokines, Liver enzymes (Bilirubin…) In general, log transform works when a
multiplicative phenomena is transformed to an additive phenomena.
Compute stats on the log scale & back transform results to original scale for final report. Since log(A)–log(B) =log(A/B), differences on the log scale correspond to ratios on the original scale. Remember
10 mean(log data) =geometric mean < arithmetic mean
monotone transformation ladder- try these
Y2, Y1.5, Y1, Y0.5=√Y,
Y0=log(Y),
Y-0.5=1/√Y, Y-1=1/Y,Y-1.5, Y-2
Multiplicity & F testsMultiple testing can create “false positives”. We
incorrectly declare means are “significantly” different as an artifact of doing many tests even if none of the means are truly different.
Imagine we have k=four groups: A, B, C and D. There are six possible mean comparisons: A vs B A vs C A vs D B vs C B vs D C vs D
If we use p < 0.05 as our “significance” criterion, we have a 5% chance of a “false positive” mistake for any one of the six comparisons, assuming that none of the groups are really different from each other. We have a 95% chance of no false positives if none of the groups are really different. So, the chance of a “false positive” in any of the six comparisons is
1 – (0.95)6 = 0.26 or 26%.
To guard against this we first compute the “overall” F statistic and its p value.
The overall F statistic compares all the group means to the overall mean (M).
__
F = ni( Yi – M)2/(k-1) =MSx = between group var
(SDp)2 MSerror within group var __ __ __
=[n1(Y1 – M)2 + n2(Y2-M)2 + …nk(Yk-M)2]/(k-1)
(SDp)2
If “overall” p > 0.05, we stop. Only if the overall p < 0.05 will the pairwise post hoc (post overall) t tests and p values have no more than an overall 5% chance of a “false positive”.
This criterion was suggested by RA Fisher and is called the Fisher LSD (least significant difference) criterion. It is less conservative (has fewer false negatives) than the very conservative Bonferroni criterion. Bonferroni criterion: if making “m” comparisons, declare significant only if p < 0.05/m.
This overall F is the same as the overall F test in regression for testing β1=β2=β3=…βk=0 (all regression coeffs=0).
Ex:Clond-time to fall off rod
0
10
20
30
40
50
60
70
time_
sec
KO-no TBI KO-TBI WT-noTBI WT-TBI
group
All Pairs
Tukey-Kramer
0.05
One way analysis of variancetime to fall data, k= 4 groups, df= k-1
R square 0.5798
Adj R square 0.5530
Root Mean Square Error=SDe 10.99Mean of Response 30.20
Observations (or Sum Wgts) 51
Source DF Sum of Squares Mean Square F Ratio Prob > F
group 3 7827.438 2609.15 21.618 <.0001*
Error 47 5672.546 120.69
C. Total 50 13499.984
Means & SDs in sec (JMP)
Level Number Mean median SD SEM
KO-no TBI 8 21.196 21.65 6.4598 2.2839
KO-TBI 7 18.659 18.47 8.7316 3.3002
WT-noTBI 15 49.197 46.93 9.9232 2.5622
WT-TBI 21 23.902 23.33 13.3124 2.9050
No model
ANOVA model, pooled SDe=10.986 sec
Level NumberMean
SEM
KO-no TBI 821.196
3.8841
KO-TBI 718.659
4.1523
WT-noTBI 1549.197
2.8366
WT-TBI 2123.902
2.3973Why are SEMs not the same??
Mean comparisons- post hoc tLevel Mean
WT-noTBI A 49.197
WT-TBI B 23.902
KO-no TBI B 21.196
KO-TBI B 18.659
Means not connected by the same letter are significantly different
Multiple comparisons-Tukey’s qAs an alternative to Fisher LSD, for pairwise
comparisons of “k” means, Tukey computed percentiles for
q=(largest mean-smallest mean)/SEd
under the null hyp that all means are equal.
If mean diff > q SEd is the significance criterion, type I error is ≤ α for all comparisons.
q > t > Z
One looks up”t” on the q table instead of the t table.
t vs q for α=0.05, large n
num means=k t q*
2 1.96 1.96
3 1.96 2.34
4 1.96 2.59
5 1.96 2.73
6 1.96 2.85
* Some tables give q for SE, not SEd, so must multiply q by √2.
Post hoc: t vs Tukey q, k=4Level - Level Mean
DiffSE diff t p-Value- no
correctionp-Value-Tukey
WT-noTBI KO-TBI 30.54 5.03 6.073 <.0001* <.0001*
WT-noTBI KO-no TBI 28.00 4.81 5.822 <.0001* <.0001*
WT-noTBI WT-TBI 25.30 3.71 6.811 <.0001* <.0001*
WT-TBI KO-TBI 5.24 4.79 1.094 0.2797 0.6952
WT-TBI KO-no TBI 2.71 4.56 0.593 0.5562 0.9338
KO-no TBI KO-TBI 2.54 5.69 0.446 0.6574 0.9700
Mean comparisons-TukeyLevel Mean
WT-noTBI A 49.197
WT-TBI B 23.902
KO-no TBI B 21.196
KO-TBI B 18.659
Means not connected by the same letter are significantly different
One way analysis of variance
comparing means across groups-ANOVA vs regr
Example: Comparing mean birth weight by race.
ANOVA via regression Coding categorical variables – dummy vs
effect coding
Below, we create two new variables, “af_am” and “other” from the variable “Race”.
Dummy coding - “white” is the referent category
Race Af_am other
White-1 0 0
Black-2 1 0
Other-3 0 1
Dummy (0,1) coded variables are usually correlated with each other even in balanced designs – not orthogonal. However, they are easier to interpret.
Effect coding, ‘white” is the referent category
Race Af_am other
White-1 -1 -1
Black-2 1 0
Other-3 0 1
In balanced designs, effect coded (-1, 0, 1) variables have zero correlation = they are orthogonal. In balanced designs, effect coded variables have sum and mean zero and cross products of zero. Under effect coding, cell means correspond to Xi = -1 or 1 and marginal means correspond to Xi=0.
ANOVA VIA REGRESSION (dummy vars)
Birth weight Overall Analysis of Variance table Sum of MeanSource DF Squares Square F Value p valueModel 2 5070608 2535304 4.97 0.0079Error 186 94846445 509927Total 188 99917053
Root MSE=SDe = 714.09 gm R-Square 0.0507Dependent Mean 2944.656
ANOVA via regression - Dummy coding
Variable df regr coef SE t p value
Intercept 1 3103.74 72.88 42.59 <.0001
af_am 1 -384.05 157.87 -2.43 0.0159
other 1 -299.72 113.68 -2.64 0.0091
Birth wt = 3104 – 384 af_am – 300 other+error
With dummy coding, regression coefficients are the mean difference from the referent group (white in this example)
ANOVA via regression (cont) effect coding for race
Overall Analysis of Variance table Sum of MeanSource DF Squares Square F Value p valueModel 2 5070608 2535304 4.97 0.0079Error 186 94846445 509927Total 188 99917053
Root MSE 714.09 R-Square 0.0507Dependent Mean 2944.656
ANOVA via regression - effect coding
Variable df regr coef SE t p value
Intercept 1 2875.82 60.13 47.83 <.0001
af_am 1 -156.12 100.76 -1.55 0.1230
other 1 -71.80 78.43 -0.92 0.3612
Birth wt= 2876- 156 af_am – 72 other + error
With effect coding, the 2875 is the mean of the race means, the unweighted overall mean. The regression coeffs are the deviations from this overall mean for each factor.
Multiway ANOVA
Balanced designs - ANOVA example
Brain Weight data, n=7 x 4 = 28, nc=7 obs/cell Dementia Sex Brain Weight (gm)
No F 1223No F 1228No F 1222
No F 1204No F 1234No F 1211No F 1217… … …
Mean brain weights (gms) in Males and Females with and without dementia
Cell mean
A balanced* 2 x 2 (ANOVA) design, nc= 7 obs per cell, n=7 x 4 = 28 obs
totalMeans
DementiaMales (1) Female (-1) Total
Yes (1) 1321.14 1201.71 1261.43
No (-1) 1333.43 1219.86 1276.64
1327.29 1210.79 1269.04
Males Females Overall
Dementia Cell Cell Margin
No dementia Cell Cell Margin
Overall Margin Margin
Terminology – cell means, marginal means
Difference in marginal sex means (Male – Female) 1327.29 - 1210.79 = 116.50, 116.50/2 = 58.25Difference in marginal dementia means (Yes – No)
1261.43 - 1276.64 = -15.21, -15.21/2 = -7.61
Difference in cell mean differences (1321.14 - 1333.43) – (1201.71 - 1219.86) = 5.86 (1321.14 - 1201.71) – (1333.43 - 1219.86) = 5.86
note: 5.86/(2x2) = 1.46 * balanced = same sample size (nc) in every cell
Brain weight via ANOVA - Effect coding (-1,1) MODEL: brain wt = sex dementia sex*dementia
Class Levels Values sex 2 -1 1 dementia 2 -1 1 28 observations Source DF Sum of Squares Mean Square F Value Pr > FModel 3 96686 32228.70 451.05 <.0001Error 24 1715 71.45 = SD2
e
C Total 27 98402
R-Square Coeff Var Root MSE Mean brain wt 0.9826 0.666092 8.453=SDe 1269.04
Source DF Type III SS Mean Square F Value Pr > F=p valuesex 1 95005.75 95005.75 1329.64 <.0001dementia 1 1620.32 1620.32 22.68 <.0001sex*dementia 1 60.04 60.04 0.84 0.3685
brain weight - via regression –effect coding Sum of MeanSource DF Squares Square F Value Pr > FModel 3 96686 32228.70 451.05 <.0001Error 24 1714.9 71.45Corrected Total 27 98401
R-Square=0.9826 Root MSE=8.453 Mean =1269.04
Parameter StandardVariable DF Estimate Error t Value Pr > |t|Intercept 1 1269.03571 1.59746 794.41 <.0001sex 1 58.25000 1.59746 36.46 <.0001dementia 1 -7.60714 1.59746 -4.76 <.0001sexdem 1 1.46429 1.59746 0.92 0.3685
Brain wt=1269 +58.3 sex-7.6 dementia + 1.46 sex dementia
Balanced designs and Effect coding
Type of person variable: dementia gender dementia*gender no dementia-Female -1 -1 1 dementia-Female 1 -1 -1 no dementia-Male -1 1 -1 dementia-Male 1 1 1 total 0 0 0
Correlations among X1=dementia, X2=gender, X3= dementia*gender Effect coding used with balanced data creates orthogonality
Dementia Gender Dementia*genderDementia 1.0 0.0 0.0Gender 0.0 1.0 0.0Dementia*gender 0.0 0.0 1.0
Relation between sum of squares (SS) and regression coefficients, SS=nb2
Factor regr coefficient (b) nb2 =Sum squares (n=28)Dementia 7.607 28(7.607)2 = 1620.32Gender -58.25 28 (58.25)2 = 95005.75Dementia*Gender 1.4643 28(1.46423)2 = 60.036
The SS are functions of the squared regression coefficient & n.
Dementia, Gender and the Dementia x Gender interaction are orthogonal. The statistical significance of each factor does not depend on whether the other factors are in the model. Makes evaluating each factor easy. Orthogonality holds if :1. Effect coding is used in the regression2. The design is balanced
ANOVA tables as a compact regression In general, if factor A has “a” levels (and “a” means), in a regression it
must be represented by a-1 dummy or effect coded variables with a-1
corresponding regression coefficients. In the ANOVA table for factor A, the sum of squares for A (SSa), is made out of the sum of squares of the a-1
regression coefficients. DF=a-1.
Ex: a=4, a-1=3, three dummy vars
SSa = constant (b1 + b2 + b3)2
So, if factor A is NOT significant in the ANOVA table, we can conclude that β1=β2=… βa-1=0 without looking at each one individually, a major simplification.
If factor B has “b” levels, there are a x b possible combinations (cells) and (a-1) + (b-1) + (a-1)(b-1)= ab-1 dummy (or effect coded) variables/ regression coefficients for A, B and the A x B interaction respectively. There are ab combinations of A and B. The squared effects of A, B and AxB are represented in a “condensed” form in the ANOVA table.
ANOVA table – summarizes ab-1 effects in three lines
Factor df Sum Squares (SS) Mean square=SS/df A a-1 SSa SSa/(a-1) B b-1 SSb SSb/(b-1) AB (a-1)(b-1) SSab SSab/(a-1)(b-1)
When is the ANOVA table useful? Dependent Variable: depression score Source DF SS Mean Square F Value overall p valueModel 199 3387.41 17.02 4.42 <.0001Error 400 1540.17 3.85Corrected Total 599 4927.58 root MSE=1.962, R2=0.687
Source DF SS Mean Square F Value p valuegender 1 778.084 778.084 202.08 <.0001race 3 229.689 76.563 19.88 <.0001educ 4 104.838 26.209 6.81 <.0001occ 4 1531.371 382.843 99.43 <.0001gender*race 3 1.879 0.626 0.16 0.9215gender*educ 4 3.575 0.894 0.23 0.9203gender*occ 4 8.907 2.227 0.58 0.6785race*educ 12 69.064 5.755 1.49 0.1230race*occ 12 62.825 5.235 1.36 0.1826educ*occ 16 60.568 3.786 0.98 0.4743gender*race*educ 12 77.742 6.479 1.68 0.0682gender*race*occ 12 59.705 4.975 1.29 0.2202gender*educ*occ 16 100.920 6.308 1.64 0.0565race*educ*occ 48 206.880 4.310 1.12 0.2792gender*race*educ*occ 48 91.368 1.903 0.49 0.9982
8 graphs of 200 depression means.
Y=depr, X=occ (occupation), X=educ.
separate graph for each gender & race
Males Females
W W
B B
H H
A A
One of the 8 graphs
Note parallelism implying no interaction
Depression-final model Sum of Source DF Squares Mean Square F overall p
Model 12 2643.981859 220.331822 56.64 <.0001 Error 587 2283.610408 3.89030 Corrected Total 599 4927.592267
R-Square Coeff Var Root MSE y Mean 0.536567 21.24713 1.972386=SDe 9.283069
Source DF SS Mean Square F Value p value gender 1 778.084257 778.08 200.01 <.0001 race 3 229.688698 76.56 19.68 <.0001 educ 4 104.837607 26.21 6.74 <.0001 occ 4 1531.371296 382.84 98.41 <.0001
Analysis shows that factors are additive (no significant interactions)
Example2 : ANOVA as a compact regression Example: Y = log pertussus antibody titer
What if the potential predictive factors are: Blood type: A-, A+, B-, B+, Ab-, Ab+, O-,O+ (8 levels) Center: LA, SF, Chicago, NY, Houston, Seattle (6 levels) Vaccine: placebo, IgA, IgG (3 levels) How many β parameters are summarized?
Factor df (= number of βs) SS MS=SS/df F p value(Intercept 1) -- Bloodtype 7 Center 5Vaccine 2Bloodtype * Center (7 x 5) = 35Bloodtype * Vaccine (7 x 2) = 14Center * Vaccine (5 x 2) = 10 BT*Center*Vaccine (7 x 5 x 2) = 70 Total model 144
If one of the "condensed" factors above is NOT significant, the entire set of βs for that factor can be removed from the model.
The "sum of squares" ANOVA table is a condensed regression table that is useful for screening, particularly screening interactions. It allows one to test "chunks" of the model.
If we also have balance, then all the parts above are orthogonal so the assessment of one factor or interaction is not affected if another factor or interaction is significant or not. This is an ideal analysis situation.
If all of the interaction terms are NOT significant, then one has proven that the influence of all the factors on the outcome is additive.
If all the interaction terms for factor “B” are not significant, then the impact of factor B on Y is additive.
Balanced versus unbalanced ANOVAbelow “nc=” denotes the sample size in each cell
unbalanced since n not same in each cell
Cell and marginal mean amygdala volumes in cc
Male Female adj marg. mean
Obs marg. mean
Dementia 0.5 (nc=10) 0.5 (nc=90) 0.5 0.5 (n=100)
No Dementia 1.5 (nc=190) 1.5 (nc=10) 1.5 1.5 (n=200)
Adjusted marg. Means 1.0 1.0
Observed marg. means 1.45 (n=200) 0.6 (n=100) n=300(10 x 0.5 + 190 x 1.5)/200=1.45, (90 x 0.5 + 10 x 1.5)/100=0.60
Gender & dementia NOT orthogonal
Repeated measure ANOVA – paired t test
example
Motivation-Jaw growth in children (cm) -Potthoff & Roy
child 8 yrs 10 yrs difference
1 21.0 20.0 -1.0
2 21.0 21.5 0.5
3 20.5 24.0 3.5
4 23.5 24.5 1.0
5 21.5 23.0 1.5
6 20.0 21.0 1.0
7 21.5 22.5 1.0
8 23.0 23.0 0.0
9 20.0 21.0 1.0
10 16.5 19.0 2.5
11 24.5 25.0 0.5
8 yrs 10 yrs | difference
Mean 21.1 22.2 | 1.0
SD 2.1 1.9 | 1.2
SE 0.64 0.57 | 0.36=SEd
unpaired paired
t 1.216 2.907
df 20 10
p value 0.2382 0.0156Paired & unpaired t test give different p values even thoughthey are using the same means. Corr is r= 0.83, not zero.
SEd=sqrt(SE12 + SE2
2 - 2 SE1 SE2 r)
Repeated measures-must add subject effect
Factor df SS MS F p value
A a-1 SSa MSa Fa p value for a
B b-1 SSb MSb Fb p value for b
AB (a-1)(b-1) SSab MSab Fab p value for ab
Subject n-1 SSsub MSsub -- --
Error (n-1)(T-1) SSe MSe
Otherwise “Error” SS is too big as it is within subject error and between subject variation combined
Factorial vs repeated measure ANOVA
Model Residual SD2e SDe
Factorial 4.07 2.02 Repeated measure 0.71 0.84
The SDe is too large if the subject effect is not taken into account. If SDe is too large, SEs are too large & p values are too large.
p values for comparing means to
zero
Type 3 Tests of Fixed Effects (ANOVA table)
Effect Num DF Den DF F Value Pr > Fyear 1 20 1.48 0.2382
Least Squares Means StandardEffect year Estimate Error DF t Value Pr > |t|year 8 21.1818 0.6080 20 34.84 <.0001year 10 22.2273 0.6080 20 36.56 <.0001
Differences of Least Squares Means StandardEffect year year Estimate Error DF t Value Pr > |t|year 8 10 -1.0455 0.8598 20 -1.22 0.2382
Repeated measure ANOVA
Correct p value for comparing means
(Co)variance Parameter EstimatesCov Parm Estimateid 3.3545 =SDp
2-between person var (controlled)Residual 0.7114 =SDe
2-within person variation(note 3.3545 + 0.7114 = 4.0659 from incorrect analysis) Type 3 Tests of Fixed Effects Num DenEffect DF DF F Value Pr > Fyear 1 10 8.45 0.0156 Least Squares Means StandardEffect year Estimate Error DF t Value Pr > |t|year 8 21.1818 0.6080 10 34.84 <.0001year 10 22.2273 0.6080 10 36.56 <.0001 Differences of Least Squares Means StandardEffect year year Estimate Error DF t Value Pr > |t|
year 8 10 -1.0455 0.3596 10 -2.91 0.0156