RESEARCH STATISTICSRESEARCH STATISTICS
Jobayer Hossain
Larry Holmes, Jr,
November 20, 2008
Analysis of Variance
Experimental Design TerminologyExperimental Design Terminology
An Experimental Unit is the entity on which measurement or an
observation is made. For example, subjects are experimental units
in most clinical studies.
Homogeneous Experimental Units: Units that are as uniform as
possible on all characteristics that could affect the response.
A Block is a group of homogeneous experimental units. For
example, if an investigator had reason to believe that age might be a
significant factor in the effect of a given medication, he might
choose to first divide the experimental subjects into age groups,
such as under 30 years old, 30-60 years old, and over 60 years old.
Experimental Design TerminologyExperimental Design Terminology
A Factor is a controllable independent variable that is being
investigated to determine its effect on a response. E.g.
treatment group is a factor.
Factors can be fixed or random
– Fixed -- the factor can take on a discrete number of values
and these are the only values of interest.
– Random -- the factor can take on a wide range of values
and one wants to generalize from specific values to all
possible values.
Each specific value of a factor is called a level.
Experimental Design TerminologyExperimental Design Terminology
A covariate is an independent variable not manipulated by the
experimenter but still affecting the response. E.g. in many
clinical experiments, the demographic variables such as race,
gender, age may influence the response variable significantly
even though these are not the variables of interest of the study.
These variables are termed as covariate.
Effect is the change in the average response between two
factor levels. That is, factor effect = average response at one
level – average response at a second level.
Experimental Design TerminologyExperimental Design Terminology
Interaction is the joint factor effects in which the effect of one
factor depends on the levels of the other factors.
No interaction effect of factor A and B
Interaction effect of factor A and B
Experimental Design TerminologyExperimental Design Terminology
Randomization is the process of assigning
experimental units randomly to different experimental
groups.
It is the most reliable method of creating
homogeneous treatment groups, without involving
potential biases or judgments.
Experimental Design Experimental Design TerminologyTerminology
A Replication is the repetition of an entire experiment or portion of
an experiment under two or more sets of conditions.– Although randomization helps to insure that treatment groups are as similar as
possible, the results of a single experiment, applied to a small number of experimental units, will not impress or convince anyone of the effectiveness of the treatment.
– To establish the significance of an experimental result, replication, the repetition of an experiment on a large group of subjects, is required.
– If a treatment is truly effective, the average effect of replicated experimental units will reflect it.
– If it is not effective, then the few members of the experimental units who may have reacted to the treatment will be negated by the large numbers of subjects who were unaffected by it.
– Replication reduces variability in experimental results and increases the significance and the confidence level with which a researcher can draw conclusions about an experimental factor.
Experimental Design TerminologyExperimental Design Terminology
A Design (layout) of the experiment includes the choice of factors and factor-
levels, number of replications, blocking, randomization, and the assignment of
factor –level combination to experimental units.
Sum of Squares (SS): Let x1, …, xn be n observations. The sum of squares of
these n observations can be written as x12 + x2
2 +…. xn2. In notations, ∑xi
2. In a
corrected form this sum of squares can be written as (xi -x )2.
Degrees of freedom (df): Number of quantities of the form – Number of
restrictions. For example, in the following SS, we need n quantities of the form
(xi -x ). There is one constraint (xi -x ) = 0. So the df for this SS is n – 1.
Mean Sum of Squares (MSS): The SS divided by it’s df.
Experimental Design TerminologyExperimental Design Terminology The analysis of variance (ANOVA) is a technique of decomposing the
total variability of a response variable into:
– Variability due to the experimental factor(s) and…
– Variability due to error (i.e., factors that are not accounted for in the experimental
design).
The basic purpose of ANOVA is to test the equality of several means.
A fixed effect model includes only fixed factors in the model.
A random effect model includes only random factors in the model.
A mixed effect model includes both fixed and random factors in the
model.
The basic ANOVA situationThe basic ANOVA situation
Two type of variables: Quantitative responseCategorical predictors (factors),
Main Question: Do the (means of) the quantitative variable depend on which group (given by categorical variable) the individual is in?
If there is only one categorical variable with only 2 levels (groups):
• 2-sample t-test
ANOVA allows for 3 or more groups
One-way analysis of VarianceOne-way analysis of Variance
One factor of k levels or groups. E.g., 3 treatment groups in a drug study.
The main objective is to examine the equality of means of different groups.
Total variation of observations (SST) can be split in two components:
variation between groups (SSG) and variation within groups (SSE).
Variation between groups is due to the difference in different groups. E.g.
different treatment groups or different doses of the same treatment.
Variation within groups is the inherent variation among the observations
within each group.
Completely randomized design (CRD) is an example of one-way analysis of
variance.
One-way analysis of varianceOne-way analysis of variance
Consider a layout of a study with 16 subjects that intended to compare 4 treatment groups (G1-G4). Each group contains four subjects.
S1 S2 S3 S4
G1 Y11 Y12 Y13 Y14
G2 Y21 Y22 Y23 Y24
G3 Y31 Y32 Y33 Y34
G4 Y41 Y42 Y43 Y44
One-way Analysis: Informal InvestigationOne-way Analysis: Informal Investigation
Graphical investigation: • side-by-side box plots• multiple histograms
Whether the differences between the groups are significant depends on
• the difference in the means• the standard deviations of each group• the sample sizes
ANOVA determines P-value from the F statistic
One-way Analysis: Side by Side One-way Analysis: Side by Side BoxplotsBoxplots
One-way analysis of VarianceOne-way analysis of Variance
Model:
Assumptions:
– Observations yij are independent.
– eij are normally distributed with mean zero and constant standard deviation.
– The second (above) assumption implies that response variable for each group is normal (Check using normal quantile plot or histogram or any test for normality) and standard deviations for all group are equal (rule of thumb: ratio of largest to smallest are approximately 2:1).
error. theis andmean general theis
group, ofeffect theis
group, ofn observatio theis where,
ij
i
ij
ijiij
e
ith
jthithy
ey
One-way analysis of VarianceOne-way analysis of Variance
Hypothesis:Ho: Means of all groups are equal.
Ha: At least one of them is not equal to other.
– doesn’t say how or which ones differ.
– Can follow up with “multiple comparisons”
Analysis of variance (ANOVA) Table for one way classified data
Sources of Variation
Sum of Squares
df Mean Sum of Squares
F-Ratio
Group SSG k-1 MSG=SSG/k-1 F=MSG/MSE
Error SSE n-k MSE=SSE/n-k
Total SST n-1
A large F is evidence against H0, since it indicates that there is more difference between groups than within groups.
Pooled estimate for variancePooled estimate for variance
sp2
(n1 1)s12 (n2 1)s2
2 ... (nI 1)sI2
n I
sp2
(df1)s12 (df2)s2
2 ... (df I )sI2
df1 df2 ... df I
The pooled variance of all groups can be estimated as the weighted average of the variance of each group:
MSEDFE
SSEsp 2
so MSE is the pooled estimate of variance
Multiple comparisonsMultiple comparisons
If the F test is significant in ANOVA table, then we intend
to find the pairs of groups are significantly different.
Following are the commonly used procedures:– Fisher’s Least Significant Difference (LSD)
– Tukey’s HSD method
– Bonferroni’s method
– Scheffe’s method
– Dunn’s multiple-comparison procedure
– Dunnett’s Procedure
One-way ANOVA - DemoOne-way ANOVA - Demo
MS Excel:
– Put response data (hgt) for each groups (grp) in side by side
columns (see next slides)
– Select Tools/Data Analysis and select Anova: Single Factor
from the Analysis Tools list. Click OK.
– Select Input Range (for our example a1: c21), mark on Group by
columns and again mark labels in first row.
– Select output range and then click on ok.
One-way ANOVA MS-Excel Data One-way ANOVA MS-Excel Data LayoutLayout
grp1 grp2 grp352.50647 47.83167 43.8221143.14426 43.69665 40.4135955.91853 40.73333 57.6574845.68187 46.56424 41.9620754.76593 53.03273 49.0805145.27999 56.41127 55.9779741.95513 43.69635 47.7141943.67319 50.92119 41.7591244.01857 40.36097 46.2185944.54295 51.95264 53.61966
46.1765 57.1638 49.6348440.02826 50.98321 57.6822958.09449 49.23148 49.0847143.25757 48.01014 40.5986642.07507 45.98231 38.9905546.80839 55.32514 54.7428643.80479 47.21837 53.7462457.60508 40.30104 44.8250742.47022 50.56327 45.85581
57.4945 55.55145 41.36863
One-way ANOVA MS-Excel output: One-way ANOVA MS-Excel output: height on treatment groupsheight on treatment groups
Anova: Single Factor
SUMMARYGroups Count Sum Average Variance
grp1 20 949.3017 47.46509 36.88689grp2 20 975.5313 48.77656 28.10855grp3 20 954.7549 47.73775 37.50739
ANOVASource of Variation SS df MS F P-value F critBetween Groups 19.15632 2 9.578159 0.280329 0.756571 3.158846Within Groups 1947.554 57 34.16761
Total 1966.71 59
One-way ANOVA - DemoOne-way ANOVA - Demo
SPSS:
– Select Analyze > Compare Means > One –Way ANOVA
– Select variables as Dependent List: response (hgt), and Factor:
Group (grp) and then make selections as follows-click on Post Hoc
and select Multiple comparisons (LSD, Tukey, Bonferroni, or
Scheffe), click options and select Homogeneity of variance test,
click continue and then Ok.
One-way ANOVA SPSS output: height One-way ANOVA SPSS output: height on treatment groupson treatment groups
ANOVA hgt
Sum of Squares df Mean Square F Sig.
Between Groups 19.156 2 9.578 .280 .757 Within Groups 1947.554 57 34.168 Total 1966.710 59
Analysis of variance of factorial Analysis of variance of factorial experiment (Two or more factors)experiment (Two or more factors)
Factorial experiment:
– The effects of the two or more factors including their interactions
are investigated simultaneously.
– For example, consider two factors A and B. Then total variation of
the response will be split into variation for A, variation for B,
variation for their interaction AB, and variation due to error.
Analysis of variance of factorial Analysis of variance of factorial experiment (Two or more factors)experiment (Two or more factors)
Model with two factors (A, B) and their interactions:
Assumptions: The same as in One-way ANOVA.
error theis
B of level andA level ofeffect n interactio theis
Bfactor theof level ofeffect theis
Afactor theof level ofeffect theis
mean general theis
)(
ijk
ij
j
i
ijkijjiijk
e
jthithβ)(
jth
ithα
ey
Analysis of variance of factorial Analysis of variance of factorial experiment (Two or more factors)experiment (Two or more factors)
Null Hypotheses:
Hoa: Means of all groups of the factor A are equal.
Hob: Means of all groups of the factor B are equal.
Hoab:(αβ)ij = 0, i. e. two factors A and B are independent
Analysis of variance of factorial Analysis of variance of factorial experiment (Two or more factors)experiment (Two or more factors)
ANOVA for two factors A and B with their interaction AB.
Kpr-1SSTTotal
MSE=SSE/kp(r-1)
kp(r-1)SSEError
MSAB/MSEMSAB=SSAB/ (k-1)(p-1)
(k-1)(p-1)SSABInteraction Effect AB
MSB/MSEMSB=SSB/p-1P-1SSBMain Effect B
MSA/MSEMSA=SSA/k-1k-1SSAMain Effect A
F-RatioMean Sum of Squares
dfSum of Squares
Sources of Variation
Kpr-1SSTTotal
MSE=SSE/kp(r-1)
kp(r-1)SSEError
MSAB/MSEMSAB=SSAB/ (k-1)(p-1)
(k-1)(p-1)SSABInteraction Effect AB
MSB/MSEMSB=SSB/p-1P-1SSBMain Effect B
MSA/MSEMSA=SSA/k-1k-1SSAMain Effect A
F-RatioMean Sum of Squares
dfSum of Squares
Sources of Variation
Two-factor with replication - DemoTwo-factor with replication - Demo
MS Excel:
– Put response data for two factors like in a lay out like in the next
page.
– Select Tools/Data Analysis and select Anova: Two Factor with
replication from the Analysis Tools list. Click OK.
– Select Input Range and input the rows per sample: Number of
replications (excel needs equal replications for every levels).
Replication is 2 for the data in the next page.
– Select output range and then click on ok.
Two-factor ANOVA MS-Excel Data Two-factor ANOVA MS-Excel Data LayoutLayout
grp1 grp2 grp3shade1 52.50647 47.83167 43.82211shade1 43.14426 43.69665 40.41359shade1 55.91853 40.73333 57.65748shade1 45.68187 46.56424 41.96207shade1 54.76593 53.03273 49.08051shade1 45.27999 56.41127 55.97797shade1 41.95513 43.69635 47.71419shade1 43.67319 50.92119 41.75912shade1 44.01857 40.36097 46.21859shade1 44.54295 51.95264 53.61966shade2 46.1765 57.1638 49.63484shade2 40.02826 50.98321 57.68229shade2 58.09449 49.23148 49.08471shade2 43.25757 48.01014 40.59866shade2 42.07507 45.98231 38.99055shade2 46.80839 55.32514 54.74286shade2 43.80479 47.21837 53.74624shade2 57.60508 40.30104 44.82507shade2 42.47022 50.56327 45.85581shade2 57.4945 55.55145 41.36863
Two-factor ANOVA MS-Excel output: height on Two-factor ANOVA MS-Excel output: height on treatment group, shades, and their interactiontreatment group, shades, and their interaction
Anova: Two-Factor With Replication
SUMMARYgrp1 grp2 grp3 Totalshade1
Count 10 10 10 30Sum 471.4869 475.201 478.2253 1424.913Average 47.14869 47.5201 47.82253 47.49711Variance 26.773 29.80231 38.11298 29.46458
shade2
Count 10 10 10 30Sum 477.8149 500.3302 476.5297 1454.675Average 47.78149 50.03302 47.65297 48.48916Variance 50.87686 26.02977 41.05331 37.84396
Total
Count 20 20 20Sum 949.3017 975.5313 954.7549Average 47.46509 48.77656 47.73775Variance 36.88689 28.10855 37.50739
ANOVASource of Variation SS df MS F P-value F crit
Sample 14.76245 1 14.76245 0.416532 0.521406 4.01954Columns 19.15632 2 9.578159 0.270254 0.764212 3.168246Interaction 18.9572 2 9.478602 0.267445 0.766341 3.168246Within 1913.834 54 35.44137
Total 1966.71 59
Two-factor ANOVA - DemoTwo-factor ANOVA - Demo
SPSS:
– Select Analyze > General Linear Model > Univariate
– Make selection of variables e.g. Dependent varaiable: response
(hgt), and Fixed Factor: grp and shades.
– Make other selections as follows-click on Post Hoc and select
Multiple comparisons (LSD, Tukey, Bonferroni, or Scheffe),
click options and select Homogeneity of variance test, click
continue and then Ok.
Two-factor ANOVA SPSS output: height on Two-factor ANOVA SPSS output: height on treatment group, shades, and their interactiontreatment group, shades, and their interaction
Between-Subjects Factors
N 1 20 2 20
grp
3 20 1 30 Shades
2 30
Tests of Between-Subjects Effects Dependent Variable: hgt
Source Type III Sum of Squares df Mean Square F Sig.
Corrected Model 52.876(a) 5 10.575 .298 .912 Intercept 138200.445 1 138200.445 3899.410 .000 grp 19.156 2 9.578 .270 .764 Shades 14.762 1 14.762 .417 .521 grp * Shades 18.957 2 9.479 .267 .766 Error 1913.834 54 35.441 Total 140167.155 60 Corrected Total 1966.710 59
a R Squared = .027 (Adjusted R Squared = -.063)
Repeated MeasuresRepeated Measures
The term repeated measures refers to data sets with multiple
measurements of a response variable on the same experimental unit
or subject.
In repeated measurements designs, we are often concerned with
two types of variability:– Between-subjects - Variability associated with different groups of subjects who
are treated differently (equivalent to between groups effects in one-way ANOVA)
– Within-subjects - Variability associated with measurements made on an
individual subject.
Repeated MeasuresRepeated Measures
Examples of Repeated Measures designs:
A. Two groups of subjects treated with different drugs for whom responses
are measured at six-hour increments for 24 hours. Here, DRUG
treatment is the between-subjects factor and TIME is the within-subjects
factor.
B. Students in three different statistics classes (taught by different
instructors) are given a test with five problems and scores on each
problem are recorded separately. Here CLASS is a between-subjects
factor and PROBLEM is a within-subjects factor.
C. Volunteer subjects from a linguistics class each listen to 12 sentences
produced by 3 text to speech synthesis systems and rate the naturalness
of each sentence. This is a completely within-subjects design with
factors SENTENCE (1-12) and SYNTHESIZER
Repeated MeasuresRepeated Measures
When measures are made over time as in example A we want to assess:– how the dependent measure changes over time independent of
treatment (i.e. the main effect of time)– how treatments differ independent of time (i.e., the main effect of
treatment)– how treatment effects differ at different times (i.e. the treatment by
time interaction). Repeated measures require special treatment because:
– Observations made on the same subject are not independent of each other.
– Adjacent observations in time are likely to be more correlated than non-adjacent observations
Res
pons
e
Time
Repeated MeasuresRepeated Measures
Repeated MeasuresRepeated Measures
Methods of repeated measures ANOVA
– Univariate - Uses a single outcome measure.
– Multivariate - Uses multiple outcome measures.
– Mixed Model Analysis - One or more factors (other than
subject) are random effects.
We will discuss only univariate approach
Repeated MeasuresRepeated Measures
Assumptions:
– Subjects are independent.
– The repeated observations for each subject follows a
multivariate normal distribution
– The correlation between any pair of within subjects levels
are equal. This assumption is known as sphericity.
Repeated MeasuresRepeated Measures
Test for Sphericity:
– Mauchley’s test
Violation of sphericity assumption leads to inflated F statistics and hence inflated
type I error.
Three common corrections for violation of sphericity:
– Greenhouse-Geisser correction
– Huynh-Feldt correction
– Lower Bound correction
All these three methods adjust the degrees of freedom using a correction factor
called Epsilon.
Epsilon lies between 1/k-1 to 1, where k is the number of levels in the within subject
factor.
Repeated Measures - SPSS DemoRepeated Measures - SPSS Demo
Analyze > General Linear model > Repeated Measures
Within-Subject Factor Name: e.g. time, Number of Levels:
number of measures of the factors, e.g. we have two
measurements PLUC.pre and PLUC.post. So, number of level is
2 for our example. > Click on Add
Click on Define and select Within-Subjects Variables (time):
e.g. PLUC.pre(1) and PLUC.pre(2)
Select Between-Subjects Factor(s): e.g. grp
Repeated Measures ANOVA SPSS Repeated Measures ANOVA SPSS output:output:
Within-Subjects Factors Measure: MEASURE_1
time Dependent
Variable 1 PLUC.pre 2 PLUC.post
Between-Subjects Factors
N 1 20 2 20
grp
3 20
Mauchly's Test of Sphericity(b) Measure: MEASURE_1
Epsilon(a)
Within Subjects Effect Mauchly's W Approx. Chi-
Square df Sig. Greenhouse-
Geisser Huynh-Feldt Lower-bound time 1.000 .000 0 . 1.000 1.000 1.000
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of Within-Subjects Effects table. b Design: Intercept+grp Within Subjects Design: time
Repeated measures ANOVA SPSS Repeated measures ANOVA SPSS outputoutput
Tests of Within-Subjects Effects Measure: MEASURE_1
Source Type III Sum of Squares df Mean Square F Sig.
Sphericity Assumed 172.952 1 172.952 33.868 .000 Greenhouse-Geisser 172.952 1.000 172.952 33.868 .000 Huynh-Feldt 172.952 1.000 172.952 33.868 .000
time
Lower-bound 172.952 1.000 172.952 33.868 .000 Sphericity Assumed 46.818 2 23.409 4.584 .014 Greenhouse-Geisser 46.818 2.000 23.409 4.584 .014 Huynh-Feldt 46.818 2.000 23.409 4.584 .014
time * grp
Lower-bound 46.818 2.000 23.409 4.584 .014 Sphericity Assumed 291.083 57 5.107 Greenhouse-Geisser 291.083 57.000 5.107 Huynh-Feldt 291.083 57.000 5.107
Error(time)
Lower-bound 291.083 57.000 5.107
Tests of Within-Subjects Contrasts Measure: MEASURE_1
Source time Type III Sum of Squares df Mean Square F Sig.
time Linear 172.952 1 172.952 33.868 .000 time * grp Linear 46.818 2 23.409 4.584 .014 Error(time) Linear 291.083 57 5.107
Tests of Between-Subjects Effects Measure: MEASURE_1 Transformed Variable: Average
Source Type III Sum of Squares df Mean Square F Sig.
Intercept 14371.484 1 14371.484 3858.329 .000 grp 121.640 2 60.820 16.328 .000 Error 212.313 57 3.725
QuestionsQuestions
Top Related