Introduction to Linear Mixed EffectsKiran PedadaPhD Student (Marketing)
March 26, 2015
Correlated Data
Forms of correlated data: Time Series data Repeated measurements Longitudinal data Spatial data
Source: http://www.stat.missouri.edu/~spinkac/stat8320/LinearMixedModels.pdf
Linear Mixed Effects Models
Mixed model analysis provides a general, flexible approach in the situations of correlated data.
•Mixed model consists of two components: Fixed effects – usually the conventional linear
regression part Random effects – associated with individual
experimental units produced at random from the data generating process.
Source: http://www.stat.cmu.edu/~hseltman/309/Book/chapter15.pdfhttp://www.mathworks.com/help/stats/linear-mixed-effects-models.html
Linear Mixed Effects Models
The standard form of a linear mixed effects model:
Y=βX+Zb+u
Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html
Fixed effect
Random effect
Error
Y is the n x1 response vector, and n is the number of observations.X is an n x p fixed-effects design matrix.β is a p x 1 fixed-effects vector.Z is an n x q random-effects design matrix.b is a q x 1 random-effects vector.u is the n x 1 observation error vector.
Random Effect and Error Vectors
Random effects vector, b, and the error vector, ε are assumed to be independent and distributed as follows :
b ~ N (0, σ2D(θ))ε ~ N (0, σ2I)
Where D is a symmetric and positive semi definite matrix, parameterized by a variance component vector θ, I is an n x n identity matrix, and σ2 is the error variance.
Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html
Bodo Winter Example
General Form of Linear Mixed Model:Y=βX+Zb+u
Bodo Winter Fixed Effect Model:
Pitch ~ politeness + sex + uBased on the general form of Linear Mixed Model, we can write the Bodo Winter Example as follows:
Y= β1X1 + β2X2 + uWhere, Y is the response variable, i.e., Pitch and X1 and X2 are the fixed effects, i.e., politeness and sex. β1 and β2 are fixed effect parameters.
Source: http://www.bodowinter.com/tutorial/bw_LME_tutorial.pdf
Mixed Effect Model
If we add one or more random effects to the fixed effect model, then model will become a Mixed Effect Model.
Let us add one random effect (for subject).
Thus, the Mixed Effect Model will look like the following:
Y= β1X1 + β2X2 + Zb + u εWhere, Z is the random effect, i.e., multiple responses per subject. And b is random effect parameter.
Matrix Notation of the Bodo Winter Mixed Model
Y = X β + Z b + u
200 X 1200 X 1 200 X 1
200 X 1
200 X 2 2 X 1 200 X 40 40 X 1
X = β = 200 X 2
2 X 1
Source: Dr. Westfall Notes
To make the example simple, let us consider 1 fixed effect and one random effect.
Let us say, there are 40 female subjects with 5 repetitions on each subject. Half of the subjects are observed in formal case (1) and other half in informal case (0).
1
200
6
Matrix Notation of the Bodo Winter Mixed Model
X β = 200 X 2 2 X 1
=
200 X 1
Source:Source: Dr. Westfall Notes
x
Matrix Notation of the Bodo Winter Mixed Model
Z = 200 X 40
Source: Dr. Westfall Notes
1
5
6
200
Matrix Notation of the Bodo Winter Mixed Model
Z b = 40 X 1200 X 40
Source: Dr. Westfall Notes
Variance-covariance Matrix
Y= β1X1 + β2X2 + Zb + u εCov(ε) = Cov (Zb + u) = Cov (Zb)+ Cov(u) = Z Cov (b) ZT+ σ2I
Source: Dr. Westfall Notes
Cov(ε) = Z Cov (b) ZT+ σ2I
Variance-covariance Matrix
Z Cov (b) ZT+ σ2I
200 X 200
Source: Dr. Westfall Notes
b b b b
b b b
b b
b
R simulation
R Simulation
Top Related