An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian...
Transcript of An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian...
![Page 1: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/1.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
An Introduction to Linear and LogitMixed Models
Day 1
Florian Jaeger
February 4, 2010
![Page 2: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/2.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Overview
I Class 1:I (Re-)Introducing Ordinary RegressionI Comparison to ANOVAI Linear Mixed ModelsI Generalized Linear Mixed ModelsI Trade-offs & MotivationI How to get started
I Class 2:I Common Issues in Regression Modeling (Mixed or not)I Solutions
I Please ask/add to the discussion any time!
![Page 3: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/3.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Acknowledgments
I I’ve incorporated (and modified) a couple of slidesprepared by:
I Victor Kuperman (Stanford)I Roger Levy (UCSD)
... with their permission (naturalmente!)I I am also grateful for feedback from:
I Austin Frank (Rochester)I Previous audiences to similar workshops at CUNY,
Haskins, Rochester, Buffalo, UCSD, MIT.
I For more materials, check out:I http://www.hlp.rochester.edu/I
http://wiki.bcs.rochester.edu:2525/HlpLab/StatsCoursesI http://hlplab.wordpress.com/ (e.g. multinomial mixed
models code)
![Page 4: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/4.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Models
Goal: model the effects of predictors (independent variables)X on a response (dependent variable) Y .
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Response
Predictors
Model parameters
![Page 5: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/5.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Models
Goal: model the effects of predictors (independent variables)X on a response (dependent variable) Y .
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Response
Predictors
Model parameters
![Page 6: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/6.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Models
Goal: model the effects of predictors (independent variables)X on a response (dependent variable) Y .
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Response
Predictors
Model parameters
![Page 7: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/7.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Models
Goal: model the effects of predictors (independent variables)X on a response (dependent variable) Y .
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Response
Predictors
Model parameters
![Page 8: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/8.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs
Assumptions of the generalized linear model (GLM):
I Predictors {Xi} influence Y through the mediation of alinear predictor η;
I η is a linear combination of the {Xi}:
η = α + β1X1 + · · ·+ βNXN (linear predictor)
I η determines the predicted mean µ of Y
η = g(µ) (link function)
I There is some noise distribution of Y around thepredicted mean µ of Y :
P(Y = y ;µ)
![Page 9: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/9.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs
Assumptions of the generalized linear model (GLM):
I Predictors {Xi} influence Y through the mediation of alinear predictor η;
I η is a linear combination of the {Xi}:
η = α + β1X1 + · · ·+ βNXN (linear predictor)
I η determines the predicted mean µ of Y
η = g(µ) (link function)
I There is some noise distribution of Y around thepredicted mean µ of Y :
P(Y = y ;µ)
![Page 10: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/10.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs
Assumptions of the generalized linear model (GLM):
I Predictors {Xi} influence Y through the mediation of alinear predictor η;
I η is a linear combination of the {Xi}:
η = α + β1X1 + · · ·+ βNXN (linear predictor)
I η determines the predicted mean µ of Y
η = g(µ) (link function)
I There is some noise distribution of Y around thepredicted mean µ of Y :
P(Y = y ;µ)
![Page 11: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/11.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs
Assumptions of the generalized linear model (GLM):
I Predictors {Xi} influence Y through the mediation of alinear predictor η;
I η is a linear combination of the {Xi}:
η = α + β1X1 + · · ·+ βNXN (linear predictor)
I η determines the predicted mean µ of Y
η = g(µ) (link function)
I There is some noise distribution of Y around thepredicted mean µ of Y :
P(Y = y ;µ)
![Page 12: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/12.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs
Assumptions of the generalized linear model (GLM):
I Predictors {Xi} influence Y through the mediation of alinear predictor η;
I η is a linear combination of the {Xi}:
η = α + β1X1 + · · ·+ βNXN (linear predictor)
I η determines the predicted mean µ of Y
η = g(µ) (link function)
I There is some noise distribution of Y around thepredicted mean µ of Y :
P(Y = y ;µ)
![Page 13: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/13.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Linear Regression
Linear regression, which underlies ANOVA, is a kind ofgeneralized linear model.
I The predicted mean is just the linear predictor:
η = l(µ) = µ
I Noise is normally (=Gaussian) distributed around 0 withstandard deviation σ:
ε ∼ N(0, σ)
I This gives us the traditional linear regression equation:
Y =
Predicted Mean µ = η︷ ︸︸ ︷α + β1X1 + · · ·+ βnXn +
Noise∼N(0,σ)︷︸︸︷ε
![Page 14: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/14.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Linear Regression
Linear regression, which underlies ANOVA, is a kind ofgeneralized linear model.
I The predicted mean is just the linear predictor:
η = l(µ) = µ
I Noise is normally (=Gaussian) distributed around 0 withstandard deviation σ:
ε ∼ N(0, σ)
I This gives us the traditional linear regression equation:
Y =
Predicted Mean µ = η︷ ︸︸ ︷α + β1X1 + · · ·+ βnXn +
Noise∼N(0,σ)︷︸︸︷ε
![Page 15: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/15.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Linear Regression
Linear regression, which underlies ANOVA, is a kind ofgeneralized linear model.
I The predicted mean is just the linear predictor:
η = l(µ) = µ
I Noise is normally (=Gaussian) distributed around 0 withstandard deviation σ:
ε ∼ N(0, σ)
I This gives us the traditional linear regression equation:
Y =
Predicted Mean µ = η︷ ︸︸ ︷α + β1X1 + · · ·+ βnXn +
Noise∼N(0,σ)︷︸︸︷ε
![Page 16: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/16.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Linear Regression
Linear regression, which underlies ANOVA, is a kind ofgeneralized linear model.
I The predicted mean is just the linear predictor:
η = l(µ) = µ
I Noise is normally (=Gaussian) distributed around 0 withstandard deviation σ:
ε ∼ N(0, σ)
I This gives us the traditional linear regression equation:
Y =
Predicted Mean µ = η︷ ︸︸ ︷α + β1X1 + · · ·+ βnXn +
Noise∼N(0,σ)︷︸︸︷ε
![Page 17: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/17.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Logistic Regression
Logistic regression, too, is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (1)
I The distribution around the mean is taken to bebinomial.
![Page 18: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/18.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Logistic Regression
Logistic regression, too, is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (1)
I The distribution around the mean is taken to bebinomial.
![Page 19: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/19.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Logistic Regression
Logistic regression, too, is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (1)
I The distribution around the mean is taken to bebinomial.
![Page 20: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/20.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing Logistic Regression
Logistic regression, too, is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (1)
I The distribution around the mean is taken to bebinomial.
![Page 21: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/21.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLM
I Poisson regression
I Beta-binomial model (for low count data, for example)
I Ordered and unordered multinomial regression.
I ...
![Page 22: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/22.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Determining the parameters
I How do we choose parameters (model coefficients) βi
and σ?
I We find the best ones.I There are two major approaches (deeply related, yet
different) in widespread use:I The principle of maximum likelihood: pick parameter
values that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 23: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/23.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Determining the parameters
I How do we choose parameters (model coefficients) βi
and σ?
I We find the best ones.I There are two major approaches (deeply related, yet
different) in widespread use:I The principle of maximum likelihood: pick parameter
values that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 24: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/24.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Determining the parameters
I How do we choose parameters (model coefficients) βi
and σ?
I We find the best ones.I There are two major approaches (deeply related, yet
different) in widespread use:I The principle of maximum likelihood: pick parameter
values that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 25: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/25.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Determining the parameters
I How do we choose parameters (model coefficients) βi
and σ?
I We find the best ones.I There are two major approaches (deeply related, yet
different) in widespread use:I The principle of maximum likelihood: pick parameter
values that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
P({βi}, σ|Y ) =P(Y |{βi}, σ)
Prior︷ ︸︸ ︷P({βi}, σ)
P(Y )
![Page 26: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/26.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Determining the parameters
I How do we choose parameters (model coefficients) βi
and σ?
I We find the best ones.I There are two major approaches (deeply related, yet
different) in widespread use:I The principle of maximum likelihood: pick parameter
values that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
P({βi}, σ|Y ) =
Likelihood︷ ︸︸ ︷P(Y |{βi}, σ)
Prior︷ ︸︸ ︷P({βi}, σ)
P(Y )
![Page 27: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/27.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Penalization, Regularization, etc.
I Modern moderns are often fit using maximization oflikelihood combined with some sort of penalization, aterm that ‘punished’ high model complexity (high valuesof the coefficients).
I cf. Baayen, Davidson, and Bates (2008) for a nicedescription.
![Page 28: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/28.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
The Linear Model
I Let’s start with the Linear Model (linear regression,multiple linear regression)
![Page 29: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/29.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I You are studying word RTs in a lexical-decision task
tpozt Word or non-word?house Word or non-word?
![Page 30: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/30.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I You are studying word RTs in a lexical-decision task
tpozt Word or non-word?house Word or non-word?
![Page 31: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/31.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I You are studying word RTs in a lexical-decision task
tpozt Word or non-word?house Word or non-word?
![Page 32: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/32.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Data: Lexical decision RTs
I Data set based on Baayen et al. (2006; availablethrough languageR library in the free statistics programR)
![Page 33: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/33.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Data: Lexical decision RTs
I Lexical Decisions from 79 concrete nouns each seen by21 subjects (1,659 observation).
I Outcome: log lexical decision latency RTI Inputs:
I factor (e.g. NativeLanguage: English or Other)I continuous predictors (e.g. Frequency).
> library(languageR)> head(lexdec[,c(1,2,5,10,11)])
Subject RT NativeLanguage Frequency FamilySize1 A1 6.340359 English 4.859812 1.38629442 A1 6.308098 English 4.605170 1.09861233 A1 6.349139 English 4.997212 0.69314724 A1 6.186209 English 4.727388 0.00000005 A1 6.025866 English 7.667626 3.13549426 A1 6.180017 English 4.060443 0.6931472
![Page 34: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/34.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I A simple model: assume that Frequency has a lineareffect on average (log-transformed) RT, and trial-levelnoise is normally distributed
I If xi is Frequency, our simple model is
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I We need to draw inferences about α, β, and σ
I e.g., “Does Frequency affects RT?”→ is β reliablynon-zero?
![Page 35: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/35.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I A simple model: assume that Frequency has a lineareffect on average (log-transformed) RT, and trial-levelnoise is normally distributed
I If xi is Frequency, our simple model is
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I We need to draw inferences about α, β, and σ
I e.g., “Does Frequency affects RT?”→ is β reliablynon-zero?
![Page 36: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/36.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I A simple model: assume that Frequency has a lineareffect on average (log-transformed) RT, and trial-levelnoise is normally distributed
I If xi is Frequency, our simple model is
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I We need to draw inferences about α, β, and σ
I e.g., “Does Frequency affects RT?”→ is β reliablynon-zero?
![Page 37: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/37.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simple example
I A simple model: assume that Frequency has a lineareffect on average (log-transformed) RT, and trial-levelnoise is normally distributed
I If xi is Frequency, our simple model is
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I We need to draw inferences about α, β, and σ
I e.g., “Does Frequency affects RT?”→ is β reliablynon-zero?
![Page 38: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/38.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs: A simple example
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I Here’s a translation of our simple model into R:> glm(RT ~ 1 + Frequency, data=lexdec,+ family="gaussian")
[...]Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.5887 0.022296 295.515 <2e-16 ***Frequency -0.0428 0.004533 -9.459 <2e-16 ***> sqrt(summary(l)[["dispersion"]])
[1] 0.2353127
α̂
β̂
σ̂
![Page 39: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/39.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs: A simple example
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I Here’s a translation of our simple model into R:> glm(RT ~ 1 + Frequency, data=lexdec,+ family="gaussian")
[...]Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.5887 0.022296 295.515 <2e-16 ***Frequency -0.0428 0.004533 -9.459 <2e-16 ***> sqrt(summary(l)[["dispersion"]])
[1] 0.2353127
α̂
β̂
σ̂
![Page 40: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/40.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs: A simple example
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I Here’s a translation of our simple model into R:> glm(RT ~ 1 + Frequency, data=lexdec,+ family="gaussian")
[...]Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.5887 0.022296 295.515 <2e-16 ***Frequency -0.0428 0.004533 -9.459 <2e-16 ***> sqrt(summary(l)[["dispersion"]])
[1] 0.2353127
α̂
β̂
σ̂
![Page 41: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/41.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Reviewing GLMs: A simple example
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I Here’s a translation of our simple model into R:> glm(RT ~ 1 + Frequency, data=lexdec,+ family="gaussian")
[...]Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.5887 0.022296 295.515 <2e-16 ***Frequency -0.0428 0.004533 -9.459 <2e-16 ***> sqrt(summary(l)[["dispersion"]])
[1] 0.2353127
α̂
β̂
σ̂
![Page 42: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/42.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Linear Model with just an intercept
I The intercept is a predictor in the model (usually onewe don’t care about).
→ A significant intercept indicates that it is different fromzero.
> l.lexdec0 = lm(RT ~ 1, data=lexdec)> summary(l.lexdec0)
[...]Residuals:
Min 1Q Median 3Q Max-0.55614 -0.17048 -0.03945 0.11695 1.20222
Coefficients:Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.385090 0.005929 1077 <2e-16 ***[...]
NB: Here, intercept encodes overall mean.
![Page 43: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/43.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Visualization of Intercept Model
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●●●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●●
●●●
●
●
●●
●
●
●
●
●●●●
●
●
●●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●
●
●●●
●
●
●
●
●
●●●
●●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●●●●
●
●
●
●
●●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
0 500 1000 1500
6.0
6.5
7.0
7.5
Predicting Lexical Decision RTs
Case Index
Res
pons
e la
tenc
y (in
log−
tran
sfor
med
mse
cs)
![Page 44: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/44.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Linear Model with one predictor
> l.lexdec1 = lm(RT ~ 1 + Frequency, data=lexdec)
I Classic geometrical interpretation: Finding slope for thepredictor that minimized the squared error.
NB: Never forget the directionality in this statement (theerror in predicting the outcome is minimized, not thedistance from the line).
NB: Maximum likelihood (ML) fitting is the more generalapproach as it extends to other types of GeneralizedLinear Models. ML is identical to least-squared error forGaussian errors.
![Page 45: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/45.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Frequency effect on RT
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●● ●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●● ●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
● ●●
●●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
● ●●
●
●
●
●
●
●●●
●●
●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●●
●●
●
● ●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
2 3 4 5 6 7 8
6.0
6.5
7.0
7.5
Predicting Lexical Decision RTs
Word Frequency (log−transformed)
Res
pons
e la
tenc
y (in
log−
tran
sfor
med
mse
cs)
![Page 46: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/46.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Linearity AssumptionNB: Like AN(C)OVA, the linear model assumes that the
outcome is linear in the coefficients (linearityassumption).
I This does not mean that the outcome and the inputvariable have to be linearly related (cf. previous page).
I To illustrate this, consider that we can back-transformthe log-transformed Frequency (→ transformationsmay be necessary).
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●●●
●●
●
●
●●●
●
●
●● ●●
●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●● ●
●
●
●
●
●
●
●● ●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●● ●
●●
●
●
●●
● ●●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●●
●
● ●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●●
●
● ●
●●
●●●
●
●
●●
●●
●●
●●
●● ●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●●
● ●●
●●●
●
● ●
●
● ●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●●
●●●
●
●●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●● ●
●●●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
● ● ●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●●●
●
●
●
●●
●● ●
●
●
●●●●
●
●●
●
●
●
●●
●
●
● ●
●
●●
●●
●
●●
●
●
●●●
●
●
●●
●
●
● ●
●
● ●
●
●●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●●
●●●●
●
●
●
●
●
● ●
●
●●
●●
● ●●
●
●●●
●
●●
●
●●●●●
●
● ● ●●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●●
●
●●
●●●
●●●
●
●
● ● ●●
●
● ●●
●
●
●
●
●●● ●
●●
●
●
●
●●●
● ●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●●
●●●
●
●
●
●
●●
●●●
●
●●
●
●●● ●
●
●
●●
●
●●
●
● ●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●● ●●
●●●
●
●
●
●
●●
●●
●
●
● ●
●
●
●● ●
●
●
●●
●●
●
● ●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
● ●●
● ●
●
●●
●
●
●
● ●●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
● ●
●
●●●
●
●●
●
●●●
●
●
● ●●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●● ●
●
●
●●
●
●●
●
●●
●●●
●
●
●
●
●
● ●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●●●
●
●
●●
●●
●
●●
●
●
●
● ●●
●
●●
●
●
●●
●
●●●
● ●
●
●●
●
●
●●
●●
●
●
●
● ●
●●
●
●
●●●
●
●
●● ●●
●
● ● ●●
●
●●
●●
●●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
● ●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
0 500 1000 1500 2000
500
1000
1500
2000
Predicting Lexical Decision RTs
Word Frequency
Res
pons
e la
tenc
y (in
mse
cs)
![Page 47: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/47.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Adding further predictors
I FamilySize is the number of words in themorphological family of the target word.
I For now, we are assuming to independent effects.
> l.lexdec1 = lm(RT ~ 1 + Frequency + FamilySize,+ data=lexdec)
Coefficients:Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.563853 0.026826 244.685 < 2e-16 ***Frequency -0.035310 0.006407 -5.511 4.13e-08 ***FamilySize -0.015655 0.009380 -1.669 0.0953 .
![Page 48: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/48.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Question
I On the previous slide, is the interpretation of the outputclear?
I What is the interpretation of the intercept?
I How much faster is the most frequent word expected tobe read compared to the least frequent word?
![Page 49: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/49.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Frequency and Morph. Family Size
Predicting Lexical Decision RTs
1 2 3 4 5 6 7 8
5.5
6.0
6.5
7.0
7.5
8.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Word Frequency (log−transformed)N
umbe
r of
mor
ph. f
amily
mem
bers
(lo
g−tr
ansf
orm
ed)
Res
pons
e la
tenc
y (in
log−
tran
sfor
med
mse
cs)
●●
●
●●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●●
●
● ●●
● ●●
●●
●●●
●
●
●●●
●
●●
●●●●
●●●●
●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●●●
●
● ●
●
●
● ●
●
●
●
●
●●● ●
●
●
●
●
●●●
●
●●
●
●● ●
●
●
●●
●
●
●●
●
●● ●
●●
●
●
●
●● ●
●
●
●
●● ●●●● ●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●● ●
● ●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●●●
●
●●
●● ●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●
●● ●
●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●●●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●●
●
●●
●●
●●
●●●
●
●●●
●●●
● ● ●● ●
●
●●
● ●●
●●
● ●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
● ●
●●
●
●
●
●● ●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●●●●
●●●
●●
●● ●●
●
●●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
● ●●●
●●●
●●●
●
●●
●
●●
● ●
●
●
●
●●● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●●
●●
●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●●
●●
●●
●
●●
●●
●
●●
●●
●●
●●
●
●●
●●●
●●● ●●
● ●
●●●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●●
●
●●
●● ●●
●
●
●
●
●●●
●●
●
●●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●●
●
●●
●
●●
●●
●
●
●
●
●
● ●●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●●●
● ●● ●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●● ●●●
●●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
● ●
●●
● ●
●
● ●
●
●●●●
● ●●
●●●●
●
●
●
●
● ●●●
●●
●●
●
●
●
●●●
●
● ●● ●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●●●
●●● ●
●
●
●
●●
●
●
●●
●
● ●
●●
●
●●
●
●●
●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●●
●
●
●●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●●
●
●●
●●
●
●
●●
●
●
● ●
●●
●●
●
●
●●
●
●
●● ●
●●
●●
●●●
●
●●
● ●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
● ●●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
● ●
●
●
●●
●
●●
●●
●●
●●
●
● ●●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●● ●
●●
● ●
●●
●
●●●
●
●
●
● ●
●
●
●
● ●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●● ●●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●
●●
●●
●
●●
●
●
●●
● ●
●
●
●●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●●● ●
●
●●
●
●●●
●
● ●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
● ●●
●
●
● ● ●●●
●●
●●
●●
●
●
●●
●
●
●●●
●
●●● ●
●
● ●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●●
●●
●
●
●
●
● ●●
● ●
●●
●●
●●
●●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●●
●
●
●
![Page 50: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/50.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Continuous and categorical predictors
> l.lexdec1 = lm(RT ~ 1 + Frequency + FamilySize ++ NativeLanguage, data=lexdec)
I Recall that we’re describing the output as a linearcombination of the predictors.
→ Categorical predictors need to be coded numerically.I The default is dummy/treatment coding for regression
(cf. sum/contrast coding for ANOVA).
![Page 51: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/51.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Adding Native Language
Predicting Lexical Decision RTs
Native Speakers (red) andNon−Native Speakers (blue)
1 2 3 4 5 6 7 8
5.5
6.0
6.5
7.0
7.5
8.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Word Frequency (log−transformed)
Num
ber
of m
orph
. fam
ily m
embe
rs (
log−
tran
sfor
med
)
Res
pons
e la
tenc
y (in
log−
tran
sfor
med
mse
cs)
●●
●
●●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●●
●
● ●●
● ●●
●●
●●●
●
●
●●●
●
●●
●●●●
●●●●
●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●●●
●
● ●
●
●
● ●
●
●
●
●
●●● ●
●
●
●
●
●●●
●
●●
●
●● ●
●
●
●●
●
●
●●
●
●● ●
●●
●
●
●
●● ●
●
●
●
●● ●●●● ●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●● ●
● ●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●●●
●
●●
●● ●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●
●● ●
●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●●●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●●
●
●●
●●
●●
●●●
●
●●●
●●●
● ● ●● ●
●
●●
● ●●
●●
● ●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
● ●
●●
●
●
●
●● ●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●●●●
●●●
●●
●● ●●
●
●●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
● ●●●
●●●
●●●
●
●●
●
●●
● ●
●
●
●
●●● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●●
●●
●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●●
●●
●●
●
●●
●●
●
●●
●●
●●
●●
●
●●
●●●
●●● ●●
● ●
●●●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●●
●
●●
●● ●●
●
●
●
●
●●●
●●
●
●●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●●
●
●●
●
●●
●●
●
●
●
●
●
● ●●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●●●
● ●● ●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●● ●●●
●●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
● ●
●●
● ●
●
● ●
●
●●●●
● ●●
●●●●
●
●
●
●
● ●●●
●●
●●
●
●
●
●●●
●
● ●● ●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●●●
●●● ●
●
●
●
●●
●
●
●●
●
● ●
●●
●
●●
●
●●
●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●●
●
●
●●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●●
●
●●
●●
●
●
●●
●
●
● ●
●●
●●
●
●
●●
●
●
●● ●
●●
●●
●●●
●
●●
● ●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
● ●●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
● ●
●
●
●●
●
●●
●●
●●
●●
●
● ●●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●● ●
●●
● ●
●●
●
●●●
●
●
●
● ●
●
●
●
● ●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●● ●●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●
●●
●●
●
●●
●
●
●●
● ●
●
●
●●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●●● ●
●
●●
●
●●●
●
● ●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
● ●●
●
●
● ● ●●●
●●
●●
●●
●
●
●●
●
●
●●●
●
●●● ●
●
● ●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●●
●●
●
●
●
●
● ●●
● ●
●●
●●
●●
●●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●●
●
●
●
![Page 52: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/52.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Question
I Remember that a Generalized Linear Model predicts themean of the outcome as a linear combination.
I In the previous figure, what does ‘mean’ mean here?
![Page 53: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/53.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interactions
I Interactions are products of predictors.
I Significant interactions tell us that the slope of apredictor differs for different values of the otherpredictor.
> l.lexdec1 = lm(RT ~ 1 + Frequency + FamilySize ++ NativeLanguage + Frequency:NativeLanguage,+ data=lexdec)Residuals:
Min 1Q Median 3Q Max-0.66925 -0.14917 -0.02800 0.11626 1.06790
Coefficients:Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.441135 0.031140 206.847 < 2e-16Frequency -0.023536 0.007079 -3.325 0.000905FamilySize -0.015655 0.008839 -1.771 0.076726NativeLanguageOther 0.286343 0.042432 6.748 2.06e-11Frequency:NatLangOther -0.027472 0.008626 -3.185 0.001475
![Page 54: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/54.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Question
I On the previous slide, how should we interpret theinteraction?
![Page 55: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/55.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interaction: Frequency & Native Language
Predicting Lexical Decision RTs
Interaction with Native Speakers (red) andNon−Native Speakers (blue)
1 2 3 4 5 6 7 8
5.5
6.0
6.5
7.0
7.5
8.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Word Frequency (log−transformed)
Num
ber
of m
orph
. fam
ily m
embe
rs (
log−
tran
sfor
med
)
Res
pons
e la
tenc
y (in
log−
tran
sfor
med
mse
cs)
●●
●
●●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●●
●
● ●●
● ●●
●●
●●●
●
●
●●●
●
●●
●●●●
●●●●
●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●●●
●
● ●
●
●
● ●
●
●
●
●
●●● ●
●
●
●
●
●●●
●
●●
●
●● ●
●
●
●●
●
●
●●
●
●● ●
●●
●
●
●
●● ●
●
●
●
●● ●●●● ●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●● ●
● ●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●●●
●
●●
●● ●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●
●● ●
●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●●●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●●
●
●●
●●
●●
●●●
●
●●●
●●●
● ● ●● ●
●
●●
● ●●
●●
● ●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
● ●
●●
●
●
●
●● ●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●●●●
●●●
●●
●● ●●
●
●●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
● ●●●
●●●
●●●
●
●●
●
●●
● ●
●
●
●
●●● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●●
●●
●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●●
●●
●●
●
●●
●●
●
●●
●●
●●
●●
●
●●
●●●
●●● ●●
● ●
●●●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●●
●
●●
●● ●●
●
●
●
●
●●●
●●
●
●●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●●
●
●●
●
●●
●●
●
●
●
●
●
● ●●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●●●
● ●● ●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●● ●●●
●●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
● ●
●●
● ●
●
● ●
●
●●●●
● ●●
●●●●
●
●
●
●
● ●●●
●●
●●
●
●
●
●●●
●
● ●● ●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●●●
●●● ●
●
●
●
●●
●
●
●●
●
● ●
●●
●
●●
●
●●
●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●●
●
●
●●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●●
●
●●
●●
●
●
●●
●
●
● ●
●●
●●
●
●
●●
●
●
●● ●
●●
●●
●●●
●
●●
● ●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
● ●●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
● ●
●
●
●●
●
●●
●●
●●
●●
●
● ●●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●● ●
●●
● ●
●●
●
●●●
●
●
●
● ●
●
●
●
● ●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●● ●●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●
●●
●●
●
●●
●
●
●●
● ●
●
●
●●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●●● ●
●
●●
●
●●●
●
● ●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
● ●●
●
●
● ● ●●●
●●
●●
●●
●
●
●●
●
●
●●●
●
●●● ●
●
● ●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●●
●●
●
●
●
●
● ●●
● ●
●●
●●
●●
●●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●●
●
●
●
![Page 56: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/56.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Linear Model vs. ANOVA
I Shared with ANOVA:I Linearity assumption (though many types of
non-linearity can be investigated)I Assumption of normality, but part of a more general
framework that extends to other distribution in aconceptually straightforward way.
I Assumption of independenceNB: ANOVA is linear model with categorical predictors.
I Differences:I Generalized Linear ModelI Consistent and transparent way of treating continuous
and categorical predictors.I Regression encourages a priori explicit coding of
hypothesis → reduction of post-hoc tests → decrease offamily-wise error rate.
![Page 57: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/57.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Hypothesis testing in psycholinguisticresearch
I Typically, we make predictions not just about theexistence, but also the direction of effects.
I Sometimes, we’re also interested in effect shapes(non-linearities, etc.)
I Unlike in ANOVA, regression analyses reliably testhypotheses about effect direction, effect shape, andeffect size without requiring post-hoc analyses if (a)the predictors in the model are coded appropriately and(b) the model can be trusted.
I cf. tomorrow
![Page 58: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/58.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
I Experiments don’t have just one participant.I Different participants may have different idiosyncratic
behavior.I And items may have idiosyncratic properties, too.
→ Violations of the assumption of independence!
NB: There may even be more clustered (repeated) propertiesand clusters may be nested (e.g. subjects ε dialects εlanguages).
I We’d like to take these into account, and perhapsinvestigate them.
→ Generalized Linear Mixed or Multilevel Models(a.k.a. hierarchical, mixed-effects).
![Page 59: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/59.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall: Generalized Linear Models
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Predictors
Model parameters
Response
![Page 60: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/60.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall: Generalized Linear Models
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Predictors
Model parameters
Response
![Page 61: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/61.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall: Generalized Linear Models
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Predictors
Model parameters
Response
![Page 62: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/62.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall: Generalized Linear Models
The picture:
θ
x1
y1
x2
y2
xn
yn· · ·
Predictors
Model parameters
Response
![Page 63: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/63.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
θ
Σb
b1 b2 bM· · ·
· · ·
x11 x1n1
y11 y1n1
· · ·
· · ·
x21 x2n2
y21 y2n2
· · ·
· · ·
xM1 xMnM
yM1 yMnM
· · ·
· · ·
Cluster-specificparameters
(“random effects”)
Shared parameters(“fixed effects”)
Parameters governinginter-cluster variability
![Page 64: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/64.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
θ
Σb
b1 b2 bM· · ·
· · ·
x11 x1n1
y11 y1n1
· · ·
· · ·
x21 x2n2
y21 y2n2
· · ·
· · ·
xM1 xMnM
yM1 yMnM
· · ·
· · ·
Cluster-specificparameters
(“random effects”)
Shared parameters(“fixed effects”)
Parameters governinginter-cluster variability
![Page 65: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/65.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
θ
Σb
b1 b2 bM· · ·
· · ·
x11 x1n1
y11 y1n1
· · ·
· · ·
x21 x2n2
y21 y2n2
· · ·
· · ·
xM1 xMnM
yM1 yMnM
· · ·
· · ·
Cluster-specificparameters
(“random effects”)
Shared parameters(“fixed effects”)
Parameters governinginter-cluster variability
![Page 66: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/66.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
θ
Σb
b1 b2 bM· · ·
· · ·
x11 x1n1
y11 y1n1
· · ·
· · ·
x21 x2n2
y21 y2n2
· · ·
· · ·
xM1 xMnM
yM1 yMnM
· · ·
· · ·
Cluster-specificparameters
(“random effects”)
Shared parameters(“fixed effects”)
Parameters governinginter-cluster variability
![Page 67: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/67.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Generalized Linear Mixed Models
θ
Σb
b1 b2 bM· · ·
· · ·
x11 x1n1
y11 y1n1
· · ·
· · ·
x21 x2n2
y21 y2n2
· · ·
· · ·
xM1 xMnM
yM1 yMnM
· · ·
· · ·
Cluster-specificparameters
(“random effects”)
Shared parameters(“fixed effects”)
Parameters governinginter-cluster variability
![Page 68: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/68.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Linear Model
I Back to our lexical-decision experiment:I A variety of predictors seem to affect RTs, e.g.:
I FrequencyI FamilySizeI NativeLanguageI Interactions
I Additionally, different participants in your study mayalso have:
I different overall decision speedsI differing sensitivity to e.g. Frequency.
I You want to draw inferences about all these things atthe same time
![Page 69: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/69.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Linear Model
I Back to our lexical-decision experiment:I A variety of predictors seem to affect RTs, e.g.:
I FrequencyI FamilySizeI NativeLanguageI Interactions
I Additionally, different participants in your study mayalso have:
I different overall decision speedsI differing sensitivity to e.g. Frequency.
I You want to draw inferences about all these things atthe same time
![Page 70: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/70.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Linear Model
I Back to our lexical-decision experiment:I A variety of predictors seem to affect RTs, e.g.:
I FrequencyI FamilySizeI NativeLanguageI Interactions
I Additionally, different participants in your study mayalso have:
I different overall decision speedsI differing sensitivity to e.g. Frequency.
I You want to draw inferences about all these things atthe same time
![Page 71: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/71.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Linear Model
I Random effects, starting simple: let each participant ihave idiosyncratic differences in reaction times (RTs)
RTij = α + βxij +
∼N(0,σb)︷︸︸︷bi +
Noise∼N(0,σε)︷︸︸︷εij
![Page 72: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/72.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed linear model with one randomintercept
I Idea: Model distribution of subject differences asdeviation from grand mean.
I Mixed models approximate deviation by fitting a normaldistribution.
I Grand mean reflected in ordinary intercept
→ By-subject mean can be set to 0→ Only parameter fit from data is variance.
> lmer.lexdec0 = lmer(RT ~ 1 + Frequency ++ (1 | Subject), data=lexdec)
![Page 73: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/73.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interpretation of the output
RTij = α + βxij +
∼N(0,σb)︷︸︸︷bi +
Noise∼N(0,σε)︷︸︸︷εij
I Interpretation parallel to ordinary regression models:
Formula: RT ~ 1 + Frequency + (1 | Subject)Data: lexdecAIC BIC logLik deviance REMLdev
-844.6 -823 426.3 -868 -852.6Random effects:Groups Name Variance Std.Dev.Subject (Intercept) 0.024693 0.15714Residual 0.034068 0.18457Number of obs: 1659, groups: Subject, 21
Fixed effects:Estimate Std. Error t value
(Intercept) 6.588778 0.026981 244.20Frequency -0.042872 0.003555 -12.06
![Page 74: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/74.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
MCMC-samplingI t-value anti-conservative
→ MCMC-sampling of coefficients to obtain nonanti-conservative estimates
> pvals.fnc(lmer.lexdec0, nsim = 10000)
$fixedEstimate MCMCmean HPD95lower HPD95upper pMCMC Pr(>|t|)
(Intercept) 6.5888 6.5886 6.5255 6.6516 0.0001 0Frequency -0.0429 -0.0428 -0.0498 -0.0359 0.0001 0
$randomGroups Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upper
1 Subject (Intercept) 0.1541 0.1188 0.1205 0.0927 0.15162 Residual 0.1809 0.1817 0.1818 0.1753 0.1879
Posterior Values
Den
sity
05
10
6.45 6.50 6.55 6.60 6.65 6.70
(Intercept)
020
4060
8010
0
−0.055 −0.045 −0.035
Frequency
05
1015
2025
0.10 0.15 0.20
Subject (Intercept)
020
4060
8010
012
0
0.170 0.175 0.180 0.185 0.190 0.195
sigma
![Page 75: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/75.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interpretation of the output
I So many new things! What is the output of the linearmixed model?
I estimates of coefficients for fixed and randompredictors.
I predictions = fitted values, just as for ordinaryregression model.
> cor(fitted(lmer.lexdec0), lexdec$RT)^2
[1] 0.4357668
![Page 76: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/76.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interpretation of the output
I So many new things! What is the output of the linearmixed model?
I estimates of coefficients for fixed and randompredictors.
I predictions = fitted values, just as for ordinaryregression model.
> cor(fitted(lmer.lexdec0), lexdec$RT)^2
[1] 0.4357668
![Page 77: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/77.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models vs. ANOVA
I Mixed models inherit all advantages fromGeneralized Linear Models.
I Unlike the ordinary linear model, the linear mixed modelnow acknowledges that there are slower and fastersubjects.
I This is done without wasting k − 1 degrees of freedomon k subjects. We only need one parameter!
I Unlike with ANOVA, we can actually look at therandom differences (→ individual differences).
![Page 78: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/78.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with one random intercept
I Let’s look at the by-subject adjustments to theintercept. These are called Best Unbiased LinearPredictors (BLUPs)
I BLUPs are not fitted parameters. Only one degree offreedom was added to the model. The BLUPs areestimated posteriori based on the fitted model.
P(bi |α̂, β̂, σ̂b, σ̂ε,X)
I The BLUPs are the conditional modes of the bi s—thechoices that maximize the above probability
![Page 79: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/79.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with one random intercept
I Let’s look at the by-subject adjustments to theintercept. These are called Best Unbiased LinearPredictors (BLUPs)
I BLUPs are not fitted parameters. Only one degree offreedom was added to the model. The BLUPs areestimated posteriori based on the fitted model.
P(bi |α̂, β̂, σ̂b, σ̂ε,X)
I The BLUPs are the conditional modes of the bi s—thechoices that maximize the above probability
![Page 80: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/80.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with one random intercept
NB: By-subjects adjustments are assumed to sum to zero,but they don’t necessarily do so (here: −2.3E-12).
head(ranef(lexdec.lmer0))
$Subject(Intercept)
A1 -0.082668694A2 -0.137236138A3 0.009609997C -0.064365560D 0.022963863...
![Page 81: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/81.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with one random intercept
I Observed and fitted values of by-subject means.
> p = exp(as.vector(unlist(coef(lmer.lexdec0)$Subject)))> text(p, as.character(unique(lexdec$Subject)), col = "red")> legend(x=2, y=850, legend=c("Predicted", "Observed"),+ col=c("blue", "red"), pch=1)
5 10 15 20
500
600
700
800
900
Subject as random effect
Subject Index
Res
pons
e la
tenc
y in
mse
cs
A1
A2
A3
C
D
I J
KM1
M2
P
R1
R2
R3S T1
T2
V
W1
W2
Z
A1
A2
A3
C
D
I J
KM1
M2
P
R1
R2
R3S T1
T2
V
W1
W2
Z
●
●
PredictedObserved
![Page 82: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/82.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with more random intercepts
I Unlike with ANOVA, the linear mixed model canaccommodate more than one random intercept, if wethink this is necessary/adequate.
I These are crossed random effects.
> lexdec.lmer1 = lmer(RT ~ 1 + (1 | Subject) + (1 | Word),+ data = lexdec)> ranef(lmer.lexdec1)
$Word(Intercept)
almond 0.0164795993ant -0.0245297186apple -0.0494242968apricot -0.0410707531...$Subject
(Intercept)A1 -0.082668694A2 -0.137236138A3 0.009609997
![Page 83: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/83.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with more random intercepts
I Shrinkage becomes even more visible for fitted by-wordmeans
0 20 40 60 80
550
600
650
700
750
Word as random effect
Word Index
Res
pons
e la
tenc
y in
mse
cs
owl
mole
cherry
pear
dog
blackberry
strawberrysquirrel
radish
almond
whale
reindeer
blueberrylemonbeegoat
eggplant
tomato
peanut
bat
mustard
magpie
stork
goose
squid
carrot
crocodile
tortoisespidergrape
foxpotato
frog
beetroot
asparagus
mouse
dolphin
pineapple
lettucesheep
cucumber
gherkin
gull
pig
camel
shark
vulture
snake
donkey
broccoliapricot
leek
clovebanana
lion
elephant
hedgehog
eagle
apple
kiwi
wasp
cat
paprika
antbunny
beaverhorse
mushroom
woodpecker
moose
orange
avocado
chicken
butterfly
olive
swan
walnut
melon
monkey
v
^
^
^
v
v
^^
v
^
v
vvvvv
v
^
^
^
^
v
vv
^
^
^
^^v
^^
v
^
^
^
v
v
^^
v
^
^
^
v
^
v
^
^
^^
v
^^
^
v
^
^
^
v
^
v
v
^^
^^
v
v
v
^
^
^
v
v
v
v
^
vv
^
^
^
v
v
^^
v
^
v
vvvvv
v
^
^
^
^
v
vv
^
^
^
^^v
^^
v
^
^
^
v
v
^^
v
^
^
^
v
^
v
^
^
^^
v
^^
^
v
^
^
^
v
^
v
v
^^
^^
v
v
v
^
^
^
v
v
v
v
^
v
![Page 84: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/84.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with random slopes
I Not only the intercept, but any of the slopes (of thepredictors) may differ between individuals.
I For example, subjects may show different sensitivity toFrequency:
> lmer.lexdec2 = lmer(RT ~ 1 + Frequency ++ (1 | Subject) + (0 + Frequency | Subject) ++ (1 | Word),+ data=lexdec)
Random effects:Groups Name Variance Std.Dev.Word (Intercept) 0.00295937 0.054400Subject Frequency 0.00018681 0.013668Subject (Intercept) 0.03489572 0.186804Residual 0.02937016 0.171377Number of obs: 1659, groups: Word, 79; Subject, 21
Fixed effects:Estimate Std. Error t value
(Intercept) 6.588778 0.049830 132.22Frequency -0.042872 0.006546 -6.55
![Page 85: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/85.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed models with random slopes
I The BLUPs of the random slope reflect the by-subjectadjustments to the overall Frequency effect.
> ranef(lmer.lexdec2)
$Word(Intercept)
almond 0.0164795993ant -0.0245297186...$Subject
(Intercept) FrequencyA1 -0.1130825633 0.0020016500A2 -0.2375062644 0.0158978707A3 -0.0052393295 0.0034830009C -0.1320599587 0.0143830430D 0.0011335764 0.0038101993I -0.1416446479 0.0029889156...
![Page 86: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/86.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed model vs. ANOVA
I A mixed model with random slopes for all its predictors(incl. random intercept) is comparable in structure toan ANOVA
I Unlike ANOVA, random effects can be fit for severalgrouping variables in one single model.
→ More power (e.g. Baayen 2004; Dixon, 2008).
I No nesting assumptions need to be made (for examplesof nesting in mixed models, see Barr, 2008 and hisblog). As in the examples, so far, random effects can becrossed.
I Assumptions about variance-covariance matrix can betested
I No need to rely on assumptions (e.g. sphericity).I Can test whether specific random effect is needed
(model comparison).
![Page 87: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/87.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Random Intercept, Slope, and Covariance
I Random effects (e.g. intercepts and slopes) may becorrelated.
I By default, R fits these covariances, introducingadditional degrees of freedom (parameters).
I Note the simpler syntax.
> lmer.lexdec2 = lmer(RT ~ 1 + Frequency ++ (1 + Frequency | Subject) ++ (1 | Word),+ data=lexdec)
![Page 88: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/88.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Random Intercept, Slope, and Covariance
Random effects:Groups Name Variance Std.Dev. CorrWord (Intercept) 0.00296905 0.054489Subject (Intercept) 0.05647247 0.237639
Frequency 0.00040981 0.020244 -0.918Residual 0.02916697 0.170783Number of obs: 1659, groups: Word, 79; Subject, 21
Fixed effects:Estimate Std. Error t value
(Intercept) 6.588778 0.059252 111.20Frequency -0.042872 0.007312 -5.86
I What do such covariance parameters mean?
![Page 89: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/89.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Covariance of random effects: An example
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
−0.2 0.0 0.2 0.4 0.6
−0.
04−
0.03
−0.
02−
0.01
0.00
0.01
0.02
Random Effect Correlation
(Intercept)
Fre
quen
cy
![Page 90: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/90.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Plotting Random Effects: Example
I Plotting random effects sorted by magnitude of firstBLUP (here: intercept) and with posteriorvariance-covariance of random effects conditional on theestimates of the model parameters and on the data.
> dotplot(ranef(lmer.lexdec3, postVar=TRUE))
M1
K
A2
J
W1
R1
I
C
A1
T1
S
A3
R3
D
R2
W2
P
V
M2
Z
T2
−0.4 −0.2 0.0 0.2 0.4 0.6
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
(Intercept)
−0.4 −0.2 0.0 0.2 0.4 0.6
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Frequency
![Page 91: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/91.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Plotting Random Effects: ExampleI Plotted without forcing scales to be identical:
> dotplot(ranef(lmer.lexdec3, postVar=TRUE),+ scales = list(x =+ list(relation = 'free')))[["Subject"]]
M1
K
A2
J
W1
R1
I
C
A1
T1
S
A3
R3
D
R2
W2
P
V
M2
Z
T2
−0.4 −0.2 0.0 0.2 0.4 0.6
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
(Intercept)
−0.06 −0.04 −0.02 0.00 0.02 0.04
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Frequency
![Page 92: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/92.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Plotting Random Effects: Example
I Plotting observed against theoretical quantiles:
Standard normal quantiles
−0.
4−
0.2
0.0
0.2
0.4
0.6
−2 −1 0 1 2
●
●
●
●● ●
● ●●
● ● ● ● ●
● ●
●
●
●
●
●
(Intercept)
−2 −1 0 1 2
−0.
06−
0.04
−0.
020.
000.
020.
04
●
●●
●
●
●● ● ●
● ● ● ● ●
●● ●
●●
●
●
Frequency
![Page 93: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/93.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Is the Random Slope Justified?
I One great feature of Mixed Models is that we canassess whether a certain random effect structure isactually warranted given the data.
I Just as nested ordinary regression models can becompared (cf. stepwise regression), we can comparemodels with nested random effect structures.
I Here, model comparison shows that the covarianceparameter of lmer.lexdec3 significantly improves themodel compared to lmer.lexdec2 with both therandom intercept and slope for subjects, but nocovariance parameter (χ2(1) = 21.6, p < 0.0001).
I The random slope overall is also justified (χ2(2) = 24.1,p < 0.0001).
→ Despite the strong correlation, the two random effectsfor subjects are needed (given the fixed effect predictorsin the model).
![Page 94: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/94.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interactions
> lmer.lexdec4b = lmer(RT ~ 1 + NativeLanguage * (+ Frequency + FamilySize + SynsetCount ++ Class) ++ (1 + Frequency | Subject) + (1 | Word),+ data=lexdec)[...]Fixed effects:
Estimate Std. Error t value(Intercept) 6.385090 0.030425 209.86cNativeEnglish -0.155821 0.060533 -2.57cFrequency -0.035180 0.008388 -4.19cFamilySize -0.019757 0.012401 -1.59cSynsetCount -0.030484 0.021046 -1.45cPlant -0.050907 0.015609 -3.26cNativeEnglish:cFrequency 0.032893 0.011764 2.80cNativeEnglish:cFamilySize 0.018424 0.015459 1.19cNativeEnglish:cSynsetCount -0.022869 0.026235 -0.87cNativeEnglish:cPlant 0.082219 0.019457 4.23
![Page 95: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/95.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interactions
> p.lmer.lexdec4b = pvals.fnc(lmer.lexdec4b,nsim=10000, withMCMC=T)> p.lmer.lexdec$fixed
Estimate MCMCmean HPD95lower HPD95upper pMCMC Pr(>|t|)(Intercept) 6.4867 6.4860 6.3839 6.5848 0.0001 0.0000NativeLanguageOther 0.3314 0.3312 0.1990 0.4615 0.0001 0.0000Frequency -0.0211 -0.0210 -0.0377 -0.0048 0.0142 0.0156FamilySize -0.0119 -0.0120 -0.0386 0.0143 0.3708 0.3997SynsetCount -0.0403 -0.0401 -0.0852 0.0050 0.0882 0.0920Classplant -0.0157 -0.0155 -0.0484 0.0181 0.3624 0.3767NatLang:Frequency -0.0329 -0.0329 -0.0515 -0.0136 0.0010 0.0006NatLang:FamilySize -0.0184 -0.0184 -0.0496 0.0109 0.2416 0.2366NatLang:SynsetCount 0.0229 0.0230 -0.0297 0.0734 0.3810 0.3866NatLang:Classplant -0.0822 -0.0825 -0.1232 -0.0453 0.0001 0.0000
![Page 96: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/96.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Visualizing an Interactions
2 3 4 5 6 7 8
6.2
6.3
6.4
6.5
6.6
6.7
6.8
Frequency
RT
English
Nat
iveL
angu
age
Other
![Page 97: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/97.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
MCMC
Posterior Values
Den
sity
02
46
8
6.3 6.4 6.5 6.6 6.7
(Intercept)
01
23
45
6
0.1 0.3 0.5
NativeLanguageOther
010
2030
40
−0.06 −0.02
Frequency
05
1015
2025
30
−0.06 −0.02 0.02
FamilySize
05
1015
−0.10 0.00
SynsetCount
05
1015
20
−0.05 0.00 0.05
Classplant
010
2030
40
−0.06 −0.02
NativeLanguageOther:Frequency
05
1015
2025
−0.05 0.00 0.05
NativeLanguageOther:FamilySize
05
1015
−0.10 0.00 0.10
NativeLanguageOther:SynsetCount
05
1015
20
−0.15 −0.050.00
NativeLanguageOther:Classplant
020
4060
0.02 0.04 0.06
Word (Intercept)
05
1015
2025
0.10 0.15 0.20
Subject (Intercept)
050
100
0.160 0.170 0.180
sigma
![Page 98: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/98.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Logit Model
I So, what do we need to change if we want toinvestigate, e.g. a binary (categorical) outcome?
![Page 99: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/99.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall that ...
logistic regression is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (2)
I The distribution around the mean is taken to bebinomial.
![Page 100: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/100.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Recall that ...
logistic regression is a kind of generalized linear model.
I The linear predictor:
η = α + β1X1 + · · ·+ βnXn
I The link function g is the logit transform:
E(Y) = p = g−1(η)⇔
g(p) = lnp
1− p= η = α + β1X1 + · · ·+ βnXn (2)
I The distribution around the mean is taken to bebinomial.
![Page 101: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/101.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Logit Models
I Mixed Logit Models are a type of Generalized LinearMixed Model.
I More generally, one advantage of the mixed modelapproach is its flexibility. Everything we learned aboutmixed linear models extends to other types ofdistributions within the exponential family (binomial,multinomial, poisson, beta-binomial, ...)
Caveat There are some implementational details (depending onyour stats program, too) that may differ.
![Page 102: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/102.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
An example
I The same model as above, but now we predict whetherparticipants’ answer to the lexical decision task wascorrect.
I Outcome: Correct vs. incorrect answer (binomialoutcome)
I Predictors: same as above
> lmer.lexdec.answer4 = lmer(Correct == "correct" ~ 1 ++ NativeLanguage * (+ Frequency + FamilySize + SynsetCount ++ Class) ++ (1 + Frequency | Subject) + (1 | Word),+ data=lexdec, family="binomial")
NB: The only difference is the outcome variable and thefamily (assumed noise distribution) now is binomial (wedidn’t specify it before because ”gaussian” is thedefault).
![Page 103: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/103.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Mixed Logit Output
[...]AIC BIC logLik deviance495 570.8 -233.5 467
Random effects:Groups Name Variance Std.Dev. CorrWord (Intercept) 0.78368 0.88526Subject (Intercept) 2.92886 1.71139
Frequency 0.11244 0.33532 -0.884Number of obs: 1659, groups: Word, 79; Subject, 21
Fixed effects:Estimate Std. Error z value Pr(>|z|)
(Intercept) 4.3612 0.3022 14.433 < 2e-16 ***cNativeEnglish 0.2828 0.5698 0.496 0.61960cFrequency 0.6925 0.2417 2.865 0.00417 **cFamilySize -0.2250 0.3713 -0.606 0.54457cSynsetCount 0.8152 0.6598 1.235 0.21665cPlant 0.8441 0.4778 1.767 0.07729 .cNativeEnglish:cFrequency 0.2803 0.3840 0.730 0.46546cNativeEnglish:cFamilySize -0.2746 0.5997 -0.458 0.64710cNativeEnglish:cSynsetCount -2.6063 1.1772 -2.214 0.02683 *cNativeEnglish:cPlant 1.0615 0.7561 1.404 0.16035
![Page 104: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/104.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interaction in logit space
2 3 4 5 6 7 8
23
45
6
Frequency
Cor
rect
==
"co
rrec
t"
English
Nat
iveL
angu
age
Other
![Page 105: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/105.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Interaction in probability space
2 3 4 5 6 7 8
0.86
0.88
0.90
0.92
0.94
0.96
0.98
1.00
Frequency
Cor
rect
==
"co
rrec
t"
English
Nat
iveL
angu
age
Other
![Page 106: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/106.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Why not ANOVA?
I ANOVA over proportion has several problems (cf.Jaeger, 2008 for a summary)
I Hard to interpret outputI Violated assumption of homogeneity of variances
![Page 107: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/107.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Why not ANOVA?
I These problems can be address via transformations,weighted regression, etc., But why should we do this isif there is an adequate approach that does not needfudging and has more power?
![Page 108: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/108.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Summary
I There are a lot of issues, we have not covered today (byfar most of these are not particular to mixed models,but apply equally to ANOVA).
I The mixed model approach has many advantages:I Power (especially on unbalanced data)I No assumption of homogeneity of variancesI Random effect structure can be explored, understood.I Extendability to a variety of distributional familiesI Conceptual transparencyI Effect direction, shape, size can be easily understood
and investigated.→ You end up getting another perspective on your data
![Page 109: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/109.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Summary
I There are a lot of issues, we have not covered today (byfar most of these are not particular to mixed models,but apply equally to ANOVA).
I The mixed model approach has many advantages:I Power (especially on unbalanced data)I No assumption of homogeneity of variancesI Random effect structure can be explored, understood.I Extendability to a variety of distributional familiesI Conceptual transparencyI Effect direction, shape, size can be easily understood
and investigated.→ You end up getting another perspective on your data
![Page 110: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/110.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Summary
I There are a lot of issues, we have not covered today (byfar most of these are not particular to mixed models,but apply equally to ANOVA).
I The mixed model approach has many advantages:I Power (especially on unbalanced data)I No assumption of homogeneity of variancesI Random effect structure can be explored, understood.I Extendability to a variety of distributional familiesI Conceptual transparencyI Effect direction, shape, size can be easily understood
and investigated.→ You end up getting another perspective on your data
![Page 111: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/111.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Modeling schema
![Page 112: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/112.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Two Methods
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I How do we fit the parameters βi and σ (choose modelcoefficients)?
I There are two major approaches (deeply related, yetdifferent) in widespread use:
I The principle of maximum likelihood: pick parametervalues that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 113: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/113.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Two Methods
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I How do we fit the parameters βi and σ (choose modelcoefficients)?
I There are two major approaches (deeply related, yetdifferent) in widespread use:
I The principle of maximum likelihood: pick parametervalues that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 114: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/114.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Two Methods
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I How do we fit the parameters βi and σ (choose modelcoefficients)?
I There are two major approaches (deeply related, yetdifferent) in widespread use:
I The principle of maximum likelihood: pick parametervalues that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
![Page 115: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/115.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Two Methods
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I How do we fit the parameters βi and σ (choose modelcoefficients)?
I There are two major approaches (deeply related, yetdifferent) in widespread use:
I The principle of maximum likelihood: pick parametervalues that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
P({βi}, σ|Y ) =P(Y |{βi}, σ)
Prior︷ ︸︸ ︷P({βi}, σ)
P(Y )
![Page 116: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/116.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Two Methods
RTij = α + βxij +
Noise∼N(0,σε)︷︸︸︷εij
I How do we fit the parameters βi and σ (choose modelcoefficients)?
I There are two major approaches (deeply related, yetdifferent) in widespread use:
I The principle of maximum likelihood: pick parametervalues that maximize the probability of your data Y
choose {βi} and σ that make the likelihoodP(Y |{βi}, σ) as large as possible
I Bayesian inference: put a probability distribution on themodel parameters and update it on the basis of whatparameters best explain the data
P({βi}, σ|Y ) =
Likelihood︷ ︸︸ ︷P(Y |{βi}, σ)
Prior︷ ︸︸ ︷P({βi}, σ)
P(Y )
![Page 117: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/117.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
Fitting Mixed Models
RTij = α + βxij +
∼N(0,σb)︷︸︸︷bi +
Noise∼N(0,σε)︷︸︸︷εij
I A couple of caveats about current implementations:I To avoid biased variance estimates, linear mixed models
are sometimes fit with Restricted MaximumLikelihood.
I There are no known analytic solutions to the likelihoodformula of mixed logit models. Instead LaplaceApproximation is used, which, however, provides adecent approximation (Harding and Hausman 2007). Inmodern implementations, this approximation can beimproved (at the cost of increased computational cost).
I Finally, and as for all models/analysis, statistics are onlya tool and, whether we can trust our results, depends onhow careful we use these tools → Tomorrow’s lecture.
![Page 118: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/118.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
RTij = α + βxij +
∼N(0,σb)︷︸︸︷bi +
Noise∼N(0,σε)︷︸︸︷εij
I Simulation of trial-level data can be invaluable forachieving deeper understanding of the data
## simulate some data> sigma.b <- 125 # inter-subject variation larger than> sigma.e <- 40 # intra-subject, inter-trial variation> alpha <- 500> beta <- 12> M <- 6 # number of participants> n <- 50 # trials per participant> b <- rnorm(M, 0, sigma.b) # individual differences> nneighbors <- rpois(M*n,3) + 1 # generate num. neighbors> subj <- rep(1:M,n)> RT <- alpha + beta * nneighbors + # simulate RTs!
b[subj] + rnorm(M*n,0,sigma.e) #
![Page 119: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/119.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
RTij = α + βxij +
∼N(0,σb)︷︸︸︷bi +
Noise∼N(0,σε)︷︸︸︷εij
I Simulation of trial-level data can be invaluable forachieving deeper understanding of the data
## simulate some data> sigma.b <- 125 # inter-subject variation larger than> sigma.e <- 40 # intra-subject, inter-trial variation> alpha <- 500> beta <- 12> M <- 6 # number of participants> n <- 50 # trials per participant> b <- rnorm(M, 0, sigma.b) # individual differences> nneighbors <- rpois(M*n,3) + 1 # generate num. neighbors> subj <- rep(1:M,n)> RT <- alpha + beta * nneighbors + # simulate RTs!
b[subj] + rnorm(M*n,0,sigma.e) #
![Page 120: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/120.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 2 4 6 8
400
600
800
1000
# Neighbors
RT
I Participant-level clustering is easily visibleI This reflects the fact that (simulated) inter-participant
variation (125ms) is larger than (simulated) inter-trialvariation (40ms)
I And the (simulated) effects of neighborhood density arealso visible
![Page 121: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/121.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 2 4 6 8
400
600
800
1000
# Neighbors
RT
Subject intercepts (alpha + beta[subj])
I Participant-level clustering is easily visibleI This reflects the fact that (simulated) inter-participant
variation (125ms) is larger than (simulated) inter-trialvariation (40ms)
I And the (simulated) effects of neighborhood density arealso visible
![Page 122: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/122.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 2 4 6 8
400
600
800
1000
# Neighbors
RT
Subject intercepts (alpha + beta[subj])
I Participant-level clustering is easily visibleI This reflects the fact that (simulated) inter-participant
variation (125ms) is larger than (simulated) inter-trialvariation (40ms)
I And the (simulated) effects of neighborhood density arealso visible
![Page 123: An Introduction to Linear and Logit Mixed Models - Day 1 · Generalized Linear Mixed Models Florian Jaeger Generalized Linear ... I I am also grateful for feedback from: I Austin](https://reader030.fdocuments.us/reader030/viewer/2022021801/5b1dc77a7f8b9ac6348bf16b/html5/thumbnails/123.jpg)
Generalized LinearMixed Models
Florian Jaeger
Generalized LinearModel
Graphical Model View
Theory
Linear Model
An Example
Geometrical Intuitions
Comparison to ANOVA
Generalized LinearMixed Model
Graphical Model View
Linear MixedModel
Getting an Intuition
Understanding MoreComplex Models
Mixed LogitModels
Summary
Extras
Fitting Models
A Simulated Example
A simulated example
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 2 4 6 8
400
600
800
1000
# Neighbors
RT
Subject intercepts (alpha + beta[subj])
I Participant-level clustering is easily visibleI This reflects the fact that (simulated) inter-participant
variation (125ms) is larger than (simulated) inter-trialvariation (40ms)
I And the (simulated) effects of neighborhood density arealso visible