1

1

Introduction to Analysis Methods for Longitudinal/Clustered Data, Part 1:

Unadjusted Tests for Paired Data

Mark A. Weaver, PhDFamily Health International

Office of AIDS Research, NIHICSSC, FHI

Goa, India, September 2009

2

2

Objectives Yesterday, we discussed methods for comparing

groups with independent data: Continuous or ordinal outcomes: randomization test,

t-test, Wilcoxon-Mann-Whitney non-parametric test Binary outcomes: randomization (Fishers exact) test,

chi-squared test Here, well discuss some simple corresponding

methods for paired data

3

3

What do I Mean by Paired Data?

Experimental units for which two related responses are made

Examples Left and right eye measurements from same person Husband and wife voting preferences Twins Same person in a cross-over design Units matched 1-1 on some criterion prior to

randomization

4

4

Why Different Methods for Paired Data?

Observations from the same pair tend to be more similar than observations from different pairs

That is, observations within pairs are correlated they do not each contribute independent information

Can also think of this as clustering at pair level But, correlation here tends to increase your power; the

more correlated paired outcomes are, more power Same is not true for clustered data that Mario will

discuss this afternoon

5

5

Example: Comparing Means

Compare 2 glaucoma treatments1. Surgery + new eyedrops2. Surgery alone

For each of 5 patients, randomize one eye to receive treatment 1, and the other treatment 2

Outcome: laser flare photon units/ms (lower better)

6

6

Example: Comparing Means

H0: 1 = 2 Difference in observed means is -4.6 Two-sample t-test: 2-sided p-value = 0.25 Whats wrong with this analysis?

30.626mean37 (r)35 (l)5

35 (l)30 (r)428 (l)25 (r)328 (l)20 (r)225 (r)20 (l)1Trt 2Trt 1Pt

7

7

Example: Comparing Means Note: we have paired data!

For each patient, eye receiving trt 1 did better Cant we use that information to perform more

appropriate analysis?

-4.630.626mean-237355-535304-328253-828202

-525201

DiffTrt 2Trt 1Pt

8

8

Paired T-Test Test the following hypothesis:

H0: 1 = 2 H0: 1 - 2 = 0 H0: = 0 where 1 and 2 are the population means for treatments 1 and 2, respectively.

Test statistic for testing these hypotheses with paired data is

where is the sample mean difference and sd is the sample standard deviation of the differences.

=

ns

dtd

d

9

9

Paired T-Test

For our data,= -4.6

sd = 2.3n = 5

t = -4.5 From t-distribution with 4 df, we find that 2-sided

p-value = 0.0111 So, reject H0! Right? But wait what are the assumptions of this

analysis?

d

10

10

Randomization Test Alternatively, we could use a randomization test

to compute an exact p-value for paired t statistic No assumptions other than randomization

For each patient, there are two possible random assignments: 1. left eye trt 1, right eye trt 22. left eye trt 2, right eye trt 1

There are 25 = 32 equally likely random assignments with these 5 patients

Under H0 of no trt difference, responses from each eye same regardless of treatment received

11

11

Randomization Test

Can calculate paired t statistic for each possible random assignment

Exact 2-sided p-value = 2/32 = 0.0625

0

1

2

3

4

5

6

7

8

9

10

-4.5 -3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5 4.5

12

12

Non-parametric Test

Could also use a non-parametric test In this case, Wilcoxon signed-rank test

Equivalent to the randomization test in most cases In fact, for this example exact 2-sided p-value = 0.0625

Which test to use? In large samples (e.g., n > 30), it doesnt matter too

much as they tend to converge Small samples, Wilcoxon signed-rank test preferred Paired t-test is directly related to confidence intervals

13

13

Paired Categorical Outcomes Example: Comparing 2 kinds of sunscreen 45 people enrolled, sunscreen will be applied to the arms Left and right arms randomized to type A or B

H0: P( burn | A) = P( burn | B) Observed proportions: PA 0.42; PB 0.22 McNemars test!

P-values: Asymptotic = 0.039; Exact = 0.064

453510Total26215No Burn19145Burn

No BurnBurn TotalType B

Type A

1

1

Introduction to Analysis Methods for Longitudinal/Clustered Data, Part 2:

Linear Mixed Models

Mark A. Weaver, PhDFamily Health International

Office of AIDS Research, NIHICSSC, FHI

Goa, India, September 2009

2

2

Objectives1. To develop a basic conceptual understanding of

what mixed models are and2. When they might be applicable3. With a focus on interpretation of results

Unfortunately, given extremely limited time, you wont learn how to actually apply them with the different software packages But, hopefully, this will give you a place to start

3

3

What are Linear Mixed Models?

Linear regression models that contain both fixed and random effects.

Relatively new tools useful for correlated continuous outcomes Nonlinear mixed models available for binary/categorical

outcomes GEE also useful tool for correlated binary/categorical data

4

4

Example Applications

Longitudinal/repeated measures data (same dependent variable measured over time for each subject)

Clustered designs (e.g., families, litters, siblings, hospitals, eyes, teeth, etc.)

Multivariate data (related, but different, outcomes from same subject)

5

5

Other Names for Linear Mixed Models

Hierarchical linear models Multilevel models Random coefficients models Growth curve models

6

6

Example Revisited Intervention: American Heart Association 8-week

School Program (among 3rd and 4th graders) Is the intervention effective to reduce BMI?

Y = BMI T = Treatment (0=Control, 1=Intervention)

New: 3 study visits1. Visit 1 baseline, pre-randomization2. Visit 2 8 weeks following start of intervention3. Visit 3 1 Year post-intervention

7

7

Broad or Wide Data Structure Broad structure one record per subject Required for old fashioned repeated measures

ANOVA methods.

0

1T

M15.815.415.7202

F18.316.417.4102

Other VARsSEXBMI3BMI2BMI1STID

8

8

Long Data Structure

Mixed models and GEE require long data structure One record per observation per subject True for all standard statistical packages

F118.33102M015.71202M015.42202M015.83202

Other VARs

F116.42102F117.41102

SEXTBMIVISITSTID

9

9

What About Subjects with Missing Data?Available BMI Observations

257306325930522593071

T = 1T = 0Visit

One benefit of mixed models and GEEBoth allow for use of all available data from each subjectNo imputation required

(e.g., no need for anything like LOCF)

10

10

BMI: Group Means by Visit

11

11

Questions of Interest

Primary question: Is intervention effective at reducing BMI over 8 weeks?

1. Re-expressed: Do the groups differ wrt change in mean BMI from baseline to post-intervention (V1 to V2), controlling for other important variables?

12

12

Questions of Interest

Secondary questions:2. Do the groups differ wrt change from baseline

to 1 year following intervention (V1 to V3), controlling for other important variables?

3. Do the groups differ wrt average change from baseline to the 2 follow-up visits, controlling for other important variables?

13

13

Review of Linear Regression Model Well start by reviewing Marios model

BMIi = 0 + 1* Ti + 2* B_BMIi + 3* METSUMi+ 4* B_AGEi + 5* MALEi + 6* URBANi + i

whereBMIi = observed BMI for ith subjectTi = Randomized treatment (0=Control, 1=Intervention)i = random error for the ith subject

In the model, s are all fixed (but unknown) constants, but i is a random variable.

i.e., s are fixed effects and i is a random effect So, in a sense, even standard linear regression models are

mixed models, although we dont usually think of them that way.

14

14

Review of Linear Regression Model

Common assumptions of this model:1.The i are normally distributed with mean 0 (i.e., E(i) = 0)

and common variance w2

i N(0, w2)

2.The i from any two observations are independent (i.e., Corr(i, j) = 0).

But, we actually have repeated (correlated) measurements from each subject

15

15

A Note on Adjusting for Baseline Response 3 ways to adjust for baseline response1. Include baseline response as a covariate (ANCOVA)

Model: Yj = 0 + 1*TRT + 2*Y0 + , j = 1, , K

2. Analyze change-from-baseline scores as the outcome Model: (Yj - Y0) = 0 + 1*TRT + 2*Y0 + , j = 1, , K

3. Include baseline response as another outcome in the model. Model: Yj = 0 + 1*TRT + , j = 0, , K

These 3 methods are not equivalent. My personal preference is for #3. Why? (Hint: missing data)

16

16

BMI for 2 Subjects in Intervention Group

Raw Means for Intervention Group

id = 106

id = 102

17

17

Subject Effects and Visit Effects Data from any one child are systematically

different from the population average. Subject 102s data all fall below the population line Subject 106s data all fall above the population line Data for some kids cross the population line

Kids differ from one another. We dont expect all children to grow at the same rate!

18

18

Subject Effects and Visit Effects

Therefore, we need to include subject effects in our model. Subject effects will be random why?

We also need to add terms for visits in our model to allow for change over time

19

19

Model with Subject and Visit Effects Added Now consider this model for BMI

BMIij = 0 + 1*Ti + 2*V2 + 3*V3 + 4*Ti *V2 + 5*Ti *V3+ (terms for control variables) + i + ij

whereBMIij = jth observation (j = 1, 2, 3) from ith childi = random effect for the ith childij = random error for jth observation from ith childthe s are fixed effects, i and ij are random effects

Assumptions: i N(0, b2)ij N(0, w2)

s and s are all mutually independent

20

20

Mixed Models in General

Mixed models actually specify both expected value (mean) models and correlation models

Important concept: 1. fixed effects contribute to the expected value

(mean) model2. random effects contribute to the correlation (or

variance/covariance) model.

21

21

Expected Value (Mean) ModelE(BMI) = 0 + 1*Ti + 2*V2 + 3*V3 + 4*Ti *V2 + 5*Ti *V3

Based on the specified model (and ignoring the control variables), the BMI means for each group at each visit are: Control GroupE(BMI | T=0, Visit = 1) = Ctl, 1 = 0E(BMI | T=0, Visit = 2) = Ctl, 2 = 0 + 2E(BMI | T=0, Visit = 3) = Ctl, 3 = 0 + 3

Intervention Group E(BMI | T=1, Visit = 1) = Int, 1 = 0 + 1E(BMI | T=1, Visit = 2) = Int, 2 = 0 + 1 + 2 + 4E(BMI | T=1, Visit = 3) = Int, 3 = 0 + 1 + 3 + 5

22

22

Correlation Model It is not difficult to show that this model implies

Var(BMIij) = Var(i) + Var(ij) = b2 + w2

Cov(BMIij, BMIik) = b2

Thus, the correlation between any two observations from the same child is b2 / (b2 + w2) Can call this the intra-cluster correlation (ICC). Correlation is the same regardless of how far apart

measurements are in time this is called compoundsymmetric correlation structure.

23

23

Lets Fit the Model! Use any appropriate software:

SAS: Proc MIXED (what I use) Stata: XTMIXED command SPSS: MIXED command

(or use point and click windows but be very careful that you understand what youre clicking and not clicking!)

24

24

Correlation Model Estimates

0.55Residual

13.60stid (student ID)EstimateCov Parm

Covariance Parameter Estimates

b2

w2

This model assumes constant BMI variance across the visits (i.e., as kids get older):

VAR( BMI ) = b2 + w2 = (13.60 + 0.55) = 14.15

And constant correlation between repeated measurements:

ICC = b2 / (b2 + w2) = 13.60 / (14.15) = 0.961

25

25

Mean Model Estimates

26

26

Answering Our Questions of Interest

Primary question: Do the groups differ wrt change in mean BMI from baseline to post-intervention?

H0: (Int, 2 Int, 1) (Ctl, 2 Ctl, 1) = 0

Plugging in the formulas for these conditional means (see slide 21), we find:(Int, 2 Int, 1) = (0+1+2+4) (0+1) = 2 + 4

(Ctl, 2 Ctl, 1) = (0+2) (0) = 2

So, (Int, 2 Int, 1) (Ctl, 2 Ctl, 1) = 4

27

27

Answering Our Questions of Interest

Thus, null hypothesis for the primary question reduces to H0: 4 = 0

Estimate of 4 = 0.16 (meaning?)

p-value = 0.068

28

28

Answering Our Questions of Interest

Secondary questions:2. Do the groups differ wrt change from visit 1 to

visit 3?

We can similarly show that test based on model parameters would reduce to H0: 5 = 0

Estimate of 5 = -0.23 (meaning?) p-value = 0.009

29

29

Answering Our Questions of Interest Secondary questions:

3. Do the groups differ wrt change from visit 1 to average of visit 2 and visit 3?

Exercise: Show that this question can be answered by testing H0: (4 + 5) / 2 = 0(See slide 21)

Stata, SPSS, and SAS allow testing hypotheses regarding linear combinations of parameters

Sometimes necessary for primary hypothesis!

Estimate = -0.197; p-value = 0.010

30

30

Alternative Correlation Structures

Recall that we just fit a model assuming compound symmetric correlation Assumes constant outcome variance over time Assumes constant within-subject correlations

Are these reasonable assumptions for this design?

Many different correlation structures available Ill show only one more that is typically

appropriate for RCTs

31

31

More Direct Approach to Model Correlation

Weve just fit a model with explicit random subject effects

Now consider this new model for BMIBMIij = 0 + 1*Ti + 2*V2 + 3*V3 + 4*Ti *V2 + 5*Ti *V3

+ (terms for control variables) + ij

where ij is the random error term for the ijth observation. ij are normally distributed with mean 0. However, now allow s for each child to be correlated,

and directly specify this correlation matrix

32

32

Compound Symmetry

1.00V3

1.00V2

1.00V1

V3V2V1

Correlation between any two observations from same child is the same no matter how far apart in time

This is exactly the same model that we just fit using random effects.

33

33

Unstructured

1.002313V3

231.0012V2

13121.00V1

V3V2V1

Correlation allowed to vary depending on visits

Additionally, variance is allowed to vary by visit

34

34

Correlation Model Estimates for New Model

1.000.960.95Visit 30.961.000.98Visit 20.950.981.00Visit 1

Visit 3Visit 2Visit 1

Correlations dont differ much at all (recall ICC was 0.96)

But, estimated variance of BMI increases over visit (from about 13 at Visits 1 and 2 to about 16 at Visit 3)

More realistic?

35

35

Mean Model Estimates for New Model

0.0010.29-1.01urban0.9540.300.02male0.0010.190.66b_age

0.8580.010.001metsum0.0300.10-0.23T*Vis30.0090.06-0.16T*Vis2

36

36

Answering Our Questions of Interest

0.0100.0083. Average change V1 to V2 and V3

0.0090.0302. Change V1 to V30.0680.0091. Change V1 to V2

p-value compound symmetry

p-value unstructured

modelQuestion of Interest

Oh no which results should we present?

Note: not necessarily which youd like to present!

37

37

Choosing Between Covariance Structures

Several methods are provided by software My method of choice, and also the easiest to

apply, is the Bayesian information criterion (BIC) Simply choose model with smallest BIC For compound symmetry, BIC = 6259 For unstructured, BIC = 6026 In this case, better model is also more significant

Suggested approach: specify a small number (2 or 3) of covariance models to compare, and also how youll choose, in the analysis plan before looking at the data

38

38

Questions?

Tab 06.00a_PairedTestsTab 06.01b_MixedModelsTab 06.02b_StatisticalTheory