Missing data - Analysis of repeated measurements...

u n i v e r s i t y o f c o p e n h a g e n d e p a r t m e n t o f b i o s t a t i s t i c s

Faculty of Health Sciences

Missing dataAnalysis of repeated measurements 2017

Julie Lyng Forman & Lene Theil Skovgaard

Department of Biostatistics, University of Copenhagen


Contents

◮ Planning statistical analyses with missing data.

◮ Missing data types.

◮ Bias due to missing data or improper handling of these.

◮ Analyzing data with missing values using multiple imputations,likelihood inference, or inverse probability weighting.

◮ Analysis of longitudinal studies with death or otherintercurrent events.

Suggested reading FLW (2011) chapters 18 +19, lecture notes.

2 / 60


Outline

What to worry about when you have missing data

Missing data types

Simple methods for handling missing data

Advanced methods for handling missing data

Missing data in population average models (binary data)

Death and other intercurrent events in longitudinal studies

Evaluation

3 / 60


What is missing data?

Most investigations are planned to be balanced but almostinevitably turn out to have intermittent missing values, orpatients who drop-out for some reason . . .

◮ Just by coincidence (sample lost or ruined).

◮ The patient moved away (may be worrysome).

◮ The patient has recovered (worrying, i.e. carryinginformation).

◮ The patient is too ill to show up (very serious, i.e. carryingunretrievable information).

4 / 60


Missing data is trouble

◮ It complicates statistical analysis.

◮ It may bias statistical results beyond repair.

◮ It compromises the causal interpretation of treatmenteffect in randomized trials.

◮ It reduces statistical power since information is lost.

The best way to handle missing data would be to prevent it,but this is often not possible.

Missing data should always be recognized as a limitation.

5 / 60


Planning statistical analyses with missing data

Missing data should be addressed already in the planning stage.

1. What are the outcomes and explanatory variables?

2. What are the parameters of interest (the study objective)?

3. Which variables may have missing values?

4. What are the likely reasons they are missing?

5. What other factors (auxiliary variables) could be associatedwith missingness? Are they also associated with the outcome?

This helps us decide:

6. What statistical methods should be used for analyses?

6 / 60


The missing data mechanism

It is important to understand WHY data is missing.

Investigate:

◮ If possible, ask the patients or investigators!

◮ Make separate spaghettiplots for completers and drop-outs.

◮ Make a table comparing the distribution of covariates andother characteristics between the drop-outs and thecompleters.

Speculate:

◮ Think about what differences there might be e.g. betweencompleters and drop-outs in terms of unmeasured outcomesand confounders.

◮ How could these affect the results of your analysis.7 / 60


Example: CKD study from lecture 1

More drop outs due to adverse events in Eplerenone group!8 / 60


Study objectives

What parameter is the target of the study or trial?

Mean of actual data had they all been collected, e.g.

◮ Change in mean over time of the entire study population.

◮ Difference in means between two populations at a given time.

◮ Difference in means between the initially randomizedtreatment groups regardless of what treatment subjectsactually received (intention to treat principle).

Mean of counterfactual data had they all been collected, e.g.

◮ Difference in means that would have been found if all subjectshad completed their assigned treatment.

◮ Difference in populations means if all had survived until end offollow-up.

9 / 60


FDA recommendations for clinical trials

In your study protocol please include a section describing how youplan to address missing data.

We recommend missing data be avoided by continuing to collect(efficacy and safety) data even from subjects who prematurelydiscontinue study drug.

Our preference is that the primary analysis 1) include all data, notjust data while adhering to study drug, and 2) for the limitedmissing data that do occur, it be represented by what theirresponse likely would have been had it been measured.

Because missing data tend to be associated with treatmentadherence, it would not be appropriate to have an analysis thatuses information from those with data who adhered to treatmentto describe what happened to those without data who did notadhere to treatment.10 / 60


Your own data

Think about the data from your own research project.

◮ Are any data missing?

◮ How many?

◮ Do you know WHY?

◮ Are the observed outcomes representative of the populationyou wanted to study, or different somehow?

◮ What exactly is your study objective, considering the missingdata?

11 / 60


Outline


Missing data types





Evaluation

12 / 60


Types and patterns of missingness

Missing data taxonomy

MCAR Missing completely at random.Unrelated to the outcome and the covariates.Note: Special case of MAR.

MAR Missing at random.Missingness is conditionally independent of themissing values given the observed data.

NMAR Not missing at random.Missingness is not conditionally independent of themissing values given the observed data.

Missing data patterns in longitudinal studies

Monotone Drops out and stays out.

Intermittent (aka non-monotone) Comes back later.13 / 60


MCAR: Missing completely at random

Examples:

◮ Sample lost in the mail.

◮ Some data too expensive/inconvenient to collect from thewhole sample, hence only collected for a random subsample.

Statistical consequences

◮ Reduced power due to reduced sample size.

◮ Data may end up being unbalanced (software problem - ?)

◮ Otherwise benign.

If missing data is MCAR, then the complete cases form a randomrepresentative subsample from the original study population.

14 / 60


Average curves

The average curve is representative of the whole population whendata is complete or missing data is MCAR.

◮ When missing data is MAR or NMAR it is likely biased.

Spaghettiplots are always ok.

15 / 60


Example of a missing mechanism MAR

Low values are good (e.g. blood pressure):

◮ When the patient learns he is doing well, he might decide heno longer needs to attend visits and staying away does notaffect his outcome.

Sample averages are biased. Mean estimates from LMM are ok.16 / 60


Example of a missing data mechanism NMAR

Low values are bad (e.g. lung function):

◮ When the person gets sufficiently ill, he drops out of thelabour market (healthy worker effect).

Sample averages are biased. Mean estimates from LMM too.17 / 60


MAR: Missing at random

R1 denotes the response indicator (1=observed, 0=missing).

Simplified DAG

potential drop outonly after baseline.

X //

��

''

Y0

��

Y1

��

// R1

~~

Y2

◮ Missingness may depend on past observed outcomes andcovariates included in the model, e.g. treatment.

◮ Missingness may not depend on current outcome neitherdirectly nor by means of unmeasured confounders.

◮ Future outcome of interest (de facto or counterfactual)after drop out may not depend on missingness.

18 / 60


MAR depends on the covariates

Assume a treatment-gender interaction (or -gene, or . . . ):

◮ Positive effect in women. Negative effect in men.

◮ Men are overall more likely to drop out.

An average positive change is found in the population if gender isnot included in the model and the interaction is not recognized!19 / 60


Case: Missing data in calcium study

Drop-outs tend to have lower BMD initially.20 / 60


Objectives of calcium study

Two possibilities for defining the target treatment effect:

1. Difference in de facto mean BMD at end of study for everyone.

2. Difference in mean BMD which would have been found hadeveryone completed their assigned treatment.

Which is closest to the effect of an intervention in the populationis not that obvious, since reasons for discontinuing treatment inreal life may be different from reasons for dropping out of thestudy. Presumably everyone tolerates both calcium and placebo socounterfactual outcomes are not unreasonable. The individualwants to know what can I expect happens to me if I take thistreatment not what happens if I don’t.

21 / 60


Case: Missing data in calcium study

Likely causes and effects of drop-out:

◮ We expect that the positive effect of calcium ceases when thegirl drops out of the trial (NMAR for target 1).

◮ Family moves away or too busy to participate (MCAR fortarget 2 unless related to unmeasured confounders).

◮ Parents learn at the visit that BMD is low and decides towithdraw because they think the girl is on placebo but needstreatment (MAR for target 2).

◮ Families with an unhealthy lifestyles are more likely towithdraw (MAR or NMAR for target 2, depending on whetherthe lifestyle factors are included as covariates in the model).

Note: What type of missing data we are dealing with has to beargued - it cannot be tested or otherwise assessed from the data.22 / 60


NMAR: Not missing at random

No statistical method can make up for data being NMAR.

It has been suggested to perform sensitivity analyses over a rangeof plausible models for the missingness / unobserved data.

However:

◮ All such models rely on unverifiable assumptions that cannever be checked with the data.

◮ We have very limited possibilities to perform such analyseswith statistical software.

23 / 60


Outline


Missing data types





Evaluation

24 / 60


Naive methods for handling missing data

Unwarranted approaches that are often substantially biased:

◮ Complete case analysis

◮ LOCF (or LVCF): Last observation (value) carried forward

◮ Mean value imputation.

◮ Predicted value imputation.

Beware: These methods are still popular among health scienceresearchers because they are so easy to use!

25 / 60


Complete case analysis

Make an analysis including only those individuals who areobserved at all available time points.

◮ Default choice for oldfashioned software (e.g. MANOVA).

◮ Valid under MCAR-assumption

Consequences:

◮ Likely biased if there are specific reasons for the missingness.

◮ Inefficient (reduced power) because partial information fromnon-completers is lost.

Use with caution

26 / 60


Per protocol analysis

Target:

◮ Estimate treatment effect in those subjects who are able tocomplete (tolerate) all of the treatments.

Suggested analysis:

◮ Complete case (possibly with additional exclusion of violators).

Limitations:

◮ Subjects may drop from e.g. active treatment and placebo fordifferent reasons (lack of effect vs adverse events).

◮ How do we identifiy those who tolerate all treatments?Run-in phase? Cross-over trial?

27 / 60


Last observation carried forward (LOCF)

If an individual has no observed value at time t, replace themissing value by the previously observed value.

◮ For drop-outs, all subsequent values will be identical.

Consequences:

◮ The estimated time effect is most likely biased.

◮ The natural variation is obscurred, so standard errors are mostlikely biased.

Definitely not recommended

28 / 60


LOCF in clinical trials

LOCF has been widely applied in clinical trials in the past.

The LOCF is the easiest imputation approach for missing data tobe understood by the non-statisticians. However, the LOCFapproach has been the target for criticisms from the statisticiansfor its lack of a sound statistical foundation and for its biases ineither direction (i.e., it is not necessarily conservative). After theNational Academies published its draft report "The prevention andtreatment of missing data in clinical trials”, using LOCF approachseemed to be out-dated and markedly out of step with modernstatistical thinking.

Reference: http://onbiostatistics.blogspot.dk/2012/05/is-last-observation-carried-forward.html

29 / 60


Predicted value imputation

Replace the missing values with more or less qualifiedguesses of what they might have been, e.g.

◮ Mean value imputation:Replace missing value with average over observed data.

◮ Model prediction imputation:Replace missing values with predicted values from regressionmodels including previous observations and other covariates.

Consequences:

◮ Estimates may be biased if the prediction model is not correct.

◮ Likely to underestimate SEs because imputed values aretreated as if they were the actual observations and becausepredictions are less variable than genuine observations.

Definitely not recommended30 / 60


Calcium: Predicted values for calcium drop-out

Predicted group mean / individual value based on the cLMM.31 / 60


Outline


Missing data types





Evaluation

32 / 60



Up-to-date methods:

◮ Likelihood inference (default in LMMs, e.g. proc mixed).

◮ Multiple imputations.

◮ Inverse probablility weighting (IPW).

Usually the best available options for handling missing data

◮ Valid under MAR (but does MAR hold?)

◮ Extensions to some particular NMAR scenarios.

BUT:

◮ Results aren’t as robust to modelmisspecification as withcomplete data. E.g. the normal distribution matters.

33 / 60


Residuals for LMMs

Raw residuals: Observed - Predicted.

◮ Note that variance may change with time.

Pearson residuals: (Obs - Pred) / SD(Obs).

◮ Same variance, but residuals on same subject are correlated.

Studentized residuals: (Obs - Pred) / SD(Obs-Pred).

◮ Same as Pearson residuals in large samples; takes estimationuncertainty in predictions into account when estimating SD.

Scaled residuals: vciry-option in proc mixed.

◮ Scales residuals by the square root of the inverse V-matrix.

◮ Independent and standard normal if the model is correct;also when there are missing data as long as MAR holds.

34 / 60


Goodness of fit, scaled residuals

Model: categorical-time effect and unstructured covariance.35 / 60


Likelihood inference

Perform the usual LMM analysis.

◮ In both SAS and R likelihood inference is default.

◮ All observations must be included, not just complete cases.

Properties:

◮ Valid under MAR if the model is correct.

◮ Efficient - makes optimal use of the available observations.

◮ Also applies to generalized linear mixed models fornon-normal outcomes (lecture 5).

◮ Not applicable to population average models (lecture 5).

Recommended.

36 / 60


The MAR assumption - again

The MAR assumption can never be verified from the data.

What you need to argue:

◮ MAR means that responses from subjects who remain arerepresentative of everyone in the study population who hassimilar characteristics (covariates) and a similar response untilthe time of drop out.

◮ WHY do subjects drop out?

But even better:

◮ Envision different NMAR-scenarios.

◮ Make sensitivity analyses to check how results are affected. . . as far as statistical software permits :(

37 / 60


Multiple imputations

Similar to predicted value imputation only random errorterms are added to predictions (and model parameters).

1. This is repeated M times and analysis is performed on eachimputed dataset.Recommendation: M = 5, 100, 1000, depending on who you ask!

2. Finally estimates are averaged and SEs are computedaccording to Rubin’s rule.

Properties

◮ Valid if the imputation model is correct.

◮ Applicable to MAR and some NMAR scenarios.

◮ But limited modeling possibilities in software.

Recommended.38 / 60


The imputation model I

Quantitative missing data after drop out are imputed sequentially

◮ conditionally on observed data before drop out,

◮ . . . and previous imputed values,

◮ . . . and other predictors (covariates, auxiliary variables).

from standard linear regression models:

Yi,j+1 = βj,0 + βj,1Yi,1 + . . . + βj,jYi,j + (other) + εi,j

If the outcome follows a multivariate normal distribution (given thepredictors), then the imputation model is consistent with this.

Technical note: Model parameters have to be estimated as an integral step inthe imputation procedure. This is technical and usually done using a so-calledMCMC-algorithm (estimation and imputation is alternated until convergence).To further reflect estimation uncertainty model parameters for each separateimputation are sampled with error. This is called proper imputations.39 / 60


The imputation model II

Multiple imputations are more difficult to perform with:

1. Data that is not quantitative or of mixed type,

2. Intermittent missing values or missing covariates.

since this involves

1. Using regression models of other/mixed type, e.g. logistic.

2. Repeating sequential imputations in a cyclic manner whileconditioning on all other input variables (other outcomes etc.).

Two different algorithms for this are available:

◮ The substantive model compatible fully conditionalspecification algorithm (R-package SMCFCS). Recommended.

◮ Multiple imputations by chained equations (R-package: mice,SAS: proc mi). Although not fully consistent with any multivariatemodel, the algorithm seems to be working well in practice.

40 / 60


Calcium study: sensitivity analyses

When a girl drops out of the calcium group . . .

◮ A. The positive effect of calcium persist after drop out; thegirl continues to gain in BMD like the remaining girls in thecalcium group (like in the counter factual world where shestayed on treatment).

◮ B. The girl will keep the gain in BMD she got while oncalcium, but any further gain she experiences after drop outwill be like in the placebo group.

◮ C. It is like she never had any calcium at all; at end of studyher BMD will be like if she had been in the placebo group allalong.

Optimistic - realistic - pessimistic in terms of treatment effect.

41 / 60


Calcium study: Multiple imputation plan.

◮ A. Standard multiple imputation are carried out for each ofthe two groups, i.e. by grp in SAS with proc mi.

◮ B. Drop outs are pooled with the placebo group and multipleimputations are performed for this group alone. Resulting datafrom each imputation is joined with the calcium completersbefore analysis.

◮ C. Same as B except that all follow-up responses are deletedfrom the calcium drop outs before pooling their data with theplacebo group.

◮ Imputed datasets are analyzed with the cLMM (proc mixed)and resulting output is synthisized with proc mianalyze.

A. is mplemented in calcium2_demo.sas42 / 60


Calcium: Multiply imputed data (scenario A)

43 / 60


Calcium study: 3 scenarios

44 / 60


Calcium: Results of sensitivity analyses

Method Estimate⋆ Std.Error P-value

complete case cLMM 18.99 6.53 0.0046

cLMM same covariance 18.95 6.26 0.0032

cLMM different covariances 18.79 6.30 0.0036

Multiple imputations A 18.72 6.37 0.0033

Multiple imputations B 16.74 6.27 0.0076

Multiple imputations C 16.04 6.44 0.0128

⋆ Difference in mean BMD-gain at final follow-up (calcium vs placebo).

Conclusion: Effect of calcium is an overalll robust finding unlesssome of the drop-outs are substantially different from other girls(any unmeasured confounders. . . ?).45 / 60


Agreement between LMM and multiple imputations

Mixed model results and multiple imputations agree if:

1. the MAR-assumption holds (as in scenario A).

2. the mixed model used for the analysis is the same as theimputation model; i.e. an unrestricted model for the mean andcovariance in each group seperately.

3. sample size is large.

In moderate and small samples multiple imputations tend to yieldconservative standard errors.

. . . but note that the p-values from proc mianalyze are based onthe normal approximation (no correction on degrees of freedom).

46 / 60


Outline


Missing data types





Evaluation

47 / 60


Case study: Amenorrhea (from lecture 5)

1151 women were randomized to a contracepting drug in either

◮ low dose of 100 mg (trt=0) / high dose of 150 mg (trt=1)

Are frequencies of amenorrhea biased due to missing data?

Analysis Variable : amenorrhea

N N

dose time Obs N Miss Mean Variance

------------------------------------------------------------------------------------

0 1 576 576 0 0.1857639 0.1515187

2 576 477 99 0.2620545 0.1937882

3 576 409 167 0.3887531 0.2382065

4 576 361 215 0.5013850 0.2506925

1 1 575 575 0 0.2052174 0.1633874

2 575 476 99 0.3361345 0.2236179

3 575 389 186 0.4935733 0.2506029

4 575 353 222 0.5354108 0.2494527

------------------------------------------------------------------------------------

48 / 60


Missing data in PA models

Missing data should not be ignored as the GEE-estimates maybecome biased.

E.g. we have a problem assessing the prevalences if

◮ Replicates are correlated; some women are more prone toexperience amenorrhea than others.

◮ Women with amenorrhea at the previous occation are morelikely to drop out.

◮ Then women with amenorrhea would be underrepresented atlater occations.

◮ Note that we are interested in the counterfactual prevalenceof amenorrhea had everyone completed the study. Presumablythe side effect ceases when a women comes off the drug, butthat is not what we are interested in.

49 / 60


Inverse probability weighting (IPW)

◮ Assign more weight to those who should be more inclined todrop out but actually stays in the trial (weighted GEE).

◮ If women with current disease are twice as likely to drop outthan those without, those who stay have to count for two.

Properties:

◮ Valid under MAR-assumption if both the model for themissingness and the model for the observations are correct.

◮ In case of monotone missingness, it suffices that one of thetwo models is correct (estimates are doubly robust).

◮ No good with small dataset or high prevalence of missing data.

Recommended when feasible.50 / 60


Modeling inverse probability weightsNeed to model the response indicators at each occation.

◮ Usually logistic regression including previous outcomes, etc.

◮ Cumulate predicted probabilities of response over occations

πij = P (Ri1 = 1) · . . . · P (Rij = 1)

◮ Input their inverses, wij = π−1

ij as weights in GEE(FLW chapter 19 describes how to do this in SAS).

Drop out according to treatment and previous outcome.

Occation2 dose amenorrhea1 N freq of drop out

-----------------------------------------------------------

0 0 469 0.16

1 107 0.21

1 0 457 0.15

1 118 0.26

------------------------------------------------------------51 / 60


Estimates with inverse probability weighting

IPW compared to complete case analysis.

Estimate GEE with IPWs Complete cases

-------------------------------------------------------------------------------

risk time 1 (low) 0.19 (0.16;0.22) 0.18 (0.14;0.22)

risk time 2 (low) 0.27 (0.23;0.31) 0.25 (0.21;0.30)

risk time 3 (low) 0.40 (0.35;0.44) 0.37 (0.32;0.42)

risk time 4 (low) 0.52 (0.46;0.57) 0.50 (0.45;0.55)

risk time 1 (high) 0.21 (0.17;0.24) 0.16 (0.13;0.20)

risk time 2 (high) 0.34 (0.30;0.39) 0.30 (0.25;0.35)

risk time 3 (high) 0.52 (0.47;0.57) 0.48 (0.43;0.53)

risk time 4 (high) 0.57 (0.52;0.62) 0.54 (0.48;0.59)

Complete case analysis underestimates prevalences of amenorrheaeven at first occation where no one has dropped out yet!

52 / 60


Outline


Missing data types





Evaluation

53 / 60


Intercurrent events

Non-existent data due to death is not the same as missing data.

In other circumstances collected data may not contain theinformation that was intended, e.g.

◮ data from non-adherent patients does not reflect optimal /true treatment efficacy.

◮ data after surgery does not reflect natural disease progression.

An intercurrent event is an event that occurs after treatmentinitiation and either preclude observation of the variable or affectsits interpretation⋆.

◮ How do we analyze data with intercurrent events - ???

⋆ ICH E9 (R1) addendum on estimands and sensitivity analyses in clinical trials

to the guideline on statistical principles for clinical trials. EMA, August 2017.

54 / 60


Dead is not missing

Scenario 1: Someone dies, no one drops out.◮ Mean of observed data is an unbiased estimate of the population

mean (since the population consists of the survivors).

Scenario 2: Someone drops out, no one dies.◮ Estimated mean from linear mixed model is unbiased,

(assuming MAR and correct model specification).55 / 60


Analyzing randomized studies with death

Potential difference in survival:

◮ Death must be considered a poor outcome.

◮ Compare summary statistics that identifies an early death as apoor outcome.

◮ E.g. survival time or AUC until death/end of follow-up.

No difference in survival:

◮ Evaluate treatment effect in survivors only.

◮ Compare summary statistics that identifies good/pooroutcome regardless of survival.

◮ E.g. Average outcome while alive.

Missing data can be handled by making multiple imputationsin strata defined by time of death.56 / 60


Case: Tumor growth in mice

Experimental treatment (n=8) compared to control (n=7).

ATT: Sacrifice mice when tumor volume exceeds 1000 mm3.

◮ Missing data is obviously MAR - why not impute?57 / 60


Case: Survival analysis

Outcome: time to sacrifice.

Alternatively: Time to doubling or quadrupling.58 / 60


Outline


Missing data types





Evaluation

59 / 60


Course evaluation

Your feedback is much appreciated!

60 / 60

Missing data - Analysis of repeated measurements...

Documents

Transcript of Missing data - Analysis of repeated measurements...