Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Flexible Discriminant Analysis Using

Multivariate Mixed Models

David Hughes

2015

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Outline

1. Motivation

2. Multivariate Generalized Linear Mixed Models (MGLMM)

3. Longitudinal Discriminant Analysis

4. ISDR Example

5. Conclusions

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.◮ Longitudinal

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.◮ Longitudinal◮ Multivariate

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.◮ Longitudinal◮ Multivariate◮ Different types of data

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.◮ Longitudinal◮ Multivariate◮ Different types of data◮ Complicated correlation structure

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Motivation

◮ Complex data.◮ Longitudinal◮ Multivariate◮ Different types of data◮ Complicated correlation structure

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Available Methods

◮ Univariate models using a classical linear mixed model (e.gBrant et al. (2003), Lix and Sajobi (2010), Tomasko et al.(1999) and Wernecke et al. (2004)).

◮ Fails to account properly for the dependence between markersin our case.

◮ Multivariate Models for continuous markers using multivariatemixed models (eg Morrell et al. (2012) using linear mixedmodels and Marshall et al. (2009) using non-linear mixedmodels).

◮ Not applicable if some of the markers are not continuous.

◮ Pairwise models for continuous and binary markers (Fieuws etal. (2008)).

◮ This method in principle is suitable for our purposes but in thistalk we outline a more flexible approach.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

A more flexible approach

◮ Typical assumption about the random effects distribution canbe relaxed by using a mixture of normal distributions (Komareket al. (2010)).

◮ This methodology only considers three continuous markers.

◮ Cluster Analysis with continuous, binary and count variableswith mixture distributions for the random effects is possible(Komarek and Komarekova (2013))

◮ In Cluster Analysis the groups are unknown whereas in our casegroups are known beforehand.

◮ Software is available in the mixAK package in R created byArnost Komarek.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Progress Map

Dataset for Analysis

Fitting of the multivariate mixed-effects model (MGLMM)

Discriminant model built using parameters of MGLMM

Allocate new patients to diagnostic groups

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Definitions

◮ Yi,r,j is the j‘th observation of the r ‘th marker for patient i andis measured at time ti,r,j .

◮ We consider r = 1, . . . , R markers on i = 1, . . . , N patients.

◮ Yi,r is a vector containing all observations of marker r forpatient i .

◮ Yi is a stacked vector containing all the observations of allmarkers for patient i .

◮ Distribution of each marker may depend on additionalcovariates such as time, Age, Gender.

◮ It is possible for each marker to be measured at different timepoints and a different number of times.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Multivariate Generalized Linear Mixed Models

◮ To allow for different types of marker we model each markerusing a generalised linear mixed model

h−1r [E (Yi,r |αr , bi,r )] = Xi,r αr + Zi,r bi,r (1)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Multivariate Generalized Linear Mixed Models

◮ To allow for different types of marker we model each markerusing a generalised linear mixed model

h−1r [E (Yi,r |αr , bi,r )] = Xi,r αr + Zi,r bi,r (1)

◮ hr is a link function used depending on the type of longitudinalmarker.

◮ αr is a vector of fixed parameters for marker r .

◮ bi,r is a vector of random effects for patient i for marker r (i.esubject specific parameters).

◮ X and Z are matrices containing covariate information for eachpatient.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Joint Distribution of the random effects

◮ The dependence between markers is captured by the jointdistribution of the random effects bi = (bi,1, . . . , bi,R),i = 1, . . . , N .

◮ The most common assumption is that the random effectsfollow a Normal distribution.

bi ∼ N(µ,D) (2)

◮ This assumption can be difficult to verify and additionalflexibility can be achieved by allowing a mixture of Normaldistributions.

bi ∼

K∑

k=1

wkN(µk ,Dk) (3)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Parameter Estimation

◮ We need to estimate the following parameters.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Parameter Estimation

◮ We need to estimate the following parameters.◮ Fixed effects α = (α1, . . . , αR )◮ Possible dispersion parameters φ = (φ1, . . . , φR)◮ Mixture weights w = (w1, . . . , wK )◮ Mean vector of random effects µ = (µ1, . . . , µK )◮ Covariance matrix of random effects (vec(D1), . . . , vec(DK ))

◮ In all, we need to estimate,

θ = (α, φ, w, µ, vec(D1), . . . , vec(DK )) (4)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

MCMC estimates

◮ Full maximum likelihood estimates are difficult to obtain dueto the complexity of the likelihood.

◮ We instead use a Bayesian approach based on MCMC.

◮ We utilise weakly informative priors and a block Gibbs sampler.

◮ A benefit of this method, not explored in this talk is thatcredible intervals for the group membership probabilities arereadily available. These could be incorporated into aclassification procedure in some cases.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Progress Map

Dataset for Analysis

Fitting of the multivariate mixed-effects model (MGLMM)

Discriminant model built using parameters of MGLMM

Allocate new patients to diagnostic groups

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Longitudinal Discriminant Analysis

◮ Fit MGLMM to data in each diagnostic group g , g = 1, . . . , G

to obtain MCMC parameter estimates, θg .

◮ Use the fitted GLMM model to derive the discriminant rulethat assigns the patients into two (or more) diagnostic groups.

◮ Let Pg ,new be the probability that a new observation Yi , isfrom group g .

◮ The prior probability of being in group g is denoted πg .

◮ Using Bayes rule it can be seen that

Pg ,new =πg fg ,new∑G−1

h=0 πh fh,new

(5)

◮ Assign new patients to disease group if Pdisease,new is

greater than a specified value. If not assign to the group

for which Pg ,new is largest.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Specifying the predictive density fg ,new

◮ Marginal Prediction

f margg ,new = p(ynew |θg ) (6)

◮ Conditional Prediction

f condg ,new = p(ynew |bnew = bg ,new , θg) (7)

◮ Random Effects Prediction

f randg ,new = p(bg ,new |θg) (8)

◮ These values are calculated using numerical integrationmethods such as Gauss Quadrature since they involve complexintegrals that cannot be solved analytically.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Diabetic Retinopathy example◮ Our motivation comes from the ISDR cohort study.

◮ We consider 12,628 patients with diabetes who were screenedbetween 2009 and 2013 for diabetic retinopathy .

◮ Various markers measured over time, HbA1c and Cholesterol

(continuous markers), retinopathy grading (treated as binarymarker), and number of GP visits (count variable).

◮ 600 patients had positive screening event within theobservation period.

Figure: Left: Image of diabetic eye without retinopathy. Right: Image ofdiabetic eye with late stage diabetic retinopathy (Kindly provided by Dr.Yalin Zheng).

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

◮ We consider two groups, 600 patients with a positive screeningevent (indicating STDR) and 12068 patients without.

◮ 80% of the patients in each group to train MGLMMs (one foreach group).

◮ 20% of patients to test the classification accuracy.

◮ End goal is to identify patients who will have a positivescreening event in one years time (so only consider datagathered up to one year before final visit.)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

We fit the following models:

E [log(HbA1c)] = α1Sex + α2Age + bi,0 + bi,1time (9)

E [log(Cholesterol)] = α3Sex + α4Age + α5time + bi,2 (10)

logE [Visit] = α6Sex + α7Age + α8time + bi,3 (11)

logitE [Grading ] = α9Sex + α10Age + bi,4 + bi,5time (12)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

Posterior Mean Standard Error Posterior Median 95% Credible IntervalNo STDR Group

α1 -2.74e-03 1.48e-06 -2.74e-03 (-3.03e-03,-2.46e-03)α2 -5.13e-03 3.73e-05 -5.14e-03 (-1.25e-02,2.07e-03) )α3 -3.27e-03 1.84e-06 -3.27e-03 (-3.65e-03,-2.93e-03) )α4 -8.76e-02 4.35e-05 -8.77e-02 (-9.61e-02,-7.9e-02)α5 -2.01e-05 2.24e-08 -2e-05 (-2.45e-05,-1.57e-05)α6 3.66e-03 3.97e-06 3.67e-03 (2.86e-03,4.44e-03)α7 -1.66e-02 1.01e-04 -1.65e-02 (-3.64e-02,3.03e-03)α8 2.99e-04 9.87e-08 2.99e-04 (2.8e-04,3.18e-04)α9 9.24e-03 2.6e-05 9.2e-03 (4.2e-03,1.45e-02)α10 1.2e-01 6.4e-04 1.2e-01 (-5.64e-03,2.45e-01)

STDR Groupα1 -6.76e-03 8.62e-06 -6.77e-03 (-8.4e-03,-5.05e-03)α2 -3.34e-02 2.45e-04 -3.33e-02 (-8.03e-02,1.49e-02)α3 -3.1e-03 7.47e-06 -3.12e-03 (-4.54e-03,-1.59e-03)α4 -8.25e-02 2.3e-04 -8.23e-02 (-1.27e-01,-3.79e-02)α5 -2.35e-05 1.71e-07 -2.33e-05 (-5.7e-05,1.05e-05)α6 9.07e-03 2.21e-05 9.1e-03 (4.77e-03,1.33e-02)α7 -2.65e-02 6.13e-04 -2.54e-02 (-1.46e-01,9.26e-02)α8 4.78e-04 5.97e-07 4.78e-04 (3.61e-04,5.95e-04)α9 -4.72e-03 1.09e-04 -4.51e-03 (-2.63e-02,1.58e-02)α10 -1.29e-01 3.43e-03 -1.24e-01 (-8.16e-01,5.33e-01)

Table: Posterior summary statistics for the fixed effects α in ourMGLMM.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

Posterior Mean Standard Error Posterior Median 95% Credible IntervalNo STDR Group

E [b0] 4.15 1.02e-04 4.15 (4.13,4.17)E [b1] 6.07e-06 3.37e-08 6.11e-06 (-5.18e-07,1.26e-05)E [b2] 1.68 1.24e-04 1.68 (1.65,1.7)E [b3] 5.1e-01 2.77e-04 5.1e-01 (4.55e-01,5.65e-01)E [b4] -2.96 1.97e-03 -2.96 (-3.35,-2.56)E [b5] -3.35e-04 6.69e-07 -3.36e-04 (-4.65e-04,-2.03e-04)

SD[b0] 2.71e-01 4.39e-05 2.71e-01 (2.63e-01,2.8e-01)SD[b1] 2.2e-04 5.85e-08 2.2e-04 (2.09e-04,2.32e-04)SD[b2] 1.83e-01 1.76e-05 1.83e-01 (1.8e-01,1.87e-01)SD[b3] 2.27e-01 7.16e-05 2.27e-01 (2.13e-01,2.41e-01)SD[b4] 2.54 1.27e-03 2.54 (2.3e,2.79)SD[b5] 9.46e-04 1.3e-06 9.5e-04 (6.91e-04,1.2e-03)

STDR GroupE [b0] 4.62 5.79e-04 4.62 (4.51,4.73)E [b1] 3.61e-05 1.88e-07 3.61e-05 (-8.23e-07,7.26e-05)E [b2] 1.65 4.96e-04 1.65 (1.55,1.74)E [b3] 3.05e-01 1.45e-03 3.01e-01 (2.17e-02,5.96e-01)E [b4] 3.81 7.94e-03 3.76 (2.38,5.44)E [b5] 8.66e-04 2.09e-06 8.88e-04 (2.82e-04,1.3e-03)

SD[b0] 3.05e-01 2.11e-04 3.04e-01 (2.64e-01,3.47e-01)SD[b1] 1.75e-04 2.55e-07 1.75e-04 (1.24e-04,2.24e-04)SD[b2] 2.06e-01 1.05e-04 2.05e-01 (1.85e-01,2.26e-01)SD[b3] 3.15e-01 3.69e-04 3.16e-01 (2.4e-01,3.86e-01)SD[b4] 2.75 3.63e-03 2.71 (2.16,3.59)SD[b5] 8.94e-04 1.59e-06 8.59e-04 (6.43e-04,1.3e-03)

Table: Posterior summary statistics for the means and standarddeviations of the random effects bi in our MGLMM.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

−2000 −1500 −1000 −500

3.5

4.0

4.5

5.0

Time (days)

Log(

HbA

1c)

No STDR Group

−2000 −1500 −1000 −500

3.5

4.0

4.5

5.0

Time (days)

STDR Group

−2000 −1500 −1000 −500

0.5

1.0

1.5

2.0

2.5

Time (days)

Log(

Cho

lest

erol

)

−2000 −1500 −1000 −500

0.5

1.0

1.5

2.0

2.5

Time (days)

Figure: Observed longitudinal profiles (in light blue) of log(HbA1c) andlog(Cholesterol) for patients without positive screening events (leftcolumn) and patients with positive screening events (right column). Theaverage profile over time of a male with median age is shown in eachgroup by the red and green lines respectively.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

−2000 −1500 −1000 −500

05

1015

Time (days)

Num

ber

of G

P V

isits

No STDR Group

−2000 −1500 −1000 −500

05

1015

Time (days)

STDR Group

−2000 −1500 −1000 −500

0.0

0.2

0.4

0.6

0.8

1.0

Time (days)

Gra

ding

−2000 −1500 −1000 −500

0.0

0.2

0.4

0.6

0.8

1.0

Time (days)

Figure: Observed longitudinal profiles (in light blue) of number of GPvisits and retinopathy grading for patients without positive screeningevents (left column) and patients with positive screening events (rightcolumn). The average profile over time of a male with median age isshown in each group by the red and green lines respectively.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

ROC Plot for Methods of Group Prediction

1−Specificity

Sen

sitiv

ity

MarginalConditionalRandom EffectsLDAQDA

Figure: ROC curve to compare the predictive abilities of the threelongitudinal methods of group membership prediction and the simpleLDA and QDA techniques.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Example: ISDR data

Marginal Conditional Random effects LDA QDACutoff 0.06 0.05 0.02 0.01 0.01

Sensitivity 0.79 0.78 0.50 0.55 0.52Specificity 0.81 0.74 0.65 0.52 0.49

PCC 0.81 0.75 0.64 0.52 0.50AUC 0.87 0.78 0.57 0.52 0.51

Table: The precision of the prediction of diagnostic groups for threelongitudinal methods and the classical LDA and QDA methods.PCC = Probability of Correct classification.AUC = Area Under Curve.LDA = Linear Discriminant Analysis.QDA = Quadratic Discriminant Analysis.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Conclusions

◮ There is a definite advantage to using longitudinal informationin comparison to simply applying LDA (or QDA) to the lastobservations for each patient.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Conclusions

◮ There is a definite advantage to using longitudinal informationin comparison to simply applying LDA (or QDA) to the lastobservations for each patient.

◮ The marginal prediction method gives the best classification forthe ISDR data (on all measures).

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Conclusions

◮ There is a definite advantage to using longitudinal informationin comparison to simply applying LDA (or QDA) to the lastobservations for each patient.

◮ The marginal prediction method gives the best classification forthe ISDR data (on all measures).

◮ Our methodology is able to obtain promising classificationresults by incorporating markers of different types.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Further work

◮ Can we make more use of the credible intervals that are readilyavailable from the MCMC procedure?

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Further work

◮ Can we make more use of the credible intervals that are readilyavailable from the MCMC procedure?

◮ Can we identify the ideal timing of the next screening interval?

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Further work

◮ Can we make more use of the credible intervals that are readilyavailable from the MCMC procedure?

◮ Can we identify the ideal timing of the next screening interval?

◮ Can we include categorical longitudinal outcomes within thisframework?

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

Acknowledgements

◮ Joint work with Arnost Komarek (Charles University inPrague), Gabriela Czanner, Christopher P. Cheyne, SimonHarding and Marta Garcıa-Finana.

◮ We are grateful for the support of the ISDR team.

◮ We acknowledge support from the Medical Research Council(Research project MR/L010909/1).

◮ Garcıa-Finana M, Czanner G, Cox T, Bonnett L, Harding S,Marson T. Discriminant Function Analysis for LongitudinalData: Applications in Medical Research (2014–2017) fundedby MRC MRP (£ 334,170)

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

References

◮ Brant, L.J., Sheng S.L., Morrell, C.H., Verbeke, G. N., Lesaffre, E.and Carter, H. B. (2003)Screening for prostate cancer by using random-effects models.Journal of the Royal Statistical Society: Series A, 166(1):51–62

◮ Fieuws, S., Verbeke, G., Maes, B., and Vanrenterghem, Y. (2008)Predicting renal graft failure using multivariate longitudinal profiles.Biostatistics, 9(3):419–431

◮ Komarek, A., Hansen, B.E., Kuiper, E.M.M., van Buuren, H.R., andLesaffre, E. (2010)Discriminant analysis using a multivariate linear mixed model with anormal mixture in the random effects distribution.Statistics in medicine, 29(30):3267–3283.

◮ Komarek A. and Komarekova, L. (2013)Clustering for multivariate continuous and discrete longitudinal data.The Annals of Applied Statistics, 7(1):177–200.

Flexible

Discriminant

Analysis Using

Multivariate Mixed

Models

D. Hughes

Motivation

MGLMM

Discriminant

Analysis

ISDR Example

Conclusions

References◮ Lix, L.M., and Sajobi, T.T. (2010)

Discriminant analysis for repeated measures data: a review.

Frontiers in psychology, 1, Article 146

◮ Marshall, G., De la Cruz-Mesıa, R., Quintana, F.A., and Baron, A.E.(2009)

Discriminant Analysis for Longitudinal Data with MultipleContinuous Responses and Possibly Missing Data.

Biometrics 65:69–80.

◮ Morrell, C.H., Brant, L.J., Sheng, S.L., and Metter, E. J. (2012)

Screening for prostate cancer using multivariate mixed-effectsmodels.

Journal of applied statistics, 39(6):1151–1175.

◮ Tomasko, L., Helms, R.W. and Snapinn, S.M. (1999)

A discriminant analysis extension to mixed models.

Statistics in medicine, 18(10):1249–1260.

◮ Wernecke, K-D., Kalb, G., Schink T., and Wegner, B. (2004)

A mixed model approach to discriminant analysis with longitudinaldata.

Biometrical journal, 46(2):246–254.