Adjusting the Generalized ROC Curve for Covariatesreiser/article/Adjusting...Adjusting the...
Transcript of Adjusting the Generalized ROC Curve for Covariatesreiser/article/Adjusting...Adjusting the...
Adjusting the Generalized ROC Curve for Covariates
By
Enrique F. Schisterman1, David Faraggi2 and Benjamin Reiser2
1. Division of Epidemiology, Statistics and Prevention, NICHD, NIH.
2. Department of Statistics, University of Haifa, Mount Carmel, Haifa, Israel
SUMMARY
Receiver Operating Characteristic (ROC) curves and in particular the area under the
curve (AUC), are widely used to examine the effectiveness of diagnostic markers.
Diagnostic markers and their corresponding ROC curves can be strongly influenced by
covariate variables. When several diagnostic markers are available, they can be combined
by a best linear combination such that the area under the ROC curve of the combination
is maximized among all possible linear combinations. In this paper we discuss covariate
effects on this linear combination assuming that the multiple markers, possibly
transformed, follow a multivariate normal distribution. The ROC curve of this linear
combination when markers are adjusted for covariates is estimated and approximate
confidence intervals for the corresponding AUC are derived. An example of two
biomarkers of coronary heart disease for which covariate information on age and gender
is available is used to illustrate this methodology.
Key words: diagnostic markers, Box-Cox transformations, best linear combination,
sensitivity, specificity.
2
1. Introduction
This paper deals with obtaining linear combinations of multiple continuous markers
adjusted for covariate information in order to better distinguish between healthy and
diseased populations.
The effectiveness of continuous markers in distinguishing between healthy and
diseased subjects is generally assessed through the use of the Receiver Operating
Characteristic (ROC) curve [1]. A subject is assessed as diseased (positive) or healthy
(negative) according to whether the subject's marker value is greater than or less than or
equal to a specified threshold value. Associated with any threshold value is the
probability of a true positive (sensitivity) and the probability of a true negative
(specificity). The resulting theoretical ROC curve is the plot of sensitivity versus
1-specificity for all possible threshold values.
The ROC curve can be estimated from sample data taken on both diseased (Y) and
healthy (X) subjects. This estimation can be carried out under parametric or non-
parametric assumptions [1]-[3].
A commonly used global summary measure of marker accuracy is the area under the
ROC curve (AUC). Bamber [4] proved that AUC=P(Y>X) with larger values of AUC
indicating higher diagnostic accuracy. The functional P(Y>X) appears in many statistical
problems not connected with marker evaluation [5]. Both parametric and nonparametric
procedures have been suggested for statistical inference on AUC [1], [5]. Faraggi and
Reiser [6] compare a number of procedures for estimating the AUC.
The effectiveness of a continuous marker can be influenced by covariates/factors such
as age, gender, general health status etc. The ROC curve itself and the summary index
AUC can be adjusted for covariate effects by regression modeling of the relationship
between the marker and the covariates [7], [8]. An alternative approach can be based on
modeling the ROC curve [9]. Faraggi [8] discusses some of the advantages in directly
modeling the markers.
When multiple markers are available a comparison of the areas under the different
ROC curve is often used to decide on which marker is best. Su and Liu [10]
recommended that instead of trying to decide on single marker one should use a linear
3
combination of all the markers. They discuss choosing a best linear combination for
which the area under the corresponding ROC curve is maximized. Reiser and Faraggi
[11] derived a confidence interval for this maximal area. This maximal area, which they
called the generalized ROC criterion, provides a measure of how well the vector of
markers distinguishes between the healthy and diseased groups. The ROC curve
corresponding to this linear combination can be termed the generalized ROC curve.
Further discussion of these linear combinations and some examples of their use can be
found in [12]-[14]. Alternative approaches to Su and Liu [10] for obtaining optimal
combinations of diagnostic markers are discussed in Baker [15] and McIntosh and Pepe
[16].
In this paper we discuss how the Su and Liu methodology can be adjusted to account
for covariate effects. In Section 2 we discuss a motivating example that deals with
oxidative stress and antioxidant biomarkers for cardiovascular disease. In Section 3 we
derive the covariate adjustments by extending the Su and Liu procedure [10]. We further
show how confidence intervals can be obtained for the generalized ROC criterion
conditional on given covariate values. In Section 4 we apply this methodology to the
example and in Section 5 provide concluding remarks.
The methodology developed in this paper is based on the assumption that the marker
values are normally distributed. When data analysis indicates that this assumption is
untenable a power transformation of the Box-Cox [17] type can be used to improve the
normal fit. This approach has been found effective in estimating the AUC and ROC
curves in a wide variety of cases [6], [12], [18]-[21].
2. Example: Oxidative Stress and Antioxidant Biomarkers
Biomarkers of individual oxidative stress and antioxidant status have been suggested
for discriminating between individuals with and without coronary heart disease (CHD)
[22].
Schisterman et al. [14] discuss data from a population-based sample of randomly
selected residents of New York State’s Erie and Niagara counties that provides
information on a number of biomarkers. We consider, for illustrative purposes, only the
4
markers TBARS (thiobarbuturic acid reacting substances) and TEAC (trolox equivalent
antioxidant capacity). Data are available for 45 diseased and 891 healthy individuals.
Examination of the data shows that both the TBARS and TEAC distributions are strongly
non-symmetric. In order to improve the normality of the data the TBARS and TEAC
values were taken to the power of –1 and 4 respectively. These powers were obtained by
applying the Box-Cox method of estimating transformations. Figure 1 provides the
estimated ROC curves for each transformed marker separately. It also presents the
generalized ROC curve for their best linear combination obtained following [10]. The
corresponding AUCs for TBARS, TEAC and their linear combination are 0.695, 0.622
and 0.751 respectively. The difference between the AUCs of TBARS and TEAC was
tested using the methodology of Liu and Schisterman [23] and found to be marginally
significant (p-value=0.05). Both the ROC curves and their AUCs show that TBARS is
better than TEAC while the linear combination resulted in an improvement in separating
the healthy and diseased populations.
However, covariate information is available for the subjects in this study. More
specifically information on the gender and age of each subject were obtained. Gender and
age may influence the discriminatory accuracy of the markers themselves and their linear
combination. In the following section we present the theory for adjusting the best linear
combination for covariates and then in Section 4 apply this theory to the data under
consideration.
3. Covariate Adjustment of the Linear Combination of Markers
3.1 Notation and Assumptions
Let X and Y be p dimensional column vectors denoting p different markers on the
healthy and diseased groups respectively on which samples of size m ( )miX i ,...,1, = and
n ( )njYj ,...,1, = are available. Further let ixZ represent a vector of size xr whose first
element is 1 and whose remaining 1−xr elements are observations on 1−xr explanatory
variables for the thi healthy subject. We assume for the healthy subjects that the marker
vector is related to the covariate vector by the usual multivariate regression model
5
ii xxXi ZBX ε+= (1)
where XB is a p by xr matrix composed of unknown coefficients and ixε is the residual
vector of size p such that ixε , mi ,...,1= are independently distributed as ( )xN Σ,0 with
0 being a vector of p zeros and xΣ a positive definite p by p residual covariance
matrix. We denote the design matrix corresponding to the model (1) by the m by xr
matrix )',...,,( '''21 mxxxx ZZZZ = , the first column of which is composed of ones.
Similarly to (1) we assume that the jY (diseased subjects) follow
jj yyYj ZBY ε+= (2).
Letting 1−yr represent the number of explanatory variables for the diseased subjects,
YB is a p by yr matrix where jyZ has yr elements. The n independent
jyε vectors are
( )yN Σ,0 variates and are further assumed to be independent of the ixε . The
corresponding n by yr design matrix is denoted by )',...,,( '''21 nyxyy ZZZZ = . Note that the
explanatory variables associated with the healthy subjects do not need to be identical to
those of the diseased subjects.
Note that the explanatory variables may include interaction terms or other functions of
the covariates which may be of interest. Standard model selection techniques for
multivariate regression can be employed as described for example in Rencher [24, pp
382-391]. Residual analysis should be used to examine the model assumptions [25].
3.2 The Covariate Adjusted Optimal Linear Combination
Let 0xZ and
0yZ be column vectors of size xr and yr of given covariate values
corresponding to X and Y respectively. Both have ones as the first element. Conditional
on these given covariate values
6
( )000
| xXxx ZBZXE == µ
and (3)
( )000
| yYyy ZBZYE == µ
while the conditional variances remain as xΣ and yΣ respectively. We want to find the
best linear combination XaU '= and YaV '= in the sense of having the largest AUC
over all linear combinations, conditional on 0xZ and
0yZ .
The results of Su and Liu [10] can be applied to show that
( ) µ1−Σ+Σ= yxa (4)
where 00 xy µµµ −= with corresponding
( )( ) 2/11' µµ −Σ+ΣΦ= yxAUC (5)
where Φ denotes the standard normal cumulative distribution function.
The AUC, µ and a are functions of the given covariate vectors 0xZ and
0yZ but for
notational convenience we do not write them as such explicitly.
The corresponding sensitivity (TP) and specificity (TN) are for given threshold value
C and given covariates 0xZ and
0yZ
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
Σ
−Φ=
aa
CaTP
y
y
'
'0
µ (6)
and
7
⎟⎟⎠
⎞⎜⎜⎝
⎛
Σ
−Φ=
aa
aCTN
x
x
'
'0
µ (7).
The conditional ROC curve for covariates 0xZ and
0yZ is the plot of TP versus 1-TN
for all possible values of C . This can alternatively be written as
( ) ( )
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
Σ
−ΦΣ+Φ=
−
aaTNaaa
TPy
x
'1'' 1µ
(8).
Thus (8) represents the covariate adjusted generalized ROC curve.
For estimation of these ROC curves and their corresponding AUCs we consider two
cases, (I) Σ=Σ=Σ yx and (II) yx Σ≠Σ .
Let XB and YB denote the standard least squares estimators of XB and YB while xS
and yS denote the multivariate regression residual sum of squares matrices for models
(1) and (2) respectively. Set 00
ˆˆ xXx ZB=µ , 00
ˆˆ yYy ZB=µ and 00
ˆˆˆ xy µµµ −= .
3.2.1 Case I ( Σ=Σ=Σ yx )
Set 2
' 1µµδ−Σ
= , then ( )2/1δΦ=AUC . Since AUC is monotonically related to δ , a
confidence interval for AUC readily follows from that for δ . yx
yxP rrnm
SSS
−−+
+=
provides an unbiased estimate of Σ=Σ=Σ yx . The linear combination coefficients given
by (4), the generalized ROC criteria (5) and the corresponding generalized ROC curve (8)
are estimated by substituting PS for xΣ and yΣ and µ for µ . All these estimators are
for given covariates 0xZ and
0yZ .
8
Confidence intervals for AUC can now be obtained similarly to the argument used in
[11] but in our case conditional on the given 0xZ and
0yZ . Letting W denote the Wishart
distribution, it follows from standard multivariate regression theory that
( ) ( )Σ−−+−−+ ,,~ prrnmWSrrnm yxPyx (9)
independent of
( )Σ== 200000
,~ˆˆ xxXxxXx aZBNZB µµ and ( )Σ== 200000
,~ˆˆ yyYyyYy aZBNZB µµ where
( )000
1''2xXXxx ZZZZa −
= and ( )000
1''2yYYyy ZZZZa −
= . This results in
( )Σ+−= )(,~ˆˆˆ 220000 yxxy aaN µµµµ (10)
or
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛Σ
++,1~ˆ1
22220000
µµyxyx aa
Naa
independently of PS . Consequently the Hotelling
2T statistic is
δµµ ˆ2'ˆ122
122
2
0000 yxP
yx aaS
aaT
+=
+= −
and using the standard connection between 2T and non central F variates we obtain that
)(~ˆ))(()1(2ˆ
1,22*
00
λδδ +−−−+−−++
+−−−+= prrnmp
yxyx
yxyx
Fprrnmaa
prrnm (11)
where the non centrality parameter 2200
2
yx aa +=
δλ .
Applying Lam [26] to (11) we obtain a confidence interval for λ and hence AUC (via
δ ) by numerically solving
9
21ˆ2Pr *
221,00
αδδ−=
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛≤⎟
⎟⎠
⎞⎜⎜⎝
⎛
++−−−+yx
prrnmp aaFob
yx (12)
2ˆ2Pr *
221,00
αδδ=
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛≤⎟
⎟⎠
⎞⎜⎜⎝
⎛
++−−−+yx
prrnmp aaFob
yx (13)
for δ and δ respectively. The resulting interval )](),([2/12/1 δδ ΦΦ provides a α−1
confidence interval for δ . If ( )( )*1,
ˆ0Pr δ<+−−−+ prrnmp yxFob is less than 2/1 α− [ 2/α ]
then there is no solution for (12) [(13)] and δ [δ ] is assigned the value zero.
3.2.1 Case II ( yx Σ≠Σ )
For this general case
)( 2/1δΦ=AUC (14)
for µµδ 1' −Σ= C , yxC Σ+Σ=Σ . x
xx rm
S−
=Σ and y
yy rn
S−
=Σ provide unbiased
estimates of xΣ and yΣ while yxC Σ+Σ=Σ ˆˆˆ estimates CΣ unbiasedly. Letting
µµδ ˆˆ'ˆˆ 1−Σ= C and substituting estimates for the parameters in (4), (8) and (14) provides
estimates for the best linear combination and its corresponding ROC curve and AUC.
The confidence interval for the AUC is more complicated in this case due to the
inequality of the residual covariance matrices and resembles the multivariate Behrens-
Fisher problem. We follow [11] and use approximations developed in the literature for
the Behrens- Fisher problem. For the general case ),(~ˆ 2200 yyxx aaN Σ+Σµµ
),,(~ xxx prmWS Σ− and ),,(~ yyy prnWS Σ− , all independently distributed. Thus
10
( ) ( ))/(,,)/(,,~ˆyyyxxxC rnprnWrmprmW −Σ−+−Σ−Σ .
Following [11] and the references they cite, we consider that approximately
⎟⎠⎞
⎜⎝⎛ Σ
MN C,~ˆ µµ & (15)
independently of
f ( )CC pfW ΣΣ ,,~ˆ & (16)
where
( )
( )yyxx
C
aatrtr
MΣ+Σ
Σ= 22
00
(17)
and the formula for f is given below. The above formulae (15) and (16) are of the same
form as (10) and (9) obtained for case I. In parallel to (11) we obtain that approximately
)(~1ˆˆ1,
* δδδ MFppf
fM
pfp +−⎟⎟⎠
⎞⎜⎜⎝
⎛ +−= & (18).
Based on a simulation study [11] recommended that f be estimated following [27]
whose method gives for our situation the formula
( ) { } { }⎥⎥⎦
⎤
⎢⎢⎣
⎡ΣΣΣ
−+ΣΣΣ
−Σ= −−−−−− 2121212121
00'1'1'1 µµµµµµ WyyW
yWxxW
xW a
rna
rmf (19)
where yyxxW aa Σ+Σ=Σ 2200
.
11
In order to obtain confidence intervals )](),([2/12/1 δδ ΦΦ for the AUC we have from
(18)
( )( )2
1ˆPr *1,
αδδ −=≤+− MFob pfp (20)
( )( )2
ˆPr *1,
αδδ =≤+− MFob pfp (21)
which need to solved numerically for δ and δ . These parallel (12) and (13). M and f in
(20) and (21) are obtained by the obvious substitutions in (17) and (19).
4. The Example Revisited
Multivariate regression was carried out on the TBARS and TEAC data using the
transformations given in Section 2. Age, gender and the age-gender interaction were used
as explanatory variables. For the diseased (Y) subjects both the interaction term (p-
value=0.198) and gender (p-value=0.13) were not found to be significant while for the
healthy (X) subjects all three terms were significant (p-value<0.001). Scatter plots of the
residuals for both the healthy and diseased groups show the “cloud” pattern typical for
normality (Figure 2a). Other residual analyses such as Q-Q plots show no reason to reject
the normality assumption for the transformed markers (Figure 2b). Consequently the
covariate-adjusted linear combinations were calculated as described in Section 3.2.2. A
referee raised the possibility that the residual covariance matrices in the multivariate
regression models (1) and (2) may depend on the covariates and thus violate the linear
model assumptions of variance homogeneity. An examination of the pattern of the
residuals when plotted against age (for males and females separately) found no violation
of the variance homogeneity assumption. For brevity these graphs are not presented.
Table I presents the weights (a’s) for the transformed TEAC and TBARS variables for
both females and males as a function of age. Figures 3a and 3b provide graphs of the
12
AUC for the best linear combination as a function of age along with point wise 95%
confidence intervals, which give an indication of the variability of the estimation process.
Figures 4a and 4b show the AUCs for both markers individually as well as their linear
combination adjusted for age for females and males respectively.
In Section 2 we found, when ignoring the age and gender, that the AUC for the linear
combination of TEAC and TBARS was 0.751. Figures 3a and 3b show that in fact the
AUCs differ greatly with age and that those for males are larger than those for females of
the same age. In addition for both males and females the AUCs first decrease and then
increase with age. For many ages the AUCs are quite higher than the ‘overall’ value of
0.751. The relatively short confidence intervals show the accuracy of the area estimates.
The weights of TBARS and TEAC (transformed) vary with age for both genders.
Their relative influence on the linear combination is made clear in Figures 4a and 4b. For
females younger than about 45 the AUC for TBARS alone is as good as that of the linear
combination and is substantially larger than that of TEAC alone. For females older than
65, TEAC alone does about as well as the combination and better than TBARS alone.
The linear combination does better than each marker alone for woman in the 45-65 age
group. The picture for males is similar although for older males the linear combination
still tends to show improvement over each marker separately.
The estimated adjusted ROC curves for both females and males are plotted in Figures
5a and 5b along with their corresponding AUCs for the linear combination of TBARS
and TEAC at ages 45, 55, 65 and 75. It is clear that for a given age the discrimination
between the healthy and diseased groups is better for males than for females. In addition
we see the strong effect of age. Comparing Figures 5a and 5b with the generalized ROC
curve in Figure 1 shows that ignoring gender and age will lead to misleading conclusions
about the discriminatory effectiveness of combining TBARS and TEAC.
5. Concluding Remarks
In many applications multiple diagnostic markers are available. Since different
markers may be sensitive to different aspects of the disease been studied, creating a
“new” marker as a linear combination of the multiple markers provides a simple readily
13
implementable procedure for improving diagnostic capability. Since diagnostic markers
will frequently be subject to covariate effects it becomes important to adjust the
combination process for covariates. The theory presented in Section 3 shows how this
adjustment can be carried out using standard multivariate regression modeling
techniques.
Schisterman et al. [14] consider a different procedure for handling explanatory
variables such as age and gender. They do not distinguish between the biomarkers and
the explanatory variables but treat them as a vector of markers and compute a linear
combination of all of them using the Su and Liu procedure. This approach has several
difficulties: (i) binary variables such as gender cannot be transformed to have an
approximate normal distribution (ii) the weighting given to each biomarker is fixed and
does not depend on the covariate values. As seen in our example such a dependency can
be very meaningful.
Although the theory in Section 3 is restricted by the normality assumption it can be
extended to many non-normal situations by using the Box-Cox type transformations.
Standard residual analysis methods should be used to examine the fit of the model.
Once a generalized ROC curve adjusted for covariates values of interest is estimated
not only the AUC but also other indices of interest such as the partial area and Youden
Index as well as the critical decision threshold value can be readily obtained.
Acknowledgement
We would like to thank the reviewers for their helpful comments
14
References
1. Zhou, X.H., Obuchowski, N.A. and McClish, D.K. (2002). Statistical Methods in Diagnostic Medicine. Wiley. New York.
2. Shapiro, D.E. (1999). The Interpretation of Diagnostic Tests. Statistical Methods in Medical Research, 8, 113-134.
3. Zou, K.H., Hall, W.J. and Shapiro, D.E. (1997). Smooth Non-Parametric Receiver Operating Characteristic (ROC) Curves for Continuous Diagnostic Tests. Statistics in Medicine, 16, 2143-2156.
4. Bamber, D.C. (1975). The Area Above the Ordinal Dominance Graph and the Area Below the Receiver Operating Characteristic Graph. Journal Math. Psychol. 12, 387-415.
5. Kotz, S., Lumelskii, Y. and Pensky, M. (2003). The Stress-Strength Model and Its Generalizations. World Scientific. Singapore.
6. Faraggi, D. and Reiser, B. (2002). Estimation of the Area Under the ROC Curve. Statistics in Medicine. 21, 3093-3106.
7. Smith, P.J., and Thompson, T.J. (1996). Correcting for Confounding in Analyzing Receiver Operating Characteristic Curves. Biometrical Journal, 38, 857-863.
8. Faraggi, D. (2003). Adjusting ROC Curves and Related Indices for Covariates. Journal of the Royal Statistical Society - Series D - The Statistician, 52, 179-192.
9. Pepe, M.S. (2000). An Interpretation for the ROC curve and inference using GLM procedures. Biometrics, 56, 352-359.
10. Su, J.Q. and Liu, J.S. (1993). Linear Combinations of Multiple Diagnostic Markers. Journal of the American Statistical Association 88, 1350-1355.
11. Reiser, B. and Faraggi, D. (1997). Confidence Intervals for the Generalized ROC Criterion. Biometrics 53, 644-652.
12. Kramar, A., Faraggi, D., Ychou, M., Reiser, B. and Grenier, J. (1999). Criteres ROC Generalises Pour L’evaluation de Plusieurs Marqueurs Tumoraux. Revue d’Epidemiologie et de Sante Publique. 47, 217-226.
13. Kramar, A., Faraggi, D., Fortune, A. and Reiser, B. (2001). mROC: a computer program for combining tumour markers in predicting disease states. Computer Methods and Programs in Biomedicine. l66, 199-207.
14. Schisterman, E., Faraggi, D., Browne, R., Freudenhime, J., Dorn, J., Muti, P., Armstrong, D., Reiser, B. and Trevisan, M. (2002). Minimal and Best Linear Combination of Oxidative Stress and Antioxidants Biomarkers in Discriminate Cardiovascular Disease. Nutrition, Metabolism and Cardiovascular Diseased, 12, 259-266.
15. Baker S. G. (2000). Identifying Combination of Cancer Markers for Further Study as Triggers of Early Intervention. Biometrics, 56, 1082-1087.
16. McIntoch, M. W. and Pepe, M. S. (2002). Combining Several Screening Tests: Optimality of the Risk Score. Biometrics, 58, 657-664.
17. Box, G. E. P. and Cox, D. R. (1964). An Analysis of Transformations. Journal of the Royal Statistical Society, Series B, 26, 211-243.
18. O'Malley, A. and Zou, K.H. (2002). Bayesian hierarchical transformation models for receiver operating characteristic curve analysis. ASA Proceedings of the Joint Statistical Meetings, 2541-2549.
15
19. Zou, K.H., and Hall, W.J. (2002). Semiparametric and parametric transformation models for comparing diagnostic markers with paired design. Journal of Applied Statistics, 29 (6), 803-816.
20. Zou, K.H. and Hall, W.J. (2000). Two transformation models for estimating an ROC curve derived from continuous data'', Journal of Applied Statistics, 27 (5), 621-631.
21. Zou, K.H., Tempany, C.M., Fielding, J.R., and Silverman, S.G. (1998). Original Smooth Receiver Operating Characteristic Curves Estimation from Continuous Data: Statistical Methods for Analyzing the Predictive Value of Spiral CT of Ureteral Stones. Academic Radiology, 5, 680-687.
22. Amstrong, D. (2002). Oxidative Stress Biomarkers and Antioxidant Protocols. Humana Press, New Jersey.
23. Liu, A. and Schisterman, E. F. (2003). Comparison of Diagnostic Accuracy of Biomarkers With Pooled Assessments. 45, 631-644.
24. Rencher, A. C. (1995). Methods of Multivariate analysis. Wiley, New York. 25. Gnanadesikan, R. (1997). Methods for Statistical Data Analysis of Multivariate
Observations. Wiley, New York. 26. Lam, Y.M. (1987). Confidence Limits for Noncentrality Parameters of Noncentral
Chi-Squared and F Distributions. ASA Proceedings of Statistical Computing Section, 441-443.
27. Yao, Y. (1965). An approximate degrees of freedom solution to the multivariate Behrens-Fisher problem. Biometrika, 52, 139-147.
16
Table I. Weights for Best Linear Combination
Age Females Males
TEAC TBARS TEAC TBARS
30 0.1528 6.256 -0.3583 8.832
40 -0.0664 4.725 -0.4639 6.440
50 -0.2856 3.195 -0.5695 4.047
60 -0.5047 1.665 -0.6751 1.655
70 -0.7239 0.134 -0.7806 -0.737
80 -0.9431 -1.396 -0.8862 -3.129
17
Figure 1. Unadjusted ROC Curves
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
1-specificity
Sen
sitiv
ity
TEAC
TBARS
LinearCombination
18
Figure 2. Residual analysis Healthy Diseased
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
-3 -2 -1 0 1 2 3 4
Residuals TEAC
Res
idua
ls T
BAR
S
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
-1.5 -1 -0.5 0 0.5 1 1.5
Residuals TEAC
Res
idua
ls T
BA
RS
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8Residuals TBARS
-0.5
-0.3
-0.1
0.1
0.3
0.5
-0.5 -0.3 -0.1 0.1 0.3 0.5Residuals TBARS
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3Residuals TEAC
-1.5
-0.5
0.5
1.5
-1.5 -0.5 0.5 1.5Residuals TEAC
a) Scatter Plots
b) normal Q-Q plots
19
Figure 3. Generalized AUC as a function of age with pointwise 95% confidence interval (a) Females
0.5
0.6
0.7
0.8
0.9
1
30 40 50 60 70 80
Age
AU
C
upperAUClower
(b) Males
0.5
0.6
0.7
0.8
0.9
1
30 40 50 60 70 80Age
AU
C
upper
AUC
lower
20
Figure 4. AUCs of Individuals Markers and Their Linear Combination. (a) Females
0.5
0.6
0.7
0.8
0.9
1
30 40 50 60 70 80
Age
AU
C
CombinationTBARSTEAC
(b) Males
0.5
0.6
0.7
0.8
0.9
1
30 40 50 60 70 80Age
AU
C
Combination
TBARS
TEAC
21
Figure 5. Adjusted Generalized ROC Curves. (a) Females
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 11-Specificity
Sen
sitiv
ity
Age 45, AUC=0.839
Age 55, AUC=0.735
Age 65, AUC=0.699
Age 75, AUC=0.785
(b) Males
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 11-Specificity
Sen
sitiv
ity
Age 45, AUC=0.904
Age 55, AUC=0.784
Age 65, AUC=0.730
Age 75, AUC=0.839