EMBC2001

Using Artificial Neural Using Artificial Neural Networks toNetworks to Predict Predict

Malignancy of Ovarian TumorsMalignancy of Ovarian Tumors

C. Lu1, J. De Brabanter1, S. Van Huffel1,I. Vergote2, D. Timmerman2

1Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium,

2Department of Obstetrics and Gynecology, University Hospitals Leuven, Leuven, Belgium

EMBC2001

OverviewOverview

Introduction Data Exploration Input Selection Model Building Model Evaluation Conclusions

EMBC2001

IntroductionIntroduction Problem

ovarian masses: a common problem in gynecology. develop a reliable diagnostic tool to discriminate

preoperatively between benign and malignant tumors. assist clinicians in choosing the appropriate treatment.

Data Patient data collected at Univ. Hospitals Leuven, Belgium, 1994~1999 425 records, 25 features. 291 benign tumors, 134 (32%) malignant tumors.

EMBC2001

IntroductionIntroduction Methods

Data exploration: Data preprocessing, univariate analysis, PCA, factor

analysis, discriminant analysis, logistic regression… Modeling:

Logistic regression (LR) models Artificial neural networks (ANN): MLP, RBF Performance measures:

Receiver operating characteristic (ROC) analysis

ROC curves constructed by plotting the

sensitivity versus the 1-specificity, or false positive rate, for varying probability cutoff level.

visualization of the relationship between sensitivity and specificity of a test.

Area under the ROC curves (AUC)

measures the probability of the classifier to correctly classify events and nonevents.

EMBC2001

Data explorationData exploration Univariate analysis:

preprocessing: descriptive statistics, histograms…

Variable (symbol) Benign MalignantDemographic Age (age)

Postmenopausal (meno)45.6 15.2

31.0 %56.9 14.6

66.0 %Serum marker CA 125 (log) (l_ca125) 3.0 1.2 5.2 1.5CDI High color score (colsc3,4) 19.0% 77.3 %Morphologic Abdominal fluid (asc)

Bilateral mass (bilat)Unilocular cyst (un)Multiloc/solid cyst (mulsol)Solid (sol)Smooth wall (smooth)Irregular wall (irreg)Papillations (pap)

32.7 %13.3 %45.8 %10.7 %8.3 %56.8 %33.8 %12.5 %

67.3 %39.0 %5.0 %36.2 %37.6 %5.7 %73.2 %53.2 %

Demographic, serum marker, color Doppler imaging and morphologic variables

EMBC2001

Data explorationData exploration Multivariate analysis:

factor analysis biplots

Fig. Biplot of Ovarian Tumor data.

The observations are plotted as points (0=benign, 1=malignant), the variables are plotted as vectors from the origin.

- visualization of the correlation between the variables - visualization of the relations between the variables and clusters.

EMBC2001

Input SelectionInput Selection Stepwise logistic regression analysis Searching in the feature space

fix several of the most significant variables, then vary combinations with the other predictive variables.

different logistic regression models with different subsets of input variables were built and validated.

subsets of variables were selected according to their predictive performance on the training set and test set.

EMBC2001

Model buildingModel building Logistic regression (LR) model Artificial neural networks

feed-forward neural networks, universal approximators:

- multi-layer perceptron (MLP)

- generalized regression network (GRNN) generalization capacity: central issue during network

design and training.

EMBC2001

Model buildingModel building- - LRLR

Parameter estimation:- maximum likelihood

- iterative procedure

. . . . .

b i a s

P r o b a b i l i t y o fm a l i g n a n c y

g

M O D E L 1 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s o l i r r e g p a pM O D E L 2 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s m o o t h p a p

)exp(1

1)(

aag

Fig. Architecture of LRs for Predicting

Malignancy of Ovarian Tumors

structure: LR1: 8-1 LR2: 7-1

EMBC2001

TrainingBayesian regularization combined with Levenberg-Marquardt optimization.

Model BuildingModel Building- ANN - MLP- ANN - MLP

M : n u m b e r o f h i d d e n n e u r o n s d : n u m b e r o f i n p u t v a r i a b l e s

. . . . .

b i a s

b i a s

P r o b a b i l i t y o f m a l i g n a n c y

g

g

g

g

M O D E L 1 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s o l i r r e g p a p M O D E L 2 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s m o o t h p a p

)exp(1

1)(

aag

M

j

d

iijij xwgwgy

0 0

)1()2(

Fig. Architecture of MLPs for Predicting

Malignancy of Ovarian Tumors

structureMLP1: 8-3-1MLP2: 7-3-1

EMBC2001

Model BuildingModel Building– ANN - GRNN– ANN - GRNN

Fig. Architecture of GRNNs for

Predicting Malignancy of

Ovarian Tumors. . . . .

o u t p u t

N

jj

N

jj

x

xt

xy

j

1

1

)(

)(

)(

t i : t a r g e t o u t p u t o fi t h t r a i n i n g d a t a

2

2

2exp)(

j

i

j h

xxx

N : # t r a i n i n g d a t a

x : i n p u t v e c t o rx : i n p u t v e c t o r

g

2 1 N

M O D E L 1 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s o l i r r e g p a pM O D E L 2 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s m o o t h p a p

… … …… … …

t 1t 2 t N

Training:

GRNN is another term for Nadaraya-Watson kernel regression. No iterative training; the widths of RBF units h act as smoothing parameters, chosen by cross-validation.

structureGRN1: 8-N-1GRN2: 7-N-1

EMBC2001

•RMI: risk of malignancy index = scoremorph× scoremeno× CA125

Training set : data from the first treated 265 patients

Test set : data from the latest treated 160 patients

Model Evaluation Model Evaluation - Holdout CV- Holdout CV

AUC estimates and standard errors from hold out CV

EMBC2001

stratified 7-fold CV for each run of 7-fold CV:

mAUC : (iAUCi)/7, i =1,…7, AUCi is the AUC on the ith validation setexpected ROC: Averaging.

Repeat 7-fold CV 30 times with different partitions => better statistical estimate

Model Evaluation Model Evaluation - K-fold CV- K-fold CV

Box plot of meanAUC from 7-fold CVExpected ROC curves from k-fold CV

EMBC2001

Multiple comparison of mAUCs:

one-way ANOVA followed by Tukey multiple comparison.

Rank ordered significant subgroups from multiple comparison on mean AUC

Models RMI LR2 LR1 GRN1 GRN2 MLP2 MLP1

mean mAUC

0.882

0.943

0.954

SD

0.003

0.939

0.003

0.941

0.004

0.003

0.944

0.003

0.944

0.003

0.003

Note: The subsets of adjacent means that are not significantly different at 95% confidence level are indicated by drawing a line under the subsets.

Model Evaluation Model Evaluation - K-fold CV- K-fold CV

EMBC2001

ConclusionsConclusions Summary

AUC is the advocated performance measure Data exploratory analysis helps to analyze the data set. MLPs have the potential to give more reliable

prediction.

Future work Develop models with kernel methods, e.g. LS-SVM ANNs are blackbox models. A hybrid methodology,

greybox models might be more promising