A decision support system to facilitate management of ...hahn/psfile/gastro.pdf · Decision support...

Decision support system to facilitate management of patients with acute GIB of 41, Chu, et al

A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Adrienne Chu, Hongshik Ahn, Bhawna Halwan, Everson LA Artifon, Alan Barkun, Michail G. Lagoudakis and Atul Kumar. (AC, HA and BH contributed equally to the study) The study was funded by 2005 Research and Outcomes Effectiveness Awards of the American Society of Gastrointestinal Endoscopy. Corresponding author: Atul Kumar, MD, MBA Staff Gastroenterologist-United States Department of Veterans Affairs Assistant Professor of Medicine-Stony Brook University Stony Brook, NY 11794. Phone: (631) 880-8510 FAX: (631) 486-6113 email: [email protected] Adrienne Chu, BS Department of Applied Mathematics and Statistics Stony Brook University Stony Brook, NY 11794. Hongshik Ahn, PhD Professor Department of Applied Mathematics and Statistics Stony Brook University Stony Brook, NY 11794. Bruce Kalmin, MD Division of Gastroenterology, Medical University of South Carolina Charleston, SC 29425. Everson LA Artifon, MD, PhD Assistant Professor Gastroenterology & Hepatology University of Sao Pualo School of Medicine Sao Paulo, Brazil. Alan Barkun, MD, MS Professor of Medicine, General Surgery, and Epidemiology Division Director, Gastroenterology Mc Gill University Montreal, Canada H3A 2T5.


Michail G. Lagoudakis, PhD Assistant Professor Intelligent Systems Laboratory Department of Electronic and Computer Engineering Technical University of Crete Kounoupidiana, 73100 Chania Hellas (Greece). Bhawna Halwan, MD, MS Assistant Professor of Medicine, Gastroenterology & Hepatology SUNY Downstate Brooklyn, NY 11203.


ABSTRACT

Objective: To develop a model to predict the bleeding source and identify the cohort

amongst patients with acute gastrointestinal bleeding (GIB) who require urgent

intervention, including endoscopy. Patients with acute GIB, an unpredictable event,

are most commonly evaluated and managed by non-gastroenterologists. Rapid and

consistently reliable risk stratification of patients with acute GIB for urgent

endoscopy may potentially improve outcomes amongst such patients by targeting

scarce healthcare resources to those who need it the most.

Design & Methods: Using ICD-9 codes for acute GIB, 189 patients with acute GIB

and all available data variables required to develop and test models were identified

from a hospital medical records database. Data on 122 patients was utilized for

development of the model and on 67 patients utilized to perform comparative

analysis of the models. Clinical data such as presenting signs & symptoms,

demographic data, presence of co-morbidities, laboratory data and corresponding

endoscopic diagnosis and outcomes were collected. Clinical data and endoscopic

diagnosis collected for each patient was utilized by to retrospectively ascertain

optimal management for each patient. Clinical presentations and corresponding

treatment was utilized as training examples. Eight mathematical models including

artificial neural network (ANN), support vector machine (SVM), k-nearest neighbor,

linear discriminant analysis (LDA), shrunken centroid (SC), random forest (RF),

logistic regression, and boosting were trained and tested. The performance of these

models was compared using standard statistical analysis and ROC curves.


Results: Overall the random forest model best predicted the source, need for

resuscitation, and disposition with accuracies of approximately 80% or higher

(accuracy for endoscopy was greater than 75%). The area under ROC curve for RF

was greater than 0.85, indicating excellent performance by the random forest model.

Conclusion: While most mathematical models are effective as a decision support

system for evaluation and management of patients with acute GIB, in our testing RF

model consistently provided the best accuracy. Amongst patients presenting with

acute GIB, mathematical models may facilitate the identification of the source of

GIB, need for intervention and allow optimization of care and healthcare resource

allocation; these however require further validation.

Key words: class prediction, cross validation, gastrointestinal bleeding, machine

learning


1. INTRODUCTION

Acute gastrointestinal bleeding (GIB) is an increasing healthcare problem due to rising

NSAID use and an aging population [1]. Further reductions in mortality will most likely

require introduction of novel strategies to aid identification of the cohort requiring

aggressive resuscitation and endoscopic intervention to prevent complications and

death from ongoing bleeding [2,3]. Delays in intervention usually result from failure to

adequately recognize the source and severity of the bleed. Although several models

and scores have been developed to risk-stratify patients, no single model aids the

identification of the subgroup of patients presenting with acute upper or lower GIB most

likely to benefit from urgent intervention [4,5]. Our goal was to utilize mathematical

models to formulate a decision support system utilizing clinical and laboratory

information available within a few hours of patient presentation to predict the source,

need for intervention and disposition in patients with acute upper, mid and lower GIB.

There are a number of well-known classification tools for analyzing data, including

artificial neural networks (ANN), k-nearest neighbor (kNN), decision trees, support

vector machines (SVM), and shrunken centroid (SC) [6,7]. These classification tools

are supervised learning methods where the algorithm learns from a training set and

establishes a prediction rule to classify new samples using statistical approaches for

class prediction. Logistic regression, an alternative to these classification methods, and

Fisher’s linear discriminant analysis (LDA) are both parametric approaches. Logistic

regression and LDA do not differ in functional form, and differ only in the estimation of

coefficients; LDA assumes normal distribution of the explanatory variables, while logistic

regression does not. If the Gaussian assumptions are met, then LDA is a more


powerful and efficient model than logistic regression. Standard statistical model building

relies on an a priori collection of predictor variables for identifying outcomes of clinical

interest. Often it may be computationally impossible for standard models to overcome

the complexities of problems with large dimensionality. Support Vector Machines

(SVM), introduced by Vapnik (1995) can overcome the high dimensionality problem

computationally and is consistently a good classifier and hence widely utilized as a

classification method. Following the recent introduction of ensemble-voting

approaches, two ensemble voting methods, boosting and bagging, have also gained

wide popularity [8,9]. An ensemble uses the predictions of multiple base classifiers

through majority voting. Boosting, a meta-classifier, combines weak classifiers and

takes a weighted majority vote of their predictors. Breiman (2001) developed the

random forest (RF) method by combining classification tree predictors [10]. The

bagging algorithm in RF uses bootstrap samples to build base trees. Each bootstrap

sample is formed by randomly sampling, with replacement, the same number of

observations as the training set. The final classification produced by the ensemble of

these base classifiers is obtained using equal weight voting.

The study objective was to develop and compare the performance of eight classification

models as described above to predict clinical outcomes in patients presenting with

acute GIB.


2. METHODS

2.1 Definitions

Upper GIB refers to gastrointestinal blood loss whose origin is proximal to the ligament

of Treitz; mid GIB whose source was below the ligament of Trietz but proximal to the

ileocecal valve and lower GIB referred to gastrointestinal blood loss emanating from a

source distal to the ileocecal valve [11]. Acute GIB was defined as bleeding of less than

5 days duration. Acute upper GIB was diagnosed if there was hematemesis, "coffee

ground" colored emesis, the return of red blood via a nasogastric tube, and/or melena

with or without hemodynamic compromise or hematochezia (bright red blood per rectum

in patients with brisk upper GIB) and the presence of a bleeding source in the upper GI

tract at endoscopy. Patients with melena and/or hematochezia along with clinical

evidence of bleeding (hemodynamic instability, anemia) and positive findings of a

bleeding source at enteroscopy or capsule endoscopy were designated as mid GIB.

Lower GIB was defined as the identification of a source of bleeding in the lower GI in

patients presenting with hematochezia. Hypotension was defined as systolic blood

pressure of less than 100, tachycardia as heart-rate of greater than 100 beats/minute,

orthostasis as a drop of systolic blood pressure of greater than 20mm Hg and an

increase in heart rate of greater than 20 mmHg, 2 minutes after sitting upright from

recumbency with legs dangling from the side of the bed. Urgent endoscopy was

defined as a procedure that was performed within 24 hours for upper GIB and within 48

hours for lower GIB.

2.2 Patients


Patients with acute GIB were identified from the hospital medical records database

using ICD-9 codes for GIB. The study was carried out in compliance with all institutional

human investigations committee guidelines. Eligible patients were those presenting

with clinical manifestations of acute upper or lower gastrointestinal bleeding and those

who had undergone endoscopy within 24 hours for suspected upper GIB and within 48

hours for suspected lower GIB or if upper endoscopy was negative. If no obvious

source of GIB was identified at either upper or lower endoscopy, the patient should

have undergone small bowel enteroscopy or capsule endoscopy within 1 week of the

initial episode of acute GIB. Records of patients for whom a definite source of bleeding

could not be identified or those with missing clinical variables required for model

building and testing were discarded. A total of 189 patients meeting inclusion and

exclusion criteria were identified retrospectively in hospital electronic medical records.

Clinical data on each patient was entered into a scannable data entry form which was

scanned into an SQL database and manually reviewed for errors. Variables to

ascertain clinical outcomes corresponding to patient data included clinical and

endoscopic data as listed in Box 1. Variables that correlate with and are predictive of

clinical outcomes amongst patients with acute GIB were identified from a review of the

literature [12, 21]. Initially only 70 patient data were available when first building and

testing the models. This was followed by building and testing models using a total of

122 patients which resulted in an insignificant improvement in performance of the

models (data not shown). A sample size of 122 patients was therefore deemed to be

adequate for building models. An additional 67 patient database was utilized for testing

and validation of the models.


2.3 Selection of independent variables to predict outcomes for test cases

Several studies in the past have evaluated variables predictive of adverse outcomes,

predictive of source, severity and outcomes in patients with acute upper and lower GIB.

Clinical correlates of source, severity and outcomes amongst patients with acute

gastrointestinal bleeding were reviewed and listed as in Box 3 [17-29].

Bleeding source: The definitive source of bleeding was the irrefutable identification of

a bleeding source at upper endoscopy, colonoscopy, small bowel enteroscopy or

capsule endoscopy. Variables utilized to predict the source of GIB included past

medical history & presenting symptoms (prior history of GIB, hematochezia,

hematemesis, melena, syncope/presyncope), risk factors (risk for stress ulceration,

cirrhosis, ASA/NSAID use) physical exam and laboratory tests (blood pressure

(SBP/DBP), heart rate (HR), orthostasis, NG lavage, rectal exam, platelet count (Plt),

creatinine (Cr), BUN, and INR).

Blood resuscitation: Urgent blood resuscitation referred specifically to the

administration of blood and blood products to correct loss of intravascular volume and

presence of coagulopathy. Variables utilized to predict this outcome included

symptoms (hematochezia, hematemesis, melena, duration of symptoms,

syncope/presyncope), prior history (unstable CAD), physical exam and laboratory tests

(blood pressure, heart rate, orthostasis, NG lavage, rectal exam, hematocrit (Hct), drop

in hematocrit, creatinine, BUN, and INR).

Urgent endoscopy: Variables to predict need for urgent endoscopy included

symptoms (hematochezia, hematemesis, melena, syncope/presyncope, duration of

symptoms), risk factors (cirrhosis, ASA/NSAID use) and physical exam and laboratory


tests (blood pressure, heart rate, orthostasis, NG lavage, rectal exam, hematocrit,

hematocrit drop, platelet count, creatinine, BUN, and INR).

Disposition: Variables utilized to predict disposition included patient demographics

(age), presenting symptoms (hematochezia, hematemesis, melena, duration of

symptoms, syncope/presyncope), comorbidities (unstable CAD, COPD, CRF, risk for

stress ulcer, cirrhosis), clinical exam/blood tests (blood pressure, heart rate, orthostasis,

NG lavage, rectal exam, hematocrit, drop in hematocrit, platelet count, creatinine, BUN,

and INR).

2.4 Algorithm for analysis of clinical data to ascertain outcomes for training set

The algorithm to ascertain the source of bleeding and appropriate clinical management

for each patient is as illustrated in Figure 1 and was determined jointly by authors (AK,

BH, BK) after careful analysis of the clinical and endoscopic data for each patient, in

accordance with current evidence and standard of care [13-15]. Source was

ascertained by review of endoscopic findings for each patient [16]. Patients with

findings of active bleeding or other high-risk stigmata at endoscopy were reasoned as

requiring intensive resuscitation, urgent endoscopy and admission to the medical

intensive care unit. Patients with low-risk stigmata at endoscopy and hemodynamic

instability (hypotension, tachycardia, orthostasis), elevated coagulation parameters

and/or hematocrit <30, were reasoned to require intensive resuscitation but no urgent

endoscopy. Conversely, patients under 60 years of age with no active bleeding or high-

risk stigmata and no co-morbidities were reasoned as suitable for discharge to home,

while patients with co-morbidities were categorized as requiring admission to general

inpatient. Example: A 60 year old patient with acute onset of profuse hematemesis,


melena, dizziness, tachycardia, hypotension, positive nasogastric lavage, hematocrit of

24, is admitted to the regular floor and undergoes endoscopy greater than 24 hours

after admission. Endoscopy reveals an oozing blood vessel within an ulcer bed. Since,

in retrospect the patient should have been resuscitated urgently with blood, undergone

urgent endoscopy and admitted to the ICU, the patient was classified as requiring

urgent resuscitation, endoscopy and admission to the ICU. Each case was similarly

analyzed to identify the corresponding optimal outcome and the database thus

developed was utilized to train the models.

2.5 Importance of each variable

For RF and ANN, the variable importance option was selected to see if these variables

were significant for ascertaining outcomes by the computer models. Variable

importance for RF is given in terms of the mean decrease in accuracy, hence higher the

number, greater the importance of the variable. ANN provides data about the

importance of a variable without any ranking. Hence the variable importance feature for

ANN was repeated ten times in order to obtain rankings for the variables and the

number of times a variable was shown to be important was counted; the closer the

number was to 10, the more important the variable. In addition, distributions of the

variables as well as correlation between variables were examined.

2.6 Models & Statistical Analysis

Eight predictive models including random forest (RF), support vector machines (SVM),

shrunken centroid (SC), linear discriminant analysis (LDA), k-nearest neighbor (kNN),

logistic regression (logistic), boosting, and artificial neural networks (ANN) were trained

and their performances compared [8,9]. All models were run in R (version 2.3.0,


downloadable from http://cran.cnr.berkeley.edu) except for ANN, which was run in

STATISTICA (version 7.1, Statsoft, Inc, Tulsa, OK).

Model training was performed on a randomly selected subset of patients and testing on

the remaining patients of a total of 122 patient database. The primary approach was to

use the selected explanatory variables to predict the response variable discarding any

patients with missing data. In addition, for predicting resuscitation and endoscopy, an

alternative strategy was adopted. The predicted value of “source of bleeding” and

selected input variables (hematochezia, hematemesis, syncope/presyncope, blood

pressure, heart rate, orthostasis, NG lavage, and INR) were utilized to predict the need

for resuscitation and endoscopy. Categorical variables were changed accordingly to

indicator variables.

Model Evaluation Step: Ten runs of 10-fold cross validation (CV) were performed for

each iteration to obtain a reliable result with low mean square error (MSE) and bias [30].

The MSE is a function of bias and variance and when the estimator is unbiased, MSE

reduces to variance because the bias term in MSE becomes zero. Each run of cross

validation is comprised of independent training and testing database, where 90% of the

data is put in the training set and the remaining 10% of the data is put into the test set.

For every 10-fold CV, the following statistics were calculated: sensitivity (SN), specificity

(SP), accuracy (ACC: the sum of correct predictions divided by total predictions),

positive predictive value (PPV: probability that the patient is truly positive given a

positive prediction), and negative predictive value (NPV: probability that a patient is truly

negative given a negative prediction). For each classification model, statistical results

of 10 repetitions of 10-fold CV were averaged and reported.


ROC curves were calculated and areas under the curve were compared for each of the

eight models. The Mann-Whitney statistic was calculated, which is equivalent to the

area under an ROC curve [31]. Additionally, performance measures between models

were compared using McNemar’s test. Using the Bonferroni correction to account for

multiple comparisons of models, an appropriate alpha value was used for each test to

control the error rate. For example, there were 8 models for the resuscitation response;

hence there were 28 pair-wise comparisons. Thus for an original alpha value of .05, the

new alpha used for a two-sided test was .05/28=.0018. A normal approximation and

Central Limit Theorem was used for underlying assumptions for the test.

Model Validation Step: After models had been trained using a 122 patient database,

an unseen before 67 patient database was utilized as a test set to validate the model.

Accuracies were compared for both the evaluation step and the validation step.

2.7 Individual Model Parameters

Random forest (package Random Forest): All default settings were used (500 trees

were grown and the number of variables randomly sampled at each node was

p where p is the number of explanatory variables). The variable option of importance

was set to true. To predict the outcome of a certain patient, voting was done without

normalizing.

Support vector machine (package e1071): All default settings were used, variables

were scaled, tolerance value to terminate algorithm was set to .001, epsilon set to 1 (for

insensitive-loss function). Both the radial basis (default) and linear kernels were

considered, but only the radial basis function is included in this comparison because the

two kernels gave similarly high accuracies.


Shrunken centroid (package pamr): An optimal threshold value for each response

was found by cross validation. The best threshold value was one that gave the highest

accuracies – a threshold of .2 was used for all responses.

Linear discriminant analysis (package sma), kNN (package class), and logistic

regression (package stats) models: All default settings were used. Depending on

what response variable was being classified, a different k was used for kNN. By

performing CV on several different k values, the k that yielded the highest accuracy was

chosen. The final values chosen were k=7 for source and resuscitation, k=11 for

endoscopy, and k=3 for disposition.

Boosting (package boost): No features were pre-selected (presel=0), boosting ran for

10 iterations (mfinal=10). The optimal settings were found by trying different

combinations of presel and mfinal. The R Boosting package contains four different

approaches: AdaBoost, LogitBoost, L2Boost and BagBoost. AdaBoost was excluded,

as often it did not classify patients correctly. Performance of the other three boosting

methods were similar. Among these three approaches, LogitBoost is included in the

comparison.

ANN (Neural Networks package): Multilayer perceptrons with back propagation was

performed. The error function used was the cross entropy function. A linear synaptic

function was used and a combination of the following four activation functions were

used: linear, hyperbolic, softmax, and logistic. The number of epochs to train the model

was set to 100 although the network always converged in less number of epochs. The

learning rate was set to .01 and there was one hidden layer in the network. For the

source of bleeding response, there were 30 input neurons, 11 hidden neurons (neurons


in the hidden layer), and 3 output neurons. For the resuscitation response, there were

28 input neurons, 10 hidden neurons, and 1 output neuron. For the endoscopy

response, there were 31 input neurons, 12 hidden neurons, and 1 output neuron. For

the disposition response, there were 34 input neurons, 34 hidden neurons, and 1 output

neuron.

3. RESULTS

3.1 Models

Tables 1 through 4 summarize the results for each outcome prediction variable for the

evaluation step. Eight models were run (only six of the eight models were utilized to

predict source, since logistic regression and boosting can be only used for 2-way

classification problems (while output source included three outcomes-upper, mid and

lower) using the primary approach. Figures 2 through 5 depict the accuracies obtained

from each individual model and each response variable for the evaluation step. Overall,

accuracies obtained using SVM, RF, and LDA were numerically superior to others, with

these models correctly predicting the source of bleeding, need for resuscitation, and

disposition correctly 88%-94% of the time. The need for endoscopy was correctly

predicted about 80% of the time using SVM, SC, and LDA; accuracy using RF was just

under 80% at 79%. Logistic regression did well for predicting resuscitation and

endoscopy, however it did not do well for disposition. This is because the algorithm for

obtaining the model rarely converged, that is, it was unstable (results not shown).

Accuracies using boosting were worst of all models. Accuracies obtained ANN model

was inferior to SVM, RF and LDA. At a significance level of α=0.05, kNN performed the

worst for predicting source and resuscitation. The linear discriminant analysis model


appeared to demonstrate a good overall performance with regards to sensitivity

specificity, PPV, NPV and accuracy (Tables 1 through 4). Boosting, on the other hand,

revealed an imbalance between sensitivity and specificity. ROC curves were

constructed (Figures 6 through 9 and Tables 1 through 4), and overall RF and ANN

have the highest AUC (area under the curve), followed by SVM and LDA. For

predicting resuscitation and endoscopy, the logistic model had excellent AUC. In the

validation step, RF consistently predicted responses with higher accuracies as

compared to the other models (Table 6). Overall, data from evaluation and validation

steps suggests that the RF model consistently performs the best.

3.2 Importance of variables

The importance of each variable in predicting outcomes in ascertaining relevant

outcomes when using random forest and ANN respectively are shown in Table 7 (using

122 patient database). About half the variables to predict outcomes were of common

importance to both RF and ANN models. For predicting source, explanatory variables

hematemesis through HR (heart rate) had significant weights and hence importance.

The remaining variables were of variable importance aside from ASA/NSAID, which did

not appear to have great influence on the performance of the models. For predicting

resuscitation, both models utilized the variables syncope, orthostasis, hematocrit,

hematocrit drop, blood pressure, heart rate, hematemesis and melena as important

predictor variables. The remainder of the variables was of limited and variable

importance in predicting outcomes.


4. DISCUSSION AND CONCLUSION

Although superior healthcare outcomes may be expected if gastroenterologists manage

all patients with acute GIB [32], it is logistically impossible for every patient with acute

GIB to be emergently evaluated and treated by a gastroenterologist, as the onset of

acute GIB is unpredictable. It is also impractical and economically unjustifiable to

subject every patient with acute GIB to intensive resuscitation and urgent endoscopy as

only 20% of patients with acute GIB may require urgent intervention and furthermore

because healthcare resources are expensive and limited. “Expert systems” may

potentially substitute for a specialist amongst patients with acute GIB and facilitate

triage the cohort most likely to benefit from urgent resuscitation and endoscopy.

“Machine learning” or computer-assisted predictive models have been successfully

utilized to optimize treatment and predict clinical outcomes in a variety of other

conditions [33-36], such as computerized interpretation of the electrocardiogram [37], to

help streamline and optimize care of patients with acute myocardial infarction [38],

especially in a busy practice or in the emergency room [39]. Our objective was to

develop a system to provide diagnostic and specific treatment recommendations for

patients presenting with acute GIB. Such models have previously been shown to

accurately predict the need for colonoscopy in patients with acute lower GIB [5].

However, they have not been used for all patients with acute GI bleeding. The

recommendations were designed to be in agreement with current evidence based

guidelines for management of acute GIB. Our models successfully provided patient

specific recommendations with accuracies exceeding 70-80%. In the present study, RF

performed well in classification of the four response variables, in agreement with


previous studies that have demonstrated the robustness of such models. RF and SVM

are designed for high-dimensional data with a large feature space (large number of

predictor variables) compared to the sample size and are likely to outperform other

methods for high-dimensional data unlike the current GIB data set [40]. Logistic

regression is a widely used standard regression model for binary data, and it can be

expanded to data with more than two classes. However, it often shows computational

instability such as failure to converge or the predicted value being extremely close to 1

or 0 due to the nature of the model. Our results also support the conclusion by Ahn et

al. in previous studies, that boosting strategies in general provide poor accuracies [40].

Furthermore, given its complexity, these are also cumbersome and unwieldy as

compared to other methods. Although not relevant to our problem, LDA and kNN

require a variable pre-selection for an optimal performance unlike RF or SVM for high-

dimensional data.

Statistical variable selection is often dependent on the criteria and is computer

intensive. With regards to the analysis of the importance of variable, we show that both

the RF and ANN models considered half of the variables to be important with the

remaining half being of varied importance. This appears to be consistent with prior

knowledge in regards to importance of variables identified to predict source and severity

of acute GIB. Every pre-selected variable was important for one model or the other and

therefore consistent with their identification in prior multivariate analysis. Different

models assigned varied importance to different variables due to the different methods

for evaluating variable importance. In RF, for every tree grown in the forest, test

samples are used to count the number of votes cast for the correct class. RF randomly


permutes the values of a selected variable in the test set and put these cases down the

tree. It finds the number of votes for the correct class in the data with this permuted

variable. It subtracts this number from the number of votes for the correct class in the

original data without permutation. The average of this number of all trees in the forest is

the raw importance score for the variable. ANN uses a Connection Weight Approach to

quantify variable importance. Given that a minimal variable set to predict outcomes is

desirable, overall our dataset appears to be optimal to predict clinical outcomes relevant

to management of patients with acute GIB.

Since the output utilizes training examples which are developed by individuals, bias can

potentially be introduced through using inaccurate training examples. The authors have

the ability to influence the recommendations by modulating outcomes associated with

each training example. Since, examples are utilized to train the models, flaws in

examples may lead to flaws in the model output: garbagae in-garbage out. Despite

these flaws, such systems have the potential to facilitate and standardize the care of

patients presenting with acute GIB. Given that computer based tools are more likely to

work if integrated with clinical care, prospective validation and integration of such a

model into an electronic medical record may potentially enhance care of patients with

acute GIB. It is also possible to train the model using fresh examples so as to adapt to

changing guidelines and varied clinical scenarios, allowing these predictive models to

be portable to a broad range of locales. The prospective development and testing of a

model using a larger patient cohort prospectively, its implementation and comparison of

its performance to physicians is underway.


In summary, predictive models such as shown above, allow the identification of the

high-risk cohort amongst patients presenting with acute GIB to allow optimal allocation

of resources to patients who may potentially benefit the most from urgent resuscitation,

endoscopy and intensive clinical care.


Box 2. Output variables

Source: Upper, Mid, Lower Resuscitation: Yes or No Endoscopy: Yes, No Disposition: not ICU (home, floor, monitor); ICU

Box 1: Independent variables

Presentation:

Hematemesis/Coffee Grounds Hematochezia/Melena Duration of symptoms Syncope/presyncope

Demographics:

Age, gender

Past history:

Prior GIB, Unstable CVD (MI, CHF), COPD exacerbation, CRF, Risk of Stress Ulcer, Cirrhosis

ASA/NSAID use, PPI, prior history of GIB

Clinical Exam:

SBP/DBP, HR, orthostasis, NG lavage, rectal exam

Laboratory Data:

Hematocrit, drop in Hct., platelet count, creatinine, BUN, PT/INR

Box 3: Explanatory variables used for each output Source of bleeding: prior history of GIB, hematochezia, hematemesis, melena, syncope/presyncope, risk for stress ulcer, cirrhosis, ASA/NSAID use, blood pressure, heart rate, orthostasis, NG lavage, rectal exam, platelet count, creatinine, BUN, INR Resuscitation: hematochezia, hematemesis, melena, duration of symptoms, syncope/presyncope, unstable CAD, blood pressure, heart rate, orthostasis, NG lavage, rectal exam, hematocrit, drop in hematocrit, creatinine, BUN, INR Endoscopy: hematochezia, hematemesis, melena, duration of symptoms, syncope/presyncope, cirrhosis, ASA/NSAIDs, blood pressure, heart rate, orthostasis, NG lavage, rectal exam, hematocrit, drop in hematocrit, platelet count, creatinine, BUN, INR Disposition: age, hematochezia, hematemesis, melena, duration of symptoms, syncope/presyncope, unstable CAD, COPD, CRF, risk for stress ulcer, cirrhosis, blood pressure, heart rate, orthostasis, NG lavage, rectal exam, hematocrit, drop in hematocrit, platelet count, creatinine, BUN, INR


Figure 1. Schematic of training examples for classification

MANAGEMENT –RESUSCITATION, ENDOSCOPY AND DISPOSITION Upper Ulcer with High risk stigmata: Active bleeding, non-bleeding visible vessel, overlying clot, oozing. Varices: Active bleeding or stigmata of recent bleed. AVM/Dieulafoy’s/MW tear: ongoing bleeding/oozing recent bleeding. Lower

Diverticula/Colitis/AVM/Dieulafoys’s/Varices: active bleeding, visible vessel, overlying clot.

Symptoms

Dizzy, unstable CAD, SBP<90, orthostasis, Hct <30, INR >1.5

No Yes

No

Yes

URGENT RESUSCITATION

ASCERTAINING SOURCE OF BLEED Requires endoscopic evidence of a bleeding source Upper: Proximal to ligament of treitz. Mid: Distal to ligament of treitz & proximal to

ileo-cecal valve. Lower: Distal to ileo-cecal valve.

Any stigmata of bleeding (active bleeding, visible vessel, overlying clot, oozing, pigmented spot, ulcer), varices with stigmata of bleed (cherry red spot, angioectasia, red wale signs), AVM’s, MW tear, dieulafoys’s, diverticula.

Yes

URGENT ENDOSCOPY

Yes

Yes or No

ELECTIVE ENDOSCOPY

INTENSIVE CARE UNIT

ADMIT TO FLOOR

DISCHARGE

HOME

Age < 60, no comorbidities

No Yes

No


Figure 2. Accuracies for output-bleeding source (evaluation step) Accuracies for source

0.940.93

0.91

0.93

0.70

0.92

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

1.00

RF SVM SC LDA kNN ANN

Model Type

Accura

cy


Figure 3. Accuracies for output-resuscitation (evaluation step) Accuracies for resuscitation

0.930.94

0.92 0.92

0.88

0.92 0.92

0.65

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

1.00

RF SVM SC LDA kNN ANN logistic LogitBoost

Model Type

Ac

cu

ra

cy


Figure 4. Accuracies for output-endoscopy (evaluation step)

0.790.80

0.81

0.83

0.80

0.780.79

0.63

0.60

0.65

0.70

0.75

0.80

0.85

0.90

RF SVM SC LDA kNN ANN logistic LogitBoost

Model Type

Acc

ura

cy


Figure 5. Accuracies for output-disposition (evaluation step)

0.880.89

0.90 0.90

0.88

0.85

0.58

0.50

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

RF SVM SC LDA kNN ANN LogitBoost

Model Type

Acc

ura

cy


Figure 6. ROC curves for predicting source of bleeding (evaluation step)


Figure 7. ROC curves for predicting resuscitation (evaluation step)


Figure 8. ROC curves for predicting endoscopy (evaluation step)


Figure 9. ROC curves for predicting disposition (evaluation step)


Table 1. Evaluation step–performance of models for output source of bleeding (standard

error)

ACC SN SP PPV NPV AUC

RF 0.943 (0.007)

0.980 (0.004)

0.932 (0.007)

0.967 (0.005)

0.959 (0.006)

0.998

SVM 0.930 (0.007)

0.965 (0.005)

0.945 (0.007)

0.973 (0.005)

0.932 (0.007)

0.979

SC 0.914 (0.008)

0.965 (0.005)

0.890 (0.009)

0.946 (0.007)

0.927 (0.008)

0.978

LDA 0.931 (0.007)

0.965 (0.005)

1.000 (0.000)

1.000 (0.000)

0.935 (0.007)

0.987

kNN 0.697 (0.013)

0.901 (0.009)

0.287 (0.013)

0.717 (0.013)

0.591 (0.014)

0.658

ANN 0.917 (0.008)

0.972 (0.005)

0.936 (0.007)

0.968 (0.005)

0.944 (0.007)

0.999


Table 2. Evaluation step–performance of models for output resuscitation (standard error)


RF 0.932 (0.007)

0.937 (0.007)

0.923 (0.008)

0.954 (0.006)

0.894 (0.009)

0.982

SVM 0.941 (0.007)

0.938 (0.007)

0.945 (0.007)

0.968 (0.005)

0.899 (0.009)

0.964

SC 0.915 (0.008)

0.929 (0.007)

0.891 (0.009)

0.936 (0.007)

0.879 (0.009)

0.920

LDA 0.922 (0.008)

0.904 (0.009)

0.955 (0.006)

0.972 (0.005)

0.852 (0.010)

0.937

kNN 0.884 (0.009)

0.903 (0.009)

0.852 (0.010)

0.914 (0.008)

0.835 (0.011)

0.890

ANN 0.921 (0.008)

0.927 (0.008)

0.910 (0.008)

0.946 (0.007)

0.880 (0.009)

0.993

logistic 0.923 (0.008)

0.939 (0.007)

0.895 (0.009)

0.940 (0.007)

0.897 (0.009)

0.985

LogitBoost 0.647 (0.014)

0.916 (0.008)

0.184 (0.011)

0.662 (0.014)

0.481 (0.014)

0.381


Table 3. Evaluation step–performance of models for output endoscopy (standard error)


RF 0.790 (0.012)

0.854 (0.010)

0.671 (0.014)

0.828 (0.011)

0.712 (0.013)

0.871

SVM 0.803 (0.011)

0.859 (0.010)

0.700 (0.013)

0.842 (0.011)

0.728 (0.013)

0.820

SC 0.811 (0.011)

0.838 (0.011)

0.760 (0.012)

0.866 (0.010)

0.717 (0.013)

0.801

LDA 0.833 (0.011)

0.821 (0.011)

0.857 (0.010)

0.914 (0.008)

0.720 (0.013)

0.843

kNN 0.796 (0.012)

0.876 (0.010)

0.648 (0.014)

0.822 (0.011)

0.737 (0.013)

0.766

ANN 0.778 (0.012)

0.801 (0.012)

0.733 (0.013)

0.849 (0.010)

0.665 (0.014)

0.913

logistic 0.787 (0.012)

0.871 (0.010)

0.831 (0.014)

0.815 (0.011)

0.726 (0.013)

0.853


0.891 (0.009)

0.138 (0.010)

0.658 (0.014)

0.403 (0.014)

0.404


Table 4. Evaluation step–performance of models for output disposition (standard error)


RF 0.883 (0.009)

0.907 (0.008)

0.843 (0.011)

0.908 (0.008)

0.841 (0.011)

0.967

SVM 0.887 (0.009)

0.929 (0.007)

0.816 (0.011)

0.896 (0.009)

0.872 (0.010)

0.922

SC 0.897 (0.009)

0.916 (0.008)

0.866 (0.010)

0.921 (0.008)

0.858 (0.010)

0.891

LDA 0.897 (0.009)

0.891 (0.009)

0.909 (0.008)

0.943 (0.007)

0.830 (0.011)

0.901

kNN 0.876 (0.010)

0.923 (0.008)

0.798 (0.012)

0.886 (0.009)

0.858 (0.010)

0.881

ANN 0.850 (0.010)

0.829 (0.011)

0.889 (0.009)

0.928 (0.007)

0.752 (0.013)

0.972


0.819 (0.011)

0.184 (0.011)

0.629 (0.014)

0.377 (0.014)

0.324


Table 5. Evaluation step–accuracies of predicting resuscitation and endoscopy using

source as a variable (standard error)

RF SVM SC LDA

Resuscitation 0.934 (.007)

0.935 (.007)

0.909 (.008)

0.922 (.008)

Endoscopy 0.783 (.012)

0.787 (.012)

0.809 (.011)

0.818 (.011)

kNN ANN logistic LogitBoost

Resuscitation 0.883 (.009)

0.915 (.008)

0.918 (.008)

0.631 (.014)

Endoscopy 0.794 (.012)

0.783 (.012)

0.792 (.012)

0.631 (.014)


Table 6. Validation step–Predictive accuracies using a 67 patient database

Source of bleeding

Resuscitation Endoscopy Disposition

RF 0.928 0.851 0.753 0.797

SVM 0.826 0.791 0.681 0.783

SC 0.855 0.866 0.696 0.768

LDA 0.768 0.821 0.681 0.797

kNN 0.783 0.821 0.667 0.783

ANN 0.884 0.821 0.638 0.754

logistic N/A 0.791 0.710 N/A

LogitBoost N/A 0.567 0.551 0.362


Table 7. Variable importance using RF and ANN

SOURCE RESUSCITATION

RF ANN RF ANN

Hematemesis 0.1388 10 Syncope 0.1097 10

NG Lavage 0.0692 10 Orthostasis 0.0782 10

Hematochezia 0.0611 10 Hct. Drop 0.0495 10

BUN 0.0484 10 DBP 0.0276 10

Rectal 0.0374 10 Hct. 0.0232 10

Melena 0.0221 9 HR 0.0139 10

Orthostasis 0.0116 10 SBP 0.0114 10

Hx. of GIB 0.0088 8 Hematemesis 0.0052 10

HR 0.0066 10 Melena 0.0032 10

Cr 0.0055 7 Cr 0.0019 1

SBP 0.0051 4 NG Lavage 0.0019 9

DBP 0.0043 9 BUN 0.0018 6

Syncope 0.0032 9 Hematochezia 0.0015 10

INR 0.0031 5 Duration 0.0011 6

Plt 0.0010 6 INR 0.0009 6

Risk for Stress Ulcer

0.0008 10 Rectal 0.0006 7

Cirrhosis 0.0004 10 Unstable CAD -0.0016 5

ASA NSAID -0.0010 1

ENDOSCOPY DISPOSITION

RF ANN RF ANN

Syncope 0.0507 10 Orthostasis 0.0585 10

Orthostasis 0.0259 10 HR 0.0431 10

Hct. 0.0223 10 Hct. 0.0340 10

HR 0.0213 9 SBP 0.0287 10

Hct. Drop 0.0188 10 Syncope 0.0271 10

DBP 0.0178 9 DBP 0.0217 10

Hematemesis 0.0133 10 Hct. Drop 0.0189 10

Rectal 0.0054 9 Rectal 0.0094 10

BUN 0.0051 5 Age 0.0070 10

SBP 0.0046 9 NG_Lavage 0.0055 8

INR 0.0043 2 BUN 0.0054 8

Cirrhosis 0.0039 8 INR 0.0051 4

Melena 0.0022 9 Risk for Stress Ulcer

0.0037 8

Plt. 0.0020 3 Plt. 0.0036 6

Risk for Stress 0.0015 9 Melena 0.0026 9


Ulcer

Duration 0.0008 8 Hematochezia 0.0022 6

Cr. 0.0003 2 Hematemesis 0.0010 7

ASA NSAID 0.00008 4 Cirrhosis 0.0007 8

Hematochezia 0.00001 9 COPD 0.0006 8

NG Lavage -0.0004 8 CRF 0.00007 4

Duration 0.00001 4

Cr. -0.00006 3

Unstable CAD -0.0002 5


References

1 T.A. Rockall, R.F. Logan, H.B. Devlin, T.C. Northfield. Incidence of and mortality from acute upper gastrointestinal haemorrhage in the United Kingdom. Steering Committee and members of the National Audit of Acute Upper Gastrointestinal Haemorrhage. British Medical Journal 311 (1995), pp. 222–226. 2 Baradarian R, Ramdhaney S, Chapalamadugu R, Skoczylas L, Wang K, Rivilis S, Remus K, Mayer I, Iswara K, Tenner S. Early intensive resuscitation of patients with upper gastrointestinal bleeding decreases mortality. American Journal of Gastroenterology. 2004 Apr; 99(4):619-22. 3 Elta GH. Urgent colonoscopy for acute lower-GI bleeding. Gastrointestinal Endoscopy. 2004 Mar;59(3):402-8. Review 4 Das A, Wong RC, Prediction of outcome of acute GI hemorrhage: a review of risk scores and predictive models. Gastrointestinal Endoscopy 2004 Jul;60(1):85-93. 5 Das A, Ben-Menacehm T, Cooper GS, Chak A, Sivak MV Jr, Gonet JA, et al. Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: internal and external validation of a predictive model. Lancet 2003;362:1261-6. 6 Vapnik, V. The nature of statistical learning theory. New York, NY: Springer Verlag. 7 Tibshirani, R, Hastie, T, Narasimhan, B, Chu, G. Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proceedings of National Academy of Science. 2002; 99, 6567-6572. 8 Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123-140. 9 Schapire, RE. The strength of weak learnability. Machine Learning. 1990; 5, 197-227. 10 Breiman, L. Random Forest. Machine Learning. 2001; 45, 5-32. 11 Prakash C, Zuckerman GR. Gastrointest Endosc. 2003 Sep;58(3):330-5. Acute small bowel bleeding: a distinct entity with significantly different economic implications compared with GI bleeding from other locations. Gastrointestinal Endoscopy. 2003 Sep;58(3):409-12.12 Barkun A, Bardou M, Marshall JK; Nonvariceal Upper GI Bleeding Consensus Conference Group. Consensus recommendations for managing patients with nonvariceal upper gastrointestinal bleeding. Annals of Internal Medicine. 2003 Nov 18;139(10):843-57. 13 Non-variceal upper gastrointestinal haemorrhage: guidelines. Gut 2002 Oct;51 Suppl 4:iv1-6. 14 Hay JA, Maldonado L, Weingarten SR, Ellrodt AG. Prospective evaluation of a clinical guideline recommending hospital length of stay in upper gastrointestinal tract hemorrhage. JAMA. 1997 Dec 24-31;278(24):2151-6. 15 Hay JA, Lyubashevsky E, Elashoff J, Maldonado L, Weingarten SR, Ellrodt AG. Upper gastrointestinal hemorrhage clinical--guideline determining the optimal hospital length of stay. The American Journal of Medicine. 1996 Mar;100(3):313-22. 16 Adler DG, Leighton JA, Davila RE, Hirota WK, Jacobson BC, Quereshi WA, Rajan E, Zuckerman MJ, Fanelli RD, Hambrick RD, Baron T, Faigel DO. ASGE guideline: the role of endoscopy in acute non-variceal upper-GI hemorrhage. Gastrointestinal Endoscopy 2004 Oct;60(4):497-504. 17 Klebl F, Bregenzer N, Schofer L, Tamme W, Langgartner J, Scholmerich J,

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=15089891




Messmann H. Risk factors for mortality in severe upper gastrointestinal bleeding. International Journal of Colorectal Disease. 2004 Aug 19. 18 Rockall TA, Logan RF, Devlin HB, Northfield TC. Risk assessment after acute upper gastrointestinal haemorrhage. Gut. 1996 Mar;38(3):316-21. 19 Rockall TA, Logan RF, Devlin HB, Northfield TC. Selection of patients for early discharge or outpatient care after acute upper gastrointestinal haemorrhage. National Audit of Acute Upper Gastrointestinal Haemorrhage. Lancet. 1996 Apr 27;347(9009):1138-40. 20 Velayos FS, Williamson A, Sousa KH, Lung E, Bostrom A, Weber EJ, Ostroff JW, Terdiman JP. Early predictors of severe lower gastrointestinal bleeding and adverse outcomes: a prospective study. Clinical Gastroenterology Hepatology. 2004 Jun;2(6):485-90. 21 Strate LL, Orav EJ, Syngal S. Early predictors of severity in acute lower intestinal tract bleeding. Archives of Internal Medicine. 2003 Apr 14;163(7):838-43. 22 Kalula SZ, Swingler GH, Louw JA. Clinical predictors of outcome in acute upper gastrointestinal bleeding. South African Medical Journal. 2003 Apr;93(4):286-90. 23 Bordley DR, Mushlin AI, Dolan JG, Richardson WS, Barry M, Polio J, Griner PF. Early clinical signs identify low-risk patients with acute upper gastrointestinal hemorrhage. JAMA. 1985 Jun 14;253(22):3282-5. 24 Mortensen PB, Nohr M, Moller-Petersen JF, Balslev I. The diagnostic value of serum urea/creatinine ratio in distinguishing between upper and lower gastrointestinal bleeding. A prospective study. Danish Medical Bulletin. 1994 Apr;41(2):237-40. 25 Zimmerman J, Siguencia J, Tsvang E, Beeri R, Arnon R. Predictors of mortality in patients admitted to hospital for acute upper gastrointestinal hemorrhage. Scandinavian Journal of Gastroenterology. 1995 Apr;30(4):327-31. 26 Terdiman JP, Ostroff JW. Risk of persistent or recurrent and intractable upper gastrointestinal bleeding in the era of therapeutic endoscopy. American Journal of Gastroenterology. 1997 Oct;92(10):1805-11. 27 Corley DA, Stefan AM, Wolf M, Cook EF, Lee TH. Early indicators of prognosis in upper gastrointestinal hemorrhage. American Journal of Gastroenterology. 1998 Mar;93(3):336-40. 28 Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000 Oct 14;356(9238):1318-21. 29 Zaragoza A, Tenias JM, Llorente MJ. Pre-endoscopic prognostic factors in non-varicose upper gastrointestinal bleeding. Development of a predictive algorithm. Rev Esp Enferm Dig. 2002 Mar;94(3):139-48. 30 Molinaro, AM, Simon, R, Pfeiffer, RM. Prediction error estimation: A comparison of resampling methods. Bioinformatics. 2005: 21, 3301-3307. 31 Bamber D. The area above the ordinal dominance graph and the area below the receiver operating graph. Journal of Mathematical Psychology 1975; 12:387-415. 32 Quirk DM, Barry MJ, Aserkoff B, Podolsky DK. Physician specialty and variations in the cost of treating patients with acute upper gastrointestinal bleeding. Gastroenterology 1997 Nov;113(5):1443-8. 33 Timmerman D, Verrelst H, Bourne TH, De Moor B, Collins WP, Vergote I, Vandewalle J.Artificial neural network models for the preoperative discrimination

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Bordley+DR%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Mushlin+AI%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Dolan+JG%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Richardson+WS%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Barry+M%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search%20&itool=pubmed_Abstract&term=%22Polio+J%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Griner+PF%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Mortensen+PB%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Nohr+M%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Moller%2DPetersen+JF%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Balslev+I%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Zimmerman+J%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Siguencia+J%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Tsvang+E%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Beeri+R%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Arnon+R%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Terdiman+JP%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Ostroff+JW%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Corley+DA%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Stefan+AM%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Wolf+M%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Cook+EF%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Lee+TH%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Blatchford+O%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Murray+WR%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Blatchford+M%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Zaragoza+Marcet+A%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Tenias+Burillo+JM%22%5BAuthor%5D

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Llorente+Melero+MJ%22%5BAuthor%5D




between malignant and benign adnexal masses. Ultrasound in Obstetrics and Gynecology 1999 Jan;13(1):17-25. 34 Rosenblatt KP, Bryant-Greenwood P, Killian JK, Mehta A, Geho D, Espina V, Petricoin EF 3rd, Liotta LA. Serum proteomics in cancer diagnosis and management. Annual Review of Medicine 2004;55:97-112. 35 Selaru FM, Xu Y, Yin J, Zou T, Liu TC, Mori Y, Abraham JM, Sato F, Wang S, Twigg C, Olaru A, Shustova V, Leytin A, Hytiroglou P, Shibata D, Harpaz N, Meltzer SJ. Artificial neural networks distinguish among subtypes of neoplastic colorectal lesions. Gastroenterology 2002 Mar;122(3):606-13. 36 Chong CF, Li YC, Wang TL, Chang H. Stratification of adverse outcomes by preoperative risk factors in coronary artery bypass graft patients: an artificial neural network prediction model. Proceedings AMIA Annual Symposium 2003:160-4 37 Lund LH. Comment on: Computerized interpretation of the electrocardiogram. Archives of Internal Medicine 2004, Aug 9-23:164(15):1698-9. 38 Kennedy RL, Harrison RF, Burton AM, Fraser HS, Hamer WG, MacArthur D, McAllum R, Steedman DJ . “An artificial neural network system for diagnosis of acute myocardial infarction (AMI) in the accident and emergency department: evaluation and comparison with serum myoglobin measurements,” Computer Methods and Programs in Biomedicine 52: 93-103, 1997. 39 Lisboa PJ. “A review of evidence of health benefit from artificial neural networks in medical intervention,” Neural Networks 15:11-39, 2002. 40 Ahn, H., Moon, H., Fazzari, M. J., Lim, N., Chen, J. J. and Kodell, R. L. Classification by Ensembles from Random Partitions of High-Dimensional Data. (2007). Computational Statistics and Data Analysis, 51: 6166-6179.







A decision support system to facilitate management of ...hahn/psfile/gastro.pdf · Decision support...

Documents

Transcript of A decision support system to facilitate management of ...hahn/psfile/gastro.pdf · Decision support...