Personalized Discovery and Treatment Recommendation

Jinsung Yoon Mihaela van der SchaarUCLA UCLA

Abstract

In the era of precision medicine, treatmentregimens are selected based on the patientsindividual characteristics to improve the re-sponse. However, given the high dimen-sionality of clinical data, discovering whichdata is relevant to consider when selectingtreatments can be challenging. We presenttwo novel approaches that discover the in-formation (patient features) from the elec-tronic health record that is most relevantfor predicting the best treatment regimen foreach specific patient. We describe two ap-proaches: the first assumes that features areindependent (DE-IF), the second allows forfeatures to be correlated (DE-CF). We eval-uate the performance of these approachesagainst other methods using a data set of50,000 cases. Different patient features arerelevant to predicting the success of differ-ent treatments; DE-IF and DE-CF can dis-cover these different relevant features and usethem to make personalized treatment recom-mendations. We determine rewards based onthe results of external knowledge (publishedliterature and clinical practice guidelines) de-scribing the effect of the same treatments onsimilar patients as the ones considered in ourstudy. In general, our approach achieves a16.6% improvement over other methods interms of kappa statistics. We also demon-strate that the performance of our methodsis robust against missing information.

1 INTRODUCTION

A key aspect of precision medicine is the ability topersonalize the selection of a treatment regimen based

Preliminary work. Under review by AISTATS 2016. Donot distribute.

on the personal characteristic of an individual patient(Richmond, 2008). The wealth of information be-ing routinely collected as part of the electronic healthrecord (EHR) provides an unprecedented opportunityto discover appropriate treatments for patients givenhistorical information about the treatments adminis-tered to similar patients and their outcomes (Shortliffeand Cimino, 2006). However, using this informationis difficult precisely because there is too much of it;what is needed is to extract the information that isactually relevant for the particular patient and treat-ment among the wealth of available information.

In this paper, we present a novel method called Dis-covery Engine (DE) for optimizing treatment selec-tion by identifying the features in the patient recordthat differentiate the individuals who receive a treat-ment and positively respond to it versus those whodo not. We describe two variants of this approach: asimplified method that assumes all features are inde-pendent (DE-IF) and one that takes into account fea-ture correlations (DE-CF). Our approaches leverageavailable contextual information about patients andenable learning from the large quantities of observa-tional clinical data to inform treatment recommenda-tions and make better decisions by learning from simi-lar patients. We show that our approaches consistentlyperform better than existing approaches in matchingindividual patients to relevant treatment regimens.

One of the biggest challenges faced by this researchor any other predictive algorithms is that the re-wards/true outcomes of treatment (e.g. five-year pro-gression free survival) are not available in the datawe have and indeed, are usually not available in mostdata (Bennett and Hauser, 2012). Moreover, even ifrewards/true outcomes of treatment were available,the counterfactuals rewards/outcomes of alternativetreatments that were not used are never available.What is available is a large medical literature that re-ports the results of a wide range of clinical referencesincluding different types of patients, different types oftreatments, and the outcomes of these treatments. Weuse the results of some of these references to constructa transfer reward, which we use as a proxy for rewards.

Manuscript under review by AISTATS 2016

This allows us to train the DE algorithms and alsoto assess their performance in comparison to existingmethods. The three primary contributions of this pa-per are:

• We describe a method for discovering the relevantinformation from the EHR that distinguishes be-tween patients that should receive one particularchemotherapy and the patients who should receiveanother. For instance, patients with observed mu-tations (e.g., EGFR mutation and HER-2 overex-pression) are more likely to respond to a specifictype of chemotherapy (e.g., erlotinib) (Zhou et al.2011).

• Using the retrospective cases in the EHR, our ap-proach discovers the optimal treatment based onthe available information (e.g. their clinical find-ings, treatment history, and outcomes).

• In lieu of having actual reward values associatedwith treatment decisions, we define the transferrewards, a method for ranking chemotherapies de-scribed in external knowledge based on their rel-evance to individual patients based on reportedcharacteristics.

2 RELATED WORKS

Current medical practice relies on manually curatedsystematic reviews and clinical guidelines that providetreatment recommendations for large groups of pa-tients rather than personalized treatments that are tai-lored to individual patients. Clinical decision supportsystems have been proposed before, but many of themdo not consider the specific characteristics of patientsand do not provide personalized treatment recommen-dations; hence, they are not very accurate and haveonly limited applicability in practice (Tsumoto, 1998,Greenspan and Syeda-Mahmood, 2012, and Xu et al.2014). Moreover, clinicians often refer to the medicalliterature available through Medline/PubMed, Visu-alDX, and UpToDate to help them associate observedfinding with possible conditions and recommended ac-tions; but, these resources are not customized to aspecific patients case.

Another strand of literature related to this work isthat on machine learning techniques, including Sup-port Vector Machines (SVMs), AdaBoost, logistic re-gression etc. are not able to accurately capture the nu-anced relationships between specific patient character-istics and specific treatments, thereby not being ableto issue accurate personalized treatment recommen-dations. (Comparisons against these methods are per-formed in the experiments section (Peterson, Doughty,

and Kann, 2013).) One of the reasons for inaccuraterecommendation is that those techniques cannot learnthe relevant features of patients that make them re-spond to specific treatments.

Our method also exhibits similarities to the contex-tual multi-armed bandit problem (MAB). However,contextual MABs are very inefficient when the num-ber of contexts (in our case patient characteristics) islarge (Tekin and van der Schaar, 2015). DE is able tosuccessfully deal with the curse of dimensionality bydiscovering what information is relevant and makingdecisions based on relevant contexts rather than theentire EHR, much of which is irrelevant to the deci-sion of whether to administer a specific treatment ornot.

3 PROBLEM FORMULATION

In this section, we introduce our method - discoveryengine (DE). Figure 1 depicts the proposed system,which issues a treatment recommendation to the physi-cian that is personalized to the patient. While the pro-posed system is applicable in general, we illustrate itsuse and performance in the context of breast cancer.Let x = (x1, x2, ..., xD) denote the patient informationwhere D is the total number of features of each patientsuch as age, tumor size, estrogen receptor informationetc.; a ∈ A ≡ {a1, a2, ..., aK} denotes the action (i.e.,treatment) that is performed on the patient. Each fea-ture is denoted as f ∈ F ≡ {f1, f2, ..., fD}. The rewardy would ideally be derived as the probability of five-year survival of a patient given the chemotherapy reg-imen. However, obtaining this reward value is difficultin practice; instead, we use a surrogate reward mea-sure which is discussed in a subsequent section. Letx(n), a(n), y(n) be the patient information, action andreward of nth patient and HN = {x(n), a(n), y(n)}Nn=1

be the information available for the N previously seenpatients. (This represents the training set.)

The success of a treatment a does not depend on allthe features (Goldstein et al. 2008): we assume thatthe success of a treatment a depends only on a sub-set of features R(a) ⊆ F which we call the relevantfeatures and let R =

⋃a∈AR(a) be the set of all rel-

evant features. A key challenge is that the featurethat are relevant for predicting the success of a treat-ment are not known a priori; they need to be dis-covered/learned. Note that such learning/discoveryis very different from existing feature selection algo-rithms which focus on the patients characteristics andnot on how these characteristics differently impact thesuccess of different treatments: our approach is capa-ble of discovering different features that are relevantto different treatments. We say that R(a) is rele-

Manuscript under review by AISTATS 2016

Figure 1: Personalized treatment recommendation system using DE

vant/informative for treatment a if the expected re-ward only depends on the information contained inR(a).

Figure 1 illustrates the aforementioned for-mulation using an example, which discoversR(Chemotherapy1)={Age, Prior Chemotherapy}, i.e.the patients response to Chemotherapy 1 depends onthe patient age and the prior chemotherapy but noton the lymph node status.

Our goal is to discover the relevant features of eachtreatment a (this may be different for each treatment)and determine the optimal treatment that correspondsto the information contained within the relevant fea-tures. The optimal recommended treatment is givenby,

a∗(xR) ≡ argmaxaE(y|a,xR(a))(y|a,xR(a))

where a∗(xR) is the treatment that yields the high-est reward (e.g., best patient outcome) for a pa-tient characterized by the relevant features (age, priorchemotherapy) xR.

4 DE ALGORITHM

4.1 Independent Features (DE-IF)

We begin by describing our approach under the sim-plifying assumption that the features are independent.The main idea of this algorithm is to find the relevantfeatures for a specific treatment that distinguish theexpected rewards for patients with specific patient in-formation (i.e., age 50+ patients) and those obtainedfor the entire population. Features that create a largereward separation are considered relevant to that spe-

cific treatment. Then, DE-IF simultaneously learnsthe relevant features that predict the success of a treat-ment and the best treatment given a patients charac-teristic. The key steps of DE-IF are outlined below:

Step 1: Define ySa (xS) and NSa (xS) as the sample

mean reward estimator and the number of patientswhose information contains xS for treatment a. In-tuitively, the sample mean reward estimator can bethought of as the average reward of treatment a forthe patients with information xS (such as age= 50+,tumor size= 4mm patients). Then, let ya be the sam-ple mean reward estimator for treatment a and Nabe the number of patients to whom treatment a wasrecommended.

Step 2: Using the reward estimates for all (a, f), thealgorithm computes a relevance metric hf (a) for eachtreatment feature pair (a, f), i.e.,

hf (a) ≡∑xf

Nfa (xf )Na|yfa (xf )− ya|

that measures the weighted variation of the reward es-timates. Recall that f was defined earlier as a featureitself.

Step 3: Using the relevance metric for each pair (a, f),we discover the relevant feature (r(a)) for each treat-ment a.

r(a) = argmaxf hf (a)

Define m(a) as the number of relevant features andR(a) as the set of the relevant features for treatmenta. We sequentially discover m(a) number of relevantfeatures (r(a)) for each treatment a

Manuscript under review by AISTATS 2016

Initialize: R(a) = ∅, F (a) = {f1, ..., fD} for each afor each treatment-feature pair (a, f) do

Compute hf (a) =∑xf

Nfa (xf )Na|yfa (xf )− ya|

end forfor each treatment a do

dor(a) = argmaxf∈F (a) hf (a)

R(a) = R(a)⋃r(a)

F (a) = F (a)\r(a)while (|R(a)| < m(a))

end forfor any patient information xR do

Compute a(xR) = argmaxa yR(a)a (xR(a))

end for

Figure 2: Psuedocode of DE-IF

Step 4: Define R =⋃a∈A R(a) as the relevant fea-

tures for all treatments. The optimal treatment withrespect to relevant features R is determined as

a(xR) = argmaxa yR(a)a (xR(a))

This optimization selects the treatment with the high-est estimated reward given a patient with patient in-formation xR. The pseudo-code is given in Figure 2.

4.2 Correlated Features (DE-CF)

We now describe our approach for the more generalcase in which features are allowed to be correlated.For example, for some cancers, tumor size and tumorgrade may be correlated, not independent. Therefore,we provide a generalization of the DE algorithm calledDE-CF that properly treats correlated features. Whilethe basic steps are similar to DE-IF, DE-CF uses adifferent relevance metric:

hS(a) ≡∑

xS

NSa (xS)Na

|ySa (xS)− ya|

Then, the most relevant tuple (consisting of m(a) fea-tures) associated with treatment a can be found as

R(a) = argmaxS:|S|=m(a) hS(a)

The pseudo-code of DE-CF is given in Figure 3.

4.3 Contrasting DE-IF and DE-CF

DE-CF suffers from the curse of dimensionality be-cause it needs to estimate rewards for each tuple offeatures and hence it needs to see many tuples, not justmany features. For instance, it needs to see many caseswith age 50+ and tumor size 4cm not just many caseswith age 50+ or tumor size 4cm. Hence, if the data set

Initialize: R(a) = ∅ for each afor each treatment-feature set (a, S) do

Compute hS(a) ≡∑

xS

NSa (xS)Na

|ySa (xS)− ya|end forfor each treatment a do

Compute R(a) = argmaxS:|S|=m(a) hS(a)end forfor any patient information xR do

Compute a(xR) = argmaxa yR(a)a (xR(a))

end for

Figure 3: Psuedocode of DE-CF

is small, DE-CF may not yield high confidence. Forthis reason, we suggest that for small data sets, it islikely better to make the perhaps incorrect assumptionof independent features and use DE-IF rather thanDE-CF.

4.4 DE with Missing Information

Electronic health records are often missing hence, DEmust be able to operate efficiently even when informa-tion is missing. When information is missing, the fea-ture information vector x can be divided into two com-ponents: the available features (xav) and the missingfeatures (xm). Thus, x = {xav,xm}. When featuresare missing, the relevance metric of DE-IF is solelycomputed based on the available information:

hf (a) ≡∑xavf

Nfa (xav

f )

Na|yfa (xavf )− ya|

If the feature f is frequently missing, hf (a) decreases,and as a result the feature f is rarely selected as arelevant feature. The relevance metric of DE-CF issimilarly modified. It should be also note that the es-

timated reward (yR(a)a (xav

R(a))) given a patients avail-

able relevant information, xavR(a)

, for each treatment a

is computed as:

yR(a)a (xav

R(a)) = E(y

R(a)a (xR(a))|xavR(a)

)

=∑

xm

R(a)

P (xmR(a)|xavR(a)

) · yR(a)a (xav

R(a),xmR(a)

)

=∑

xm

R(a)

P (xmR(a)|xavR(a)

) · yR(a)a (xR(a))

We can estimate the conditional probability,P (xm

R(a)|xavR(a)

), based on the probability distri-

bution of the features in training set.

For example, let us assume that there are two relevantfeatures (menopausal status and tumor stage) for thetreatment a. Suppose now that a patients tumor stage

Manuscript under review by AISTATS 2016

is G1 and her menopausal status is missing. Then tocompute the estimated reward for selecting a treat-ment for that patient, we use the conditional proba-bility of being premenopausal given that the patienthas a G1 tumor stage. Let us assume that this is 0.7(as computed based on the patients in the trainingset), then DE estimates the reward of treatment a forthis patient as:

yR(a)a (G1) = 0.7 · yR(a)

a (Pre,G1)+0.3 · yR(a)a (Post,G1)

5 TRANSFER REWARDS:ESTIMATING REWARDS BASEDON MEDICAL LITERATURE

As we have said in the introduction, the most naturalrewards would be patient outcomes (survival rates) butthese outcomes are very difficult to obtain in practice.Instead, we use a proxy for outcomes based on exter-nal knowledge which consists of published literatureand clinical practice guidelines. Let us refer to all ex-ternal knowledge as the term reference for the rest ofthe paper from here on. The idea is to match patientsto relevant reference given patient characteristics. Foreach patient (represented by each features) and eachreference, we define the amount of information thatreference provides about that patient; we then aggre-gate this information across all references. From thisinformation, we compute the posterior probability thatthe treatment used in a given reference is the besttreatment for that patient. This defines the transferrewards, which we take as the reward for that treat-ment when applied to that patient

To compute the transfer rewards, we first find relevantpatient features for each reference; we do this using amodified feature selection algorithm based on the mu-tual information (Peng, Long, and Ding, 2005). Thedefinition of the mutual information between the ith

reference (Ei) and the kth feature (fk) is:

I(fk;Ei) =∑x∈χk

P (x|Ei) log P (x|Ei)P (x)

where P (x|Ei) is the probability of feature x in refer-ence i, P (x) is the probability of feature x across theentire set of references, and χk is context space of fk.

The second step is to compute the transfer rewards asa posterior probability. Given the nth patient, charac-terized by the feature vector xn = {x1(n), ..., xD(n)},we compute the posterior probability that the treat-ment in reference i would be successful when appliedto a patient with these features; we abuse notation bywriting this as, P (Ei|X1 = x1(n), ..., XD = xD(n)).We compute this posterior probability via Bayes rule;it is computationally convenient to take logarithms:

log (P (Ei|X1 = x1(n), ..., XD = xD(n))

= log (P (X1=x1(n),...,XD=xD(n)|Ei)P (X1=x1(n),...,XD=xD(n)) · P (Ei))

∼= logP (Ei) +∑l∈Rel log P (Xl=xl(n)|Ei)

P (Xl=xl(n))

where with similar abuse of notation we write P (Ei)as the probability of selecting the ith reference as thebest treatment for the entire population.

We computed all the transfer rewards for each ref-erence/treatment. These transfer rewards provide acomplete ranking of each treatment for each patient(characterized by the relevance features). The treat-ment with the highest transfer reward is the best treat-ment for the given patient.

6 EXPERIMENTS

6.1 Dataset

We perform experiments on a de-identified data set of50,000 patient cases which underwent screening anddiagnostic testing at a large academic medical center.The patient data is characterized by 15 features sum-marized in Table 1.

Table 1: Summary of patient information type

Features RangeAge 30s - 60+

Menopausal Pos/Neg

Race White/Black/Other

Estrogen Receptor Pos/Neg

Progesteron Receptor Pos/NegHER2NEU Pos/Neg/Neu

Tumor Stage T1 - T4

Tumor Grade G1 - G3

PLNC 0 - 10+

Lymph Node Status Pos/Neg

WHO Score 0 - 5

Surgery Type BCT/MRM/No

Prior Radiotherapy Experience / No

Prior Chemotherapy Experience/No

Histology Ductal/Mix/Lobular

From an initial set of 2,353 references (performinga narrow search of therapies using PubMed Clini-cal Queries), six chemotherapies listed in Table 2,the standard chemotherapy regimen for breast cancer(Fleeger et al. 2015, Hamel and Johnson, 2015, andLevine, 2001), described in 32 references were selectedfor further analysis. The sample size of reported refer-ences ranged from 50 to 3,934 with a total populationof 27,118 individuals.

Manuscript under review by AISTATS 2016

We utilize a training set of 4,000 patients. The re-maining patients provide the test set. Standard 10-foldstratified cross-validation was applied, and therefore,no training data were used during testing of the model,but 10 different models were used to derive the finaltest results.

Table 2: Code for each chemotherapy regimen

Code Specific Chemotherapy RegimenAC Doxorubicin + Cyclophosphamide

ACT Doxorubicin + Cyclophosphamide +

Taxanes

AT Doxorubicin + Taxanes

CAF Cyclophosphamide + Doxorubicin +

5-Fluorouracil

CEF Cyclophosphamide + Epirubicin +

5-Fluorouracil

CMF Cyclophosphamide + Methotrexate +

5-Fluorouracil

6.2 Benchmarks

We compare the performance of DE-IF with six ex-isting popular machine learning algorithms describedbelow:

• Correlation Feature Selection (CFS): a well-known feature selection algorithm (Hall, 1998);

• All Contextual Learning (ACL): a well-knowncontextual learning algorithm which uses all fea-tures. This is a modified offline version of thecontextual bandit algorithm of Slivkins. (2014);

• Logistic Regression;

• Linear Regression;

• Support Vector Machines (SVM); we use a radialbasis function (RBF) kernel SVM;

• Adaptive Boosting (AdaBoost);

6.3 Measuring Success

Given a patient, our algorithm or any of these otheralgorithms recommend a course of treatment corre-sponding to a particular reference. If a patient hasexactly the same feature information as the patientin that reference, we regard the algorithm in questionas making the correct recommendation for that pa-tient; i.e. it has recommended the best course of treat-ment. (Notice that the best course of treatment maynot promise a good outcome: some cancers are nottreatable.) We take the fraction/percentage of correct

recommendations to be the success rate for the algo-rithm in question.

Given the success rate for the algorithm, we applytwo performance metrics: simple percent agreement(p0) and Cohens kappa coefficient (κ). Simple percentagreement (po) is the success rate (the fraction of timesthe personalized treatment prediction coincides withthe recommendation provided in the medical literaturefor the patients with the same characteristics). Cohenskappa coefficient (κ) is a metric which measures inter-rater agreement. It is usually considered a more robustmeasure than a simple percent agreement, because κmeasures the improvement over chance agreements. Ifpe is the probability of agreement by chance, then,κ = po−pe

1−pe .

As the bar graphs in Figure 4 show, the firstchemotherapy recommendation of DE-IF is success-ful (as defined above) 73.4% of the time and one ofthe first two recommendations is successful 88.4% ofthe time. This is 7.7% better than the next best ap-proach (i.e., SVM) in terms of selecting the optimalchemotherapy on its first choice, and 5.6% better interms of matching the optimal chemotherapy withinthe first 2 choices. This is already a significant im-provement. In terms of kappa statistics, the improve-ment is even greater: our algorithm works 16.6% bet-ter than SVM. This is because SVM indiscriminatelyrecommends the popular chemotherapies and is not ro-bust when classifying the less popular chemotherapies.Given robustness considerations, which are essential inmedical treatment recommendations, kappa statisticsis more often used as a performance metric in medicalinformatics.

It is especially useful to compare our algorithm withother algorithms that rely on feature selection. Again,note that other algorithms use feature selection, butthey do not select relevant features for specific treat-ments and it is by doing this that our algorithmachieves improvement. CFS achieves only a 48% ofsimple percent agreement because it cannot use theefficacy of the treatment to discriminate the relevantfeatures and hence the technique is entirely unsuper-vised. ACL succumbs to the curse of dimensionalitybecause there are 15 features, resulting in over 500million combinations to test. Logistic regression, lin-ear regression, and SVM perform worse than DE-IFbecause they do not consider the relevant patient in-formation for selecting treatments.

DE-CF performs 9.4% worse than DE-IF in termsof simple percent agreement. This is not surprising:4,000 cases are simply not enough for DE-CF to es-timate reward of feature tuples. The 95% confidenceintervals of DE-IF and DE-CF are 4.4% and 5.1% re-

Manuscript under review by AISTATS 2016

Figure 4: Performance analysis with benchmark algorithms (a) Kappa statistics, (b), (c) Simple percent agree-ment (1st and 2nd)

spectively. Note also that all the other algorithms withthe exception of DE-IF and DE-CF cannot issue suchconfidence estimates.

Because a large academic medical center may be ableto obtain a sufficient cohort to train treatment recom-mendation algorithms such as the one presented, it isoften informative to know the amount of cases neededto ensure that the near-optimal treatment is recom-mended with a high probability. Our simulation re-sults (Table 3) show that: DE-IF requires 4,000 casesand DE-CF requires 40,000 cases to reach near-optimaltreatment recommendation. After 35,000 cases, DE-CF works better than DE-IF because DE-CF has morepractical assumption for patients features and 35,000cases are enough to discover relevance in this applica-tion.

Table 3: Simple percent agreement analysis with var-ious number of cases

Past Patients Number (×103)% 2 4 10 30 40 45

DE-IF 67.4 73.4 73.0 73.3 73.3 73.4

DE-CF 57.9 64.0 69.4 72.7 74.5 74.9

AdaBo 54.3 55.4 56.3 57.9 56.9 57.1

6.4 Relevant Features for EachChemotherapy

Table 4 shows the top 4 ranked relevant features dis-covered by DE-IF - tumor stage, positive axillarylymph node number (PLNC), estrogen receptor etc.-for recommending AC, ACT, AT, CAF, CEF andCMF chemotherapy. As it can be seen from Table 4,DE-IF is able to discover the different relevant featuresthat are relevant for different chemotherapy.

Importantly, the relevancy of the features discov-

Table 4: Discovered relevant features for eachchemotherapy

Chemo Relevant

Feature1

Relevant

Feature2

Relevant

Feature3

Relevant

Feature4

AC PLNC Tumor

Stage

Estrogen

Receptor

Age

ACT Tumor

Stage

Prior

Chemo

PLNC Estrogen

Receptor

AT Prior

Chemo

PLNC Surgery Age

CAF Surgery Tumor

Stage

Age Tumor

Grade

CEF PLNC Estrogen

Receptor

Tumor

Stage

Age

CMF Estrogen

Receptor

PLNC Radio

therapy

Tumor

Stage

ered by DE is confirmed by clinical studies. First,note that the six considered chemotherapies are com-monly recommended to node positive breast cancerpatients, i.e. patients where cancer has been foundin the lymph nodes (Hamel, and Johnson, 2015).PLNC is the most relevant feature to differentiateamong node positive or node negative breast cancer.Hence, PLNC is correctly identified by DE to be rel-evant. Second, the menopause status is considered asan important factor because medications affect can-cer differently for premenopausal and postmenopausalwomen (Mark, 2001). More specifically, the CEFchemotherapy is only recommended to premenopausalwomen. Although the menopausal status is not in-cluded in this relevant feature set, women over theage of 50 are usually considered as postmenopausal(Mark, 2001). Therefore, age was correctly identi-fied by DE to be a discriminative feature for select-

Manuscript under review by AISTATS 2016

ing among these chemotherapies. Third, tumor stageis another important feature to discriminate amongchemotherapy treatments as confirmed in (Fleeger etal. 2015). Chemicals A(Doxorubicin), T(Taxotere),E(Epirubicin) are highly recommended to advancedbreast cancer and our top five chemotherapies includemore than one of these chemicals. Therefore, DE hascorrectly discovered that the features that are relevantfor these chemotherapies contain tumor stage informa-tion. Finally, chemical T(Taxotere) is usually recom-mended to patients having breast cancer which do notreact to the current chemotherapy. Thus, the priorchemotherapy information is indeed correctly discov-ered by DE to be relevant for AT and ACT therapies.

6.5 Performance When Patient Informationis Missing

As mentioned before, information is often missing froma patients EHR. Moreover, studies have shown thatthe missing information is often not random (Botsis etal. 2010). For example, the age of the patient is easyto record and is often verified so it is typically nei-ther missing nor incorrect. However, her2neu may notbe recorded depending on the diagnostic tests orderedand capabilities of a clinic. Figure 3 shows the perfor-mance degradation of DE and benchmark algorithmsas a function of the average degree of incompleteness(Botsis et al. 2010). (We did not use the percent-age of missing features as a metric as the featuresare not randomly missing. Which features are miss-ing features and the percentages of missing featuresare described in the supplementary materials.) Fig-ure 5 shows that the performance of DE-IF degradesfrom 73.4% to 63.0% (when the average degree of in-completeness is 50%. However, DE-IF continues tooutperform the other methods. DE discovers relevantfeatures with low missing probability, and is able toestimate the missing feature information based on theavailable feature information, so the impact of missinginformation is minimized. In fact, DE does better thanmost other algorithms even when DE misses significantamounts of information from the EHR while the otheralgorithms make their decision without missing infor-mation. Hence DE is robust to missing information.

7 CONCLUSION AND FUTUREWORK

We describe a novel approach that discovers the rele-vant information from the EHR that is important indetermining which chemotherapy to assign to a patientand use this information to provide personalized treat-ment recommendations to the physician. Our resultsdemonstrate that DE is capable of outperforming ex-

Figure 5: Performance analysis with missing informa-tion

isting machine learning and prediction techniques. Wealso show that our method is robust against missinginformation.

Future work will consider that various chemotherapytreatments contain similar chemicals, and will extendDE to incorporate the correlations of the treatments totransfer the knowledge of one to another treatment. Inaddition, variables such as the tumor size and PLNCnumber may change over time. These changes may in-fluence the duration of a therapy and the selection offuture therapies. Currently, we only consider a singleaction, identifying what information is important indeciding on a single treatment. Another DE extensionwill consider the global sequence of treatment deci-sions that optimize long-term outcomes (e.g., overallsurvival).

In conclusion, we believe that our proposed contextuallearning approach demonstrates promise towards pro-viding personalized treatment recommendations andleveraging the external knowledge to provide rewardestimation. As new therapies are evaluated and ap-proved for use, clinicians will have an increasingly dif-ficult time determining which treatment is most ef-fective for an individual patient. DE provides a steptowards providing computational methods for person-alized treatment recommendation.

Manuscript under review by AISTATS 2016

References

M. E. Trudeau (2001). Optimizing adjuvant breastcancer chemo-therapy: rationale for the MA. 21 study.Oncology 15(5 Suppl 7), 7-13.

L. J. Goldstein, A. O’Neill, J. A. Sparano, E. A. Perez,L. N. Shulman, S. Martino, N. E. Davidson (2008).Concurrent doxoru-bicin plus docetaxel is not moreeffective than concurrent doxoru-bicin plus cyclophos-phamide in operable breast cancer with 0 to 3 pos-itive axillary nodes: North American Breast CancerIntergroup Trial E 2197. Journal of Clinical Oncology26(25), 4092-4099.

A. Badanidiyuru, J. Langford, A. Slivkins (2014).Resourceful contextual bandits. arXiv preprintarXiv:1402.6779.

H. Peng, F. Long, C. Ding (2005). Feature selec-tion based on mutual information criteria of max-dependency, max-relevance, and min-redundancy.Pattern Analysis and Machine Intelligence, IEEETransactions on, 27(8), 1226-1238.

K. S. Albain (2004). Adjuvant Chemotherapy forLymph NodeNegative, Estrogen ReceptorNegativeBreast Cancer: A Tale of Three Trials. Journal ofthe National Cancer Institute, 96(24), 1801-1804.

M. Levine (2001). Clinical practice guidelines for thecare and treatment of breast cancer: adjuvant sys-temic therapy for node-positive breast cancer (sum-mary of the 2001 update). Canadian Medical Associ-ation Journal, 164(5), 644-646.

C. Tekin,M. van der Schaar (2015). RELEAF: An al-gorithm for learning and exploiting relevance. IEEEjournal of Selected Topics in Signal Processing, 9(4),716-727

T. D. Richmond (2008). The current status and futurepotential of personalized diagnostics: streamlining acustomized process. Bio-technology annual review, 14,411-422.

A. X. Garg, N. K. Adhikari, H. McDonald, M. P.Rosas-Arellano, P. J. Devereaux, J. Beyene, R. BHaynes (2005). Effects of computerized clinical de-cision support systems on practitioner performanceand patient outcomes: a systematic review. Jama,293(10), 1223-1238.

E. H. Shortliffe, J. J Cimino (2013). Biomedicalinformat-ics: computer applications in health care andbiomedicine. Springer Science and Business Media.

C. C. Bennett, K. Hauser (2013). Artificial intelligenceframework for simulating clinical decision-making: AMarkov decision process approach. Artificial intelli-gence in medicine, 57(1), 9-19.

H. G. H. Mller, T. Syeda-Mahmood (2013). MedicalContent-Based Retrieval for Clinical Decision Support.

S. Tsumoto (1998). Automated extraction of medicalexpert system rules from clinical databases based onrough set theory. Information sciences, 112(1), 67-84.

A. Hyvrinen, E. Oja, (2000). Independent compo-nent analysis: algorithms and applications. Neuralnetworks, 13(4), 411-430.

T. A. Peterson, E. Doughty, M. G. Kann (2013). To-wards precision medicine: advances in computationalapproaches for the analysis of human variants. Journalof molecular biology, 425(21), 4047-4063.

C. Zhou, Y. L. Wu, G. Chen, J. Feng, X. Q. Liu, C.Wang, C. You (2011). Erlotinib versus chemother-apy as first-line treatment for patients with advancedEGFR mutation-positive non-small-cell lung cancer(OPTIMAL, CTONG-0802): a multicentre, open-label, randomised, phase 3 study. The lancet oncology,12(8), 735-742.

B. Kaplan (2001). Evaluating informatics application-sclinical decision support systems literature review.International journal of medical informatics, 64(1), 15-37.

J. Xu, D. Sow, D. Turaga, M. van der Schaar (2014).Online Transfer Learning for Differential Diagnosis De-termination. AAAI Workshop on the World WideWeb and Public Health Intelligence

A. Slivkins (2014). Contextual bandits with similarityinformation. The Journal of Machine Learning Re-search, 15(1), 2533-2568.

T. Lu, D. Pl, M. Pl (2010). Contextual multi-armedbandits. In International Conference on Artificial In-telligence and Statistics, 485-492.

M. A. Hall (1999). Correlation-based feature selectionfor machine learning . Doctoral dissertation, The Uni-versity of Waikato

F. M. Fleegler, J. Griggs, B. Reiner, B. Re-ville, S. F. Schnall, M. C. Weiss, L. Weissmann(2015). Chemotheray Medicine, Re-trieved fromhttp://www.breastcancer.org/treatment/chemotherapy/medicines

L, Mark (2001). Clinical practice guidelines for thecare and treatment of breast cancer: adjuvant sys-temic therapy for node-positive breast cancer (sum-mary of the 2001 update). Canadian Medical Associ-ation Journal, 644-646.

P. J. Hamel, P. Johnson (2015). ChemotherayRegimen for Breast Cancer, Retrieved fromhttp://www.healthcentral.com/breast-cancer/chemo-

Manuscript under review by AISTATS 2016

regimen.html

Y. Yan, R. Rosales, G. Fung, F. Farooq, B. Rao, J. G.Dy (2012). Active learning from multiple knowledgesources. In International Conference on Artificial In-telligence and Statistics, 1350-1357.

T. Botsis, G. Hartvigsen, F. Chen, C. Weng (2010).Second-ary use of EHR: data quality issues and in-formatics opportuni-ties. AMIA summits on transla-tional science proceedings, 1.