Microarray Analysis with Bayesian Networks

29
Microarray Analysis with Bayesian Networks Using BayesiaLab for Cancer Type Classication Stefan Conrady , [email protected] Dr. Lionel Jouffe, [email protected] March 15, 2011 Conrady Applied Science, LLC - Bayesia’s North American Partner for Sales and Consulting

Transcript of Microarray Analysis with Bayesian Networks

Page 1: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 1/29

Microarray Analysis with Bayesian Networks

Using BayesiaLab for Cancer Type Classication

Stefan Conrady, [email protected] 

Dr. Lionel Jouffe, [email protected]

March 15, 2011

Conrady Applied Science, LLC - Bayesia’s North American Partner for Sales and Consulting

Page 2: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 2/29

Table of Contents

Introduction

About the Authors 2

Stefan Conrady 2

Lionel Jouffe 2

Case Study & Tutorial

Background 3

Database 3

Notation 4

Classication Model 4

Data Import 4

Supervised Learning: Augmented Markov Blanket 10

Performance Evaluation 12

Network Complexity 12

Inference 16

Target Dynamic Prole 17

Bayes Factor 19

Target Interpretation Tree 20

Conclusion 21

Appendix 23

Markov Blanket 23

Comparison of Classication Performance with Golub et al. (1999) 24

References 26

Contact Information 27

Conrady Applied Science, LLC 27

Bayesia SAS 27

Copyright 27

Introduction to Bayesian Networks

www.conradyscience.com| www.bayesia.com i

Page 3: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 3/29

Introduction

In our recent white paper about breast cancer classication, we have used Bayesian networks and BayesiaLab for feature

identication and prediction of class membership. This study was based on 569 cases and 10 attribute variables, which

allowed estimating a classication model with a very high prediction accuracy.

In this new study, we turn to the eld of cancer classication by means of microarray analysis. Microarray analysis is a

technique for gene expression proling of cell samples. Expression proles indicate which genes are currently active

among thousands of genes. The activation of certain genes can indicate the type and the current state of a cell.

In our case, we want to use the expression proles of cell samples from cancer patients to distinguish between different

types of leukemia. Leukemia is a type of cancer of the blood or bone marrow characterized by an abnormal increase of 

white blood cells. Clinically and pathologically, leukemia can be divided into a number of groups, of which we will ex-

amine two types of acute leukemia, namely acute lymphoblastic leukemia (ALL) and acute myelogenous leukemia 

(AML).

The correct classication of the subgroup of leukemia is critical for the selection of the most ef cient therapy, which

may include chemotherapy and radiation, and for minimizing side effects. In general, the progress in correct cancer clas-

sication in recent years has been crucial for improving the overall treatment success.

One of the challenges in microarray analysis is the sheer number of genes, which could potentially be predictors in a

classication model. At the same time, the number of observations tends to be small. So, it is not uncommon to have

thousands of predictors while only having a few dozens of samples. It is precisely the opposite of what one would hope

to have for a traditional statistical analysis.

As a result, many new statistical techniques have emerged in recent decades and one of them is described in detail in

Golub et al. (1999). This study demonstrates that cancer classication is feasible on the basis of gene expression data

alone. Since its publication, it has been widely cited and further disseminated, e.g in Slonim et. al (2000) and Dudoit et

al. (2002). Also, the underlying dataset has been made publicly available to any interested researcher by the Broad

Institute.1 Given the seminal nature of the Golub study and its excellent pedagogical qualities, we have chosen it as our

reference point for a new case study and BayesiaLab tutorial.

Our objective is to show that our modeling approach with Bayesian networks (as the framework) and BayesiaLab (as

the software tool) can quickly and effectively generate models of equal or better classication performance compared to

models documented in literature, while only requiring a minimum of specication effort from the research analyst.

We expect that this new approach will allow researchers to focus a greater portion of their efforts on the subject matter

of their studies, e.g. the biological interpretation, and less on the technicalities of statistical models. Furthermore, the

sheer speed of model creation facilitates a much faster and broader review of existing research data, perhaps leading to

new insights. For instance, users of BayesiaLab should be able to replicate all modeling steps in this case study within a

few hours.

Finally, we should emphasize that our case study is focused exclusively on the modeling aspect of this subject matter,

without providing any medical or biological interpretations. It is not our objective to make a contribution to the medical

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 1

1 http://www.broadinstitute.org/cgi-bin/cancer/publications/pub_paper.cgi?mode=view&paper_id=43

Page 4: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 4/29

literature, but rather to showcase a new computational method on the basis real-world data from the eld of biostatis-

tics. Hence, any medical references in this paper are paraphrased from existing research to provide context for the

reader.

About the Authors

Stefan Conrady

Stefan Conrady is the cofounder and managing partner of Conrady Applied Science, LLC, a privately held consulting

rm specializing in knowledge discovery and probabilistic reasoning with Bayesian networks. In 2010, Conrady Applied

Science was appointed the authorized sales and consulting partner of Bayesia SAS for North America.

Stefan Conrady studied Electrical Engineering and has extensive management experience in the elds of product plan-

ning, marketing and analytics, working at Daimler and BMW Group in Europe, North America and Asia. Prior to estab-

lishing his own rm, he was heading the Analytics & Forecasting group at Nissan North America.

Lionel Jouffe

Dr. Lionel Jouffe is cofounder and CEO of France-based Bayesia SAS. Lionel Jouffe holds a Ph.D. in Computer Science

and has been working in the

eld of Arti

cial Intelligence since the early 1990s. He and his team have been developingBayesiaLab since 1999 and it has emerged as the leading software package for knowledge discovery, data mining and

knowledge modeling using Bayesian networks. BayesiaLab enjoys broad acceptance in academic communities as well as

in business and industry. The relevance of Bayesian networks, especially in the context of consumer research, is high-

lighted by Bayesia’s strategic partnership with Procter & Gamble, who has deployed BayesiaLab globally since 2007.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 2

Page 5: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 5/29

Case Study & Tutorial

Background

To provide the correct medical context for our white paper, we quote Golub et al. (1999), who conducted the original

research and described a new cancer classication approach based on global gene expression analysis:

The challenge of cancer treatment has been to target specic therapies to pathogenetically distinct tumor types,

to maximize ef cacy and minimize toxicity. Improvements in cancer classication have thus been central to

advances in cancer treatment. Cancer classication has been based primarily on morphological appearance of 

the tumor, but this has serious limitations. Tumors with similar histopathological appearance can follow sig-

nicantly different clinical courses and show different responses to therapy.

For […] tumors, important subclasses are likely to exist but have yet to be dened by molecular markers. For

example, prostate cancers of identical grade can have widely variable clinical courses, from indolence over dec-

ades to explosive growth causing rapid patient death. Cancer classication has been dif cult in part because it 

has historically relied on specic biological insights, rather than systematic and unbiased approaches for recog-

nizing tumor subtypes.

Although the distinction between AML2 and ALL3 has been well established, no single test is currently suf -

cient to establish the diagnosis. Rather, current clinical practice involves an experienced hematopathologist’s

interpretation of the tumor’s morphology,4 histochemistry, immunophenotyping, and cytogenetic analysis, each

performed in a separate, highly specialized laboratory. Although usually accurate, leukemia classication re-

mains imperfect and errors do occur.

We will use the Golub dataset and create a new classication model within the framework of Bayesian networks as an

alternative to models already documented in the relevant literature. An extensive survey of existing models is provided

in Dudoit et al. (2002).

Database

The Golub study is based on two datasets containing an initially available training set of 38 samples and a second set of 

34 samples, which subsequently served as a test set. These datasets contain measurements corresponding to ALL and

AML samples taken from bone marrow and peripheral blood of leukemia patients. For details about the experimental

method and microarray analysis protocol, readers are referred to the original paper.

For all sample cases in the test and training set, we have 7,129 variables, each representing the expression level of an

individual gene. The expression levels are recorded as continuous numerical values. The dependent variable is Leukemia

Type, which can have one of two categorical states, “AML” and “ALL.” Furthermore, the variable Data Type indicates

whether a sample belongs to the Test or the Training set. Finally, Case ID serves a Row Identier.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 3

2 AML: Acute Myelogenous Leukemia

3 ALL: Acute Lymphoblastic Leukemia

4 An image-based morphology analysis of cell samples was presented in our recent white paper, Breast Cancer Diagnos-

tics with Bayesian Networks.

Page 6: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 6/29

With the benet of having both datasets available simultaneously at the time of our study, we will take advantage of the

additional observations in Golub’s test set.5 Otherwise, we use these data sets in their original format without any fur-

ther transformation, so our results can be compared to results of earlier studies.

Notation

To clearly distinguish between natural language, software-specic functions and study-specic variable names, the fol-lowing notation is used:

• BayesiaLab-specic functions, keywords, commands, etc., are capitalized and shown in bold type.

• Attribute/variable/node names are italicized.

Classication Model

Data Import

Our modeling process begins with importing the database from a CSV format into BayesiaLab. The Data Import Wizard guides the analyst through the required steps.

It is common practice for gene expression variable names to be recorded as row headers, while cases are identied by

column headers, which is just the opposite of the typical arrangement of research data. To accommodate this alternative

format, we can check the Transpose box and thus achieve compatibility with the Data Import Wizard.

For the next step, we need to identify Case ID as Row Identier. Given their values, BayesiaLab will automatically rec-ognize Leukemia Type as discrete and all gene variables (AFFX-BioB-5_at through Z78285_f_at ) as continuous. As we

will use the training and test set combined, we declare the variable Data Type as Not Distributed.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 4

5 We provide a direct comparison of model performance, based on 38 samples only, in the appendix.

Page 7: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 7/29

The following step in the Data Import Wizard normally denes missing values processing, which is not required in our

case. So, the following screen can be skipped entirely by clicking Next.

The following step, however, is critical in the import process. It denes the Discretization and Aggregation process,

which is necessary, as in BayesiaLab Bayesian networks are only dened for discrete states.

As the exclusive objective of this model is classication, we will choose the Decision Tree algorithm, which will discre-

tize each variable for an optimum information gain with respect to the target variable Leukemia Type. This is particu-larly helpful, as we do not have any a-priori knowledge about the relevance or the meaning of any of the expression

levels.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 5

Page 8: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 8/29

Theoretically, we could dene the discretization algorithm for each variable individually, but with over 7,000 variables

this is obviously not practical. So we will click Select All Continuous and the pick Decision Tree from the drop-down

menu.

Furthermore, we need to dene the maximum number of interval levels for the discretization. Given the very small

number of observations in the dataset, we recommend no more than 3 discretization levels, although the algorithm may

subsequently further reduce the number of intervals to 2 for individual variables.

Upon clicking Finish, BayesiaLab will proceed with the import process and report its progress via a status bar.

However, we will soon receive a warning that the “Decision Tree 3” discretization was not possible for one of the vari-

ables. This means that this variable could not be discretized in a way that would optimize the information gain for the

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 6

Page 9: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 9/29

target variable. As a fallback option, we will choose the K-Means Discretization algorithm with 3 intervals. As this situa-

tion might apply to other variables as well, we can check Remember My Choice.

The analyst can observe the status of the discretization via a progress bar.

Upon completion, we have the option of displaying the Import Report, which will display the nal discretization type

for every single variable in the database.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 7

Page 10: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 10/29

The following screenshot shows 7 out of 7,130 variables with the associated intervals and the type of discretization ob-

tained (red indicates Decision Tree Discretization, blue indicates K-Means Discretization).

Upon closing the report, we will see a representation of the newly imported database in the form of a fully unconnected

Bayesian network. Each variable is now represented as a blue node in the Graph Panel of BayesiaLab. Given the large

number of variables, we will only show a small portion of them in the screenshot below.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 8

Page 11: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 11/29

It is also good practice to “spot check” some of the newly created nodes. We can do that by simply double-clicking on

any node in the network. The Node Editor will open up and allow the analyst to review and edit any of the properties, if 

necessary.

For instance, we may have some a-priori expert knowledge that one particular variable can only assume either one of 

two states, e.g. high/low, and that there is a specic threshold, which separates those two states. The Node Editor allows

us to change the discretization thresholds via point-and-click directly in the probability density chart of the variable. Just

for illustration purposes, we have arbitrarily created a discretization threshold at the median value in the following

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 9

Page 12: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 12/29

screenshot. We have the ability to return to the Node Editor at any time and re-discretize (or change other attributes) of 

variables as needed.

Supervised Learning: Augmented Markov Blanket

As the starting point for the generation of our Bayesian network model, we will dene Leukemia Type as the Target

Variable. This can be done by right-clicking the Leukemia Type and selecting Set As Target Node. Double-clicking the

node while pressing the “T” key will do the same.

Beyond the ability to predict class membership of future samples based on their expression levels, we are very interestedin nding a manageable subset of variables that can be used as predictors, i.e. we want perform a feature selection.

The Markov Blanket algorithm is suitable for this kind of application and its speed is particularly helpful when dealing

with thousands of variables.6 Furthermore, BayesiaLab offers the Augmented Markov Blanket, which starts with the

Markov Blanket structure and then uses an unsupervised search to nd  the probabilistic relations that hold between

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 10

6 See appendix for a denition of the Markov Blanket

Page 13: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 13/29

each variable belonging to the Markov Blanket.7 This unsupervised search requires additional computation time but

generally results in an improved predictive performance of the model.

The learning process can be started by selecting Learning>Target Node Characterization>Augmented Markov Blanket 

from the menu.

After a few seconds, we will see the result of the machine learning process. Our Target Node  Leukemia Type is now

connected to all variables in its Markov Blanket.

To show these connections, we need to zoom out and at this level this individual nodes are barely visible dots on the

screen. However, the connections in the Markov Blanket are very prominent and we can see that only 55 variables out

of over 7,000 were selected as predictors.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 11

7 Intuitively, the “augmented” part of the network plays the same role as the interaction terms in a regression.

Page 14: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 14/29

Performance Evaluation

In order to see whether this selection proves to be adequate for classication purposes, we switch into the Validation

Mode by pressing the F5 key and start the performance evaluation.

As we do not have a separate test and training set, we will need to use Cross-Validation for evaluation purposes. Cross-

validation is a technique for assessing how the predictions of a model will generalize to an independent data set. One

round of Cross-Validation involves partitioning a sample of data into complementary subsets, estimating the model on

one subset, and then validating the analysis on the other subset. To reduce variability, multiple rounds of  Cross-

Validation are performed using different partitions, and the validation results are averaged over the rounds.

To start the process, we select Tools>Cross Validation>Targeted from the menu.

The performance report shows that all of the 47 ALL cases, 45 were correctly classied and of the 25 AML, 24 cases

were correctly identied, too. This yields a total precision of almost 96%.

Network Complexity

Beyond precision, parsimony is a key objective in most modeling tasks. This need for simplicity is particularly obvious in

the presence of thousands of potential predictors. Golub’s initial model used those 50 genes as predictors, which were

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 12

Page 15: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 15/29

most correlated with the AML-ALL distinction. Our Augmented Markov Blanket model found 55 predictors, and, as it

turns out, 14 out of the 55 predictors are in common with Golub’s list of the 50 most correlated variables.

Although the selection of 55 predictors would perhaps be suf cient for a practical application, we can investigate

whether a selection of fewer predictors can still yield reliable results. BayesiaLab allows us to manage network complex-

ity via the Structural Coef cient (SC). By default, the value of SC is set to 1, however, we can increase its value to force a

simpler network structure, which, in the case of our model, would mean fewer predictors. We will set SC arbitrarily to 2and use the Augmented Markov Blanket learning again.

The resulting network now only contains 16 predictors:

To evaluate the performance of this network we will repeat the Cross-Validation .

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 13

Page 16: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 16/29

Interestingly, despite the smaller number of predictors, the classication performance has actually improved slightly and

now stands at 97%.

Given this performance of a simpler model, we may wish to increase the SC further and see whether we will still have a

reasonable model. Outside a case study and in a real-world situation, the analyst would have to make the determination

as to how far to take this. Our pursuit of an even simpler structure only serves demonstration purposes.

We now set SC=3 and once again learn the Augmented Markov Blanket. The number of predictors is now narrowed

down to 5.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 14

Page 17: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 17/29

Repeating the Cross-Validation yields that the precision remains at the same level as before, i.e. at 97%

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 15

Page 18: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 18/29

At rst glance it might be tempting to chose this very simple model with only ve predictors for practical application.

However, practical considerations may actually suggest otherwise. Given the delicate nature of microarray analysis,

missing values, measurement errors, etc. are to be expected. A model with more predictors, including redundant predic-

tors, will be less sensitive to noise and thus more robust overall.

InferenceWith the small number of predictors in our most recent network, we can inspect them more closely and review their

states in the Monitor Panel. We can display their Monitors by rst selecting the nodes in the Graph Panel and then by

right-clicking on Monitor from the contextual menu.

We can now see their states, the thresholds between the states and their marginal distribution.

 

So far, we have only made inference about the state of Leukemia Type based on the states of the predictors, i.e. P(Leu-

kemia Type | gene).8 We can use a key property of Bayesian networks here, namely omnidirectional inference. This al-

lows to compute the posterior probability of the states of the predictors, given the state of Leukemia Type:

P(gene | Leukemia Type).

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 16

8 gene = (e1, e2, e3, …, en), i.e. a vector consisting of n expression levels. Here, n is the number of predictors.

Page 19: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 19/29

For Leukemia Type=ALL, we obtain the following posterior distributions:

Conversely, for Leukemia Type=AML, we obtain these posterior distributions:

The very manageable number of predictors certainly makes it easy to further examine their specic roles in the classica-

tion model. BayesiaLab provides a number of tools that assist with interpreting the variables.

Target Dynamic ProleThe Target Dynamic Prole function is typically used to search for the states of predictor variables, which optimize the

desired state of the target variable. In our case, however, we are interested in those states of the predictor variables,

which provide the greatest amount of information for distinguishing between the states AML and ALL of the Leukemia

Type variable.

The function can be invoked by selecting Analysis>Reports>Target Dynamic Prole from the menu.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 17

Page 20: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 20/29

In the following dialogue we select Probability, Criterion Maximization, Take Into Account the Joint Probability and 

Hard Evidence.

The rst result shows ALL vs. AML. This means that observing the lowest state (1/3) of  M23791 would change the

probability of ALL from 65% (a priori) to 97.5% (a posteriori).

Conversely, the probability of AML increases from 34% (a priori) to 91.5% (a posteriori), given that the higher state (2/ 

2) of X95735 is observed. A second piece of evidence, the higher state (2/2) of U46499, would further increase the AML 

probability to 99%.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 18

Page 21: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 21/29

Bayes Factor 

BayesiaLab offers an additional metric for interpreting the impact of observing a specic piece of evidence, namely by

providing values of the Bayes Factor for each variable.

We dene the Bayes Factor here as

K  = log2

P(H 

∣E )

P(H )⎛ ⎝ ⎜ ⎞ ⎠ ⎟ ,

which, from a statistical perspective, can be seen as the strength of the observed evidence E with respect to the hypothe-

sis H . From an information theory perspective, the Bayes Factor quanties of the modication of the state variable un-

certainty once we have the evidence E.

For instance, after observing the evidence of M23197 = 1/3, which is the rst of three states (see green bar in the associ-

ated Monitor),

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 19

Page 22: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 22/29

we can observe the impact on all the other variables by selecting Analysis>Report>Evidence Analysis.

The rightmost column shows the Bayes Factor. For the state ALL of variable Leukemia Type it is 0.5832 and for state

AML it is -3.7942. This means that this observation provides much stronger evidence against AML versus positive evi-

dence for ALL.

As we noted earlier, inference within Bayesian networks is always omnidirectional, so we obtain Bayes Factors for allother variables in the network.

Target Interpretation Tree

So, observing the specic states of any of these genes should update one’s belief about the likely state of Leukemia Type.

BayesiaLab can also represent this sequence of “belief updating given evidence” in the form of a tree. We can generate

such a tree by selecting Analysis>Target’s Interpretation Tree.

To show this Target Interpretation Tree, we have limited the number of pieces of evidence to three and omitted the rest,

so the tree can t on one page and still remain legible.

By reading the tree from left to right, we can see how each piece of evidence updates the probabilities of  AML versus

ALL. For instance, starting at the root and observing the middle value (2/3) for M23197 we would follow the center

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 20

Page 23: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 23/29

path and see that our new conditional probability for ALL is now 40.6%, i.e. it now lower than the original marginal

probability of 65%. Observing the next evidence, U46499, can change the picture again. Given the lower value (1/2) for

U46499, the conditional probability for ALL would increase to 98.5%, and so on.

 

Conclusion

We have demonstrated that, with Bayesian networks as the framework and BayesiaLab as a software tool, we can rap-

idly generate a reliable classication model on the basis of gene expression data.

Beyond the good classication performance, the Augmented Markov Blanket model provides an effective means of fea-

ture selection among thousands of potential predictors, with a minimum of specication effort. Furthermore, the ex-

treme speed of feature selection allows the researcher to move quickly from data acquisition to interpretation.

Finally, the user-friendly interface of BayesiaLab makes the required workow easily accessible to any subject matter

expert and without requiring to write any program code, which is typically required in this domain.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 21

Page 24: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 24/29

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 22

Page 25: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 25/29

Appendix

Markov Blanket

In many cases, the Markov Blanket algorithm is a good starting point for any predictive model, whether used for scoring

or classication. This algorithm is extremely fast and can even be applied to databases with thousands of variables and

millions of records.

The Markov Blanket for a node A is the set of nodes composed of A’s parents, its children, and its children’s other par-

ents (=spouses).

The Markov Blanket of the node A contains all the variables, which, if we know their states, will shield the node A from

the rest of the network. This means that the Markov Blanket of a node is the only knowledge needed to predict the be-

havior of that node A. Learning a Markov Blanket selects relevant predictor variables, which is particularly helpful

when there is a large number of variables in the database (In fact, this can also serve as a highly-ef cient variable selec-

tion method in preparation for other types of modeling, outside the Bayesian network framework).

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 23

Page 26: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 26/29

Comparison of Classication Performance with Golub et al. (1999)

In order to facilitate a direct comparison of our approach with the performance of the Golub model, we now constrain

our data set to the initial 38 cases. Based on this smaller dataset, we learn the Augmented Markov Blanket, which selects

20 variables as predictors.

Subsequently, we follow Golub’s approach of applying a leave-one-out cross-validation for performance evaluation.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 24

Page 27: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 27/29

We obtain a 97% precision, correctly classifying 37 out of the 38 cases. The Golub model correctly identi ed 36 cases

and declared the remaining two cases as uncertain. Within our model, we cannot detect any “borderline” cases in terms

of classication strength, which means that one case was indeed misclassied.

 

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 25

Page 28: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 28/29

References

Conrady, Stefan, and Lionel Jouffe. “Breast Cancer Diagnostics with Bayesian Networks”. Conrady Applied Science,LLC, March 5, 2011. http://www.conradyscience.com/index.php/wbcd.

Dudoit, S., J. Fridlyand, and T. P Speed. “Comparison of discrimination methods for the classication of tumors usinggene expression data.” Journal of the American statistical association 97, no. 457 (2002): 77–87.

Friedman, N., M. Linial, I. Nachman, and D. Pe’er. “Using Bayesian networks to analyze expression data.” Journal of computational biology 7, no. 3-4 (2000): 601–620.

Gentleman, Robert. “Reproducible Research: A Bioinformatics Case Study.” Statistical Applications in Genetics andMolecular Biology 4, no. 1 (2005). http://www.bepress.com/sagmb/vol4/iss1/art2.

Golub, T. R. “Molecular Classication of Cancer: Class Discovery and Class Prediction by Gene Expression Monitor-ing.” Science 286, no. 5439 (1999): 531-537.

Husmeier, Dirk. “Bayesian Networks for Analysing Gene Expression Data.” Bayesian Networks for Analysing GeneExpression Data, August 2001. http://www.bioss.ac.uk/~dirk/essays/GeneExpression/bayes_net.html.

“Markov Blanket.” Wikipedia. http://en.wikipedia.org/wiki/Markov_blanket.

Slonim, Donna K, Pablo Tamayo, Jill P Mesirov, Todd R Golub, Eric S Lander, and Eric S L. “Class Prediction and Dis-covery Using Gene Expression Data” 2000 (2000): 263--272.

Torgo, Luis. Data Mining with R: Learning with Case Studies. 1st ed. Chapman and Hall/CRC, 2010.

Microarray Analysis with Bayesian Networks and BayesiaLab

www.conradyscience.com| www.bayesia.com 26

Page 29: Microarray Analysis with Bayesian Networks

8/7/2019 Microarray Analysis with Bayesian Networks

http://slidepdf.com/reader/full/microarray-analysis-with-bayesian-networks 29/29

Contact Information

Conrady Applied Science, LLC

312 Hamlet’s End Way

Franklin, TN 37067

USA

+1 888-386-8383

[email protected]

www.conradyscience.com

Bayesia SAS

6, rue Léonard de Vinci

BP 119

53001 Laval Cedex

France

+33(0)2 43 49 75 69

[email protected]

www.bayesia.com

Copyright

© 2011 Conrady Applied Science, LLC and Bayesia SAS. All rights reserved.

Any redistribution or reproduction of part or all of the contents in any form is prohibited other than the following:

• You may print or download this document for your personal and noncommercial use only.

• You may copy the content to individual third parties for their personal use, but only if you acknowledge Conrady

Applied Science, LLC and Bayesia SAS as the source of the material.

• You may not, except with our express written permission, distribute or commercially exploit the content. Nor may

you transmit it or store it in any other website or other form of electronic retrieval system.

Microarray Analysis with Bayesian Networks and BayesiaLab