Neural Network Ensemble based on Feature Selection for Non-Invasive Recognition of

Liver Fibrosis StageBartosz KRAWCZYK, Michał WOŹNIAK, Tomasz ORCZYK,

Piotr PORWIK, Joanna MUSIALIK, Barbara BŁOŃSKA-FAJFROWSKA

Presentation agenda Overview. Current diagnostic methods. Proposed method.

Analyzed data. Data analysis methods. Result comparison.

Conclusions.

Overview Liver fibrosis:

Accumulation of tough, fibrous scar tissue in the liver. ~1,75% of Poland’s population is infected with HCV. Unthreated may cause Liver Cirrhosis and death.

Risk factors: Chronic infection with hepatitis C or hepatitis B virus (HCV,

HBV). Immune system compromise (HIV or immunosuppressive

drugs). Heavy alcohol consumption.

Gradation indexes: Knodell Histological Activity Index (HAI Score). Ishak system. METAVIR system.

Current diagnostic methods Invasive

Liver biopsy Risk of health complications or even death. Up to 45% uncertainty depending on bioptate quality

and size. Still assumed as a „gold standard”.

Non-invasive ELF Test FibroTest & FibroScan

Expensive Not very accurate

Proposed method Non-invasive

Blood test based

Inexpensive Only regular blood tests

Comparable with other non-invasive methods Similar error level to FibroTest

Proposed method: Analyzed data and problems

Data characteristics: 127 patients mostly with HCV (70%) and Liver

Fibrosis. All patients otherwise healthy and not under

therapy. 34 parameters measured.

Problems: Low data samples count. Unequal distribution of diagnosed fibrosis level. Incomplete records. Many poor quality biopsies.

Proposed method: Analyzed data and problems

Age* (years) 50 (13)

Male, n(%) 75 (59%)

Biopsy result, n(%)

F0 2 (2%)

F1 35 (28%)

F2 5 (4%)

F3 16 (13%)

F4 67 (53%)

HCV/HBV/other 70% / 9% / 21%

HB* (g/L) 14 (1.91)RBC* (106/UL) 5 (0.74)WBC* (103/UL) 6 (2.31)PLT* (103/UL) 161 (70.75)PT* (sec.) 13 (9.04)PTP* (%) 90 (17.82)APTT* (sec.) 38 (12.53)INR* 1 (0.26)ASPT* (IU/L) 65 (51.01)ALAT* (IU/L) 72 (61.81)ALP* (IU/L) 104 (55.11)BIL* (mg/dL) 2 (2.69)GGTP* (IU/L) 89 (94.43)KREA* (mg/dL) 1 (0.23)GLU* (mg/dL) 95 (19.02)Na* (mmol/L) 138 (3.48)

K* (mmol/L) 5 (5.16)Fe* (mmol/L) 104 (70.23)CRP* (IU/L) 4 (25.38)TG* (mg/dL) 107 (50.83)CHO* (mg/dL) 189 (51.04)Ur. acid* (mg/dL) 6 (1.39)TP* (g/dL) 7 (0.81)TIBC* 322 (120.47)Neutr* (103/UL) 3 (1.35)Lymph* (103/UL) 2 (0.55)Mono* (103/UL) 1 (0.19)Eos* (103/UL) 0 (0.13)Baso* (103/UL) 0 (0.02)Albu* (%) 58 (7.79)Glb. α1

* (%) 3 (1.33)Glb. α2

* (%) 8 (2.52)Glb. β* (%) 11 (2.43)Glb. γ* (%) 19 (7.21)

Proposed method: Neural Network Ensemble

The introduced method of classifier ensemble design consists of three main steps:

Building the pool of individual classifiers.

Pruning the acquired pool by discarding redundant predictors.

Using a sophisticated trained fuser to deliver the ensemble.

Proposed method:Building the pool of classifiers

Models should be complementary to each other, exhibiting at the same time high accuracy and high diversity.

There is no single optimal approach for feature selection task and results obtained on the basis of different methods may differ significantly.

Instead of selecting a single best feature selection method we use several of them to reduce the dimensionality of the feature space.

Proposed method:Ensemble pruning

There are several different ways in the literature on how to select valuable members to the committee.

Ideal ensemble consists of classifiers of high individual accuracy and high diversity.

Among diversity measures there are two major types: Pairwise (shows how two classifiers differ from each

other). Non-pairwise (measure the diversity of the whole

ensemble). For measuring the diversity of whole ensemble

we used the entropy measure.

Proposed method: Fusion of individual classifiers

Classifier fusion algorithms can make decisions on the basis of class labels given by individual classifiers or they can construct new discriminant functions on the basis of individual classifier support functions: The first group includes voting algorithms. The second group is based on discriminant

analysis.

The design of improved fusion classification models, especially trained fusers, is the focus of current research.

Proposed method: Fusion of individual classifiers

Assume that we have K classifiers in a pool after the pruning procedure.For a given object each individual classifier decides for class based on the values of discriminants. Let denote a function that is assigned to class i for a given value of x, and that is used by the l-th classifier . The combined classifier uses the decision rule , where .The weights can be set dependent on the classifier and class number: weight is assigned to the l-th classifier and the i-th class, and given classifier weights assigned to different classes may differ.

)()2()1( ,...,, K

Xx Mi ,...,1

xiF l ,)(

)(l

),(ˆmax),(ˆ)( xkFxiFifixMk

K

i

lK

l

ll andxiFxiF1

)(

1

)()( 1),(),(ˆ

)()( il

Proposed method: Feature selection algorithms

Eight different feature selection algorithms were used, namely: ReliefF, Fast Correlation Based Filter, Genetic Wrapper, Simulated Annealing Wrapper, Forward Selection, Backward Selection, Quick Branch & Bound, Las Vegas Incremental.

Neural network architecture was as follows: the number of neurons in the input layer was equal to the number of selected features, the number of output neurons was equal to the number of classes and the number of hidden neurons was equal to the half of the sum of number of neurons from the former layers.

Proposed method:Set-up

As reference methods we have selected most popular ensembles - Bagging, Boosting, Random Forest and Random Subspace.

Additionally we have compared our method with the single best classier from the pool, all classifiers from the pool and with simple majority voting.

The combined 5x2 CV F test [1] was carried out to asses the statistical signicance of obtained results.

Proposed method:Results

Proposed method: Results

The proposed neural network ensemble, based on feature selection methods, has outperformed all the previously used MCS for this problem.

The weakest results were returned by single best model approach, which highlights the usefulness of utilizing more than one classier to fully exploit the outputs of feature selection methods.

Second biggest accuracy boost lies in the used fuser-trained fusion of individual classiers allows to derive an optimal linear combination of them.

The pruning step had smallest but still statistically significant impact on the ensemble design.

Conclusions: The presented paper shows that, despite some

problems it is possible to reach similar or even lower error level than commercial tests.

It is also worth to mention that liver biopsy result, according to the other research, is also only a prediction with classification error varying from 35% up to 45% , depending on the sample size and count.

we proved that each of the three steps embedded in the proposed committee design has an important impact on the quality of the final prediction and thus should not be omitted.

Thank youfor your attention

Contact:bartosz.krawczyk@pwr.wroc.pl

tomasz.orczyk@us.edu.pl