Complex system
-
Upload
munnangi-anirudh -
Category
Engineering
-
view
97 -
download
1
description
Transcript of Complex system
Colon Tumor Classification using various Neural Network Models
coupled with Multi-Objective Evolutionary Optimization Schemes
Anirudh Munnangi
Chandrasekar Venkatesh
Ahmed Sageer Cheriya Melat
This project is an implementation of Neural Networks, with hybrid evolutionary algorithm optimizing multiple objectives for classification.
Optimizing three objectives: Pareto non dominated sorting genetic algorithm based optimization is done on norm of the weights, mean norm square errors and complexity of the network.
The evolutionary algorithm is applied to Radial Basis Function Networks (RBFNs) and on Multi-Layer Perceptron networks (MLPs), these algorithms are applied to classify real world two class colon tumor data.
Abstract
Data is real world two class colon tumor data
Data set consist of 62 data points
Data points have feature space of 2000
By principal component analysis method, the dimension of data points is reduced to 47 feature space
Data processing
Important Concepts
• Radial Basis Function Networks• Multi Layer Perceptron• Pareto Optimality• Genetic Algorithm
Radial Basis Function Networks
Radial Basis Function NetworksThe following equation defines the process which is followed by the RBFN map.
𝑌 2
𝑌 1
1
1
W20
W21
W22
Wm2
Wm1
W12
W10W11
∑
∑
X1
Xn
Input: 47 dimensional points
Output: 2 bits representing class
Number of hidden layers: 1Number of neurons in each layer: 20 or more
Multi Layer Perceptron
Multi Layer Perceptron
Xn
W2n
∑
∑
∑
𝑌 2
𝑌 1
W21
W1n
Wmn
Wm1
W11
∑
∑
X1
Input: 47 dimensional points Output: 2 bits
representing class
Number of hidden layers: 1Number of neurons in each layer: 20 or more
Pareto Optimality
Pareto Optimality
The solution is pareto optimal if
Objective functions considered in this project: - Mean norm square error
- Complexity map of the neural network
- norm of all the weights in the network
Genetic Algorithm
Genetic Algorithm
• Population size = 20• Each member represents Mean norm square error, complexity
of the network and norm of weights• For each iteration
• Perform Non dominated sort (Pareto optimization)• Choose fittest, crossover and form child population• Combine parent and child population, perform non-
dominated sort• Form new population
Program Flow
2.6 2.8 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.60
10
20
30
40
50
60
f3=||w||
f1=
Mean n
orm
Square
Err
or
Convergence of Pareto Front
NSGA using RBFN
Optimized RBFN with the best trade-off between objectives
0 5 10 15 20 25 30 35 40 452
2.5
3
3.5
4
4.5
5
5.5
6
6.5
7Convergence of Pareto Front
Objective function f2
Obje
ctive f
unction f
1
RBFN with NSGA
f1=MSE f2=Active hidden kernels
f1=MSE f2=||w||f1=Active hidden kernels f2=||w||
Performance in different runs
0 10 20 30 40 50 60 702
3
4
5
6
7
8
9
10Convergence of Pareto Front
f3=||w||
Mea
n N
orm
Squ
are
Err
or
RBFN NSGA
MLP NSGA
Comparative analysis of the performance of RBFN Vs MLP
INFERENCE• For the data set we used, optimized RBFN model seems to perform
better than MLP.
• Pareto optimization for multiple objectives works well with will all objectives being achieved.
FURTHER WORK• Can try more objectives, different parameters to optimize• Try optimizing other network models apart from RBFN and MLP, maybe
try optimizing for a combination of networks.
REFERENCES
1. Sultan Noman Qasem, Siti Mariyam Shamsuddin, Azlan Mohd Zain “Multi-objective hybrid evolutionary algorithms for radial basis function neural network design” Knowledge-Based Systems 27 (2012) 475–497 25 November 2011
2. Sultan Noman Qasema, Siti Mariyam Shamsuddina, “Memetic Elitist Pareto Differential Evolution algorithm based Radial basis function networks for classification problems” Neurocomputing; Applied Soft Computing 11 (2011) 5565–5581- 6 May 2011
3. Aimin Zhou, Bo-Yang Qu, Hui Li, Shi Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, Qingfu Zhang, “Multiobjective evolutionary algorithms: A survey of the state of the art” Swarm and Evolutionary Computation 1 (2011) 32–49; 16 March 2011
4. Illya Kokshenev, Antonio Padua Braga, “An efficient multi-objective learning algorithm for RBF neural network. Neurocomputing 73(2010)2799–2808, 22August2010
5. Sultan Noman Qasem and Siti Mariyam Shamsuddin, Radial Basis Function Network based on time variant multiobjective particle swarm optimization for medical disease analysis.
6. Jonathan E. Fieldsend and Sameer Singh, “Pareto Evolutionary Neural Networks” IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 16, NO. 2, MARCH 2005
7. Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and T. Meyarivan, “A Fast and Elitist Multiobjective Genetic Algorithm- NSGA-II” IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 6, NO. 2, APRIL 2002
8. David Lahoz and Pedro Mateo, “Neural Network Ensembles for Classification Problems using Multiobjective Genetic Algorithms”9. U. Alon, et al. "Broad Patterns of Gene Expression Revealed by Clustering Analysis of Tumor and Normal Colon Tissues Probed by
Oligonucleotide Arrays", PNAS, 96:6745-6750, 1999