Computational Intelligence Group Projects & Research...
Transcript of Computational Intelligence Group Projects & Research...
Computational Intelligence GroupProjects & Research Interests
http://cig.felk.cvut.czDepartment of Computer Science and Engineering
Faculty of Electrical EngineeringCzech Technical University in Prague
EUROSIM 2007
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 2
Team & Scope
● 5 employees
● 6 PhD students
● Datamining, computational intelligence, artificial neural networks, evolutionary algorithms
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 3
Cooperation
● IBM Research Czech Republic
● Sun Microsystems
● Seznam (czech information portal)
● National museum
● 1st and 2nd Medical faculty, Charles University
● Faculty for Human Studies, Charles University
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 4
Upcoming Events
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 5
CIG (Software) Projects
● FAKE GAME – open source data mining tool
● CIV toolkit – advanced algorithms for Cell processor
● MathSC – Mathematica softcomputing toobox
● BlueCar – mobile robot for intelligent rooms
● SiMoNNe – simulator of modular Neural Nets
Being prepared:
● Java OPT – nature inspired optimization package
● PREPit – automated data preprocessing
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 6
FAKE GAME
● FAKE (Fully Automated Knowledge Extraction)
● by GAME (Group of Adaptive Models Evolution)
● Inductive modeling datamining tool
● Implemented in Java, opensourced in 2007 http://sourceforge.net/projects/fakegame/
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 7
CIV Toolkit
● Computational Intelligence and Voice Processing Toolkit on IBM Cell Broadband Engine
● 3x PlayStation3
● HMM, DTW, PSO, SOM, Neural Gas, Genetic Alg.
● http://axon.felk.cvut.cz/civtoolkit
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 8
Data Preprocessing
● Data preprocessing is a corner stone of successful data mining and modelling.
● It involves among others– Data transformation
– Outliers detection and treating
– Missing data imputation
– Data reduction
– Feature selection/Feature ranking
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 9
FAKE GAME Preprocessing Module
● Some of basic preprocessing methods are implemented in the FAKE GAME project.
● The wizard is implemented to guide user through basic preprocessing steps.
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 10
Automated Preprocessing
● Selection and setup of preprocessing methods is very complex.
● To automate selection of preprocessing methods the genetic approach is involved.– Simple Genetic Algorithm
– Linear Genetic Programming
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 11
Introduction to Feature Ranking and Selection
How important is each feature?
Feature Ranking
1. P-length2. P-width3. S-length4. S-width
Reduction
Knowledge
Feature Selection of
dimensionality
Ranks
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 12
Feature Ranking(FR) in FAKE-GAME
● FAKE-GAME tool creates GAME models using Niching Genetic Algorithm(NGA)
● Importance of each feature can be obtained as a side effect of NGA by computing utilization in model building process
● This approach also causes selection of important features by ignoring redundant and irrelevant.
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 13
Example of Feature Selection using FAKE-GAME
● Exapmle with Hypercube data set from UCI data repository
– UNC is a number of unique chromosomes used for feature ranking inside NGA (from 2 to All unique chromosomes)
– First row with bold numbers shows correct rank of features
– Gray background cells are unused features
UNC 1 2 3 4 5 6 7 8 9 102 1 2 3 4 5 6 7 8 9 103 1 2 3 4 5 6 7 8 9 10
1 / 4 1 2 3 4 5 6 7 8 9 101 / 3 1 2 3 4 5 6 7 8 9 101 / 2 1 2 3 4 5 6 7 8 9 102 / 3 1 2 3 4 5 6 7 8 9 10All 1 2 3 4 5 6 7 8 9 10
● Every feature has correct rank
● With growing number of UNC is Feature Selection fewer restrictive
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 14
Parallel Processing of Recurrent Neural Networks
● Recurrent neural networks– Fully connected
– Next state depends on previous state
● Proposed solutions– Reduce number of temporal connection to lower
communication overhead (Brain cortex architecture is similar)
– Introduce the data set parallelism (ensembles of networks)
yt1=d−∑i=1d
x jt−wij
t 2∑k=1
n
ykt mik
t
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 15
Motivation for Use of Multicore Parallel Systems
● Mainstream today
● Powerful, inexpensive hardware for consumer electronics and game consoles
● Effective use of resources? (most programs runs on single core)
● Highly available and with general purpose programming
● Specialized ASICs more powerful then FPGAs
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 16
Suitable Parallel Platforms
● Sony/IBM based (Cell)● 2 AltiVec CPUs, 8 SIMD SPU cores
● Intel based (x86)● 2-8 core CPUs, 80 VLIW cores in the future● SMP, cache coherent, SIMD
● nVidia based (GPU)● High performance computing initiative Tesla● nVidia CUDA C environment● 96-128 cores / chip, NUMA
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 17
Current Work
● Scalable parallel processing of recurrent neural networks (THSOM as the possible representative)
● Usage of multicore CPUs with care to specific architecture constraints– General purpose x86 CPUs (SMP)
– nVidia CUDA capable GPU (NUMA)
– Sony PS3 with Cell CPU
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 18
Algorithms
● GAME – evolution of hybrid inductive models
● THSOM – temporal data clustering
● CEA – continuous evolution of individuals
● DEANN – evolution of neural networks
● ANTCAST – ant colony with castes
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 19
Continual Evolution Algorithm
● Hybrid Genetic Algorithm– Combination of the genetic algorithm and
– gradient-based optimization method
– Variable population size
– Age parametr of the individual
– Sequential replacement of individuals
– Separated encoding of structure and behavior
– Evolution in the continual time dimension,
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 20
Two dimensional evolution in CEA
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 21
Neural Networks Construction
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 22
Evolution Control Process
● Probability functions– Death and Reproduction probabilities
● Balancing Functions
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 23
Evolution Control Process
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 24
TWEANNs
● TWEANN (Topology and Weight Evolving
Artificial Neural Network) algorithms.
● Topology and parameters (weights) are
optimized simultaneously,
– no need to „guess“ the right topology,
– optimal topology is likely to be found.
● Use of Evolutionary Algorithms (EAs).
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 25
(In)direct encodings
● With classic approaches (direct encodings) only relativly small neural networks are possible-> curse of dimensionality
● Indirect encodings allow the compression of information -> small genome encodes large (regular) neural nework.
● Inspiration in nature -> human genome consists of 30 000 genes which encode about 1011 neurons and 1014 synapses!
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 26
DEANN cellular encoding
● Cellular encoding – based on cellular (neural) growth, cell division etc., the program to build a neural network is encoded as a tree.
● Our algorithm DEANN (Developmental Evolution of Artificial Neural Networks)-> the cellular growth is controlled by a biology inspired model of a Gene Regulatory Network.
small tree
encodes large neural network
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 27
Ant Colony Continuous Optimization
● Optimizing parameters of one neuron in GAME
x1, x
2, ... , x
n є R
hybridizing existing ANT methods with gradient search
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 28
Ant Colony Optimization with Castes
Improving ant algorithms using
groups of ants with different
behaviour
● Solving: (A)TSP, SOP, phylogenetic trees
Spaeth, Cooper, Ferguson (2003)
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 29
Applied Computational Intelligence
● Colabroute – datamining from GPS tracks
● Robospace – shape reconstruction from laser scans
● Spiral – parkinson disease recognition
● BlueCar – mobile robot for intelligent rooms
● ParrotTalk – parrot speech analysis
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 30
Colabroute
● Datamining from GPS tracks
● Automated construction of routable road maps
● Automated extraction of points of interrest (Fuel stations, dangerous crossroads, ...)
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 31
Robospace
● Reconstruction from unstructured vector clouds
● Self-organizing Maps
● Application in mobile robotics
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 32
Parkinson Disease Recognition
● Analysis of spirals drawn by hand
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 33
Parrot Speech Analysis
● Speech Recognition Methods applied to analysis of voices of grey parrots.
● Clustering of samples by Self-organizing Maps
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 34
Mining Biological Signals
● Interest in data mining mainly but not limited to medical applications.– Sleep stages recognition based on EEG
– Heart contractions shapes classification based on ECG
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 35
Parallel Subsolutions for TSP
● Solving subproblems in parallel on Cell
● Updating pheromone on main CPU
clustering: k-means http://www.playstation2.cz
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 36
Anthropological data modelling
In this project, we focus on processing Anthropological data by means of several data mining methods. The goal is to predict an age of individuals described by a set of parameters measured on their skeletons. Data in this project are problematic due to very high noise. Methods are tuned and parameterized to give best possible performance on data. The performance of methods is compared and the recommendation, how to process noisy and partially inconsistent data will be one of the final conclusions of this project.
Computational Intelligence Group, 19. 6. 2008Projects & Research Interests 37
Estimation of Fetal weight
● Find accurate model of fetal weight prediction ● Based on sonography measured data during
pregnancy shortly before delivery
● EFW = 0,0504AC2*16,427AC + 38,867FL + 284,074