Artificial Neural Networks Examples - cvut.cz · 01001110 01100101 01110101 01110010 01101111...
36
01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001 00100000 01101011 01100001 01110100 01100101 01100100 01110010 01111001 00100000 01110000 01101111 01100011 01101001 01110100 01100001 01100011 01110101 00101100 00100000 01000110 01000101 01001100 00100000 01000011 01010110 01010101 01010100 00101100 00100000 01010000 01110010 Artificial Neural Networks Examples Jan Drchal [email protected] Computational Intelligence Group Department of Computer Science and Engineering Faculty of Electrical Engineering Czech Technical University in Prague
Transcript of Artificial Neural Networks Examples - cvut.cz · 01001110 01100101 01110101 01110010 01101111...
01001110 0110010101110101 0111001001101111 0110111001101111 0111011001100001 0010000001110011 0110101101110101 0111000001101001 0110111001100001 0010000001101011 0110000101110100 0110010101100100 0111001001111001 0010000001110000 0110111101100011 0110100101110100 0110000101100011 0111010100101100 0010000001000110 0100010101001100 0010000001000011 0101011001010101 0101010000101100 0010000001010000 0111001001100001 0110100001100001 00000000
Artificial Neural Networks Examples
Computational Intelligence GroupDepartment of Computer Science and Engineering
Faculty of Electrical EngineeringCzech Technical University in Prague
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Outline
● Learning artificial neural networks (ANNs).
● Task to solve with ANNs.
● ANN applications.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
ANN, Learning & Recall
● ANN is a black box performing transformation.
● ANNs work most frequently in two phases:– Learning phase – adaptation of ANN's
internal parameters.
– Evaluation phase (recall) – use what was learned.
•••
•••
Artificial
Neural
Network
Input vectors Output
x1
x2x3
xn
y1y2
ym
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Supervised learning
● Learning by examples:
– given a set of example pairs Pi=(x
i,y
i),
– find transformation f which approximates y
i=f(x
i) for all i.
http://sugiyama-www.cs.titech.ac.jp/~sugi/figs/supervised-learning.png
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Unsupervised learning
● Self-organization, no teacher.● SOM, ART...
http://en.wikipedia.org/wiki/Self-organizing_map
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Reinforcement learning
● Teaching examples not available → they are generated by interactions with the environment (mostly control tasks).
http://anji.sourceforge.net/polebalance.htm
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Reinforcement Learning Demonstration
● 1994, Karl Sim's: Evolving Virtual Creatures video.
● Evolves both creature bodies (including sensors) and controlling networks...
● You can play with http://www.framsticks.com/
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Financial Applications
● Stock market time series forecasting.● Buy/sell timing detection and stock portfolio
selection.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Medical
● EEG/ECG processing – e.g. sleep disorder● Survival analysis – e.g. breast cancer● Pattern/Image recognition – MR, CT
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Compression by ANNs
http://www.cse.unr.edu/~looney/cs773b/NNimage-compress.pdf
cut here
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Neocognitron
Prof. Fukushima (1980)handwritten character recognition
similar to structures in brain
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Convolutional Neural Networks: LeNet
● Yann LeCun
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
CNN for Speech Recognition
● Abdel-Hamid et al. (2014)
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
LSTM for Translation
● Sutskever et al. (2014)
● Long-Short Term Memory
● English → internal representation → French
http://arxiv.org/abs/1409.3215
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
CNN & LSTM for Activity Recognition
● Baccouche et al. (2011).
Baccouche, Moez, et al. "Sequential deep learning for human action recognition." Human Behavior Understanding. Springer Berlin Heidelberg, 2011. 29-39.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Playing Atari 2600 Games
● Mnih et al. (2015).
Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Playing Atari 2600 Games
● Mnih et al. (2015).
Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Playing Atari 2600 Games
● Mnih et al. (2015).
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Speechreading (Lipreading)
● Günter Mamier, Marco Sommerau & Michael Vogt, Universität Stuttgart.
● A neural classifier detects visibility of teeth edges and other attributes.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Detection and Tracking of Moving Targets (ICBMs)
● The moving target detection and track methods here are "track before detect" methods.
● They correlate sensor data versus time and location, based on the nature of actual tracks.
● The track statistics are "learned" based on artificial neural network (ANN) training with prior real or simulated data.
● Reduce false alarm rates by up to a factor of 1000 based on simulated SBIRS data for very weak ICBM targets against cloud and nuclear backgrounds.
http://tralvex.com/pub/nap/#Detection and Tracking of Moving Targetshttps://web.archive.org/web/20011204214308/http://www.ca.defgrp.com/detect.html
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
ANNs and Game AI
● AI opponents.● Modeling player behaviour.● Strategy estimation.● Realistic motion.● Cheat detection.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Blondie 24 Chess (1999)
● David B. Fogel.
● Combination of minmax. & neuroevolution.
● Coevolution.
● Defeated 99.61% of 165 online players (including masters).
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Colin McRae Rally 2.0 (2001)
http://www.ai-junkie.com/misc/hannan/hannan.html
● Opponents AI.● MLPs, RPROP learning.● ANN driving model follows
optimal track. ● Different models for different cars and
road conditions● Different networks for steering and
speed control.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Forza Motorsport 2 (2007)
● Opponent AI.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Parrot Speech (FEE CTU)
● Classification of parrot sounds → parrot speech consists of 41 “words”. Self Organization Map (SOM).
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Colabroute (FEE CTU)
● GPS Data Mining - THSOM.
● Automatic detection of crossroads.
● Detection of interesting places (gas stations, dangerous crossroads,...)
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
ShapeSOM (FEE CTU)
● Reconstruction of a room model out of unordered clusters of points.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Bone Age Modelling (FEE CTU)
● Modelling of age based on bone measurements.
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
Fetal Weight Prediction (FEE CTU)
● EFW = 0,0504AC2*16,427AC + 38,867FL + 284,074
A4M33BIA 2016
Jan Drchal, [email protected], http://cig.felk.cvut.cz
RoboNEAT (FEE CTU)
● Neuroevolution of robotic controllers.● HyperNEAT – large-scale ANNs.
