Training Neural Networks using Evolution Strategies · Training Neural Networks using Evolution...
15
WWW.CRIM.CA Principal partenaire financier Training Neural Networks using Evolution Strategies FRANK GOUINEAU RESEARCH AGENT EMERGING TECHNOLOGIES AND DATA SCIENCE TEAM MAY 5, 2017
Transcript of Training Neural Networks using Evolution Strategies · Training Neural Networks using Evolution...
W W W . C R I M . C A
Principal partenaire financier
Training Neural Networks using Evolution Strategies
FRANK GOUINEAURESEARCH AGENTEMERGING TECHNOLOGIES AND DATA SCIENCE TEAM
MAY 5, 2017
2
WHY GPUS ARE BETTER THAN CPUS FOR NEURAL NETWORKS?
What is a deep neural network?
- a pretty picture?
X H1 H2 H3 Y
Source: http://neuralnetworksanddeeplearning.com/chap5.html
3
WHY GPUS ARE BETTER THAN CPUS FOR NEURAL NETWORKS?
What is a deep neural network?
- a mathematical formula?
Yes! With a lot of matrix multiplications!AlphaGO smallest neural network has 13 hidden layers and 48 images of 19x19 points = 17,328 input neurons!
X = inputH1 = W1*X + B1 H2 = W2*H1 + B2H3 = W3*H2 + B3Y = W4*H3 + B4
Where Wi are the weight matrices and Bi are the biasesY = f(B4 + W4*(
f(B3 + W3*(f(B2 + W2*(
f(B1 + W1*X))))))f = tanh, sigmoid, relu, …
4
WHY GPUS ARE BETTER THAN CPUS FOR NEURAL NETWORKS?
Matrices operations
We can compute each ABij or f(Aij) independently, in parallel !!!
( ) == ⋯⋮ ⋱ ⋮⋯
( ) = ( ) ⋯ ( )⋮ ⋱ ⋮( ) ⋯ ( )
5
WHY GPUS ARE BETTER THAN CPUS FOR NEURAL NETWORKS?
CPU vs GPU
Source: http://www.nvidia.com/object/what-is-gpu-computing.html
6
WHY GPUS ARE BETTER THAN CPUS FOR NEURAL NETWORKS?
CPU vs GPU
Source: https://www.nvidia.com/en-us/geforce/products/10series/titan-xp/
Source: https://ark.intel.com/fr/products/93790/Intel-Xeon-Processor-E7-8890-v4-60M-Cache-2_20-GHz
7
CATEGORIES OF NEURAL NETWORKS
Supervised Learning = We know what we the want
Unsupervised Learning = We don’t know the result
Semi-supervised Learning = The environment know what is good or bad (Fitness)
- Reinforcement Learning
DeepQ, AlphaGo, …
- Evolution Strategies
8
WHAT ARE EVOLUTIONS STRATEGIES?
Darwin’s theory
9
WHAT ARE EVOLUTIONS STRATEGIES?
From wolfs to dogs
Wolfs
firstGen1 firstGen2 firstGen3
scGen2
sndGen1 sndGen2 sndGen3
Trained Wolf
10
WHAT ARE EVOLUTIONS STRATEGIES?
Genetic algorithms
Neural Network
init1 init2 init3
scGen2
comb1 comb2 comb3
Trained Network
11
WHAT ARE EVOLUTIONS STRATEGIES?
Neural Networkinit0
Gaussian1 Gaussian2 Gaussian3
scGen2Gaussian1
Step1Gaussian2
Step1Gaussian3
Step1
Trained Network
1000s steps later
Computer1 Computer2
• Evolution Strategies
Salimans, T., Ho, J., Chen, X., & Sutskever, I. (2017). Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXivpreprint arXiv:1703.03864.
12
EVOLUTION STRATEGIES TO TRAIN DEEP NEURAL NETWORKS
Source: https://blog.openai.com/evolution-strategies/
13
EVOLUTION STRATEGIES TO TRAIN DEEP NEURAL NETWORKS
“On Atari, ES trained on 720 cores in 1 hour achieves comparable performance to A3C trained on 32 cores in 1 day"
“we were able to solve one of the hardest MuJoCo tasks using 1,440 CPUs across 80 machines in only 10 minutes. As a comparison, in a typical setting 32 A3C workers on one machine would solve this task in about 10 hours.”
« Almost Embarrassingly Parallel Optimization »
Source: http://www.inference.vc/evolutionary-strategies-embarrassingly-parallelizable-optimization/
14
EVOLUTION STRATEGIES TO TRAIN DEEP NEURAL NETWORKS
• Pros:– Parallel Optimization– Faster results
• Cons:– Local optimum– Supervised Learning problems are not concerned “ES on MNIST digit recognition …1,000 times slower”
• Future work:– GPUs testing = Faster computation? faster simulation? Faster results?– Evolving the network structure in addition to the parameters
Suivez-nous :
Dialoguez avec nous
Suivez-nous
@CRIM_ca
wwwCRIMca
Frank GouineauAgent de Recherche – Science des donnéesTESDCRIM – Centre de recherche informatique de Montréal
Le CRIM est un centre de recherche appliquée en TI qui développe, en mode collaboratif avec ses clients et partenaires, des technologies innovatrices et du savoir-fairede pointe, et les transfère aux entreprises et aux organismes québécois afin de les rendre plus productifs et plus compétitifs localement et mondialement. Le CRIMdispose de quatre équipes de recherche en TI de calibre mondial et œuvre principalement dans les domaines des interactions et interfaces personne-système, del’analytique avancée et de la science et technologie du logiciel. Détenteur d’une certification ISO 9001:2008, son action s’inscrit dans les politiques et stratégies pilotéespar le ministère de l'Économie, de la Science et de l'Innovation, son principal partenaire financier.
Tous droits réservés © 2017 CRIM. 405, avenue Ogilvy, bureau 101, Montréal (Québec) H3N 1M3 514 840-1234 / 1 877 840-2746
WWW.CRIM.CA
