Teaching Recurrent Neural Networks using Tensorflow (May 2016)
-
Upload
rajiv-shah -
Category
Data & Analytics
-
view
1.648 -
download
0
Transcript of Teaching Recurrent Neural Networks using Tensorflow (May 2016)
![Page 1: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/1.jpg)
Teaching Recurrent Neural Networks
using Tensorflow Rajiv Shah
twitter: rajcs4 [email protected] www.rajivshah.com
![Page 2: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/2.jpg)
![Page 3: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/3.jpg)
![Page 4: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/4.jpg)
Teaching Recurrent Neural Networks using Tensorflow
• How they are used
• Theory - http://playground.tensorflow.org/
• Learn a sine wave - http://tinyurl.com/raj-sine
• Learn to add - http://tinyurl.com/raj-add
• Learn handwriting - https://github.com/hardmaru/write-rnn-tensorflow
![Page 5: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/5.jpg)
![Page 6: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/6.jpg)
![Page 7: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/7.jpg)
![Page 8: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/8.jpg)
![Page 9: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/9.jpg)
![Page 10: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/10.jpg)
Theory
![Page 11: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/11.jpg)
![Page 12: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/12.jpg)
Recurrent Neural Networks: Theory
![Page 13: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/13.jpg)
Unrolling an RNN
![Page 14: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/14.jpg)
![Page 15: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/15.jpg)
RNN
![Page 16: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/16.jpg)
LSTM
![Page 17: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/17.jpg)
![Page 18: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/18.jpg)
Examples of RNNs
![Page 19: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/19.jpg)
Little more theory . . . but fun
![Page 20: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/20.jpg)
Tensorflow Playground• http://playground.tensorflow.org/
![Page 21: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/21.jpg)
![Page 22: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/22.jpg)
![Page 23: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/23.jpg)
Sine Wave
![Page 24: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/24.jpg)
![Page 25: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/25.jpg)
High Level Architecture
• Function to get batches
• Determined type of RNN cells and initialize them
• Decide on cost function and hyper parameters (dropout, learning rate, optimizer, batch size, sequence length)
• Run session
![Page 26: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/26.jpg)
Go to the python notebook
![Page 27: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/27.jpg)
Friendly Tensorflow - Using R / Shiny
• Wrapped calls to tensorflow using the rPython language
• Shiny provided a nice visual interface and the ability to quickly change different sorts of inputs / models
• https://github.com/rajshah4/tensorflow_shiny
![Page 28: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/28.jpg)
Adding
![Page 29: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/29.jpg)
Teaching Kids to Add
![Page 30: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/30.jpg)
![Page 31: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/31.jpg)
Teaching to Add(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)
![Page 32: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/32.jpg)
Go to the python notebook
![Page 33: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/33.jpg)
Learning Handwriting
![Page 34: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/34.jpg)
![Page 35: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/35.jpg)
Links
• http://www.cs.toronto.edu/~graves/handwriting.html
• https://github.com/hardmaru/write-rnn-tensorflow
![Page 36: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/36.jpg)
Take Aways
• RNNs are sequential learners and state of the art
• Understand why deep learning is powerful
• See how a RNN learns to model a sine wave, add, learn handwriting . . .
![Page 37: Teaching Recurrent Neural Networks using Tensorflow (May 2016)](https://reader031.fdocuments.us/reader031/viewer/2022030302/587d15a81a28abae148b6909/html5/thumbnails/37.jpg)
Teaching Recurrent Neural Networks
using Tensorflow Rajiv Shah
twitter: rajcs4 [email protected] www.rajivshah.com