Deep learning
50
Location: QuantUniversity Meetup December 21 st 2016 Boston MA Deep Learning : An introduction 2016 Copyright QuantUniversity LLC. Presented By: Sri Krishnamurthy, CFA, CAP www.QuantUniversity.com [email protected]
-
Upload
quantuniversity -
Category
Data & Analytics
-
view
577 -
download
0
Transcript of Deep learning
Location:
QuantUniversity Meetup
December 21st 2016
Boston MA
Deep Learning : An introduction
2016 Copyright QuantUniversity LLC.
Presented By:
Sri Krishnamurthy, CFA, CAP
www.QuantUniversity.com
2
Slides and Code will be available at: http://www.analyticscertificate.com
- Analytics Advisory services- Custom training programs- Architecture assessments, advice and audits
• Founder of QuantUniversity LLC. and www.analyticscertificate.com
• Advisory and Consultancy for Financial Analytics• Prior Experience at MathWorks, Citigroup and
Endeca and 25+ financial services and energy customers.
• Regular Columnist for the Wilmott Magazine• Author of forthcoming book
“Financial Modeling: A case study approach”published by Wiley
• Charted Financial Analyst and Certified Analytics Professional
• Teaches Analytics in the Babson College MBA program and at Northeastern University, Boston
Sri KrishnamurthyFounder and CEO
4
5
Quantitative Analytics and Big Data Analytics Onboarding
• Trained more than 500 students in Quantitative methods, Data Science and Big Data Technologies using MATLAB, Python and R
• Launching the Analytics Certificate Program in September
6
• January 2017▫ 19th, Deep Learning Lecture Part II
• February 2017▫ Deep Learning Workshop (Date TBD)
Events of Interest
7
Dr. Victor ShnayderFellowQuantUniversity
Prior Experience:Product Manager EdX (March 2013-June 2016)
Harvard UniversityPhD, Computer Science
8
Start with labeled pairs (Xi, Yi)
( ,“kitten”),( ,“puppy”)
…
9
Success: predict new examples
( ,?)
10
https://commons.wikimedia.org/wiki/Neural_network
“kitten”
“puppy”
“has fur?”
“pointy ears?”
“dangerously cute?”
11
12
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
13
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Weighted sum
14
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Non-linear “activation” function
15
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learning = “find good weights”
16
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learning = “find good weights”
How? Gradient descent!
18
1. Our labeled datasets were thousands of times too small.
2. Our computers were millions of times too slow.
3. We initialized the weights in a stupid way.
4. We used the wrong type of non-linearity.
- Geoff Hinton
Neural nets were tried in the 1980s. What changed?
https://youtu.be/IcOMKXAw5VA?t=21m29s
19
http://www.rsipvision.com/exploring-deep-learning/
20
http://www.asimovinstitute.org/neural-network-zoo/
21
22
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
23
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
24
Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans http://www.nature.com/articles/srep24454/figures/1
25
Towards End-to-End Speech Recognition with Recurrent Neural Networks http://www.jmlr.org/proceedings/papers/v32/graves14.pdf
26
https://www.technologyreview.com/s/544651/baidus-deep-learning-system-rivals-people-at-speech-recognition/
27
https://research.googleblog.com/2014/11/a-picture-is-worth-thousand-coherent.html
28
http://cs.umd.edu/~miyyer/data/deepqa.pdf
29
30
http://blog.ventureradar.com/2016/03/11/10-hot-startups-using-artificial-intelligence-in-cyber-security/
32
https://www.engadget.com/2016/03/12/watch-alphago-vs-lee-sedol-round-3-live-right-now/
33
https://www.youtube.com/watch?v=kMMbW96nMW8
34
35
How is deep learning special?
Given (lots of) data, DNNs learn useful input representations.
D. Erhan et al. ‘09http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/247
36
37
Hardware
38
Data
http://www.theneweconomy.com/strategy/big-data-is-not-without-its-problems
40
41
• Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently
• Performs efficient symbolic differentiation• Leverages NVIDIA GPU (Claim 140X faster than CPU)• Developed by University of Montreal researchers and is open-source• Works on Windows/Linux/Mac OS
• See https://arxiv.org/abs/1605.02688
Theano
42
• GPU vs CPU▫ Theano Test
▫ See Theano Test.ipyb
Demo
43
• Logistic Regression
Theano
See Theano-Logistic Regression.ipyb
44
MLP
45
Convolutional Neural Networks
Convolution
46
Convolutional Neural Networks
Sparse connectivityWeight sharing-Max-pooling layer
See Theano-Conv-Net.ipynb
47
• Keras is a high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation.
• Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility).
• Supports both convolutional networks and recurrent networks, as well as combinations of the two.
• Supports arbitrary connectivity schemes (including multi-input and multi-output training).
• Runs seamlessly on CPU and GPU.
Keras
48
• Keras Examples▫ Testing Keras: See KerasPython.ipynb
▫ Running Convolutional NN on Keras with a Theano Backend See Keras-conv-example-mnist.ipynb
Demo
49
• A case study for Convolutional Neural Networks
• Recurrent Neural Networks
• Auto Encoders
• Best Practices
Coming on January 21st - Part II
50
Q&A
Thank you!Members & Sponsors!
Sri Krishnamurthy, CFA, CAPFounder and CEO
QuantUniversity LLC.
srikrishnamurthy
www.QuantUniversity.com
Contact
Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not bedistributed or used in any other publication without the prior written consent of QuantUniversity LLC.
51
