Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning
-
Upload
sam-putnam-deep-learning -
Category
Software
-
view
408 -
download
3
Transcript of Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning
Executing Deep Learning Strategies
Sam Putnam, Enterprise Deep Learning, LLC
July 26, 2017
Day 1
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Morning: Introduction to deep learning - High-level overview of why enterprises are using deep learning 9:00 – 10:00 Introduction to machine learning and its applications10:00 - 11:00 Introduction to deep learning and how it is currently being used by enterprises11:00 - 12:00 Question and answer session with each student on deep learning's present or future role in their business12:00-1:00 LunchAfternoon: Overview of deep learning strategies
1:00-2:00 Awareness building - educating specific groups within your enterprise on the maturity of the technology2:00-3:00 Specialized to particular tasks - bespoke, custom, made-to-order solutions3:00-4:00 Using deep learning across teams - capturing and re-using insights, running a deep learning-first enterprise4:00-5:00 Question and answer session with each student on what their business would need out of a deep learning strategy
July 26 2017
Part 1 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
July 26 2017
Introduction to Machine Learning and its Applications
Part 2 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Introduction to Deep Learning and How It Is Being Used By Enterprises
July 26 2017
Part 3 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Question and Answer Session on Deep Learning’s Current or Future Role In Your Business
July 26 2017
Part 4 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Awareness Building - Educating Specific Groups Within Your Enterprise on the Maturity of the Technology
July 26 2017
Part 5 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
July 26 2017
Part 6of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Using Deep Learning Across Teams - Capturing and Re-using Insights, Running a Deep-Learning First Enterprise
July 26 2017
Part 7 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Question and Answer Session on What Your Business Would Need Out Of A Deep Learning Strategy
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Anomaly Detection - detecting an insider trade, flagging a keyword - detecting a fraudulent credit card transaction - detect money laundering
Machine Learning Applications
Speech Recognition - detecting specific words for translation systems - recognizing your voice, for biometric systems - identify whale sounds, so ships do not hit them - detect bird species from a bird call
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Face Detection
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Facial Recognition
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Image Classification - tagging images to create a searchable database - monitoring a stream of images - augmenting security, helping those who label data - segment objects, crop photos
Machine Learning Applications
Topic Modeling - automatic tagging of news articles, written content - detecting a fraudulent transaction,
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Anti-Spam - detecting (lazy) malware, use system logs/packets - detect spam in your email, want few false positives - detecting spam blogs, SEO aggressors
Machine Learning Applications
Genetics - use clustering to find disease-predicting genes
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Search
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Ads
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Translation - translate another language to native language - translate obscure text/writing into readable English
Machine Learning Applications
Language understanding - understand intents/slots from a query - understand text, forward to appropriate recipient - branch to different department, issue tracking - summarize long documents
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Video compression
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Weather forecasting
Preview - Slide Available at deeplearningconf.com Market segmentation
Preview - Slide Available at deeplearningconf.com July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Price premiums - forecast the lifetime cost of an insurance customer
Machine Learning Applications
Trade stocks and derivatives - buy, clean, and backtest data with algorithms - extract true patterns and detect buy signals
Predict risk of investment - use key performance indicators - test and mine valuation models July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Forecast housing/auction sale prices
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Predict ratings
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Recognize characters
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Self-driving cars
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Factory equipment maintenance
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Wait times
Preview - Slide Available at deeplearningconf.com
July 26 2017
Other forms of fraud
Preview - Slide Available at deeplearningconf.com
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
In-person market segmentation
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Chemical drug discovery
Preview - Slide Available at deeplearningconf.com July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Music recommendations
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
News or product recommendations
Preview - Slide Available at deeplearningconf.comMore recommendations
Preview - Slide Available at deeplearningconf.com July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Employee permissions - granting or revoking access to specific applications - employee attrition
Machine Learning Applications
Medical event premonition - analyze doctors’ notes to predict heart failure - predict emergency room admissions - predict premature births
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Call for help
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Video Event Detection/Anomaly Event detection
Preview - Slide Available at deeplearningconf.com July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Design suggestions - recommend layouts that match color palettes - colorize black and white images using context - generate variations of drawing created by user - create a full photo from a sketch
Machine Learning Applications
Identify songs - hear a few seconds of a song and give the title
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Generate handwriting - create personalized experiences
Machine Learning Applications
Generating Text - automatically caption images - generate new ads from previous ad clicks/social - Fill in missing parts of a legal document - Generate coherent arguments from legal documents
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Synthesize sound
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Generating & Organizing Music
Preview - Slide Available at deeplearningconf.com July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Question answering - interact with a chatbot to inquire about an item - ask your phone questions, receive useful answers - feature suggestions, gradually expose user via chat
Machine Learning Applications
Understand emotion - See images from a video stream and read emotions - Identify email or chat messages that are sales leads - Triage users who need special care or attention - Personality detection and compatibility July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Individual detection - enable actions for specific people in a seat in a car - personalize/identify user of a family internet account
Machine Learning Applications
Predict rising/trending stars - Mine tags in a geographic area for keywords - Predict if a product launch will be successful - Identify credibility of a source or thought leader
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Flag noteworthy news
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Authentication
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Security - predictive policing using crime data - imaging and video systems for airport security - predict psychopathy from internet usage
Machine Learning Applications
Energy - estimating demand requirements, load balancing - initiating iOT devices to turn on at low peak times
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Predict bad loans - predict refinancers and defaulters - analyze credit risk and automate loan processing - conversely reward those most likely to pay back
Machine Learning Applications
Combinations of Artistic styles - combine a famous painting with a camera photo - modify a live video stream with artistic style
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Detect new market segmentations
Preview - Slide Available at deeplearningconf.com
Machine Learning Applications
Text to speech and speech to text
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Machine Learning Applications
Pretty much anything that a normal person can do in <1 sec, we can now automate with AI. – Andrew Ng (@AndrewYNg), 19 October 2016
(with enough developers, even physical motion)July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
What is Machine Learning?
July 26 2017
A: It Is Teaching a Machine
B: It’s The Engine Behind Artificial Intelligence (AI)
You Might Believe:
C: It’s What Makes Google Still Relevant and What Makes New Startups Interesting Enough To Compete (This is a cop out Answer)
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
What is Machine Learning?
A: It Is Teaching a Machine
B: It’s The Engine Behind Artificial Intelligence (AI)
You Might Believe:
C: It’s What Makes Google Still Relevant and What Makes New Startups Interesting Enough To Compete (This is a cop out Answer)
D: It’s writing code that allows a computer to mine vastquantities of data and make fast, intelligent decisions
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
What is Machine Learning?
A: It Is basic intelligence that an analyst can do, but takes time
B: It is finding simple patterns in large amounts of data that humans can’t physically process
often sub-human or comparable performance that saves money
Other Adequate answers:
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
What is Machine Learning?
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- You coulduse only common
sense, whichyou may havea lot of, if youhave domainknowledge
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
So, you pick a decision threshold July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- You can classifydata by only onefeature, and it is
scientific tolook at one
variable, but it’s very limiting!
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- You can classify data by only two
features, and it isinterpretable, but it’s
still limiting!
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- Question: Why is it interpretable?
- Answer: You can draw the decision boundary (see the larger green and smaller blue rectangles
here), and therefore visualize the separation between your two
classes.
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- So, you could add more features.
- Here, you have seven features, with the relationship between the features plotted. Look at the top row, what have you learned? Perhaps still nothing more than the fact that houses in SF (green) are at higher elevations than houses in NY (blue). But, In most of the plots, the boundaries that delineate the classes are not immediately clear. Machine Learning solves this problem for you.
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
What is Deep Learning?
Automate Machine Learning
Automated feature selection
Look at larger quantities of data
Improve accuracy even more in industries that can benefit from small to medium improvements
July 26 2017
Part 2 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Introduction to Deep Learning and How It Is Being Used By Enterprises
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Compliance - Bank Secrecy Act Anti-Money Laundering BSA AML
- cluster suspicious transactions
- improvement on rule-based alert(s), which do not evolve
https://skymind.ai/case-studies/bsa-aml
- can have basic machine learning system that adds rules
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
SIM Box Fraud
https://skymind.ai/case-studies/orange
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
https://skymind.ai/case-studies/canonical
SysAdmin Monitoring
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
SysAdmin Monitoring Process
https://skymind.ai/case-studies/canonical
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Fraud Monitoring
https://skymind.ai/case-studies/finance
- rule based systems used because tens to hundreds of authorizations per second are sent to be vetted
- instead of sampling fraud instances and losing information, collect and learn from all of the information
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- roll your own system, rather than relying on legacy, catch-all analytics systems
- adaptively prioritize high probability cases of frauds in ways frozen decision tree models do not
Deep Learning and How It Is Being Used By Enterprises
Fraud Monitoring
https://skymind.ai/case-studies/finance
- finance is already open source - Linux, Hadoop/Spark
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
https://skymind.ai/case-studies/image
Image classification for online experiences, e-commerce, and security
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
https://skymind.ai/case-studies/image
Image classification for security and enforcement
- detect logos in images for copyright infringement
- monitor video streams and detect weapons under clothing
- automobile industry is a user of deep learning, detect scratches on vehicles as they go down assembly line
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
https://skymind.ai/case-studies/image
Image classification for security and enforcement
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
https://skymind.ai/commerce
Recommender Systems
- deal with mix of browsing behavior metadata, support chat history, and transaction history
- analyze information about pages viewed, text in descriptions, product images, type of music played in store
- constantly refine clustering algosJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 1: Data and Architecture)
Keywords: Supervised, Regression, Classification, Artificial Neural Networks, Hidden Layers, Weights
https://www.youtube.com/watch?v=bxe2T-V8XRs
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 2: Forward Propogation)
Keywords: Hyperparameters, Activation Function, Forward Propagation, Hidden Layers
https://www.youtube.com/watch?v=UJwK6jAStmg
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 3: Gradient Descent)
Keywords: Cost Function, Training, Dimensionality, Brute Force, Gradient Descent, Non-Convex, Stochastic Gradient
Descent
https://www.youtube.com/watch?v=5u0jaA3qAGk
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 4: Backpropagation)
Keywords: Chain Rule, Activation, Backprogagating, Batch Gradient Descent, Deep Neural Network,
https://www.youtube.com/watch?v=GlcnxUlrtek
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 5: Numerical Gradient Checking)
Keywords: The Definition of the Derivative, Vectors, Perturb Weights, Norm
https://www.youtube.com/watch?v=pHMzNW8Agq4
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 6: Training)
Keywords: Convergence, Local Minima, Optimization, BFGS, Overfitting,
https://www.youtube.com/watch?v=bxe2T-V8XRs
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep Learning and How It Is Being Used By Enterprises
Intro to Neural Networks (Part 7: Overfitting, Testing, and Regularization)
Keywords: Signal vs. Noise, Training Data, Testing Data, Regularization
https://www.youtube.com/watch?v=bxe2T-V8XRs
Preview - Slide Available at deeplearningconf.com
July 26 2017
Part 3 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Question and Answer Session on Deep Learning’s Current or Future Role In Your Business
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
1) What is your name, again, and what is your business?
2) What do you see as deep learning’s current or future role in your business?
3) What questions do you have about deep learning in production that we can we talk through?
July 26 2017
Part 4 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Awareness Building - Educating Specific Groups Within Your Enterprise on the Maturity of the Technology
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- No decision is the default for machine learning systems - that is Ok
If the current system is A, then the team would be unlikely to switch to B. If the current system is B, then the team would be unlikely to switch to A.
This seems in conflict with rational behavior: however, predictions of changing metrics may or may not pan out, and thus there is a large risk involved with
either change.
Start with Expectations and Metrics
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- You may have a number of project requirements and metrics, but you start with one machine learning objective function
- If you optimize for number of clicks, you are likely to see the time spent increase. So, keep it simple and don’t think too hard about balancing different metrics when you can still easily increase all the metrics.
Have 1 Objective Only
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Launch first, get data and do machine learning after
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Do best to capture all information
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Keep it simple
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Have specific people own a feature/column and document how it is populated and what it is
- Although many feature columns have descriptive names, it's good to have a more detailed description of what the feature is, where it came from, and how it is expected to help.
Segment ownership of the project to data engineering and data science
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Prioritize the freshness requirement of your model, if it generates a majority of your revenue and you've seen a major
performance hit when its unwatched, watch it more often
- Ad systems receive new ads each day, so generally must update daily
Freshness is paramount
July 26 2017
Part 5 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- 5.5 million face videos from 70 countries
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Quantify gaze fixation duration, but also discrete emotions (happy, sad, perplexed)
- Partner company, HireVue uses tech to rank video interviews
- Recently started to use voice data as well, another modality, conversational commerce,
healthcare proven by other companiesJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Use temporal model, expressions unfold over time
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Using Convolutional Neural Nets, training on identical images with different lighting for robustness to daylight/nighttime
- All data goes to Amazon S3- Certified labelers spit into training and validation, if agree
- Also use active machine learning, so machine in the loop, humans decide outliers
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Typically train on 200 examples, run experiments every night
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Think car on autopilot, in seconds needs to know if you’re awake or not to take control
- Think application that reads your emotions as you view a billboard, car iOT
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Matters because techniques are very applicable to audio, using analog detectors to get audio waveform out, extracting
the signature of a particle, as in speech
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Cofounder recorded himself 24/7 for two weeks, uploaded the data every day
- Speech to Text didn’t work well enough, average microphone caused high word error rate
- Google had worked on this problem in the past, turning political speeches to text, but didn’t work July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Google went back to text to speech
- Deepgram folks tried to recognize features directly, started at 20% accuracy, went to 90% six months later
- Think searching/indexing recorded business calls that would otherwise be useless low quality audioJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Not using text as an intermediary, building the index out of activations in the deep neural network
- Similar to using phonemes as features, but doesn’t exactly learn same “phonemes”
- Can force an ASR system to use phonemes to get slightly better performance in cases July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Same idea as CNNs, hierarchical representations, deeper is higher level, as in phonemes versus indiscernible sounds
- Can train with labeled topics and part of speech to further restrict and control what the activations are
- Can look at 2D FFT and recognize patterns, like images
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- In the tens of layers could go deeper with much more computation, but need to think where to put skips and 90% to
92% not that attractive at this time
- Service is an API, send in query and get all mentions
- Can get customized model built, needs 50 to 100 ten minute to one hour long files at least July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Own model runs on top of Theano, Caffe, and TensorFlow, can add 2D convolutional layers, another with batch norm,
recurrent layers with batch norm, then dense layer that predicts the word
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Three types of cybersecurity categories to consider: Malware, threat detection, and stream detection
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Ground truth is really expensive, unlike the image and video data applications, where data is plentiful
- To get the “bad” data, pay a provider for a threat database
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Clients are antivirus companies and police organizations
- Need an expert to tell you that is a threat, must reverse engineer it
Preview - Slide Available at deeplearningconf.comJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Have a labeled database, so run an ensemble regression on the inputs, use scoring altos to see how serious the data is
coming from the feed/stream
- Also collect data with “honeypots” - vulnerable machines put out as bait, get info on attackers
July 26 2017
- Have both host and network based data to look at, common to look at function names and regex
for “bad” strings
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Common technique is to look at the file hash - the filename description, in large quantities, to classify a threat
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Bad actors might add gibberish to a file, that can mislead tools
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Plus, calls to 1000 domains, picks one or two to attahk, modifies domains using domain generation algos, which
humans can easily see through, but not hundreds of thousands a day
Preview - Slide Available at deeplearningconf.com
July 26 2017
- Problem: adversaries follow conferences, two to three weeks see big change in variance in
random domains
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- A regex can’t identify these pseudorandom domain names, so unsupervised learning is used
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Develop predictive models that cover all major trading markets in the world, 30k stocks, forecast over 7 days, 14
days, and 30 days, seeing 60% to 70% accuracy four months in, so far
Preview - Slide Available at deeplearningconf.comJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Macroeconomic effects - Something in Asia ripples through the US financial markets
- Bringing technology to hedge fund quants as early adopters
Preview - Slide Available at deeplearningconf.comJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Ultimately make accessible to investment managers - want capabilities exposed via natural language
Preview - Slide Available at deeplearningconf.comJuly 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Traditional model can’t represent graph-directed edges, monitoring a million pieces of data a day
- Structured financial data and unstructured regulatory filings
- 300k news articles a day also - topic modeling July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Cover 80k publicly traded companies, 15 million private companies, “track” 200 million people, current litigation,
who investors are
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Entity resolution, disambiguating between sea shells and Shell the company, record linkage links datasets together, connecting things based off of location, or mention of an
iPhone, even when Apple not mentioned
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Grab every patent of every US company, check if address matches other company filings, look at these bodies of
evidence to prove a linkage
- Automated information extraction a real challenge, use deep learning with TensorFlow for language understanding, and to extract similar entities
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Get better results using thousands of highly specialized models than one general AI model
- First network picks out a concept/topic from the text, then a more sophisticated model determines that the document
is about, say, a corporation. July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Given word embeddings, corporation is near M & A, strong relationship there, so can identify that original company is
involved in M & A activity.
- Think automated tone identification to augment committees of users who currently evaluate CFO’s body
language and mannerisms July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
- Bottom line - deep learning at the onset for topic modeling and summarization, interpretable models at the user-facing
end for the forecasting
- Launched product, using an ensemble model so that is more “reverse engineerable”, retrain when you find you
overweighted interest rates, but try to be hands free, 60% is good enough oner an entire portfolio
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Specialized to Particular Tasks - Bespoke, Custom, Made-to-Order Solutions
Preview - Slide Available at deeplearningconf.com
July 26 2017
Part 6of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Using Deep Learning Across Teams - Capturing and Re-using Insights, Running a Deep-Learning First Enterprise
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Use Analytics
- Use analytics, choose simple features you don’t know you will need yet, track your current system with metrics
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Choose simple features so that you can be assured they are reaching your algorithm correctly when you have a mix of live
and offline-collected features
Create your pipeline
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Have permission to manually inspect data to double check? Do it.
- Have tests for examples in training and serving and check that the score is the same for an example
Test, independently
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- If data is coming from different services in your company, include a feature that specifies this
Include most data
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Use preexisting systems for preprocessing for your new system, a sender that has been blacklisted should be labeled
such, not learned - Create a feature that encodes the heuristic, a relevance
score can be encoded as a feature for a search system, just as a tax assessment can be encoded for a housing prices
model
Use what you have previously learned
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Do create composite heuristics that well represent your scoring function, for example, multiple the rating by the
number of installs
- Try breaking apart a composite heuristic of unrelated features, say one that sums installs and number of characters in an app description, and use each of those parts as features
Cautiously use combined heuristics
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- If you have a significant difference between training performance and holdout performance, look to training the next iteration of your model using data that is from different
days- Codify positional features, you will sidestep feedback loops
that come from placing, for an example, an ad in the first position and seeing it get clicked and therefore weighted more
heavily
Check for data collection missteps and feedback loops
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Have higher regularization on features that correspond to more than one query, that way you can emphasis results that
respond only to specific queries
- Only allow features to have positive weights; therefore, only “good” features will be used, no feature will have a more
negative impact than an unknown feature
Make informed choices for parameters
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- The data bottleneck becomes apparent with unbalanced classes. If you sampling a tiny percentage, 0.01%, and you
have few spam examples, you must uphold the sampling rate between classes. You can work with as few as 10k examples,
sometimes fewer
You don’t need all the data, if you have a ton
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Use a basic metric, and separate from production initially
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Gradually decaying performance can indicate a table is not being refreshed, so a feature has significantly and/or older
examples, so a refresh can improve performance vastly more than improving the model
Even the giants fail, conspicuously in this case
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Pick one proxy objective and stick with it at the onset
- If your objective is increased, but you have chosen not to launch the product, reevaluate or pick a new objective that
when increased will result in a launch
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
1. Was this ranked link clicked? 2. Was this ranked object downloaded? 3. Was this ranked object forwarded/replied to/emailed? 4. Was this ranked object rated? 5. Was this shown object marked as spam/pornography/
offensive?
1. Choose direct objectives:
Pick a simple objective
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
1. Did the user visit the next day? 2. How long did the user visit the site? 3. What were the daily active users?
1. Avoid modeling indirect effects, these are metrics, to be used during A/B testing during launch, not objectives:
Watch out for roundabout or indirect objectives
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Get as many metrics as possible, combine and recombine features to create new features, just like that embraced by Kaggle
Iterate on features for your models
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Use deep features after you have a baseline system using manually selected features, as a deep model can give a
different solution the next time it is trained, so it needs to be tested more thoroughly
Use deep features after you have developed a baseline
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- It is expected that some of the labels will be wrong or cover different amounts of data, use a lot of these simple features anyway. You can use “regularization”, that is, add noise, to learn in a generalizable way, despite this.
Use all the data you have, but nothing you wouldn’t have
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Deep learning across the teams must be calculated
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Diversity, relevance, and personalization are not as correlated with popularity, that is, clicks, as one might think
Look at just your objective
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Look for new data sources once you plateau
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- If you can use one programming language across training and serving, do it, it will allow you to share code and better
confirm performance
Simplify and do not cross models
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Pick a number of features that represent the length, such as number of unique words, number of words, number of characters, number of pages, for example, let the system decide what is important.
Pick multiple features, let the system learn
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Engineering time is too valuable to spend time guessing at the importance of certain features as compared to others
Spend engineering time engineering
Preview - Slide Available at deeplearningconf.com
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Get rid of features that are not clean or interpretable as to how they were collected, or that cover a tiny percentage of the
examples
- At the same time, definitely keep those features that cover a tiny percentage of the examples but a large percentage of a certain class
Clean up your features if you can
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- The number of feature weights you can learn, that is, the number of parameters, is roughly proportional to the amount of
data you have
- Yes, more data means more features, which tends to mean better performance. As a general rule, go two orders of magnitude down from number of examples to number of features
Follow general rules
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
- Preprocess data using “discretizations" and “crosses”.
- Discretizations create boundaries of histograms, pick a discrete integer to represent a grouping
- With a large amount of data, use crosses to create new features that are unions/overlaps between 2 or more features
Use transformations to augment data
July 26 2017
Part 7 of 7
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
Question and Answer Session on What Your Business Would Need Out Of A Deep Learning Strategy
July 26 2017
Sam PutnamExecuting Deep Learning Strategies
@edeeplearning
1) What do you see as your company’s deep learning strategy?
2) What questions do you have about deep learning strategies that we can we talk through?
3) What is something that you are going to take away from this conference and apply to your business?
July 26 2017
Thank you
Sam Putnam
Questions/Comments: [email protected]
Thank you to Google and others who have published guidelines. Slides are for today only.
Executing Deep Learning Strategies
@edeeplearning
July 26 2017