Adventures in Azure Machine Learning from NE Bytes
-
Upload
derek-graham -
Category
Technology
-
view
166 -
download
2
Transcript of Adventures in Azure Machine Learning from NE Bytes
![Page 1: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/1.jpg)
Adventures!in Azure Machine
Learning@deejaygraham
![Page 2: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/2.jpg)
derek graham!principal developer @sage!
special product responsibility for Azure bits
![Page 3: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/3.jpg)
![Page 4: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/4.jpg)
![Page 5: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/5.jpg)
Sorry!!live coding!
live portall-ing
![Page 6: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/6.jpg)
Machine learning is not telling
computers what to do but letting them
learn from examples or past experience
![Page 7: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/7.jpg)
![Page 8: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/8.jpg)
Which means you can…
• Analyse historic or current data • Find patterns and trends • Make predictions about future events
![Page 9: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/9.jpg)
![Page 10: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/10.jpg)
![Page 11: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/11.jpg)
![Page 12: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/12.jpg)
![Page 13: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/13.jpg)
Project Adam
![Page 14: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/14.jpg)
![Page 15: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/15.jpg)
![Page 16: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/16.jpg)
![Page 17: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/17.jpg)
![Page 18: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/18.jpg)
Azure Machine Learning
• A "new" cloud-based service from Microsoft • Integrates with existing Cloud technologies • Use ready-made algorithms • Program custom algorithms tuned to your problem • You can evaluate it for free
![Page 20: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/20.jpg)
![Page 21: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/21.jpg)
![Page 22: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/22.jpg)
• Browser based • Drag n Drop • Flowchart-y • Example data sets • Use R or Python • Excellent intro wizard
ML Studio
![Page 23: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/23.jpg)
![Page 24: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/24.jpg)
• Import data • Filter and aggregate data • Create machine learning models • Run experiments • Publish finished model
Provides tools to:
![Page 25: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/25.jpg)
![Page 26: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/26.jpg)
The Learning Process• Define a problem you want to solve • Design a solution • Experiment! !
• Identify your data • Train the model with the data • Evaluate against expected results (speed and
accuracy) • Adapt data or algorithm (or both) • Repeat !
• Save the best model • Publish • Run with live data
![Page 27: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/27.jpg)
Proof!
![Page 28: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/28.jpg)
Imagine…• "Business" Software
• Azure hosted
• PaaS
• Multi-tenanted
![Page 29: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/29.jpg)
Open-ended Workflow• Monday Morning Login
• Friday Reports
• In between?
• Weekends?
• Holidays?
![Page 30: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/30.jpg)
Balancing• User demand
• User experience
• Compute resources
• Cost
![Page 31: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/31.jpg)
Scaling• Instances auto-scale based on the CPU% metric
using Azure’s standard scaling model.
• Azure standard scaling is slow
• Once auto scaler notices we need more capacity, the demand has often disappeared!
• Not a good user experience
![Page 32: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/32.jpg)
![Page 33: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/33.jpg)
Experiment• Customer use is not regular...
• ...but, is it predictable?
![Page 34: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/34.jpg)
Hackathon!• Can we build a better autoscaler?
• Spin-up before high demand
• Tear-down when idle
• Better Cost vs UX
![Page 35: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/35.jpg)
![Page 36: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/36.jpg)
Requirements• What will "we" need on a given date or time?
• Do "we" need to take action now to compensate for what will happen in 20 minutes time?
• Number of instances
• Predicted CPU
![Page 37: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/37.jpg)
Best Predictor of Demand?
• Sessions?
• Instance Memory Use?
• Instance CPU?
![Page 38: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/38.jpg)
![Page 39: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/39.jpg)
Table Storage Diagnostics• Too slow
• Purging
• ML queries all or nothing
• ML Data Reader stops after 4GB
• GB !!!!
• ML times-out after ~3 Hours
![Page 40: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/40.jpg)
CPU
![Page 41: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/41.jpg)
Event Hubs• Application log sink
• Low overhead
• Highly scalable
• Time-based
• Disposable
![Page 42: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/42.jpg)
![Page 43: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/43.jpg)
Neural Net Experiments!
• Feed Forward NN
• Written using R libraries
• Good predictor for 10-20 minute window
• Too inaccurate after that
• Best compromise between precision and speed
• Recurrent NN better at forecasting
• RNN execution time too long
• Need to reduce data to optimal subset
![Page 44: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/44.jpg)
Stream Analytics!
• Real-time data analysis
• Fast
• Sql-like syntax
• Range of inputs and outputs
• Interesting development
![Page 45: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/45.jpg)
![Page 46: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/46.jpg)
![Page 47: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/47.jpg)
![Page 48: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/48.jpg)
![Page 49: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/49.jpg)
![Page 50: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/50.jpg)
Anomalies• Dev Process is painful
• Syntax Errors
• “Test” Import Behaviour
• Starting and Stopping and Starting and Stopping
![Page 51: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/51.jpg)
![Page 52: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/52.jpg)
Compromise
![Page 53: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/53.jpg)
Closing the Loop
![Page 54: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/54.jpg)
Publish…• ReST Web service
• Client Worker Role
• Management Service API
![Page 55: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/55.jpg)
…& Be Damned• Too much data crashes model
• Fine in ML Studio
• 500
• Out of memory?
![Page 56: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/56.jpg)
![Page 57: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/57.jpg)
Finished!
![Page 58: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/58.jpg)
![Page 59: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/59.jpg)
Result!
![Page 60: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/60.jpg)
![Page 61: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/61.jpg)
![Page 62: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/62.jpg)
What we learned
![Page 63: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/63.jpg)
Bugs• We were pushing the environment quite hard
• YMMV
• ML studio has bugs
• Parallel tasks !Parallel
• ML portal missing functionality preventing it being production ready
![Page 64: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/64.jpg)
#DevOps• Sharing models is "public" - Gallery
• No export support
• No support (yet) for model deployment
• Still Drag n Drop
• PowerShell for EventHubs and Stream Analytics
![Page 65: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/65.jpg)
Machine Learning• Parallel R processing library would help
• Finding an appropriate solution often requires a data science specialist
• Solution is only as good as your data
• You may need to compromise on accuracy for speed
• Cost
• Hosting
• Each call to the service
![Page 66: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/66.jpg)
References
http://studio.azureml.net/
![Page 67: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/67.jpg)
E-Book• Microsoft Azure Essentials: Azure Machine
Learning
• Download from: https://mva.microsoft.com/ebooks
![Page 68: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/68.jpg)
Titanic• Jennifer Marsman - https://
blogs.msdn.microsoft.com/jennifer/2016/02/19/using-azure-machine-learning-to-predict-who-will-survive-the-titanic/
• Data Science! https://www.kaggle.com/
• Amy Nicholson @AmyKateNicho https://blogs.technet.microsoft.com/amykatenicho/
![Page 69: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/69.jpg)
#DevOps• https://azure.microsoft.com/en-gb/documentation/articles/event-
hubs-programming-guide/
• https://azure.microsoft.com/en-gb/documentation/articles/service-bus-event-hubs-manage-with-ps/
• https://azure.microsoft.com/en-us/documentation/articles/stream-analytics-dotnet-management-sdk/
• https://azure.microsoft.com/en-us/documentation/articles/stream-analytics-monitor-and-manage-jobs-use-powershell/
![Page 70: Adventures in Azure Machine Learning from NE Bytes](https://reader031.fdocuments.us/reader031/viewer/2022022414/5875df6e1a28ab7d5a8b4d91/html5/thumbnails/70.jpg)
Questions ?