David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All...
Transcript of David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All...
![Page 1: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/1.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
GPU Technology Conference (GTC) 2019 [S9810]March 19, 2019
Ian Hummel, Data Architect, Office of the CTODavid Eis, Senior Software Engineer, AI Group
Machine Learning@ BloombergBuilding on Kubernetes
![Page 2: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/2.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Agenda
• Finance and technology
• Machine Learning at Bloomberg
• Data Science Platform infrastructure
• Results and future work
![Page 3: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/3.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Bloomberg
![Page 4: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/4.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
About Bloomberg
• Founded in 1981 • 325,000 Terminal subscribers • Customers in 170 countries• Over 19,000 employees in 192
locations• Over 5,000 software
engineers/programmers• More reporters than The New York
Times + Washington Post + Chicago Tribune
![Page 5: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/5.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Finance and technology
![Page 6: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/6.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Finance and technology
https://en.wikipedia.org/wiki/Bulla_(seal)https://en.wikipedia.org/wiki/Semaphore_telegraph
![Page 7: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/7.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Finance and technology today
![Page 8: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/8.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Understanding what’s going on out there
![Page 9: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/9.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
SEC Announcement
First Bloomberg HeadlineNew York Times Story
12%
<20 minutes
Natural Language Processing & Events
![Page 10: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/10.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Over $136 billion was wiped out inminutes
The risks of being wrong
https://www.bloomberg.com/news/articles/2013-04-23/a-fake-ap-tweet-sinks-the-dow-for-an-instant
![Page 11: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/11.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
The future is ripe for ML
![Page 12: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/12.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Machine Learning at Bloomberg
![Page 13: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/13.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Financial named entity extraction
JP
Bear Stearns
![Page 14: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/14.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
“In 1971, Siegl, Bowker and Baldwin established the Starbucks Coffee Company.”Explaining Knowledge
Graph relationships
![Page 15: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/15.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Table Extraction
![Page 16: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/16.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Automating the news
![Page 17: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/17.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
HELP HELP - Query routing
![Page 18: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/18.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
News QA
![Page 19: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/19.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Charts QA
![Page 20: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/20.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Why build a Data Science Platform?
![Page 21: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/21.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
ML opportunitiesWhat questions do teams ask before starting a new project?
• What data is relevant to my business problem?• What does the data look like?• What features should I create from my data?• What metric should I use to measure my model performance?• What algorithms or parameters should I test out? • How is my model doing on live data in production?
![Page 22: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/22.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Challenges for ML projectsEven in an established enterprise, ML teams face the same challenges again and again...
• Where is my data located?• How do I get permissioned for my data?• How do I load this data in a native visualization tool?• How do I get this toolkit installed and operational?• How do I schedule my long-running job?• How much compute power is available for me to exploit?• How do I deploy my model?
![Page 23: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/23.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
The model development lifecycle
![Page 24: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/24.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Different roles, different priorities
Customers• Smarter Functionality• Privacy• Correctness• Uptime/Stability
ML Practitioners• Keep up with latest trends in
academia and forefront of Deep Learning innovations
• Access to specialized hardware• Rapid iteration over research
ideas• Focus on business problem, not
building infrastructure
Engineers• Less operational burden • Support for multi-tenancy • Ease of data access
― But also secure and properly privileged• Avoid duplicated code• Reproducibility of results
![Page 25: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/25.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Blueprint for an ML infrastructure...
Foundational Building Blocks• Specialized Hardware• Standardized Runtimes• Batch Scheduling
Data Liquidity• Data Access• Security
Developer Experience• Ease of Use• Rapid Iteration• Native Tooling
ML Specifics• Experiment Management• Hyperparameter Tuning• Result Reproducibility
![Page 26: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/26.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes to the rescue!
![Page 27: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/27.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Legend
COMPUTE
DS Platform architecture
- scheduled job
- GPU
API Server Platform Controller
Web UI
DATA + MODELS
S3
HDFS
Fetch data/store models
USER
CLI
Job history UI
- yaml spec
![Page 28: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/28.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
What is Kubernetes?“Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.”
- https://kubernetes.io/
![Page 29: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/29.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
USERAPI Server Controller
![Page 30: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/30.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
Watch Resource ChangesUSER
API Server Controller
![Page 31: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/31.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
React to Updates
USERAPI Server Controller
![Page 32: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/32.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
Watch Resource ChangesUSER
API Server Controller
![Page 33: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/33.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
Pod Resource
USERAPI Server Controller
![Page 34: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/34.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes
Kubernetes life cycle
Schedule Pod
USERAPI Server Controller
![Page 35: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/35.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Deployment resource (YAML)apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deploymentspec: replicas: 3 template: spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
![Page 36: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/36.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Deployment resource: kindapiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deploymentspec: replicas: 3 template: spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
![Page 37: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/37.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Deployment resource: spec (replicas)apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deploymentspec: replicas: 3 template: spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
![Page 38: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/38.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Deployment resource: spec (pod template)apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deploymentspec: replicas: 3 template: spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
![Page 39: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/39.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Benefits of Kubernetes• Declarative deployment• Hardware indifference• Monitoring• Replication• Autoscaling• Security Extensions
― Ingress + SSL termination― Istio service mesh
![Page 40: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/40.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Custom resources• Resource - a set of API objects in Kubernetes, such as
Deployments or Pods.
• Custom Resource - an extension of the Kubernetes API defined by a user that declares new types of API Objects.
![Page 41: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/41.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/fancy_model
![Page 42: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/42.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 43: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/43.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 44: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/44.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 45: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/45.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 46: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/46.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 47: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/47.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 48: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/48.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Kubernetes meets ML: TensorFlowJobapiVersion: ds.bloomberg.com/v1kind: TensorFlowJobmetadata: name: tf-test annotations: ai.bloomberg.com/project: foo ai.bloomberg.com/experiment: abcdespec: framework: tensorflow-1.13-python-3 identities: - hadoop: id: davideis pipPackages: [ai.bloomberg.com.myteam.gpu_tftraining, numpy] module: gpu_tftraining size: GpuLarge args: --data-dir hdfs://CLUSTER/projects/news/news_index --output-dir hdfs://CLUSTER/users/deis/abcde/foo/1
![Page 49: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/49.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
A model for the ML ecosystem• Other Types of Custom Resources
― Spark― Python ― JVM― Jupyter Notebook― Hyperparameter Tuning
• Foundation for reproducibility and automation
![Page 50: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/50.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Tooling
![Page 51: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/51.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Achieving nirvana (reprise)
Foundational Building Blocks• Specialized Hardware• Standardized Runtimes• Batch Scheduling
Data Liquidity• Data Access• Security
Developer Experience• Ease of Use• Rapid Iteration• Native Tooling
ML Specifics• Experiment Management• Hyperparameter Tuning• Result Reproducibility
![Page 52: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/52.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Helping teams succeed with the DS Platform
![Page 53: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/53.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Building infrastructure is not for the faint-hearted
• Demanding users• Multi-tenancy is hard• New hardware designs hard to incorporate• ML projects require consulting• Effective tooling means team requires broad skill set• Open source moving incredibly fast• What’s your value-add??
![Page 54: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/54.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Strong emphasis on metrics
• Number of teams using the platform• Number of users using the platform• How often they submit jobs• Duration of jobs• Types of jobs submitted• Hardware utilization• Net Promoter Score (NPS) surveys
![Page 55: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/55.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
What’s next for us?
• More tooling• Kubernetes batch/job queueing• KNative serving• Dataset and model management• Additional ML services like data drift detection
![Page 56: David Eis, Senior Software Engineer, AI Group Ian Hummel ... · © 2019 Bloomberg Finance L.P. All rights reserved. GPU Technology Conference (GTC) 2019 [S9810] March 19, 2019 Ian](https://reader033.fdocuments.us/reader033/viewer/2022042220/5ec660815ba11250ea75d1cd/html5/thumbnails/56.jpg)
© 2019 Bloomberg Finance L.P. All rights reserved.
Thanks!
Reach out to us [email protected] and [email protected]