SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq...
23
SciOps, because Data Scientists need DevOps too!
Transcript of SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq...
SciOps, because Data Scientists need DevOps too!
What happens when an idea is born?
What actually happens
Data scientist should have full control over the solutions they implement
A move to useful solutions from brilliant ideas
Frustration!
Engineer Data Scientist
Boring Long iterations
Repetitive No control
No time for Innovation Lots of waiting
● Single library import will turn any function into a high performance server
● Works best with Docker and Kubernetes● Facilitate DevOps culture
AssafSimple Model Serving Platform
Centralized services
Request queues
Response queues
Models
Assaf infrastructure on Kubernetes
Connectors
Major Design Decisions
Request queues
Response queues
Message Queues
● Request queue: messages are picked up by the connector on the designated process
● Response queue: messages are picked up and served to clients
● Both: one-way message queues
Request queues
Response queues
Benefits
● Decoupling of producers and consumers ● Resilient communications - even if services are down
requests are persisted by the queues ● Language agnostic - just connect to rabbitmq
● Connectors handle parallelism ● Prefer using native message passing
○ Go -> Channels○ Scala -> Actors
● ØMQ is used when messages are not supported natively
Message Passing For Concurrency
Connectors
Anatomy an assaf connector
● Embarrassingly Parallel Applications● Write procedural, run in parallel
○ Requests must be atomic ● High Availability out of the box● Infinite horizontal scaling
Benefits
Centralized Services
● High performance collector
● Tracing
● Monitoring Centralized
services
Benefits
● Freedom to choose best tool for
the job
● Change once, enjoy everywhereCentralized
services
Demo
What improvements did we deliver??
Centralized application
Cohesive process for shipping to production
Low effort deployment
Incremental improvements
Propagation of improvements
Responsibility stays with the implementer
CI/CD
Easy to use library
