Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config...
Transcript of Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config...
![Page 1: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/1.jpg)
Velociraptoran open-source app deployment system by YouGov
![Page 2: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/2.jpg)
YouGov
• Top 25 research company
• Based in London
• Global offices
• Gathers beliefs and behaviors from millions
• Data + expertise → What the World Thinks™
• Heavily invested in Python
![Page 3: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/3.jpg)
motivation
• repeatable deployment
• developer driven
• language agnostic
• Python
• PaaS
![Page 4: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/4.jpg)
eggmonster• Setuptools multi-installed packages
• Managed by central config server
• YAML configs with inherited properties
• Violated repeatability
• one app might update another’s packages
• No GUI
• Fixed ports and manual routing
![Page 5: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/5.jpg)
Consideration
• Juju
• ActiveState Stackato
• Based on CloudFoundry and Docker
![Page 6: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/6.jpg)
Inspiration
• Heroku!
• 12-Factor Methodology
• http://12factor.net
• Buildpacks
• Docker (lxc)
![Page 7: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/7.jpg)
Implementation• Django
• GUI
• Postgres database models
• TastyPie API
• MongoDB
• Images
• Slugs
• Celery workers
• Supervisord to host processes (procs)
• Command-line Interface
![Page 8: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/8.jpg)
Features
• Zero downtime deployment
• Pluggable Balancers
• Managed Config
• Repeatable Deployments
• Uptests!
![Page 9: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/9.jpg)
Platform• Stacks / Images
• Buildpacks
• Apps
• Squads
• Ingredients (config)
• Builds
• Releases
• Swarms
![Page 10: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/10.jpg)
Stacks
• Runtime environment
• Akin to Heroku stacks
• Linux images
• base
• provisioning script
• Images built through GUI
![Page 11: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/11.jpg)
Buildpacks
• Inspired by Heroku
• Industry standard for building apps
• Simple protocol (detect, compile, release)
• Supports many languages and techniques
![Page 12: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/12.jpg)
Apps
• Repository (Git or Mercurial)
• Relevant buildpack (autodetection)
• Relevant stack
![Page 13: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/13.jpg)
Squads
• Pool of hosts
• Commodity
• Identically provisioned
• Colocated
![Page 14: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/14.jpg)
Config Ingredients
• Re-usable Application Config
• YAML
![Page 15: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/15.jpg)
Builds
• Buildpack output
• Freezes dependencies
• Slug without config
• Combine with config to create a Release (slug)
![Page 16: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/16.jpg)
Swarms• Primary target for Velociraptor
• Logical combination of
• a Process
• within an App
• at a Tag (designates build)
• with Config (ingredients + direct)
• targeting a Squad
• routed in a Balancer
![Page 17: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/17.jpg)
Balancers• Internet-facing hosts
• Route traffic to procs in Pools
• Supports:
• Nginx
• Varnish
• Stingray
• Pluggable, extensible
![Page 18: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/18.jpg)
Uptests
• Sanity check
• Arbitrary executable
• Guard routing until proc validated
![Page 19: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/19.jpg)
Demo
![Page 20: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/20.jpg)
Getting Started
• walkthrough
• Vagrant + Virtualbox
• Mercurial
• test suite
• py.test
![Page 21: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/21.jpg)
Production Deployment
• Create hosts with supervisord and proc_publisher.
• Create a virtualenv for bootstrapping (similar to development model).
• Configure and provision production databases.
• Use bootstrap instance to build/deploy a production instance of Velociraptor.
![Page 22: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/22.jpg)
Alternatives• ActiveState Stackato (http://www.activestate.com/
stackato)
• Google Kubernetes (http://kubernetes.io)
• Ansible Tower (http://www.ansible.com/home)
• Gilliam (http://gilliam.github.io/)
• Tsuru (http://docs.tsuru.io/en/latest/)
• openroku (https://github.com/openruko)
![Page 23: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/23.jpg)
Questions?
![Page 24: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/24.jpg)
Why not Docker?
• Docker is opinionated about its process management role.
• Conflicts with use of supervisord.
• https://github.com/docker/docker/issues/503
![Page 25: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/25.jpg)
Advanced Features
• Shell in application context.
• Volume mounts.
![Page 26: Velociraptor · eggmonster • Setuptools multi-installed packages • Managed by central config server • YAML configs with inherited properties • Violated repeatability •](https://reader034.fdocuments.us/reader034/viewer/2022042222/5ec9638287a89d58691d0720/html5/thumbnails/26.jpg)
References
• Repository https://bitbucket.org/yougov/velociraptor