OSDC 2014: Tobias Schwab - Continuous Delivery with Docker
description
Transcript of OSDC 2014: Tobias Schwab - Continuous Delivery with Docker
![Page 1: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/1.jpg)
Continuous Delivery with Docker
Tobias Schwab
![Page 2: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/2.jpg)
Myself• Tobias Schwab
• www.dynport.de
• twitter.com/tobstarr
• github.com/tobstarr
![Page 3: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/3.jpg)
Philosophie• continuous delivery: deploy multiple times a day
• canary releases
• “never touch a running system”
• “Immutable Infrastructure and Disposable Components"
• don’t fix it, if it can be replaced
![Page 4: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/4.jpg)
Theory• AWS
• AMI based deployments
• Elastic Load Balancer
• AutoScaling Groups
• S3, RDS, …
![Page 5: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/5.jpg)
Reality• privacy concerns: AWS not an option
• hoster we could not pick
• first no, then proprietary and unreliable API
• flash based infrastructure management
• limited capacity
• we were the biggest customer
![Page 6: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/6.jpg)
Docker• build, distribute and deploy container based
applications
• creator: dotcloud
• initial release: March 13, 2013
• license: Apache 2.0
• 11k stars on Github (top 50)
• golang client/server
![Page 7: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/7.jpg)
Container Virtualization• os level
• shared kernel
• cgroups: isolate CPU, Memory, Block IO, Network
• lxc: cgroups + application namespaces
• lightweight and fast
![Page 8: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/8.jpg)
Images • blueprints for containers
• tarball of os installation/packages
• read only
• stateless
• layered
![Page 9: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/9.jpg)
Containers• instances of images
• copy on write / union file system
• running or exited
• goal: stateless and immutable
• can be “saved” (docker commit) as images
• created to be thrown away
![Page 10: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/10.jpg)
Containers and images
Source: http://docs.docker.io/en/latest/terms/container/
![Page 11: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/11.jpg)
Demo
![Page 12: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/12.jpg)
Build• manual
• start and attach container
• install required packages
• checkout application code
• run build management tool
• bad: not reproducible
• bad: does not utilise caching
![Page 13: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/13.jpg)
Build• chef/puppet/…
• start an attach container
• run chef/puppet/… client
• good: automated and documented
• bad: does not utilise caching
![Page 14: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/14.jpg)
Dockerfile• simple, plain text script to create images
• commands:
• FROM: base image to use
• RUN: execute shell command
• ENV: set environment variable
• ADD: write local file to image
• ENTRYPOINT: start command for containers
• others: MAINTAINER, EXPOSE, CMD, USER, VOLUME, WORKDIR, ONBUILD
![Page 15: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/15.jpg)
Dockerfile
![Page 16: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/16.jpg)
Dockerfile
![Page 17: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/17.jpg)
Caching• statement based: each step creates a new image
• existing steps (command tree exists) are re-used
• tricky: “non functional” commands (e.g. apt-get update/upgrade)
• use ENV or comments to break caching of non functional commands
![Page 18: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/18.jpg)
Caching
![Page 19: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/19.jpg)
Configuration Management• “store config in the environment” (http://12factor.net/config)
• dependency injected with start of container
• same image for
• development
• testing
• staging
• production
![Page 20: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/20.jpg)
Don’ts• full blown VMs
• ssh daemon inside containers
• syslog daemon inside containers (sometimes needed)
• user management: everything can run as root
• chef/puppet/… => makes caching useless
![Page 21: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/21.jpg)
Build Management Tools• candidates: bundler, pip, mvn, carton, composer, …
• problem with caching: bmt are slow when started with “clean slate”
• option 1: add bmt manifest before code
• bmt needs to run only when manifest changes
• option 2: use pre-bundled base images
• bmt only needs to work the delta
• re-build base images from time to time
• option 3: combine option 1 and option 2
![Page 22: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/22.jpg)
• Problems
• unicorn: Rack HTTP server for fast clients
• static assets
• logging: default ruby syslog library uses syscall (needs local syslog daemon)
• Solution
• run 3 daemons in 1 container: unicorn, nginx and rsyslogd
• upstart
• ENTRYPOINT [“/sbin/init”]
• load ENV from /proc/1/environ
• foreman
Use Case: Ruby on Rails
![Page 23: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/23.jpg)
Multi-Host
• image distribution via docker registry
• weighted load balancing via HAProxy
• SSL termination via nginx in front of HAProxy
![Page 24: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/24.jpg)
Registry
• push and pull images
• public
• private
• backends: local, S3, Elliptics, Google Cloud Storage, hosted
![Page 25: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/25.jpg)
Load Balancing• HAProxy
• license: GPL v2
• pool configuration stored in redis/etcd
• config update
• compile config files from stored configuration
• upload via ssh
• verify on remote hosts
• replace current config with verified one
• reload
![Page 26: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/26.jpg)
HAProxy
![Page 27: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/27.jpg)
HAProxy
![Page 28: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/28.jpg)
Deployment Pipeline• commit triggers new image build
• build suite executed with image
• image is pushed to registry if tests passed
• optional: start image with staging ENV settings for manual testing
• start image with production ENV for last pre-flight tests
• deploy image to more hosts
• update load balancer (canary or green/blue)
• monitor new containers/image
![Page 29: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/29.jpg)
Deployment PipelineNginx
HAProxy
Nginx
HAProxy
Docker
Container
Container
Container
Docker
Container
Container
Container
Docker
Container
Container
Container
Docker
Container
Container
Container
Docker Registry
Docker Build
2 push
3 pull + run
1 build
4 update4 update
Route 53
![Page 30: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/30.jpg)
Logging
• host: docker host, container_id
• code: image_id, revision
• request: request_id, action, status_code, etag, times, calls
• NOT inside containers
• remote syslog (when possible)
• alternative: local syslog relay inside container
![Page 31: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/31.jpg)
Metrics• OpenTSDB
• “distributed, scalable Time Series Database”
• license: LGPLv2.1+
• HBase
• Tags / Dimensions
• from syslog via udp (StatsD “like”)
• rickshaw.js for graphs
• compare status codes, counts and times between actions of two revisions
![Page 32: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/32.jpg)
OpenTSDB
![Page 33: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/33.jpg)
Metrics
![Page 34: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/34.jpg)
Metrics
request counts by revision
![Page 35: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/35.jpg)
Metrics
![Page 36: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/36.jpg)
Metrics
![Page 37: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/37.jpg)
Metrics
![Page 38: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/38.jpg)
Docker reduces• external dependencies (“rubygems/github slow/unreliable/down”)
after image is built
• “did work on my machine/staging”: same OS package versions, configuration and code in all stages
• unused CPU cycles
• number of hosts
• feedback times
• time to get new host online
• bottlenecks: hosts are more flexible
![Page 39: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/39.jpg)
VS. AWS• HAProxy much more flexible
• multiple containers per host
• balancing weights
• faster build process
• faster deployments
• instance flexibility
![Page 40: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/40.jpg)
Resources• docker.io
• opentsdb.net
• haproxy.1wt.eu
• continuousdelivery.com
• chadfowler.com/blog/2013/06/23/immutable-deployments/
• 12factor.net
![Page 41: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/41.jpg)
Questions?!?
![Page 42: OSDC 2014: Tobias Schwab - Continuous Delivery with Docker](https://reader034.fdocuments.us/reader034/viewer/2022051314/54c644f84a795906378b459a/html5/thumbnails/42.jpg)
Thank you!