Docker open stack boston

27
Application deployment on Openstack using Containers and Docker BOSTON OPENSTACK MEETUP: August 14, 2013

description

Presentation at OpenStack Meetup in Boston on Docker,

Transcript of Docker open stack boston

Page 1: Docker open stack boston

Application deployment on Openstack using Containers and DockerBOSTON OPENSTACK MEETUP:

August 14, 2013

Page 2: Docker open stack boston

Contents

• Introduction to Docker (10 min)• Docker and OpenStack (5 min)• Demo of Docker (10 minutes)• Demo of Docker + OpenStack (10 minutes)• Q&A

Page 3: Docker open stack boston

In the 4.5 months since we launched

• >50,000 pulls• >4,500 github stars• >100 significant contributors• >150 projects built on top of docker

• UIs, mini-PaaS, Remote Desktop….

• 1000’s of Dockerized applications• Memcached, Redis, Node.js…and

Hadoop

• Integration in Jenkins, Travis, Chef, Puppet, Vagrant and OpenStack

• Meetups arranged around the world…with organizations like Ebay, Cloudflare, and Rackspace presenting on their use of Docker

Page 4: Docker open stack boston

Why all the excitement?

Page 5: Docker open stack boston

Static website

Web frontend

User DB

Queue Analytics DB

Background workers

API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8

hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recovery

Contributor’s laptop

Production Servers

The ChallengeM

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps

interact appropriately?

Can I migrate

smoothly and quickly?

Page 6: Docker open stack boston

Results in matrix from hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM

QA Server Single Prod Server

Onsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Page 7: Docker open stack boston

Mul

tiplic

ity o

f Goo

dsM

ultip

ilici

ty o

f m

etho

ds fo

r tr

ansp

ortin

g/st

orin

g

Do I w

orry about how

goods interact (e.g. coffee beans

next to spices)

Can I transport quickly and sm

oothly(e.g. from

boat to train to truck)

Cargo Transport Pre-1960

Page 8: Docker open stack boston

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Also a matrix from hell

Page 9: Docker open stack boston

Mul

tiplic

ity o

f Goo

dsM

ultip

licity

of

met

hods

for

tran

spor

ting/

stor

ing

Do I w

orry about how

goods interact (e.g. coffee beans

next to spices)

Can I transport quickly and sm

oothly(e.g. from

boat to train to truck)

Solution: Intermodal Shipping Container

…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another

A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.

Page 10: Docker open stack boston

Static website Web frontend User DB Queue Analytics DB

Development VM

QA server Public Cloud Contributor’s laptop

Docker is a shipping container system for code M

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps

interact appropriately?

Can I migrate

smoothly and quickly

…that can be manipulated using standard operations and run consistently on virtually any hardware platform

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…

Page 11: Docker open stack boston

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM

QA Server Single Prod Server

Onsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

Docker eliminates the matrix from Hell

Page 12: Docker open stack boston

Why Developers Care

• Build once…(finally) run anywhere*• A clean, safe, hygienic and portable runtime environment for your app.• No worries about missing dependencies, packages and other pain points during

subsequent deployments.• Run each app in its own isolated container, so you can run various versions of

libraries and other dependencies for each app without worrying• Automate testing, integration, packaging…anything you can script • Reduce/eliminate concerns about compatibility on different platforms, either your

own or your customers. • Cheap, zero-penalty containers to deploy services? A VM without the overhead of

a VM? Instant replay and reset of image snapshots? That’s the power of Docker

• * Today—we require a modern Linux kernel and AUFS. These requirements will be relaxed significantly with the 0.8 release of Docker. See http://blog.docker.io/2013/08/getting-to-docker-1-0/

Page 13: Docker open stack boston

Why Devops Cares?

• Configure once…run anything• Make the entire lifecycle more efficient, consistent, and repeatable• Increase the quality of code produced by developers. • Eliminate inconsistencies between development, test, production, and

customer environments• Support segregation of duties• Significantly improves the speed and reliability of continuous deployment and

continuous integration systems• Because the containers are so lightweight, address significant performance,

costs, deployment, and portability issues normally associated with VMs

Page 14: Docker open stack boston

AppA

Containers vs. VMs

Hypervisor (Type 2)

Host OS

Server

GuestOS

Bins/Libs

AppA’

GuestOS

Bins/Libs

AppB

GuestOS

Bins/Libs

App A’

Docker

Host OS

Server

Bins/Libs

App ABins/Libs

App B

App B’

App B’

App B’VM

Container

Containers are isolated,but share OS and, whereappropriate, bins/libraries

GuestOS

GuestOS

…result is significantly faster deployment, much less overhead, easier migration, faster restart

Page 15: Docker open stack boston

Why are Docker containers lightweight?

Bins/Libs

AppA

Original App(No OS to takeup space, resources,or require restart)

App Δ

Bins/

AppA

Bins/Libs

AppA’

GuestOS

Bins/Libs

Modified App

Union file system allowsus to only save the diffsBetween container A and containerA’

VMsEvery app, every copy of anapp, and every slight modificationof the app requires a new virtual server

AppA

GuestOS

Bins/Libs

Copy ofApp

No OS. CanShare bins/libs

AppA

GuestOS

GuestOS

VMs Containers

Page 16: Docker open stack boston

What are the basics of the Docker system?

Source Code

Repository

DockerfileFor

A

Docker Engine

DockerContainer

Image Registry

Build

Docker

Host 2 OS (Linux)

Container A

Container B

Container C

Container A

Push

Search Pull

Run

Host 1 OS (Linux)

Page 17: Docker open stack boston

Changes and Updates

Docker Engine

DockerContainer

Image Registry

Docker Engine

Push

Update

Bins/Libs

AppA

App Δ

Bins/

Base Container

Image

Host is now running A’’

Container Mod A’’

App Δ

Bins/

Bins/Libs

AppA

Bins/

Bins/Libs

AppA’’

Host running A wants to upgrade to A’’. Requests update. Gets only diffs

Container Mod A’

Page 18: Docker open stack boston

Docker Futures

• Stabilizing: Docker 0.8 in Early October• Shrink and stabilize Core• Provide stable, pluggable API

• Expanding runtime environments:• OpenStack: announced• Docker as various flavors of VM

(AMI, vdf, etc.)-This week• Increase compatible kernel set-

October• Increase compatible Union FS

set-October• IaaS compatibility-Ongoing

• Dev/devops tools integrations: Ongoing• Chef, Puppet, Vagrant, GitHub,

Travis, Buildbot, • Various flavors of PaaS & IDE

Page 19: Docker open stack boston

OpenStack / DockerNew hypervisor to enable Nova to deploy Linux containers

Page 20: Docker open stack boston

Why Docker + OpenStack

• Alternative to VMs within OpenStack• Easier deployment of OpenStack itself• Cross cloud application deployment• At OpenStack Summit we hope to show:• Building and testing an application from source• Running on a laptop• Running it, without modification or noticeable downtime, on a

public cloud• Running it, without modification or noticeable downtime, on an

openstack cluster• Doing all of the above using Nova and Horizon

Page 21: Docker open stack boston

Why a new hypervisor?

• Nova a computing controller for OpenStack• Nova support for containers is minimal (via LibVirt)• Enables control of Docker through OpenStack projects

(ex: deploy containers via Horizon Web UI)

Page 22: Docker open stack boston

Current state of the project

• Nova blueprint accepted• Code review in progress• Testing in Devstack• Code available:• https://github.com/dotcloud/openstack-docker

Page 23: Docker open stack boston

Deploying container with Horizon

Page 24: Docker open stack boston

Access Docker Registry through Glance

Page 25: Docker open stack boston

Under the hood

• Implements Nova’s driver API (start, stop, deploy, destroy, etc…)• Using Docker HTTP Rest API to control containers

Page 26: Docker open stack boston

Want to learn more:

• www.docker.io:• Documentation• Getting started: interactive tutorial, installation instructions, getting

started guide, • About: Introductory whitepaper:http://www.docker.io/the-whole-story/

• Github: dotcloud/docker• IRC: freenode/docker/• Google groups: groups.google.com/forum/#!forum/docker-user• Twitter: follow @docker• OpenStack Code: https://github.com/dotcloud/openstack-docker

• Meetups: Scheduled for Boston, San Francisco, Austin, London, Paris, Boulder…and Nairobi. Go to website for details

Page 27: Docker open stack boston

www.docker.io