The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

Deploy servicesreliably & consistently

• If it works locally, it will work on the server

• With exactly the same behavior

• Regardless of versions

• Regardless of distros

• Regardless of dependencies

Just like the real thing

• Work in dev environment

• Other services (databases etc.) in containers

• Whenever you want to test « for real »:

• Build in seconds

• Run instantly

Better! Faster! Stronger!

Better! Faster! Stronger!

15 Months Later: An Incredible Platform and Ecosystem

Official Repos & 14K+ Dockerized Apps

Community 460+ Contributors 250+ Meetups on Docker 2.75M Downloads 6.7K Projects on GitHub

Support Enterprise Support Robust Documentation Implementation, Integration, Training Network of Partners

The Docker Platform ! Docker Engine

Docker Hub !Build, Ship, and Run

Partners

Content

Users

55 People and a Turtle

Now up to 55 people (and our pet turtle, Gordon)

…to Build, Ship, and Run

Build Ship

Run

Dev

QA

Source

Staging

Physical

Virtual

Cloud

Infrastructure Management

Infrastructure Management

DockerFile

Source Code Repository

TESTTEST

TESTTEST

TEST

GCE RAX IBM

Mac/Win Dev Machine

Boot2Docker

Doc

ker

Analytics DB

Prod Machine

Linux OS

Doc

ker

Doc

ker

++

Users Collab

Provenance Policy

Docker Hub

Registries

Public Curated Private

Docker Hub API

Third Party Tools

Prod Machine

Linux OS

Doc

ker

Doc

ker

Prod Machine

Linux OS

Doc

ker

Doc

ker

VM

Doc

ker

Doc

ker

VM

Doc

ker

Doc

ker

VM

Doc

ker

Doc

ker

QA Machine

Linux OS

Doc

ker

Doc

ker

An Open Platform…

API

Engine Hub

open source software at the heart of the Docker platform

cloud-based platform services for distributed applications

API

An Introduction to the Docker Engine

It’s an image builder.

Database

HTTP Service HTTP Service HTTP Service

Haproxy Haproxy

Quagga Quagga

BGP / OSPF

Database

{{{

image #1:your-favorite-lb

image #2:apache, nginx, etc

image #3:mysql, zookeeper, etc

Zookeeper

HTTP Service HTTP Service HTTP Service

Haproxy Haproxy

Quagga Quagga

BGP / OSPF

Zookeeper

{{

image #1:your-favorite-lb

image #2:consensus-web-service

Zookeeper

Do it!‣ Satisfied with your local build?

‣ Push it to a registry (public or private)

‣ Run it (automatically!) in CI/CD

‣ Run it in production

‣ Happiness!

‣ Something goes wrong? Rollback painlessly!

Installing Docker on EC2#!/bin/bash -xaws ec2 run-instances \ --image-id ami-e55a648c \ --key-name mykey \ --user-data \ ”#include https://get.docker.io"

https://get.docker.io

#!/bin/bash -x aws ec2 run-instances \ --image-id ami-e55a648c \ --key-name my-key \ --user-data "#include https://get.docker.io” ip=$(aws ec2 describe-instances \ --output json \ --filter Name=instance-state-name,Values=running | python \ -c 'import json; import sys; print json.load(sys.stdin)[“Reservations”][0]["Instances"][0]["PublicIpAddress"]') ssh ubuntu@$ip sudo docker run cirros

https://get.docker.io

An ecosystem

libcontainer libchan

libswarm45Tuesday, June 10, 14

Engine + + DockerHub

libcontainer… a story of Linux namespaces

libcontainer…

• Standalone project

• Contributors:

• RedHat

• Google

• Parallels (OpenVZ)

• Ubuntu / LXC

User namespace

Security

misconceptions

• Docker is not secure

• Docker should be compared to VM security

a security product

• Docker Engine is a security product.

• It provides a wrapper around processes

• Provides a path toward attestation of arbitrary processes (Trusted Compute)

• You can use VMs to wrap containers

a work in progress…

That said…

Security was not a priority for the 1.0 release

Security is a priority post-1.0

Big issues are still be discovered at a rapid clip…

Big issues are being resolved at a rapid clip.

libchana lightweight communication

protocol for distributed systems

25Tuesday, June 10, 14

d

libchanLike Go channels over the network

Simple message passing

Synchronization without sharing state

Raw socket passing: channels as gateways to any other protocol

Nesting: channels can send channels


libchan

Available transports:SPDY/TLS,websocket,raw TCP,high-perf unix sockets (with fd passing),in-memory go channels.

Designed to be simple and portable


libswarmA minimalist toolkit

to compose network services


libswarmA standard interface to combine and organize

services in a distributed system.

Compose complex architectures from standard building blocks

Avoid vendor lock-in by swapping any service out with another

Pick services from a built-in library, or write your own with a simple API.


Shipper or Mesos or Coreos/FleetGeardor

or or orConsul Helios Centurion

EC2 Rackspace GCE Orchard Tutum


Images… and a new way of looking

at infrastructure

Images on HWis usually mutable

Hardware

Image

Linux

Chef

Installs Chef

Image'

Linux

Chef

Creates

Replaces

Runs

Ephemeral environmentsare (somewhat) immutable.

Hypervisor

Image

Linux

Chef

Runs

Image'

Linux

Chef

Chef

Runs

Configures

VM

Accesses

COW

Containers are like ephemeral VMs

Docker

Image

Linux

Chef

Runs

Image'

Linux

Chef

Chef

Runs

Configures

Container

Accesses

COW

Hypervisor

Image

Linux

Chef

Runs

Image'

Linux

Chef

Chef

Runs

Configures

VM

Accesses

COW

Hypervisor

VM Docker

Container

Chef-for-runtime

$ cat Dockerfile FROM fedora RUN yum update; \ yum -y install chef

ADD http://x/receipes.tar.gz /opt/chef"ADD solo.rb /etc/chef/solo.rb"CMD chef-solo -c /etc/chef/solo.rb ; \! apachectl start

Containersare

THINGS

Servers vs Things

Pets vs Cattle

LET US BAKEIMAGES!

Let us

BAKE

images!

Burning configuration into images.

Docker ContainerInitiates Creates

Image

Linux

Chef

Chef

Runs

Configures

Build Creates

Bakery Chef

$ cat Dockerfile FROM fedora RUN yum update; \ yum -y install chef"ADD http://x/receipes.tar.gz /opt/chef"ADD solo.rb /etc/chef/solo.rb"RUN chef-solo -c /etc/chef/solo.rb

Expanded view:Burning configuration into

images.

Docker Image tagInitiates

Image'

Linux

Chef

Chef

Build

Image

Linux

Chef

Creates

Creates

Runs Creates

References1

2

Anatomy of a Docker+Chef build & run

Docker ContainerInitiates Creates

Image

Linux

Chef

Chef

Runs

Configures

Chef

Runs

Configures

Build Creates

Stage 1 Stage 2

For All The Things!

$ cat Dockerfile FROM fedora RUN yum update; \ yum -y install chef ADD http://x/receipes.tar.gz /opt/chef"ADD solo-stage1.rb /etc/chef/solo-stage1.rb"ADD solo-stage2.rb /etc/chef/solo-stage2.rb"RUN chef-solo -c /etc/chef/solo-stage1.rb"CMD chef-solo -c /etc/chef/solo-stage2.rb; \" apachectl start

Does it converge?

$ docker build —rm . $ echo $? # pass or fail

(This is great use of Docker as an alternative to VMs for testing Chef recipes targeting non-Docker production systems)

Managing Docker at scale

Creating Containers is Easy

Managing them SUCKS

needs improvement

Management Ecosystem

Mesos

FlynnClockerClusterHQ

PaaS ecosystem

Configuration / Infrastructure Management

• Chef

• Puppet

• Salt

• Ansible

• CFEngine

• etc…

Container Inventory

• discoverd / sdutil • serf • skydock • others?

Change Management Tools

Chef# using https://github.com/bflad/chef-docker $ cat cookbooks/docker-registry/default.rb# Pull latest image docker_image 'samalba/docker-registry' !# Run container exposing ports docker_container 'samalba/docker-registry' do detach true port '5000:5000' env 'SETTINGS_FLAVOR=local' volume '/mnt/docker:/docker-storage' end $ knife ec2 server create # yada yada yada

Chef container$ knife container docker init docker -r 'recipe[apache2]' -z -b

Puppetdocker::run { 'helloworld': image => 'base', command => '/bin/sh -c "while true; do echo hello world; sleep 1; done"', ports => ['4444', '4555'], links => ['mysql:db'], use_name => true, volumes => ['/var/lib/couchdb', '/var/log'], volumes_from => '6446ea52fbc9', memory_limit => 10485760, # bytes username => 'example', hostname => 'example.com', env => ['FOO=BAR', 'FOO2=BAR2'], dns => ['8.8.8.8', '8.8.4.4'], restart_service => true, }

http://example.com

Ansible- hosts: web sudo: yes tasks: - name: ensure redis container is running docker: image=crosbymichael/redis name=redis - name: ensure redis_ambassador container is running docker: image=svendowideit/ambassador ports=6379:6379 links=redis:redis name=redis_ambassador_ansible

Orchestration

fig - local orchestration——fig.yml——web: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis redis: image: orchardup/redis

figleaf - containerized fig

Testing / dev: $ docker run --privileged -v $PWD:/opt/figapp ewindisch/figleaf

Production - image generation: $ echo “FROM ewindisch/figleaf” >> Dockerfile$ docker build -t my_img .$ docker run —privileged my_img

figleaf images may be plugged into “dumb” orchestration.

7KH�'RFNHU�SOXJLQ�IRU�+HDW%\�XVLQJ�WKH�SOXJLQ��+HDW�FDQ�WDON�GLUHFWO\�WR�'RFNHU

Orchestration for Dockerwith OpenStack Heat

DockerInc::Docker::Container

VMs

Baremetal

Heat Workflow

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

heat_template_version: 2013-05-23 description: shared volumes example resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io ftp_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: mikz/vsftpd ports: [ “21:21” ] volumes: [ “/ftp” ] name: “FTP”

apache_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: fedora/apache ports: [ “80:80” ] volumes-from: “FTP” cmd: “rm -rf /var/www; ln -s /ftp /var/www; /run-apache.sh”

OpenStack’s new container service…

nova-api

nova-compute

instance

instance

docker

containers-api

swarm-proxy

docker

keystone

neutron

User

OpenStack’s new container service…

nova-api

nova-compute

instance

instance

docker

containers-api

swarm-proxy

docker

keystone

neutron

User

swarmd

OSC%Client%Use%Case%

$%source%./openrc%

$%osc%containerIcreate%IIport%pub:222:22%IIdaemon%IIimage%<uuid>%IIcmd%“/usr/sbin/sshd%–D”%

$%osc%containerIshow%DEADBEEF%|%grep%ports%ports:%[12.34.56.78:222]%$%ssh%Ip%222%12.34.56.78%foo$%

From: https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf

https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf

Docker%Client%Use%Case%

$%source%./openrc%

$%export%DOCKER_SERVER=h:ps://…%$%docker%run%Ip%pub:222:22%Id%foo%/usr/sbin/sshd%ID%DEADBEEF%

$%osc%containerIshow%DEADBEEF%|%grep%ports%ports:%[12.34.56.78:222]%$%ssh%Ip%222%12.34.56.78%

foo$%

From: https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf

https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf

<insert here>Mesos,

Clocker,

Fleet,

Flynn,

Deis,

Kubernetes,

etc

Conclusion…

Containersare

THINGS

Docker is a valuable component in your

security story.

Q & A

@ewindisch

The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

Software

Transcript of The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration