Orchestration  for  SysadminsPhu  Nguyen  Anh  – Einsights

Orchestration for Sysadmins

Andrea Luzzardi / Victor VieuxSoftware Engineers @ docker

Source: Recap from DockerCon US 2015

Agenda

• Introduction to Docker Swarm

• Swarm Fault Tolerance: Replication

• Integrating Swarm with Mesos

• Provisioning Swarm with Docker Machine

Source: Recap from DockerCon US 2015

“Running containers on multiple hosts.”

Source: Recap from DockerCon US 2015

Swarm in a nutshell

• Exposes several Docker Engines as a single virtual Engine

• Serves the standard Docker API

• Extremely easy to get started

• Batteries includes but swappable

(PhuNA edited)

Source: Recap from DockerCon US 2015

(Added by Phu)

TimelineOct

Nov

Dec

Jan

Feb

Jun

Proof of Concept

DockerCon EU Open Repository

First Release Candidate

Swarm Beta Release

Global Hack Day Open Proposal

0.2.0 Release

0.3.0 Release

Apr

Source: Recap from DockerCon US 2015

Setting up Swarm

• Create a cluster: - $ swarm create

• Add nodes to a cluster: - $ swarm join --advertise=<node_ip> token://<token>

• Start Swarm: - $ swarm manage -H <swarm_ip> token://<token>

Or you can use your own etcd, zookeeper or consul

Source: Recap from DockerCon US 2015

Swarm Scheduler

• Resource Management - Memory, CPU, Network

• Fit containers - Apply filters to exclude nodes - Use a strategy to rank and pick the best node

• Scheduling Knobs - Constraints - Affinities

$ docker run -m 1g -c 2 -p 80:80 ...

$ docker run -e constraint:storage==ssd ...

$ docker run -e affinities:container!=redis* ...

(Constraint, Affifinity, Port, Dependency, Health)

(spread, binpack, random)

(PhuNA edited)

(PhuNA edited)

Source: Recap from DockerCon US 2015

(Added by Phu)

(Added by Phu)

(Added by Phu)

“Batteries Included but Swappable”

Source: Recap from DockerCon US 2015

Docker Swarm + Mesos

• Experimental • Docker Swarm act as a Mesos Framework • In collaboration with

$> swarm manage --cluster mesos-experimental <mesos_master_url>

Source: Recap from DockerCon US 2015

Mesos Cluster

+

Mesos CLI

Mesos Slaves

+

+

+

+

+

+

+

+

+

+

+

+

+

Mesos Masters

Marathon

Source: Recap from DockerCon US 2015

Mesos Cluster + Docker Swarm

+

Mesos CLI

Mesos Slaves

+

+

+

+

+

+

+

+

+

+

+

+

+

Mesos Masters

Marathon

DockerCLI

Source: Recap from DockerCon US 2015

Mesos Cluster + Docker Swarm

+

Mesos CLI

Mesos Slaves

+

+

+

+

+

+

+

+

+

+

+

+

+

Mesos Masters

Marathon

DockerCLI

shipyard

kitematic compose

Source: Recap from DockerCon US 2015

BY

Source: Recap from DockerCon US 2015

Swarm Manager Replication

Source: Recap from DockerCon US 2015

Replication

DockerCLI

Swarm Cluster

• Swarm Manager is a Single Point of Failure

API Request

Manage

Source: Recap from DockerCon US 2015

Replication

DockerCLI

Swarm Cluster

• Upon failure it becomes impossible to manage the cluster

• Cluster continues to run un-managed

API Request

Manage

Source: Recap from DockerCon US 2015

Replica

Replication

Primary

DockerCLI

Swarm Cluster

Forward

API Request

Manage

Replica

• One primary, many replicas

• Leader Election built on top of Consul, etcd, ZooKeeper

• Replicas forward API calls to primary

• CLI can talk to any instance

Source: Recap from DockerCon US 2015

Replica

Replication

Primary

DockerCLI

Swarm Cluster

Forward

API Request

Manage

Replica

• Replicas monitor health of primary instance

• Leader gets re-elected upon primary failure

Source: Recap from DockerCon US 2015

PrimaryReplica

Replication

Replica

DockerCLI

Swarm Cluster

Forward

API Request

Manage

• One replica promoted to primary

• Other replicas notified of leadership update

• CLI can continue talking to same instance

Source: Recap from DockerCon US 2015

Demo

Source: Recap from DockerCon US 2015

“From Zero to Docker”

Source: Recap from DockerCon US 2015

Machine in a nutshell

• Host management for a container centric world

• Provisions Virtual Machines

• Installs & Manages Docker Engine

• Swarm Cluster Provisioning

Source: Recap from DockerCon US 2015

Machine Provisioning

5 Local Providers

9 Cloud Providers

7 Operating Systems

Source: Recap from DockerCon US 2015

Machine Management

• Install, Configure & Upgrade Docker Engine - docker-machine upgrade node-{1,2,3}

• PKI Management - TLS Encryption & Authentication by default - CA generation, Key signing and Re-keying - Seamless integration: Auto-configure the CLI

• Bring Your Own Machine - Generic Driver

Source: Recap from DockerCon US 2015

Machine Operations

• Operate - List, Start, Stop and Kill machines

• Debug - docker-machine ssh node-2

• Script - docker-machine toolkit: ip, url, …

Source: Recap from DockerCon US 2015

Machine Clustering

• Integration with Docker Swarm

• Provision Swarm management nodes

• Pre-configure machines to join a Swarm cluster

• Tweak Swarm options

Source: Recap from DockerCon US 2015

Demo

Source: Recap from DockerCon US 2015

Thank you, Questions?Andrea Luzzardi - @aluzzardi

Victor Vieux - @vieux

Source: Recap from DockerCon US 2015