Patrick Chanezon, @chanezon

David Chung, @dchungsf

Mindy Preston, @mindypreston

Build your own container-based system

with the Moby project

May 2017

French

Polyglot

Platforms

Software Plumber

San Francisco

Developer Relations

@chanezon

Docker

The world needs

tools of mass innovation

A programmable Internet would be the ultimate

tool of mass innovation

A commercial product,

built on

a development platform,

built on

infrastructure,

built on

standards.

Docker is building a stack to program the Internet

Docker is building a stack to program the Internet

CE

EE

enterprise edition

Ubuntu

Fedora

Mac

Azure

CentOS

Windows 10

AWS

Debian

community edition

Ubuntu

Windows Server

Azure

CentOS

Suse

Red Hat

AWS

Oracle Linux

Orchestration

Container Runtime

OS

Infrastructure Management

Container Platform Layers

Application Services

Docker is a platform made of components

Raft StoreNode

IdentitySecrets

Routing

Mesh

Overlay

Networking

Swarm Orchestration

Engine

Application Services

12,000,000,000

11,000,000,000

10,000,000,000

9,000,000,000

8,000,000,000

7,000,000,000

6,000,000,000

5,000,000,000

4,000,000,000

3,000,000,000

2,000,000,000

1,000,000,000

Notary

runC

containerd

HyperKit , VPNKit, DataKit

SwarmKit

libcontainer

libnetwork

InfraKit

2013 2014 2015 2016 2017

1M2014

PULLS

1B2015

PULLS

6B2016

PULLS

12B2017

PULLS

linuxKit

LinuxKitA toolkit for building secure, portable and lean operating systems for containers

Taking Dockermulti-platform

“I want Docker for X”

Desktop Server Cloud

I want Docker for…

Not every platform provides a Linux subsystem

Not every platform provides a Linux subsystem

Orchestration

Container Runtime

Linux Subsystem

Infrastructure Management

Application Services

The container movement needs asecure, lean, portable subsystem

The container movement needs

a secure, lean, portable Linux subsystem.

introducing

Only works with

containers

- Smaller attack

surface

- Immutable

infrastructure

- Sandboxed system

services

- Specialized patches

and configuration

Incubator for

security innovations

- Wireguard,

Landlock, KSPP

- MirageOS type

safe system

daemons

Community-first

security process

- Linux is too big

for any one

company to

secure it

- Participate in

existing Linux

security efforts

1. LinuxKit: a SECURE Linux subsystem

- Minimal size, minimal boot time

- All system services are containers

- Everything can be removed or

replaced

2. LinuxKit: a LEAN Linux subsystem

- Desktop, server, IoT, mainframe

- Intel & ARM

- Bare metal & virtualized

3. LinuxKit: a PORTABLE Linux subsystem

Docker and Microsoft collaborate to bringLinux containers to Windows

+ +

https://github.com/linuxkit/linuxkit

Get Started with LinuxKit

MobyAn open framework to assemble specialized container systems without reinventing the wheel.

Pioneers 2013 - 2014

Production Model: open-source!

Use case: cloud native apps on Linux server

Early Adopters 2015 - 2016

Production Model: OPEN COMPONENTS

Mainstream 2017 - 2018Containers are spreading to every category of computing:

server, datacenter, cloud, IoT, desktop, mobile…

Case study:

Specializing Docker for the mainstream

Desktop Server Cloud

The open component model shows its limits…

The auto industry has solved this problem: COMMON ASSEMBLIES.

Scaling the Docker production model: share components AND

ASSEMBLIES.

It’s time to take our ecosystem to the next level…

By collaborating on components AND COMMON ASSEMBLIES.

– Library of 80+ components

– Package your own

components as containers

– Reference assemblies

deployed on millions of nodes

– Create your own assemblies

or start from an existing one

A framework to assemble

specialized container

systems without

reinventing the wheel.

Docker uses Moby for its

open-source

– Thousands of contributors,

hundreds of patches/week

– Component development

– Specialized assembly

development

– Integration tests

– Architecture design

– Integration with other projects

– Experimentation and bleeding

edge features

Docker uses Moby for its

open-source...

and so can you!

– Community-run

– Open governance inspired by

the Fedora project

– Plays well with existing

projects - no donation

necessary!

Moby and Docker

What it means for you

Moby helps you

innovate without tying

you to Docker

System BuildersDocker Users

Docker will better leverage

the ecosystem to innovate

faster for you

Moby transforms multi-month R&D projects into weekend projects.

locked-down Linux with remote attestation

Weekend project #1:

Notary

custom CI/CD stack

Weekend project #2:

Notary Registry Docker Builder

+

custom CI/CD stack + Debian+ Terraform

Weekend project #3:

Notary Docker Builder

+

Registry

“RedisOS”

Weekend project #4:

"RedisOS"for Windows

"RedisOS"for Mac

"RedisOS"for bare metal

HyperKit

bare metal

Etcd clustering on Google Cloud

Weekend project #5:

SSHD

Kubernetes on the Mac

Weekend project #6:

HyperKit

Getting Started

- Blog https://mobyproject.org/blog

- Twitter @moby

- Github moby/moby

Let’s take containers mainstream!

InfraKitA toolkit for building declarative, self-healing infrastructure.

What is it?

53

• Launched at LinuxCon, Berlin in October, 2016.

• Toolkit for building declarative, self-managing

distributed applications

• Active management with active controllers

• scaling groups, rolling updates

• monitoring / health checks

• connecting nodes to L4 / ingress

• Declarative infrastructure

Architecture

CLI

API

container orchestration

Where does it fit?

55

kubectl run nginx --image=nginx

gcloud container node-pools list --zone us-

central1-f --cluster MyWorkers

aws autoscaling update-auto-scaling-group

--auto-scaling-group-name MyWorkers

docker create service nginx …

infrakit group describe workers

az vmss create --resource-group vmss-

test-1 --name MyWorkers

container orchestration

infrastructure orchestrationinfrastructure orchestration

list, err :=

group.Controller.Describe(“workers”)

App Opscontainer orchestrationApp Ops

One console across environments

56

kubectl run nginx --image=nginx docker create service nginx …

infrakit group describe workers

container orchestration

infrastructure orchestration

list, err :=

group.Controller.Describe(“workers”)

AWS RackHDAZ GCP OneVIEWMAASKVM VMW

Cloud Ops Hardware OpsCluster Ops

Configuration

Example config file (zk.conf): Group configuration = Instance + Flavor

{"Properties": {

/* raw configuration */

}}

{"groups" : {

"my_zookeeper_nodes" : {"Properties" : {

"Instance" : {"Plugin": "instance-vagrant","Properties": {

"Box": "bento/ubuntu-16.04"}

},"Flavor" : {

"Plugin": "flavor-zookeeper","Properties": {

"type": "member","IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"]

}}

}}

}}

Current Status

Support more platforms

59

• Compute:

• Bare-metal: HP OneView, MAAS, RackHD

• Public cloud: AWS, GCP

• MacOS X (HyperKit); Docker containers

• Coming soon: Azure, IBM, Digital Ocean,

Packet, libvirt

• Other resource types

• AWS - vpc, subnets, gateways, etc.

Improve usability

60

• Templates

• Complex scripts and configuration in any format;

no more escape quotes in JSON

• Fetch templates from remote repositories

• Playbooks

• CLI - flags, prompts — config driven and

dynamic

• Share “playbooks” from remote repositories

Improve core system

61

• High Availability — Swarm Mode or etcd

• New Plugin types — Metadata and Events

• Metadata: cluster-wide sysfs and reflection

• Events - publish / subscribe

• Remote client access: infrakit -H host:port to remote cluster

Road Map

Use Cases

63

• Support container orchestration

• bootstrapping + day N management

• API for cluster autoscaling

• k8s, Docker Swarm Mode

• Bare-metal + GPU provisioning

• IoT — LinuxKit integration / custom kernel

deployment

Improve usability

64

• Finalize API / Schema for 1.0

• Make it easy to consume

• Simplify setup - fewer daemons and binaries

• Embeddable / vendor API

• Sensible CLI for stable / experimental features

• Make it easy to extend / contribute

• metadata / instance plugins

• playbooks / reusable templates

• community CI / compatibility testing

• Documentation

Improve core system

65

• Provisioning of diverse resource types

• networks / proxies / load balancers

• GPU

• Stability / performance of core controllers

• Asynchronous messaging - mqtt, natsd, amqp

• Monitoring + Health check SPI

Support more platforms

66

• Direct libvirt / KVM / CUDA

• Better bare-metal / hardware ops integration

• Kernel image build pipeline — LinuxKit

Build, test, and deploy clusters from infrastructure

definitions to kernel images

Get involved

https://github.com/docker/infrakit

dockercommunity.slack.com: #infrakit

Learn More at OSCON

- Mindy Preston, Amir Chaudhry’s

“MirageOS 3: Smaller, lighter, and more transparent”

Wednesday 4:15 pm

- David Chung, Bill Farner

“InfraKit: A toolkit for infrastructure orchestration”

Thursday 11 am

THANK YOU