CoreOS Intro

CoreOS

@superstructor / joukou.com

Commanding the Fleet

Warning: Live demos

1st Solution: Ubuntu Server

Internet is full of

servers running years-

old software with

dozens of

vulnerabilities

(Shellshock,

Heartbleed etc)

2nd Solution: Ubuntu + Docker

● Isolation of services

● Rapid service

deployment

● Portable

● Version control

● Reuse

● Lightweight & fast

Docker = Standardized Deployment of Services

CoreOS Cluster Dev Environment

or

git clone https://github.com/coreos/coreos-vagrant.git

CoreUpdate

● Make upgrading the default, seamless optiono Regular

o Reliable

o Automatic

● The “Google Chrome” of server operating

systems

CoreUpdate: How does it work ?

● Omaha protcol (check-in/retrieval)o Simple XML-over-HTTP protocol developed by

Google to facilitate polling and pulling updates from

a server

● Active/passive read-only rootfs partitionso One for running live system, one for updates

CoreUpdate: Active/passive rootfs

CoreUpdate: Active/passive rootfs /usr

● /usr is a single image containing most of the

OSo mounted read-only

o rootfs / is mounted read-write on top (persistent

data)

o parts of /etc dynamically generated at boot

o a lot of work moving default config from /etc to /usr

Atomic Updates

● Entire OS is a single read-only imageo e.g. “touch /usr/bin/foo”

● Easy to verify cryptographically

● No chance of inconsistencies due to partial

upgrades

Cloud Config

Automate customizing a CoreOS machine

through user-data.

● YAML

● configure etcd, fleet...

● e.g. EC2 user-data

Example Cloud Config

Etcd

● key-value store

● basis foro shared configuration

o service discovery

o distributed locking

● REST API

● Raft algorithm: raftconsensus.github.io

systemd is great!

● Automatically handles:o Process daemonization

o Service inter-dependencies

o Resource isolation/containment (cgroups)

e.g. MemoryLimit=512M

o Health-checking, restarting failed services

o Logging (journal)

applications can just write to stdout, systemd

adds metadata

o Timers, socket activation...

systemd (and docker) are… not so great

● Problem: poor integration with Dockero Docker is de-facto standard

o Docker and systemd do not always place nice

together

o Both Docker and systemd manage cgroups and

processes, so when the two are trying to manage

the same thing the results are mixed

Example: sending signals to a container

Given a simple container...

[Service]

ExecStart=/usr/bin/docker run busybox /bin/bash -c “while true; do echo Hello

World; sleep 1; done”

● Try to kill it with systemctl kill hello.service

● … Nothing happens

● Kill command sends SIGTERM, but bash in a Docker container has PID1,

which happily ignores the signal...

Example: sending signals to a container

● OK, SIGTERM didn’t work, so escalate to SIGKILL:

systemctl kill -s SIGKILL hello.service

● Now the systemd service is gone:

hello.service main process exited, code=killed, status=9/KILL

● But… the Docker container still exists ?

# docker ps

CONTAINER ID COMMAND STATUS NAMES

feb755848a9a /bin/sh -c ‘whi Up 31 seconds hello

# ps -ef|grep ‘[d]ocker run’

root 24231 1 0 03:49 ? 00:00:00 /usr/bin/docker run -name hello….

Why ?

● Docker client does not run containers itself; it

just sends a command to the Docker

daemon

● systemd expects processes to fork directly

● Since Docker daemon’s cgroup is entirely

seperate systemd cannot keep track of the

forked container

systemd-cgls

Solution: systemd-docker

● systemd-docker - small app that moves

cgroups of Docker containers back under

systemd’s cgroup

● Uses Docker for images, but systemd-

nspawn for runtime

● (proposed) Docker standalone mode: client

starts container directly rather than via

daemon

Fleet: systemd for the cluster

Fleet Overview

Fleet + systemd

● systemd takes care of things so fleet doesn’t

have to

● fleet configuration is just systemd unit files

● fleet extends systemd to the cluster-level,

and adds some features of its own (using [X-

Fleet])o Template units (run n identical copies of a unit)

Fleet Scheduling

● Global (run a unit everywhere in the cluster)

● MachineMetadata (run only on certain

machines)

● Conflicts (prevent a unit from being

collocated)

● MachineOf (run only on a machine that hosts

a specific unit)

Fleet + etcd

● fleet needs a consistent view of the cluster to

make scheduling decisions: etcd provides

this viewo What units exist in the cluster ?

o What machines exist in the cluster ?

o What are their current states ?

● All unit files, unit state, machine state and

scheduling information is stored in etcd

Fleet Reconciler Model

In a loop, run periodically until stopped:

1. Retrieve current state (how the world is) and

desired state (how the world should be) from

etcd

2. Determine necessary actions to transform

current state -> desired state

Fleet Unit & fleetctl

Examples

Service Annoucement & Discovery

+ SkyDNS v2

Flannel

Intra-machine Docker overlay network backed

by etcd.

● one subnet per machine

● get rid of Docker port mapping

● containers talk to each other directly

● pluggable transport backends:o encapsulate packets in UDP

o VXLAN implementation coming

Questions ?

● Everything is open source under Apache 2.0 - join us!

○ https://github.com/joukou

○ https://github.com/coreos

● Any more questions, feel free to

○ email [email protected]

○ or tweet @superstructor

CoreOS Intro

Technology

Transcript of CoreOS Intro