Docker for HPC in a Nutshell

© 2013 Cassini Consulting

Andreas Schmidt | @aschmidt75

Docker: User Friendly Application & Service Containers for HPC Environments Docker In A Nutshell

2 16.07.15 Cassini Consulting


vs. Pets Cattle

16.07.15 Cassini Consulting 6

Technical Perspective

Developer Perspective

Security

Ecosystem & Partners




Security


Docker Daemon

Docker Engine core components


Containers

Images

Containers

Docker Daemon

Docker Engine core components


Containers

Images

Containers

API

Docker Client

Docker Client

The Docker workflow & commands


Image



Registry

Image

pull



Registry

Image

Docker- file

pull

build



Registry

Image

Docker- file

push pull

build



Registry

Image

Container

Docker- file

push pull

run

build



Registry

Image

Container

Docker- file

push pull

run

commit

build



Registry

Image

Container

Docker- file

push pull

run

commit

start, stop, ...

build



Registry

Image

Container

Docker- file

✖

push pull

run

commit

rm

start, stop, ...

build



Registry

Image

Container

Docker- file

✖ ✖

push pull

run

commit

rm rmi

start, stop, ...

build



Registry

Image

Container

Docker- file

External Tarball

✖ ✖

push pull

run

commit

rm rmi

start, stop, ...

build

export

Docker Engine technology foundation

Isolation through Kernel Namespaces Linux Capabilities Resource limitation through Linux control groups Filesystem isolation, Copy-On-Write & Union FS


Docker Engine technology foundation

21 16.07.15 (*) https://www.docker.com/whatisdocker

Virtual Machines vs. Containers

22 16.07.15 (*) https://www.docker.com/whatisdocker




Security


From Development to Production: Challenges ahead!


What Docker gives to developers


Easy sandbox approach

Easy way to deliver software

Dev / Prod parity

Easy Sandbox approach

§  Create build and test environments

§  Choose Libraries and Framework Dependencies, per application

§  Deal with incompatible version mismatches (Ruby 1.9 vs. Ruby 2.1, including libs, bundler, version switchers, ...)

§  Lightweight alternative to using virtual machines


Easy Sandbox approach

A Docker Image contains its own userland libraries and binaries

§  separated from other images

§  take exactly the versions of libraries you need

§  leave out things you do not need

§  reproducible, lightweight, easily testable

§  look at it as a unit of delivery




Code Deliverables (i.e. RPM packages) are not runnable.

An installed instance (i.e. in a VM) is hard to transport.


https://twitter.com/joyent/status/565243828718678016


Dockerfile as a Contract between Development and Operations


what to base from

set environment params

prepare the image, i.e. install something, configure it

describe the interface

what to run


Dockerfile as the blueprint for reusable building blocks


what to base from

Redis is an official repository at dockerhub

Pave the way for Dev/Prod parity


§  From Applications to Systems

§  Describe not only compute, but also storage and networking.

§  Example: docker-compose




Security



Is Docker secure?

Let's ask this question more specific.

Is the docker daemon secure? Are images transported securely? Are images built in a secure fashion? Are containers as secure as virtual machines? Are application processes more or less secure when containerized?


Are containers as secure as virtual machines?

Control group-separated, chroot-like, namespaced resources, running on a shared kernel.


Application Containers Virtual Machines

Virtualized pieces of hardware, running its own kernel with process/user/network spaces separated on hypervisor level.

Are application processes more or less secure when containerized?

Definitely more secure, if "used properly(*)" Docker Container ~ Application process, ideally a single process, without management daemons Smaller attack surface Namespaced process, network, FS mounts, ...: ~ Application cannot see "the outside OS world" Reduced Linux Capabilities, can be fine tuned Additional isolation mechanisms at hand: SELinux Type Enforcement, AppArmor profiles, Libseccomp System Call Filtering

38 16.07.15 Cassini Consulting (*) http://container-solutions.com/is-docker-safe-for-production

Docker Hardening – Docker Security Benchmark


§  Extensive Guide on hardening docker hosts, images and containers, including checks

§  Automated tools are in development

§  benchmarks.cisecurity.org




Security


Tooling around Docker


Where to run Specialised Operating systems

Where to pull images from Registries (Private, On Premise, ...)

How to operate it Orchestration, Scheduling, Management, Monitoring

From Infrastructure to Applications

Platform-As-a-Service

How to build containers Config Management, Developer Tools

Technical topics Networking, Security, Storage

Competitors & Container Runtime Alternatives

LXC + LXD

CoreOS Rocket (rkt), Application Container Spec (appc)

Open Container Project (www.opencontainers.org)



Technical Perspective Developer Perspective


Technical Perspective Developer Perspective Security


Technical Perspective Developer Perspective Security Ecosystem & Partners


Technical Perspective Developer Perspective Security Ecosystem & Competitors Thank you!

Docker for HPC in a Nutshell

Technology

Transcript of Docker for HPC in a Nutshell