State-of-the-art

Container Management Platform

08 / 2016

About me

● Name: Chu Duc Minh - Age: 32

● Cloud Chief Architect @ VCCorp

● Head of Cloud Solutions department @ VCCloud

● Expertise: SDN, Storage, OpenStack, Kubernetes, Hashicorp tools

● Passion: Distributed System, Cloud &

Infrastructure Technologies

Ok, now we have containers...

Isolation: Keep services from interfering with each other

Scheduling: Where should my service be run?

Lifecycle: Keep my service running

Discovery: Where is my job now?

Constituency: Which (containers) is part of my service?

Scale-up/down: Making my services bigger or smaller

Auth{n,z}: Who can do things to my service?

Monitoring: What’s happening with my service?

Health: How is my service feeling? (well or sick?)

Google and Containers

Kubernetes (short-name: k8s)

● Project was started by Google in 2014.

● Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts.

● Provides container grouping, load-balancing, auto-healing, scaling.

● Can run on anywhere: Public Cloud (AWS, GCE,..), Private Cloud (OpenStack), Baremetal, v.v…

● Extensible: Modular & Pluggable & Hookable architecture

Community

Design Overview

“ Kubernetes is primarily targeted at applications composed of multiple containers, such as elastic, distributed micro-services.

It is also designed to facilitate migration of non-containerized application stacks to Kubernetes.

…[Kubernetes] provides ways for containers to find and communicate with each other in relatively familiar ways. ”

https://github.com/kubernetes/kubernetes/tree/master/docs/design

How does it work?

Kubernetes Architecture

Key concepts

● Pod - A group of co-living containers

● Labels - For identifying pods

● Replication Controller - Manages replication of pods

● Service - A logical set of pods and way to expose them

● Namespaces - Way to seperate environments, projects,

applications,…

● Service Discovery - By cluster-DNS

Pod

Small group of containers & volumes

Tightly coupled: same node

The atom of cluster scheduling & placement!

Shared network namespace: share IP address & localhost

Example: Pod 1: data puller & web server

Pod 2: web server & log shipper

Pod

Pod

Pod networking

Pod IPs are routable

Docker default is private IP

Pods can reach each other

without NAT

even across nodes

No brokering of port numbers

This is a fundamental requirement several SDN solutions

Labels

➢ Label is simple key/value pair

➢ Attached to any API object

➢ Generally used for represent identity

➢ Queryable by selectors: think SQL ‘select ... where ...’

➢ The only grouping mechanism of K8s: pods under a ReplicationController

pods in a Service

capabilities of a node (constraints)

Replication Controller

Replication Controller

- Desired = 4

- Current = 4

Replication Controller

Replication Controller

- Desired = 4

- Current = 4

Replication Controller

Replication Controller

- Desired = 4

- Current = 3

Replication Controller

Replication Controller

- Desired = 4

- Current = 4

Replication Controller

Service

A group of pods that act as one == Service group == selector

Gets a stable virtual IP and port called the service portal

also a DNS name

VIP is captured by kube-proxy watches the service constituency

updates when backends change

● By teams

● By projects

● By operators

● By environments

○ Dev

○ Test

○ Staging

○ Production

Server-side Service Discovery

Server-side Service Discovery

Rolling-update

...is a deployment pattern.

Update microservices to new version gracefully!

Also rollback too.

Deployment

More deploy patterns is supported!

Blue/Green

Canary

with many customizable options.

Ref: http://kubernetes.io/docs/user-guide/deployments/

Secrets Management

“Secrets” like username/password, API key, SSL certificate, v.v..

Secrets only stored in Kubernetes and only used by

allowed services.

[Encrypted-] secrets no more stored in gitRepo, sysadmin’s laptop,

volume on a storage, v.v…

→More secure!

Pod can access to allowed secrets via:

Files (in pod’s mounted-volume)

ENV vars

Secrets Management

Secrets Management

Monitoring

Metrics

Log

Pluggability

You can choose [almost] any technology you want!

networking (Flannel, Calico, OpenContrail, Weave, Romana, v.v..)

storage (NFS, GlusterFS, amazonEBS, gcePersistentDisk, RBD, v.v..)

container (Docker, rkt, HyperContainer)

And any cloud-provider you want!

AWS / GCE / Azure

OpenStack / CloudStack

And extend K8s’s scheduler via multi mechanisms

& over multi dimensions.

Integrate with OpenStack (/AWS/GCE/Azure)

Auto-provision and configure:

❖ Load-balancer (LBaaS @ OpenStack)

❖ Volume (Cinder @ OpenStack)

❖ etc

…magically!

Minikube

➢ Minikube starts a single node kubernetes cluster locally for purposes

of development and testing.

➢ Packages and configures a Linux VM, Docker and all Kubernetes

components, optimized for local development.

➢ Supports:

○ DNS

○ NodePorts

○ ConfigMaps and Secrets

○ Dashboards

➢ Does not support cloud-provider functionality

(LoadBalancers, PersistentVolumes, Ingress)

Conclusion

Kubernetes is a Toolkit for running distributed systems in production!

Co-locating helper processes Naming and discovery

Mounting storage systems Load balancing

Distributing secrets Rolling updates

Application health-checking Resource monitoring

Replicating application instances Log access and ingestion

Horizontal auto-scaling Support for introspection

and debugging

Q & A

This page is intentionally left blank.

One last thing...

Kubernetes-based PaaS

References

https://www.youtube.com/watch?v=06InE1XTlIo

http://www.slideshare.net/brendandburns/defrag-2014-41815642

https://www.youtube.com/watch?v=qCxYjq7EBHc

http://www.slideshare.net/imesh/an-introduction-to-kubernetes

http://www.slideshare.net/KasperNissen1/google-cloud-platform-and-kubernetes

https://www.nginx.com/blog/service-discovery-in-a-microservices-architecture/

http://martinfowler.com/articles/microservices.html

https://www.youtube.com/watch?v=DGlQgNmobuc