Docker Enables DevOps - Boston

Docker Enables DevOps

Boyd E. Hemphill @behemphi @stackengine

Lets Get Acquainted

Background

BackgroundOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged), Evangelist

BackgroundOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged) Started Austin DevOps In 2012

BackgroundOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged) Started Austin DevOps In 2012 Co-Organizer of DevOps Days Austin

BackgroundOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged) Started Austin DevOps In 2012 Co-Organizer of DevOps Days Austin Became involved in Docker Austin in August.

BackgroundOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged) Started Austin DevOps In 2012 Co-Organizer of DevOps Days Austin Became involved in Docker Austin in August. Launching the First Ever Container Days in Austin - http://bit.ly/1CBppRV

Pespective

What is The Goal of Your

Company?

What is The Goal of Your

Company?Make Money!

So What is DevOps?

Is DevOps a Process?

Is DevOps an intersection of overlapping concerns?

Is it what Gene Kim says?

Is DevOps a Culture?

So What is DevOps?

DevOps is a Philosophy

So What is DevOps?

DevOps is a Philosophy All of the previous are models for implementation of that philosophy

DevOps:DevOps is the way in which a technology organization embeds itself in a business to the benefit of that business.

Profit, Revenue & Cost

Profit = Revenue - Cost


Profit = Revenue - Cost Drive Cost to $0 and you are out of business


Profit = Revenue - Cost Drive Cost to $0 and you are out of business Increasing Revenue has no theoretical cap

Tools vs. Technology

Tools are the result of implementing a DevOps model Technology enables revenue creation

Tools vs. Technology

Tools are the result of implementing a DevOps model Technology enables revenue creation Technology enables the creation of new DevOps models.

Tech v. ToolsVirtualization Configuration Mgmt Continuous Integration Continuous Delivery Service Discovery Containers

Vmware, AWS, Heroku CFEngine, Puppet, Chef, Go, Jenkins, Travis CircleCI Artifactory, Nexus Zookeeper, etcd, consul LXC, Docker, Rocket

Ideally

We do ourselves a disservice by naming technology with tools.

IdeallyWe do ourselves a disservice by naming technology with tools. We should be talking about solving a config management problem, not writing Chef code

Examples

Realistically

Good tools enable a technology to be consumed by mere mortals

RealisticallyGood tools enable a technology to be consumed by mere mortals CFEngine has been around a long time, but Puppet and Chef raised the config management conversation

RealisticallyGood tools enable a technology to be consumed by mere mortals CFEngine has been around a long time, but Puppet and Chef raised the config management conversation VMware is world class virtualization. Slice host made it affordable. AWS brought it to the masses.

RealisticallyGood tools enable a technology to be consumed by mere mortals CFEngine has been around a long time, but Puppet and Chef raised the config management conversation VMware is world class virtualization. Slice host made it affordable. AWS brought it to the masses. Twitter, Facebook, Google, Pantheon have all be using containers for some years. Docker brings containers to all phases of the SDLC

Docker - A Technology

Docker - Opportunity & Consequence

Density Factoring Build and Test System Architecture

Density

Density - Defined

The amount of idle compute on a host tends to zero

Density - Benefits

Density - Benefits

Reduces VM consumption thus reducing cost

Density - Benefits

Reduces VM consumption thus reducing cost Reduces power consumption in a physical setting

Density - Concerns

Density - Concerns

Spare capacity on hosts not there to save you during usage spikes

Density - ConcernsSpare capacity on hosts not there to save you during usage spikes YACL - Yet another complexity layer: containers on vms on hardware

Density - ConcernsSpare capacity on hosts not there to save you during usage spikes YACL - Yet another complexity layer: containers on vms on hardware Container Sprawl

Density - Business

Density - Business

Reduces VM consumption thus reducing cost

Density - Business

Reduces VM consumption thus reducing cost Doesnt impact revenue.

Density - BusinessReduces VM consumption thus reducing cost Doesnt address revenue. Probably not compelling

Density - Adoption

Density - Adoption

Purely a production concern

Density - Adoption

Purely a production concern Discussed a great deal, but implementation implications too large

Density - AdoptionPurely a production concern Discussed a great deal, but implementation implications too large Revolution, not evolution

Density - AdoptionPurely a production concern Discussed a great deal, but implementation implications too large Revolution, not evolution Tools for production management are just not there yet

Density - Tools

Density Tools Gap

Scheduling that is location aware - bin packing problem

Density Tools Gap

Scheduling that is location aware - bin packing problem Inventory management

images containers hosts

Density Tools Available

StackEngine Tutum Fleet Dies Control Center Docker Red Hat Google AWS

Factoring Distributed Applications

Factoring - Defined

Reduce your production topology to a single machine

Factoring - Defined

Reduce your production topology to a single machine Works great for most applications

Factoring - Defined

Reduce your production topology to a single machine Works great for many applications Vagrant is a killer tool

Factoring - Benefits


Vagrant multi-machine is resource hungry. Run a single VM with multiple containers


Vagrant multi-machine is resource hungry. Run a single VM with multiple containers Developer, not Ops, driven


Vagrant multi-machine is resource hungry. Run a single VM with multiple containers Developer, not Ops, driven Developers need not learn config management, only the Dockerfile

Factoring - Concerns


Impedence: How do Build, QA and Ops teams become aware of config change


Impedence: How do Build, QA and Ops teams become aware of config change Is it necessary, or just cool? (sharding)

Factoring - Business


Unclear


Unclear Could speed up development, but is only a local optima


Unclear Could speed up development, but is only a local optima Vast majority of apps factor to a single machine


Unclear Could speed up development, but is only a local optima Vast majority of apps factor to a single machine Avoid the new-cool-hotness arguments

Factoring - Adoption


By far the most common adoption path


By far the most common adoption path Typically seen in shops where Vagrant perceived as complex or heavy


By far the most common adoption path Typically seen in shops where Vagrant perceived as complex or heavy Often gains traction in Build/QA

Factoring - Tools

Factoring - Tools Gap

Only old fashion brain power to consider application modeling


Only old fashion brain power to consider application modeling Workflow management


Only old fashion brain power to consider application modeling Workflow management Containerizing existing apps

Factoring - Tools Available

Boot2Docker Fig Vagrant Docker

Build and Test Grids

Build and Test Grids - Defined

Testing a number of language versions and environments in parallel


Testing a number of language versions and environments in parallel Very important to installed software


Testing a number of language versions and environments in parallel Very important to installed software Example Testing on Centos 6.5, Ubuntu 14.04 and CoreOs, with the last three stable Docker releases

Build and Test Grids - Benefits


Containers come up fast making for shorter builds


Containers come up fast making for shorter builds Multiple containers on a build agent improves density


Containers come up fast making for shorter builds Multiple containers on a build agent improves density Makes it possible to test many more permutations of system environments


Containers come up fast making for shorter builds Multiple containers on a build agent improves density Makes it possible to test many more permutations of system environments Potential for more build parallelism

Build and Test Grids - Concerns

Build and Test Grids - ConcernsIs a container based test environment close enough to production?

Build and Test Grids - ConcernsIs a container based test environment close enough to production? Impedance: how does the app get to production if not by container image?

Build and Test Grids - Business


Increased grid density reduces costs


Increased grid density reduces costs Reducing build times increase innovation


Increased grid density reduces costs Reducing build times increase innovation Reducing build times increase development velocity


Increased grid density reduces costs Reducing build times increase innovation Reducing build times increase development velocity Increase test speed keeps QA from becoming a bottleneck to increase development velocity


Increased grid density reduces costs Reducing build times increase innovation Reducing build times increase development velocity Increase test speed keeps QA from becoming a bottleneck to increase development velocity Optimization now occurring more holistically. Revenue impact is more likely.


A Unique Perspective

Development Velocity is Revenue


A Unique Perspective

Development Velocity is Revenue Blog: Laundry Ops - http://bit.ly/1E2gMUS


A Unique PerspectiveDevelopment Velocity is Revenue Blog: Laundry Ops - stackengine.com/laundryops-practical-devops-at-home/ Now were talking disruption

Build and Test Grids - Adoption

Build and Test Grids - Adoption

Next most common adoption path

Build and Test Grids - AdoptionNext most common adoption path See as an efficient way to bring up many copies of a test environment efficiently

Build and Test Grids - AdoptionNext most common adoption path See as an efficient way to bring up many copies of a test environment efficiently Surprisingly few producing a container from the build system

Build and Test Grids - AdoptionNext most common adoption path See as an efficient way to bring up many copies of a test environment efficiently Surprisingly few producing a container from the build system The final mile

Build and Test Grids - AdoptionNext most common adoption path See as an efficient way to bring up many copies of a test environment efficiently Surprisingly few producing a container from the build system The final mile Production adoption creating impedance

Build and Test Grids - Tools

Build and Test Grid - Tools Gap

Build systems not container aware

Build and Test Grid - Tools GapBuild systems not container aware Build systems do not produce docker images

Build and Test Grid - Tools GapBuild systems not container aware Build systems do not produce docker images Build systems do not treat images as artifacts

Build and Test Grid - Tools GapBuild systems not container aware Build systems do not produce docker images Build systems do not treat images as artifacts Deployment systems are still, as a whole, immature

Build and Test Grid - Tools GapBuild systems not container aware Build systems do not produce docker images Build systems do not treat images as artifacts Deployment systems are still, as a whole, immature Private registries very immature

Build and Test Grids - Tools Available

Jenkins - plugin Bamboo Circle CI Docker Registry Quay.io

System Architecture

System Architecture - Defined

Overloaded term


Overloaded term Is concerned with how the various services of a software system interact


Overloaded term Is concerned with how the various services of a software system interact Network, Data flow, request path, job management

System Architecture - Benefits


A separation of concerns leads to a code to the interface paradigm


A separation of concerns leads to a code to the interface paradigm Micro teams micro-services can move at their own pace


A separation of concerns leads to a code to the interface paradigm Micro teams micro-services can move at their own pace Only coordination between teams is on breaking changes.

System Architecture - Concerns


Very few coders out there who get it


Very few coders out there who get it Very few models for mere mortals to reason from

System Architecture - Business


Extraordinary increase in Development Team velocity


Extraordinary increase in Development Team velocity True competitive advantage


Extraordinary increase in Development Team velocity True competitive advantage Because of difficult in adoption, advantage will be lasting

System Architecture - Adoption


Micro service architecture is very rare in the wild (unicorns)


Micro service architecture is very rare in the wild (unicorns) Investment to move existing applications is high risk


Micro service architecture is very rare in the wild (unicorns) Investment to move existing applications is high risk Most shops are not mature/agile enough to realize the benefit

System Architecture - Tools

System Architecture - Tools Gap

Meaningful materials on micro service architectures

System Architecture - Tools Gap

Meaningful materials on micro service architectures Meaningful materials on async systems

System Architecture - Tools Available

12factor.net ?

Food For Thought

Nourishment

Black box production instrumentation - Care only about the container (tools dont exist)

Nourishment

Black box production instrumentation - Care only about the container (tools dont exist) A/B Testing for Marketing

NourishmentSmother Deployment (Continuous Delivery) Black box production instrumentation - Care only about the container (tools dont exist) A/B Testing for Marketing (The Lean Startup)

Nourishment

The future was already here

Nourishment

The future was already here Pantheon/Rackspace blog: http://bit.ly/15a2UIW

NourishmentThe future was already here Pantheon/Rackspace blog: http://bit.ly/15a2UIW Note the date: October 7th, 2013

NourishmentThe future was already here Pantheon/Rackspace blog: http://bit.ly/15a2UIW Note the date: October 7th, 2013 Docker is the _tool_ that brings containers to the masses

Robots _AND_ UnicornsDocker - containers for mere mortals.

Closing Thoughts

Closing Thoughts - Business

BusinessDeveloper adoption of Docker is only valuable as a first step. There is not enough benefit from it alone to justify the effort, it must inform system architecture and production operations (over time)

BusinessDeveloper adoption of Docker is only valuable as a first step. There is not enough benefit from it alone to justify the effort, it must inform system architecture and production operations (over time) Dockers system architecture ramifications have the potential to provide a significant and lasting competitive advantage

BusinessDeveloper adoption of Docker is only valuable as a first step. There is not enough benefit from it alone to justify the effort, it must inform system architecture and production operations (over time) Dockers system architecture ramifications have the potential to provide a significant and lasting competitive advantage Unlike most ops driven improvements derived from applying DevOps thinking, this must be developer driven since its greatest benefit is derived from system architecture

BusinessDeveloper adoption of Docker is only valuable as a first step. There is not enough benefit from it alone to justify the effort, it must inform system architecture and production operations (over time) Dockers system architecture ramifications have the potential to provide a significant and lasting competitive advantage Unlike most ops driven improvements derived from applying DevOps thinking, this must be developer driven since its greatest benefit is derived from system architecture Horses are starting to become unicorns. Evolve or die.

Closing Thoughts - DevOps

DevOps

DevOps thought leaders are responsible for the holistic impact of technology decisions at the business level!

DevOpsDevOps thought leaders are responsible for the holistic impact of technology decisions at the business level! DevOps thought leaders should be working with peers and collaborators in their company to determine if they can derive the proposed business benefits

DevOpsDevOps thought leaders are responsible for the holistic impact of technology decisions at the business level! DevOps thought leaders should be working with peers and collaborators in their company to determine if they can derive the proposed business benefits Models must be developed that provide sensible direction for implementation (evolution not revolution)

DevOpsDevOps thought leaders are responsible for the holistic impact of technology decisions at the business level! DevOps thought leaders should be working with peers and collaborators in their company to determine if they can derive the proposed business benefits Models must be developed that provide sensible direction for implementation (evolution not revolution) Tools are not there yet. Companies are showing up with the mission to address this, but it is very early days.

Should you be Considering

Docker Adoption?

audible.com - Maybe the best DevOps Tool Yet?

The Lean Startup - Reis (tech) The Goal - Goldratt (Theory of Constraints) Its not Luck - Goldratt (Theory of Constraints) Good to Great - Collins (Culture) The No Asshole Rule - Sutton (Culture) The Phoenix Project - Kim (*) (tech) Continuous Deliver - Humble (*) (hardcore tech) The Lean Enterprise - Humble (*) (tech)

Boyd Hemphill @behemphi

@stackengine

Thank You for Your Time and Comments

Docker Enables DevOps - Boston

Technology

Transcript of Docker Enables DevOps - Boston