Introduction to Continuous Delivery (BBWorld/DevCon 2013)

Slide 1

Introduction to Continuous Delivery

Mike McGarr, Director of Learn DevOpsBlackboard, Inc.

About me

Mike McGarrDirector of Learn DevOpsBlackboard [email protected] have been at Blackboard for 8 months.Software Engineer for 12+ yearsFounder of the DC Continuous Integration, Delivery and Deployment Meetup

What we are going to learn today

Overview of Continuous DeliveryWhy everybody should careWhere to get startedSneak Peek at some tools/techniques

The Dreaded Release

Software Releases are painful

They can take up hours or days to complete

They tend to eat into our personal lives

Note: A release is defined as any change to a production system

4

Why are releases hard?

They break thingsThis is production, so we have to be carefulThey dont always workA lot changed, so we need to verify everything is ok

In-FrequentReleases

5

The family Vacation

Lets use an example

The family vacation!Once a yearMust be relaxingHas to be perfect

http://www.sun-shots.com/data/photos/824_1family_professional_beach_portrait_photography.jpg

6

Vacation disaster

Have you ever had a family vacation ruined?

If something goes wrong, the whole vacation could be ruined

7

A day at the beach

Releases should be a Non-Event

Lives near the beachGoes everydayIf something goes wrong, no worries

A day at the beach is almost a non-event

http://www.tairiretreats.com/#/surfing/4555515307

8

The last mile

Manual DeploymentsInsufficient Configuration ManagementInfrequent, Error Prone Deployments

What is continuous delivery?

Continuous delivery is a set of practices and principles aimed at, building, testing and releasing software faster and more frequently

cycle time

Cycle Time

How long would it take your organization to deploy a change [to production] that involves just one single line of code? Do you do this on a repeatable, reliable basis?Mary and Tom Poppendieck,Implementing Lean Software Development

Quality or Speed

Frequent automated deployments

http://flic.kr/p/29Ree

13

Faster = Less risk

Image: http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg

14

Always Production Ready

Applying Continuous Delivery

Principles:

Improve continuouslyReleasing software MUST be repeatable and reliableAutomate everything!If it hurts, do it more oftenKeep everything in version controlDone means releasedBuild quality in!Everybody has responsibility for the release process

Agile (Lean)

ConfigurationManagement

Continuous Integration

Testing

Deployment Pipelines

Continuous Deployment

Deployment Automation

Practices:

16

Agile (Lean)

Agile (Lean)



Testing




Culture

Process

Hacking Culture

Lean KanbanScrumXPLean StartupChange Management

http://blog.crisp.se/2008/10/19/henrikkniberg/1224451800000

20

Visualize Flow

Configuration management

Agile (Lean)



Testing




Version Control

23

Version Everything

Tool should stay out of the way

Good Branching Strategy

Dont check-in binary files

Version Everything! (except build output)

Dependencies build tools can help you here. Componentize app. Import Libraries.Configurations os, db, middleware. Environment specific. Keep separate than source.Data Schema and Data

24

Automate the Build

No IDE Builds!KISSFail Fast (but smartly)No Wikis to BuildPick an appropriate TechnologyKeep it organizedDependency Management

All use dependency Mgmt.Ant is most verbose, maven, then gradle

25

Build Once, Deploy Many

Check-in DailyCommit to Trunk (no Branching)Automate the BuildKeep the Build FastEvery Commit results in BuildTest in Clone of ProductionAutomate Deployment

Binary Repositories

26



Testing

Agile (Lean)



Testing




28

Testing is not a Phase

http://flic.kr/p/6bcg

Requirements/TestingAre two sides of same coin

29

Types of Tests

Version ----- Meeting Notes (6/15/11 12:34) -----change picture

30

Testing Pyramid

Version ----- Meeting Notes (6/15/11 12:34) -----change picture

31

Unit Testing

FastIndependentRepeatableSelf-VerifyingTimely

Fast runs in less than 1/10 secondIndependent test one thing only (single assert)Repeatable reliably repeatable (no external dependencies)Self-Verifying red or greenno human interpretationTimely test written before production code

32

Test Driven Development

http://reddevnews.com/articles/2007/11/01/testdriven-development-tdd.aspx

33

Specification by Example

CollaborationExecutable Specification Living DocumentUbiquitous Language


Agile (Lean)



Testing





Check-in DailyCommit to Trunk Automate the BuildKeep the Build FastEvery Commit results in BuildTest in Clone of ProductionAutomate Deployment

35

Check-in DailyCommit to Trunk (no Branching)Automate the BuildKeep the Build FastEvery Commit results in BuildTest in Clone of ProductionAutomate Deploymenthttp://hounddog.github.io/blog/continuous-deployment/#/

How to get started

Continuous Integration on a Dollar a Day by James Shore (http://bit.ly/13CUiVn)

Setup:Old ComputerRubber ChickenDesk BellAutomated BuildDrink the Kool-AidLaunch

How it works:Run build locallyTake Rubber ChickenGet latest from SCMCheck-in codeCheck-out on Old ComputerRun build on old computerIf it fails, revert changesRing bell if passes

CI Servers

Human ProofBuild per commitNotifications (Email alerts, RSS, etc.)Dashboards & ChartsPlugins and IntegrationsHistorical DataWeb Accessible


Agile (Lean)



Testing




Code Deployments

39

Push New Binary to ServerStop ServerArchive Old BinaryDeploy New BinaryStart Server



Version your Database

40

VersionControl

DBChange

DBChange

DB Change

BuildServer

Production

Test

Dev

.sql

HTML

DBA



Liquibase Example

Infrastructure as code

Deployment pipelines

Agile (Lean)



Testing




44


A Deployment Pipeline is an automated manifestation of your process for getting software from version control into the hands of your users.

45

Deployment Pipelines(aka Build Pipelines)

Pipelines in jenkins

http://www.centrumsystems.com.au/2011/02/build-pipeline-hudson-jenkins-plugin-1-0-0-released/

46

Continuous deployment

Agile (Lean)



Testing




48

Continuous what?

ContinuousDelivery

ContinuousDeployment

ContinuousIntegration

commit =a build

commit = shippable product

commit =production release

Wanted to clear up any confusion betweenContinuous IntegrationContinuous DeliveryContinuous Deployment

They can be seen as a spectrum of increasing- complexity, automation, frequency of deployments

49


Continuous deployment

Dark ReleasesDeploy != Software ReleaseTest in ProductionFeature FlagsBranch by AbstractionCanary ReleasesA/B Testing

http://www.slideshare.net/mikebrittain/mbrittain-continuous-deploymentalm3public

50

51

Metrics and alerting

https://github.com/monitoringsucks

Thank you!

Mike McGarrDirector of Learn DevOpsBlackboard [email protected]://earlyandoften.wordpress.com@jmichaelmcgarr

53

Further Reading

Continuous Delivery: Reliable Software Releases through Build, Test and Deployment Automation, by Jez Humble and David Farley - http://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912Continuous Integration, By Paul M Duvall - http://www.amazon.com/Continuous-Integration-Improving-Software-Reducing/dp/0321336380 Refactoring Databases, By Scott Ambler and Pramod Sadalage - http://www.amazon.com/Refactoring-Databases-Evolutionary-paperback-Addison-Wesley/dp/0321774515 Refactoring, By Martin Fowler - http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672 Growing Object-Oriented Software, Guided by Tests, by Steven Freeman and Nat Pryce - http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627

Version

54

Further Reading

Extreme Programming Explained, by Kent Beck - http://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0201616416 Release It!, by Michael Nygard - http://pragprog.com/book/mnee/release-it xUnit Test Patterns, by Gerard Meszaros - http://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054Working Effectively with Legacy Code, by Michael Feathers - http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 Agile Testing, by Lisa Crispin and Janet Gregory - http://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468

Version

55

Further Reading

Test Driven Development (TDD) http://en.wikipedia.org/wiki/Test-driven_developmentIntroducing BDD, by Dan North http://dannorth.net/introducing-bdd/Agile Manifesto http://agilemanifesto.org/Scrum http://www.scrumalliance.org/learn_about_scrumContinuous Integration, by Martin Fowler http://martinfowler.com/articles/continuousIntegration.htmlSpecification by Example, by Gojko Adzic - http://specificationbyexample.com/Build Pipelines - http://www.magpiebrain.com/2009/12/13/a-brief-and-incomplete-history-of-build-pipelines/Implementing Lean Software Development, by Mary and Tom Poppendieck - http://www.amazon.com/Implementing-Lean-Software-Development-Concept/dp/0321437381

Version

56

Further Reading

Maven Releases on Steriods, by Axel Fontaine http://www.axelfontaine.com/2011/01/maven-releases-on-steroids-adios.htmlWhat is in a Name? Usually a version number, actually., by James Betteley - http://jamesbetteley.wordpress.com/2011/07/07/what-is-in-a-name-usually-a-version-number-actually/Build Once, Deploy Many - http://earlyandoften.wordpress.com/2010/09/09/build-once-deploy-many/Evolutionary Design - http://martinfowler.com/articles/designDead.htmlContinuous Deployment - http://timothyfitz.wordpress.com/2009/02/08/continuous-deployment/ Sonars Technical Debt Calculation - http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/Gherkin - https://github.com/cucumber/cucumber/wiki/Gherkin

Version

57

Further Reading

Branch by Abstraction, by Paul Hammant http://paulhammant.com/blog/branch_by_abstraction.html Making Large Scale Changes Incrementally with Branch by Abstraction, by Jez Humble - http://continuousdelivery.com/2011/05/make-large-scale-changes-incrementally-with-branch-by-abstraction/ What is DevOps?, by Damon Edwards - http://dev2ops.org/2010/02/what-is-devops/ Monitoring Sucks - https://github.com/monitoringsucks

Version

58

Tools

Git - http://git-scm.com/Subversion - http://subversion.tigris.org/Mercurial - http://mercurial.selenic.com/Rational ClearCase - http://www-01.ibm.com/software/awdtools/clearcase/Serena Dimensions CM - http://www.serena.com/products/dimensions-cm/index.htmlAnt - http://ant.apache.org/Ivy - http://ant.apache.org/ivy/Maven - http://maven.apache.org/Gradle - http://gradle.org/JUnit http://www.junit.org/Mockito http://code.google.com/p/mockito/Hamcrest http://code.google.com/p/hamcrest/Spock http://code.google.com/p/spock/dbUnit http://www.dbunit.org/Unitils http://unitils.org/summary.html

Version

59

Tools

Findbugs http://findbugs.sourceforge.net/PMD http://pmd.sourceforge.net/Checkstyle http://checkstyle.sourceforge.net/JIRA http://www.atlassian.com/software/jira/overviewGitHub https://github.com/Jenkins - http://jenkins-ci.org/TeamCity http://www.jetbrains.com/teamcity/Nexus http://www.sonatype.org/nexus/Artifactory http://www.jfrog.com/products.phpSonar http://www.sonarsource.org/FitNesse http://fitnesse.org/ Concordion http://www.concordion.org/Cucumber http://cukes.info/easyb http://www.easyb.org/jBehave - http://jbehave.org/geb - http://www.gebish.org/

Version

60

Tools

Liquibase www.liquibase.org/ Flyway http://code.google.com/p/flyway/Escape http://code.google.com/p/escservesconfig/Capistrano - https://github.com/capistrano/capistrano Deployinator - https://github.com/etsy/deployinator Puppet http://puppetlabs.com/Chef http://www.opscode.com/chef/Ansible - http://ansible.cc/ Vagrant http://vagrantup.com/JMeter http://jmeter.apache.org/Nagios - http://www.nagios.org/Logstash - http://logstash.net/ StatsD https://github.com/etsy/statsd Metrics http://metrics.codahale.com/ Ganglia http://ganglia.sourceforge.net/ Graphite - http://graphite.wikidot.com/

Version

61

Testing Pyramids

http://fabiopereira.me/blog/2012/03/05/testing-pyramid-a-case-study/http://www.takadevelop.com/2013/02/09/testing-pyramid-for-agile-development/http://blogs.agilefaqs.com/2011/02/01/inverting-the-testing-pyramid/http://martinfowler.com/bliki/TestPyramid.htmlhttp://mifos.org/contributors/qa-testing/future-quality-improvements-mifoshttp://www.ibm.com/developerworks/java/library/j-aopwork11/

Version

Introduction to Continuous Delivery (BBWorld/DevCon 2013)

Technology

Transcript of Introduction to Continuous Delivery (BBWorld/DevCon 2013)