50 production deployments a day, at least

www.techarchday.fiwww.techarchday.fi

50 production deployments a day, at least

Oscar RenaliasSenior Technology Architect

v10

@oscarrenaliasgithub.com/oscarrenaliasgithub.com/[email protected]/oscarrenaliaswww.slideshare.net/oscarrenalias

2009 2014 2015

50+ deploys per day10 deploys per day

Deploys to production every

11.6 seconds

Requirements

Design &Build

QA &Test

Operate

QA &Test

Operate Best Guess

Design &Build

Design &Build

QA &Test

Operate Best Guess

Feedback

Experiments

QA &Test

Operate Best Guess

Design &Build

1 YEAR6 MONTHS1 MONTH2 WEEKS1 WEEKS1 DAY1 HOURMINUTES

Run over200 concurrent

experiments on any given day

http://www.exp-platform.com/Pages/ControlledExperimentsAtLargeScale.aspx

THIS MANY EXPERIMENTS

Our objective: 50 deployments a day with no service impact

Requirements

Design &Build

QA &Test

Operate

ContinuousImprovement

SAY CONTINUOUS IMPROVEMENT

ONE MORE TIME

Continuous Delivery

AutomationArchitecture

Culture

Value Stream

Idea Value

Release

Ideas Values

Value Released

Time

Guessed Well

Release

Ideas Values

Value Released

Time

Guessed Well

Guessed Badly

Value Released

Time

Value Stream

Idea Value

Value Stream

Idea Value

Value Stream

Idea Value

Rapid Feedback

Optimise

Release

Ideas Value

“Hotfixes”

"Release""Release"

"Release" "Release"

"Release"

"Release""Release""Release"

"Release"

"Release"merg

e

merge mer

ge

mergemerge

merge

merge

mergemerge

Ideas Value

OverheadRelease

Requirements

QA &Test

Operate

Design &Build

ContinuousIntegration

CI – what good looks like…

Changes constantly pushed to version control

Fixing a broken build is always the priority

Nobody goes home on a broken build

Requirements

29

Design &Build

Operate

`ContinuousTest

QA &Test

Pipeline! Pipeline! Pipeline!

Build Static Analysis ST Regression



PerformanceTest Security Test






Effort and schedule

Test Phase

Comprehensive

Fast






Effort and schedule

Test Phase

Pre-commit1 min

Component1 hour

Acceptance4-8 hours

Commit10 mins






Effort and schedule

Test Phase

Stop the assembly

line!

What about tools?

THEY WANTED MORE AGILITY,SO WE SOLD THEM $2M IN DEVOPS TOOLS

Continuous Delivery PrinciplesEach check-in is a candidate production release

The same processes and tools in ALL environments

A failure at any stage stops the production line

It’s not about the tools!

Continuous Delivery


Culture

I CAN HAZ CONTINUOUS DELIVERY?

Fail-fast

Microservices

Metrics and Logs

Independently deployable and

scalable components

Failure is unavoidable. Deal with it.

Proactive monitoring

Continuous Delivery


Culture

RELEASE TIME!

Oscar’s unscientific observations of release failures

Human failures

ELIMINATE ALL HUMANS

Pets vs Cattle

Build Static Analysis ST Regression PerformanceTest Security Test Deployment

Test & QA

Automation

Infrastructure as codeTest, test, test

Eliminate the human factor

Continuous Delivery


Culture

WORKED FINE IN DEV

OPS PROBLEM NOW

Smooth Collaboration Fully Embedded Infrastructure-as-a-Service

DevOps as a Service Separate DevOps Team

DevOps Culture: Which one is right for you?

Source: http://www.devopstopologies.com

FIRED YOUR EXISTING TEAM ANDREPLACED THEM WITH A DEVOPS

TOOL?

THAT MUST BE GOING WELL

Organisation and Culture

Align to business priorities

Quality is everybody’s responsibilityDefine your own DevOps Culture

People over processes and tools

http://bit.ly/AccentureDevOpsPlatform

50 production deployments a day, at least

Technology

Transcript of 50 production deployments a day, at least