Continuous Delivery at Netflix
-
Upload
rob-spieldenner -
Category
Technology
-
view
1.501 -
download
1
description
Transcript of Continuous Delivery at Netflix
![Page 1: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/1.jpg)
Continuous Delivery at NetflixSpeed at Scale
Monday, December 23, 13
![Page 2: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/2.jpg)
Rob Spieldenner
Senior Software Engineer on Engineering Tools at Netflix
@robspieldenner -> Twitter
Monday, December 23, 13
![Page 3: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/3.jpg)
Netflix Culture
Monday, December 23, 13
![Page 4: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/4.jpg)
Culture Deckhttp://jobs.netflix.com/
Monday, December 23, 13
![Page 5: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/5.jpg)
Freedom & Responsibility
Monday, December 23, 13
![Page 6: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/6.jpg)
Context, not Control
Monday, December 23, 13
![Page 7: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/7.jpg)
Self Service
Monday, December 23, 13
![Page 8: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/8.jpg)
Developers on call for their services
Monday, December 23, 13
![Page 9: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/9.jpg)
Netflix OSShttp://netflix.github.io/
Monday, December 23, 13
![Page 10: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/10.jpg)
OODA Loop - Observe, Orient, Decide, Act
Monday, December 23, 13
![Page 11: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/11.jpg)
Commit Unit TestBuild Bake Deploy To Test
Integration & Acceptance
Tests
Red/Black Push
Continuous Delivery Pipeline
Monday, December 23, 13
![Page 12: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/12.jpg)
Commit and Code Review
Monday, December 23, 13
![Page 13: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/13.jpg)
Build
simplify build file with plugins
add metadata
calculate version
output jar/war, sources, javadoc, rpm/deb
Monday, December 23, 13
![Page 14: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/14.jpg)
Ant
build.xml<project name="example"> <property environment="env"/> <import file="${env.CBF_HOME}/library.xml"/></project>
ivy.xml
list dependencies
list publications
Monday, December 23, 13
![Page 15: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/15.jpg)
Gradle
build.gradleapply plugin: ‘nebula’publications { JAR JAVADOC SOURCES}dependencies { testCompile ‘junit:junit:4.+’}
Monday, December 23, 13
![Page 16: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/16.jpg)
Jenkins
Monday, December 23, 13
![Page 17: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/17.jpg)
Job-DSLhttps://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
Monday, December 23, 13
![Page 18: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/18.jpg)
Job DSL Example
exampledef project = 'quidryan/aws-sdk-test'def branchApi = new URL(“https://api.github.com/repos/${project}/branches”)def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader())branches.each { def branchName = it.name job { name “${project}-${branchName}”.replaceAll(‘/’, ‘-’) scm { git(“git://github.com/${project}.git”, branchName) } steps { gradle(‘test’) } }}
Monday, December 23, 13
![Page 19: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/19.jpg)
Bake
Monday, December 23, 13
![Page 20: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/20.jpg)
aminatorhttps://github.com/Netflix/aminator
Monday, December 23, 13
![Page 21: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/21.jpg)
chef for base image
Monday, December 23, 13
![Page 22: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/22.jpg)
Apply deb/rpm, snapshot
Monday, December 23, 13
![Page 23: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/23.jpg)
Unit of deployment
Monday, December 23, 13
![Page 24: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/24.jpg)
Deploy
Monday, December 23, 13
![Page 25: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/25.jpg)
Asgardhttps://github.com/Netflix/asgard
Monday, December 23, 13
![Page 26: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/26.jpg)
Monday, December 23, 13
![Page 27: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/27.jpg)
UI or REST API
Monday, December 23, 13
![Page 28: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/28.jpg)
Canary Deploys
Monday, December 23, 13
![Page 29: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/29.jpg)
Red / Black Push
Monday, December 23, 13
![Page 30: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/30.jpg)
Testing
Webdriver
Geb
Betamax
and manual
Monday, December 23, 13
![Page 31: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/31.jpg)
Monitoring
How many people are watching?
Can the new version handle load?
Errors and logging
suro, jmx
Monday, December 23, 13
![Page 32: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/32.jpg)
Alert on issues
Monday, December 23, 13
![Page 33: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/33.jpg)
Maybe automatic rollback
Monday, December 23, 13
![Page 34: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/34.jpg)
Resiliency / Anti-Fragility
Monday, December 23, 13
![Page 35: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/35.jpg)
Simian Armyhttps://github.com/Netflix/SimianArmy
Chaos Monkey
Chaos Gorilla
Chaos Kong
Latency Monkey
Janitor Monkey
Conformity Monkey
Monday, December 23, 13
![Page 36: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/36.jpg)
Assume services will break, disappear, slow
Monday, December 23, 13
![Page 37: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/37.jpg)
Books
Monday, December 23, 13
![Page 38: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/38.jpg)
Adrian Cockcroft Flowcon Keynote - Velocity and Volume (or Speed Wins) - http://www.youtube.com/watch?v=wyWI3gLpB8o
Roy Rapoport - Cloud Operations at Netflix - http://www.youtube.com/watch?v=7779Wrun5fo
AWS re:Invent - http://reinvent.awsevents.com/recap2013.html
Other Presentations
Monday, December 23, 13
![Page 39: Continuous Delivery at Netflix](https://reader031.fdocuments.us/reader031/viewer/2022020110/54b6ccf14a79596f468b4592/html5/thumbnails/39.jpg)
Netflix is Hiringhttp://jobs.netflix.com/
Monday, December 23, 13