Practical Methods for Adopting DevOps - Michael Stahnke
-
Upload
puppet-labs -
Category
Technology
-
view
1.731 -
download
4
Transcript of Practical Methods for Adopting DevOps - Michael Stahnke
![Page 1: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/1.jpg)
Practical Methods
for Adopting
DevOps
Michael Stahnke, Director of Engineering Services
@stahnma
![Page 2: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/2.jpg)
• Introduction
• DevOps Overview
• When It’s Bad
• Making It Better – 5 Methods
• Questions
Next Week & PuppetConf Overview
![Page 3: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/3.jpg)
Next Week & PuppetConfIntroduction
• Server architecture
• Infrastructure
automation
• Release
management
![Page 4: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/4.jpg)
It’s impossible to exceed expectations.
![Page 5: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/5.jpg)
Nobody ever got a high-five for a server
staying up.
![Page 6: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/6.jpg)
Next Week & PuppetConfDevOps Culture
• Lots of press, talks,
hype
• Echo chamber
• Does it apply to all of
us?
• 10 deploys/day
• Rockstar ninja pirates
• Write your own
tools
• Awesome app
developers
• Startups
![Page 7: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/7.jpg)
![Page 8: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/8.jpg)
Next Week & PuppetConfSetting the Stage
• Large company size with competing
priorities
• IT teams that don’t sit in the same
building
• Average time in role: 2-3 years
• No dev … “application owners”
• Everybody really “nice”
![Page 9: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/9.jpg)
Next Week & PuppetConfSetting the Stage
• Call the vendor for problems
• Customize for us, our needs are
unique
• Success means everything stays the
same
• Just looking for approvals
• No time to debate
![Page 10: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/10.jpg)
Next Week & PuppetConfSetting the Stage
• Enterprise-y: Tivoli, BMC, CA, IBM
• Apps: Oracle RAC, Oracle EBS,
Websphere AS, MQ Series, Weblogic,
Legacy stuff
• HP-UX, Solaris, AIX, RHEL, VMware
Tool
s
![Page 11: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/11.jpg)
Next Week & PuppetConfInfrastructure Team
• Small team – 4 of you quit; that’s 50%
of the team
• Information is tribal – What happens
when the tribe leaves?
• 16 hour days for some admins
• Server team blamed for EVERYTHING
This Silo? What Silo?
![Page 12: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/12.jpg)
Next Week & PuppetConfInfrastructure Team
• No transfers
• Windows team doesn’t work with Unix
team, who doesn’t talk to Storage team
• Exiled to different buildings
This Silo? What Silo?
![Page 13: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/13.jpg)
Next Week & PuppetConfIs it all bad?
• Industry experience
• Be the best sysadmin
• Open Source tooling
• LDAP
Motivations
![Page 14: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/14.jpg)
![Page 15: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/15.jpg)
Next Week & PuppetConfDatabases Rule the World
• Variance
• 25 db servers
• 24 unique configurations
• At least 6 cluster pairs
![Page 16: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/16.jpg)
![Page 17: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/17.jpg)
![Page 18: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/18.jpg)
Reduce the standard deviation value. Then
raise the mean.
![Page 19: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/19.jpg)
Next Week & PuppetConf Method 1
Reduce variability.
![Page 20: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/20.jpg)
Next Week & PuppetConf Method 1
Standardize & automate what’s variable
or inconsistent to reduce
deviation.
![Page 21: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/21.jpg)
Do you want systems to have planned or
unplanned downtime? Because you’re going
to get one or the other.
![Page 22: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/22.jpg)
Next Week & PuppetConf Results
• Failing in known ways
• Better uptime
• Fewer configurations
• Lots of tickets on disk management
![Page 23: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/23.jpg)
Collaboration?
![Page 24: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/24.jpg)
Next Week & PuppetConf Collaboration
• Sudo for disk mgmt
• Shared responsibility for outages
• Proper classification (production, test,
dev)
• If I’m awake, you’re awake
![Page 25: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/25.jpg)
Next Week & PuppetConfCollaboration
• Share the pain
• People who carry the pager will make
better design decisions when not on-
call
![Page 26: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/26.jpg)
Next Week & PuppetConf Method 2
Stop. Collaborate. Listen. (Break down
silos)
![Page 27: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/27.jpg)
Next Week & PuppetConf Method 2
Integrate your Ops engineers early in the
ALC and collaborate on interfaces &
handoffs
![Page 28: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/28.jpg)
![Page 29: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/29.jpg)
Next Week & PuppetConfRoot Cause Analysis Meetings
• What’s the root cause of the outage?
• What’s the exact impact?
• What do we do differently next time?
![Page 30: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/30.jpg)
![Page 31: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/31.jpg)
Next Week & PuppetConf Method 3
Shout your failures. (Honesty builds
credibility)
![Page 32: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/32.jpg)
Next Week & PuppetConf Method 3
Listen and learn about failures so they
aren’t repeated.
![Page 33: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/33.jpg)
Next Week & PuppetConfRoot Cause Analysis Meetings
• Shout out your failure
• Hand out a written report & timeline of
what happened
• Remediation plan
• Meeting lasts 10 minutes
![Page 34: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/34.jpg)
Next Week & PuppetConf Setbacks
• Everything isn’t better overnight
• Time passes, new issues
• Lots of failures
• Config Mgmt messes
Don’t expect perfection
![Page 35: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/35.jpg)
Next Week & PuppetConfExperiments
• What can you try while you’re fixing
process?
• Scrum teams
• SDLC
• Inventory service
• Puppet
![Page 36: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/36.jpg)
Next Week & PuppetConf Method 4
Experimentation matters. (Failures are
still valid data)
![Page 37: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/37.jpg)
Next Week & PuppetConf Method 4
This means Ops engineers need
dev/test environments for monitoring &
tooling
![Page 38: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/38.jpg)
Next Week & PuppetConfPeriodic Review
• Find biggest pain points – optimize
• Improving anything that isn’t the
biggest bottleneck in a process is a
wasted effort!
• Drive value up the stack
• Find metrics people understand
• (Use Puppet)
![Page 39: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/39.jpg)
The cost of retiring any technology is much
higher than introducing it.
![Page 40: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/40.jpg)
Next Week & PuppetConf Learning
• Python, Ruby, Puppet, git, REST,
APIs, Web Services, Go, Containers
• Retrospectives
• Daily standups
• Lead by example
![Page 41: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/41.jpg)
Are we satisfied with how this works, or are
we trying to make it better?
![Page 42: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/42.jpg)
Next Week & PuppetConfImprovements
• Influence other teams
• Automation help
• Monitoring
• Design
• Requests to transfer to your team
![Page 43: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/43.jpg)
Next Week & PuppetConf Method 5
Solve causes not symptoms.
(Continuous improvement)
![Page 44: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/44.jpg)
Next Week & PuppetConf Method 5
Focus Ops Engineers on proactive fixes
to root causes, not fighting the
symptoms
![Page 45: Practical Methods for Adopting DevOps - Michael Stahnke](https://reader034.fdocuments.us/reader034/viewer/2022052602/55a9ad641a28ab81738b463e/html5/thumbnails/45.jpg)
• Reduce variability.
• Stop. Collaborate. Listen.
• Shout your failures.
• Experimentation matters.
• Solve causes not symptoms.
Next Week & PuppetConf Recap