Leveraging Cloud Transformation to Build a DevOps Culture | AWS Public Sector Summit 2016
-
Upload
amazon-web-services -
Category
Technology
-
view
295 -
download
1
Transcript of Leveraging Cloud Transformation to Build a DevOps Culture | AWS Public Sector Summit 2016
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Emil Lerch, Sr Consultant, AWS Professional Services
J.R. Storment, Chief Customer Officer, Cloudability
June 20, 2016
Leveraging Cloud Transformation to Build a DevOps Culture
The term “DevOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between development and IT operations, resulting in the fast flow of planned work (i.e., high deploy rates), while simultaneously increasing the reliability, stability, resilience, and security of the production environment.
—Gene Kim, author of The Phoenix Project
What is DevOps
Through Security as Code, we have and will learn that there is simply a better way for security practitioners, like us, to operate and contribute value with less friction. We know we must adapt our ways quickly and foster innovation to ensure data security and privacy issues are not left behind because we were too slow to change.
—DevSecOps Manifesto
DevSecOps
Why does DevOps matter?
• High-performing IT organizations deploy 30x more frequently with 200x shorter lead times; they have 60x fewer failures and recover 168x faster.
• Lean management and continuous delivery practices create the conditions for delivering value faster, sustainably.
• High performance is achievable whether your apps are greenfield, brownfield, or legacy.
(source: puppet labs 2015 State of Devops Report)https://puppetlabs.com/sites/default/files/2015-state-of-devops-report.pdf
How do we transition to DevSecOps culture?
People/Process TechnicalReorganization: cross-discipline team Continuous integrationReorganization by vTeams Continuous deliveryDocumented release process Continuous deploymentDocumented testing processes Automated testingCross-discipline training Automated monitoring and log analysisCross-discipline social events Configuration managementRotation programs
Conway’s Law:Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
Melvyn Conway, 1967http://www.melconway.com/Home/Conways_Law.html
Inverse Conway Maneuver:In what could be termed an “inverse Conway maneuver,” you may want to begin by breaking down silos that constrain the team’s ability to collaborate effectively.
Jonny Leroy/Matt Simons, 2010http://jonnyleroy.com/2011/02/03/dealing-with-creaky-legacy-platforms/
Two-pizza teams
Full ownership
Full accountability
Aligned incentives
DevSecOps maturity model
Commit Accept Capacity Exploratory Production
Commit Accept Capacity Exploratory Production
Commit Accept Capacity Exploratory Production
Commit Accept Capacity Exploratory Production
Commit Accept Capacity Exploratory Production
DevSecOps maturity Deployment pipelines
Leve
l 5Le
vel 3
Leve
l 4Le
vel 2
Leve
l 1 Revision Control SystemConvergence (Configuration Management) System
Infrastructure Provisioning System
Artifact Management System
Build & Continuous Integration System
Feedback System
Strategies for migration from level 1–level 5
• Greenfield: Start full pipeline on pilot projects• Roll processes/tools to all new projects once verified
• Brownfield: Gradually apply DevSecOps principles• Large organizations usually implement a combination
• Pilot project/center of excellence• ”Back port” lessons onto existing code base
Sample strategy: existing applications
1. Setup CI/CD server2. Development automates builds3. Development/Operations automate deployments4. QA automates tests5. Operations automate infrastructure build/teardown
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
3. BUILD
DEVELOPER
4. DEPLOY TO TEST
5. DOCUMENT DEPLOYMENT
OPERATIONS
7. DEPLOY TO PROD
TEST SERVER PRODUCTION SERVER
QA
6. TEST
8. TEST
SOURCE CODEREPOSITORY
SOURCE CODEREPOSITORY
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
4. BUILD
DEVELOPER
5. DEPLOY TO TEST
6. DOCUMENT DEPLOYMENT
OPERATIONS
8. DEPLOY TO PROD
CONTINUOUS INTEGRATION SERVER
3. CHANGENOTIFICATION
TEST SERVER PRODUCTION SERVER
QA
7. TEST
9. TEST
SOURCE CODEREPOSITORY
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
4. BUILD
DEVELOPER
5. DEPLOY TO TEST
TEST SERVER
OPERATIONS
PRODUCTION SERVER
7. DEPLOY TO PROD
QA
6. TEST
8. TEST
CONTINUOUS INTEGRATION SERVER
3. CHANGENOTIFICATION
SOURCE CODEREPOSITORY
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
4. BUILD
DEVELOPER
5. DEPLOY
APPLICATION SERVER
QA
6. TEST
CONTINUOUS INTEGRATION SERVER
3. CHANGENOTIFICATION
SOURCE CODEREPOSITORY
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
4. BUILD/TEST
DEVELOPER
5. DEPLOY
APPLICATION SERVER
CONTINUOUS INTEGRATION SERVER
3. CHANGENOTIFICATION
SOURCE CODEREPOSITORY
PROJECT MANAGEMENT SERVER1. PICK
TASKS
2. SUBMITCODE
4. BUILD/CREATE ENVIRONMENT/TEST/TEARDOWN
DEVELOPER
5. DEPLOY
APPLICATION SERVER
CONTINUOUS INTEGRATION SERVER
3. CHANGENOTIFICATION
Cloud software development lifecycle
AWS Elastic Beanstalk
AWS OpsWorks
AmazonCloudWatch
AWS CloudFormation
AWS CodeDeploy
AWS CodeCommit
AWS CodePipeline
Code Build Test Deploy Provision Monitor
AWS and DevSecOpsOpportunity AWS Services
Marketplace offerings and Competency Partners
AWS CloudFormatio
n
AWSCodeDeploy
AWSOpsWorks
AWS ElasticBeanstalk
• IT shops fully embracing DevSecOps, can be orders of magnitude more productive than those that don’t.
• AWS offers an array of powerful services to enable DevSecOps.
• Using AWS CloudFormation to repeatedly and quickly deploy dev/test environments, and then shut them down immediately when tests complete, is helping customers:
Save money and time Increase quality Increase agility
AWSCodeCommit
AWSCodePipeline
DevSecOps, self service, and cost managementAutomation empowers individuals; however:
Individuals spending OPM can spend too much
AWS services can help: AWS Identity and Access Management
(IAM) restrictions Cost Explorer Detailed billing reports Budgets Cost and usage reports Billing alerts
AWS Partners can provide more analytics and assist in cost control
Bridging the gap from DevOps to financeJ.R. Storment, Chief Customer Officer at [email protected]
What DevSecOps brings to the table
Breaking down silos Collaboration between cross-disciplinary teamsMove faster in refreshing your infrastructureConstant adjustment to changeAutomated monitoring and alerting
Effect—cost goes up and with a more complex financial audit trail
AWS CodeCommit
AWS CodeDeploy
AWS CodePipeline
AWSCloudFormation
AWSOpsWorks
AWSService Catalog
AWSDevice Farm
AWSMobile Hub
AmazonSNS
AmazonSQS
Amazon CloudWatch
AmazonS3
Amazon ECR
Amazon ECS
AWS Elastic Beanstalk
AWSLambda
Amazon EC2
Amazon Redshift
Amazon Elasticsearch
Service
IAM AWS KMS
AWS CloudHSM
AWS Certificate Manager
Explosion of SKUs and metadata increasing reporting complexity
DevOps has decentralized deployment of resources to more engineers and involved finance in the planning decisions
CI/CD shortening feedback loops and creating opportunities to refresh infrastructure and improve efficiency
Cross-discipline teams (dev+ops+finance) now jointly
responsible for bill…
Engineers Finance Operations Capacity Execs
Finance a part of the process now
DevOps Finance
measurebuy
align learn
delivery pipeline
feedback loop
Cloud efficiency lifecycle
What is DevSecOps?
developers customers
releasetestbuild
plan monitor
delivery pipeline
feedback loop
Software development lifecycle
The term “FinOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between DevOps and Finance, resulting in an iterative data-driven management of infrastructure spending (i.e., lowering the unit economics of cloud), while simultaneously increasing the cost efficiency and ultimately profitability of the cloud environment.
What is FinOps?
—J.R. Storment, chief customer officer at Cloudability
FinOps czar (n) A person or team focused on looking at the AWS billing data each month to identify opportunities to save money (e.g., with Reserved Instance coverage)
FinOps/RI czar
Why appoint one?
Proper purchasing of RIs can save 30–60% on your AWS bill
Assuming a $1 M/yr spend, there’s a potential savings of $300 K+ year.
Usually is a technically minded person in finance, procurement, or vendor management
How do you build a FinOps culture?
Put data in the hands of the people
Enact policies and evangelize best practices
Cross-train teams on shared knowledge and reporting tools
Visibility
Allocation Efficiency
Savings
Unit cost
I. Cost visibility
Tips for cost visibility
Get each stakeholder the spending fundamentals daily
Let each team see other teams’ spending habits
Create broadly available dashboards
Visibility
Allocation Efficiency
Savings
Unit cost
II. Allocation
• Tags are highly flexible, but 100% coverage is difficult due to compliance• Linked accounts offer clean chargeback but limit reporting options
Consolidation of accounts to achieve volume discounts driving centralized management of finance optimization
Pro tips: allocating costs
Get consensus on the taxonomy (but let Finance drive)
Define 2–3 mandatory tags like “project” or “environment”
Consider a “tag or terminate” rule to enforce compliance
Visibility
Allocation Efficiency
Savings
Unit cost
III. Efficiency
Don’t run the cloud like a data center:65% of the hours in a month are
nights and weekends
Tips for encouraging efficient behavior
1. Automate weekly waste reporting for each team
2. Gamify cleanup by creating a visible leaderboard
3. Do a monthly, company-wide waste review
Visibility
Allocation Efficiency
Savings
Unit cost
IV. Savings
Rapid infrastructure changes driving need for iterative price optimization
Visibility
Allocation Efficiency
Savings
Unit cost
V. Unit cost
Focus on reducing unit cost, even at total cost grows
Se-ries1
0
30
60
90
120
150
Unit cost Total cost
Thank you!Emil Lerch, Senior Cloud Architect at Amazon Web Services,
J.R. Storment, Chief Customer Officer at [email protected]