(ENT205) AWS and VMware: How to Architect and Manage Hybrid Environments | AWS re:Invent 2014
Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS
-
Upload
amazon-web-services -
Category
Technology
-
view
4.633 -
download
1
description
Transcript of Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS
![Page 1: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/1.jpg)
Chris Munns, Solutions Architect, Chris Barclay, Senior
Product Manager, and Mike Limcaco, Solutions Architect
Continuous Deployment Practices, with Production, Test
and Development Environments Running on AWS
![Page 2: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/2.jpg)
This talk:
• Not going to spend a lot of time talking about Continuous Integration(CI) and Continuous Deployment(CD) philosophy
• Will spend more time talking about how AWS can help you if practicing CI and CD are your goals
• Examples to get you thinking, but not the only way
• AWS + Open Source solutions
![Page 3: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/3.jpg)
• Continuous Integration
• Continuous Deployment
![Page 4: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/4.jpg)
• Continuous Integration
• Continuous Deployment
Techniques and tools to
implement continuous
processes of applying quality
control in general small
pieces of effort, applied
frequently, to improve the
quality of software, and to
reduce the time taken to
deliver it.
![Page 5: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/5.jpg)
• Continuous Integration
• Continuous Deployment
Techniques and tools to
improve the process of
software delivery, resulting in
the ability to rapidly, reliably,
and repeatedly push out
enhancements and bug fixes
to customers at low risk and
with minimal manual
overhead.
![Page 6: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/6.jpg)
• Continuous Integration
• Continuous Deployment
Getting code from
developers’ brains,
through their fingers,
to production quickly
and efficiently, with
positive results.
![Page 7: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/7.jpg)
Continuous Integration & Deployment on AWS
• Treat infrastructure as code
• Automate the testing/deploy process end to end
• Make sure environments mimic each other as closely as possible
• Use repeatable patterns between environments at a different scale
• Use different cost models where it makes sense
• Simplify and streamline the deploy process
• Let AWS services handle control flows
• Track everything (instance metrics, application metrics, logs)
![Page 8: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/8.jpg)
es
![Page 9: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/9.jpg)
In today’s infrastructure, everything is code.
From the applications developers are writing, to
your configuration management tools, to things like
CloudFormation templates or scripts that call AWS APIs.
![Page 10: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/10.jpg)
Since Infrastructure is code, let’s treat it like code!
– Not JUST Revision control!
– Make use of bug tracking/ticketing systems
– Peer reviews of changes before they happen
– Establish infrastructure code patterns/designs
– Test infrastructure changes like code changes
![Page 11: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/11.jpg)
Let’s talk about
the journey our
code is going to
take to production
![Page 12: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/12.jpg)
1.Code gets written
2.Code gets tested
3.Code gets deployed
4.Code gets consumed
![Page 13: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/13.jpg)
1.Code gets written – Someone writes code and commits to revision
control system
– Hooks in revision control, system kicks off CI work
2.Code gets tested – Unit tests, integration tests, db tests, smoke tests,
UI tests
– “Light green, trap clean” OR GOTO STEP 1
3.Code gets deployed – Ship out that code
4.Code gets consumed – Customers use it, love it, victory, profit, vacation in
Bora Bora
![Page 14: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/14.jpg)
1.Software ( tools, services, scripts )
2.Infrastructure Environments ( dev, test, prod )
3.Process ( deploy, monitor, alert, track )
![Page 15: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/15.jpg)
1.Software ( tools, services, scripts )
2.Infrastructure Environments ( dev, test, prod )
3.Process ( deploy, monitor, alert, track )
We need tools to help work with all of the above quickly and more efficiently
![Page 16: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/16.jpg)
First stop on our journey: Continuous Integration-ville
• Help prove code quality and function repeatedly with predefined results
• Lots of options; self hosted, open source, closed source, and SaaS
![Page 17: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/17.jpg)
Continuous Integration - Jenkins
“An extendable open source continuous integration server”
• Open Source • Well established and used by many • Has plugins for EC2/SQS/SNS/CloudFormation!
• Supports spot pricing! • Supports the ability to put workers into a
“standby” mode by stopping instead of terminating
• Scales well • Easily add more EC2 instances as workers
• Flexible • Easy to get started
Internal Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
![Page 18: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/18.jpg)
Test Chef Cookbooks w/ FoodCritic after each Git Commit.
![Page 19: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/19.jpg)
Test Chef Cookbooks w/ FoodCritic after each Git Commit.
Blue dot is good!
![Page 20: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/20.jpg)
Where is our code going?
![Page 21: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/21.jpg)
Infrastructure Environments
A bad thing people do:
“Developers develop locally on their laptops, mostly OS X based, then deploy to
production, which is Ubuntu. Each laptop has a slightly different setup, and we
don’t maintain software versions across the whole team.”
– Dev and prod not in sync
– Dev not in sync with all of dev
– No testing tier between dev and prod
![Page 22: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/22.jpg)
Infrastructure Environments
A bad thing people do:
“Developers develop locally on their laptops, mostly OS X based, then deploy to
production, which is Ubuntu. Each laptop has a slightly different setup, and we
don’t maintain software versions across the whole team.”
– Dev and prod not in sync
– Dev not in sync with all of dev
– No testing tier between dev and prod
“it worked fine on my laptop”
![Page 23: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/23.jpg)
Infrastructure Environments
A bad thing people do:
“Developers develop locally on their laptops, mostly OS X based, then deploy to
production, which is Ubuntu. Each laptop has a slightly different setup, and we
don’t maintain software versions across the whole team.”
– Dev and prod not in sync
– Dev not in sync with all of dev
– No testing tier between dev and prod
“it worked fine on my laptop”
![Page 24: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/24.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance Read
Replica
Instance Instance
![Page 25: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/25.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance Read
Replica
Instance Instance
![Page 26: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/26.jpg)
Dev Environment VPC Subnet
DEV WEB ELB
Dev Stack Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance DEV APP ELB
Our Development Infrastructure
![Page 27: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/27.jpg)
Dev Environment VPC Subnet
DEV WEB ELB
Dev Stack Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance DEV APP ELB
Our Development Infrastructure
![Page 28: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/28.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance
Read Replica
Instance Instance
![Page 29: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/29.jpg)
Dev Environment VPC Subnet
DEV WEB ELB
Dev Stack Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance DEV APP ELB
Our Development Infrastructure
![Page 30: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/30.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance Read
Replica
Instance Instance
![Page 31: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/31.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance Read
Replica
Instance Instance
![Page 32: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/32.jpg)
Dev Environment VPC Subnet
DEV WEB ELB
Dev Stack Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance DEV APP ELB
Our Development Infrastructure
![Page 33: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/33.jpg)
Our Development Infrastructure
Developers &
Operations
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
![Page 34: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/34.jpg)
Our Development Infrastructure
Developers &
Operations
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
![Page 35: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/35.jpg)
Our Development &Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
![Page 36: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/36.jpg)
Infrastructure Environments
• Be prepared to be running multiple environments
– Development
– Testing/QA
– Staging/Pre-prod
– Production
• They should be running as close to the same stack as possible
• Use configuration management and infrastructure orchestration tools
• No one off hosts
• A goal: Go from nothing to fully running instances without human intervention
![Page 37: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/37.jpg)
This all seems like a lot of work,
and potentially costly.
![Page 38: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/38.jpg)
But it doesn’t need to be!
![Page 39: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/39.jpg)
Infrastructure Automation
We want to be able to rapidly stand up environments as we need to.
Sounds like we need some automation tools?
– CloudFormation
– Elastic Beanstalk
– OpsWorks
– Chef
– Puppet
![Page 40: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/40.jpg)
AWS CloudFormation
"WebServer" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, "64" ]}, "SpotPrice" : { "Ref" : "SpotPrice" }, "InstanceType" : m1.large", "SecurityGroups" : [{ "Ref" : "InstanceSecurityGroup" }], "KeyName" : { "Ref" : "KeyName" }, "UserData": { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bash -v\n", "yum update -y aws-cfn-bootstrap\n”, "curl -L http://www.opscode.com/chef/install.sh | bash\n", "cd /etc/chef\n", "/usr/bin/wget http://",{ "Ref" : "ChefServerIP" },"/chef/validation.pem\n", "/usr/bin/wget http://",{ "Ref" : "ChefServerIP" },"/chef/client.rb\n", "/bin/chown -R chef:chef /etc/chef\n",
![Page 41: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/41.jpg)
Our Development &Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
![Page 42: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/42.jpg)
Our Development/Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
![Page 43: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/43.jpg)
AWS Elastic Beanstalk & OpsWorks
Elastic Beanstalk:
• Application container framework similar to a PaaS
• Deploy your application into Elastic Beanstalk and it takes care of building a self healing, auto-scaling, multi-AZ infrastructure
• Allows you to turn some of the knobs under the hood to tweak
• Considered one of the easiest places to start with hosting an application on AWS
OpsWorks:
• Build multi-layer application stacks
• Ties in with Chef for a large degree of flexibility and customization
• Makes deploying applications easier
• More flexible than Elastic Beanstalk, but requires a bit more knowledge
![Page 44: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/44.jpg)
Our Development/Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
Elastic Beanstalk or OpsWorks
![Page 45: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/45.jpg)
Region
Amazon Route 53
Amazon CloudFront
Customer Traffic
Amazon S3
Availability Zone
Availability Zone
Availability Zone
Internet Gateway
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
VPC Subnet VPC Subnet VPC Subnet VPC Subnet
Instance Instance
Instance
NAT
Bastion/Chef
ELB ELB
RDS DB Instance
RDS DB Instance Standby (Multi-AZ) Instance
AWS CloudFormation
Amazon CloudWatch
Amazon SNS
Potential RDS DB Instance Read
Replica
Instance Instance
Elastic Beanstalk or OpsWorks
![Page 46: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/46.jpg)
Imagine you had an infrastructure you could turn on/off on
demand, make use of spare capacity at a lower cost,
and/or make a reservation for capacity based on your
usage needs and save money doing so.
![Page 47: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/47.jpg)
Oh right, on AWS you do.
![Page 48: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/48.jpg)
Using the Right Cost Model – EC2
• On Demand
• Reserved Instance ( RI ) – 40%+ savings
• Spot – 80%+ savings
Each has its place. For development infrastructure, there
are often places for each:
• On Demand – Developer instances started/stopped daily
• Reserved Instances – Code repository, CI master, DBs
• Spot – CI workers, tiers of dev infrastructure that can tolerate going
away for a bit
![Page 49: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/49.jpg)
Our Development &Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway
VPN Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
RESERVED INSTANCES
SPOT/ON-DEMAND
RRS S3, CloudFront Price Classes,
DynamoDB RC
![Page 50: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/50.jpg)
Now that we know where our code is going,
how is it getting there?
![Page 51: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/51.jpg)
Ship that code!
• How are we going to deploy our code?
– File shipping:
• Just text files
• Binaries
– Package bundling:
• RPMs
• Tarballs
– As an AMI:
• Bundle one of the above into an AMI
• How fast do we need to do this?
• Across how many instances?
• How do we roll back (or forward)?
Yes, you can technically ship your code to AWS in a box. See Import/Export.
![Page 52: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/52.jpg)
File Shipping Deploy Method
• Can be easier to work with than AMI method and package bundling
• Push out the code
– From Git/SVN/staging host
• Rolling restarts for web/application servers
• Leave existing hosts in place
• Have to worry about the cut over period
• Have to worry about feasibility of roll back/forward
• Can do deploy time schema changes (though a bad idea!)
• Have to worry about tracking what version is live for building new hosts
![Page 53: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/53.jpg)
Deploying – Package Building
• Depending on the language/deployment method, you might need to take the
time to package your code.
– RPM
– Deb
– Something else?
• Throw this in as a step after a successful CI run.
Look at using tools like FPM to manage building packages for different
distributions.
![Page 54: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/54.jpg)
AMI Deployment Method
• Code gets bundled into an AMI, we then deploy that AMI
– Pluses
• Very atomic
• New shouldn’t effect older versions
• Can deploy alongside current
• Easy tools to automate
– Cons
• Bit more work involved
• Have to think about where your data is persisting
• Schema updates potentially harder to package in
• Leverage configuration management tools in automation process
![Page 55: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/55.jpg)
A quick aside - Schema updates
Schema changes tied to deployments are a huge blocker to moving fast.
– Hard to undo a change
– Can take a long time on SQL-based databases
Unlink this from code deploys:
– Flag on/off new features that touch the database in new ways
– Don’t make destructive database changes until no code touches that data
• No deletes, alters to live data! Ever!
– When altering existing data, opt instead to create a parallel column, copy data to new
column, then delete old
– Use “shadow queries” to test new functions/data sources for a percentage of users
before turning live to all
![Page 56: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/56.jpg)
AMI Deployment Method - Building
![Page 57: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/57.jpg)
AMI Deployment Method - Building
Fully Functional AMI
OS-Only AMI
Partially Configured AMI
![Page 58: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/58.jpg)
AMI Deployment Method - Building
Fully Functional AMI
OS-Only AMI
Partially Configured AMI
Least flexible to maintain
![Page 59: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/59.jpg)
AMI Deployment Method - Building
Fully Functional AMI
OS-Only AMI
Partially Configured AMI
Most amount of post-boot work
Least flexible to maintain
![Page 60: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/60.jpg)
AMI Deployment Method - Building
Fully Functional AMI
OS-Only AMI
Partially Configured AMI
Most amount of post-boot work
Least flexible to maintain
Try and find a happy medium here
![Page 61: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/61.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
100%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 62: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/62.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
EC2 Instances
ELB
90% 10%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 63: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/63.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
EC2 Instances
ELB
50% 50%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 64: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/64.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
EC2 Instances
ELB
0% 100%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 65: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/65.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
EC2 Instances
ELB
0% 100%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 66: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/66.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
100%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 67: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/67.jpg)
AMI Deployment Method - Deploying
Blue/Green Deploys
– We stand up a duplicate part of our
infrastructure and slowly cut traffic
over to it
• Shift via DNS
• Makes it easy to do testing of new
features
• Makes it easy to roll back
– As we shift more traffic over, let auto-
scaling grow/shrink our instances of
the new or old application
• Shut down the old when no traffic
there
Amazon Route 53
EC2 Instances
ELB
100%
DynamoDB MySQL RDS Instance
ElastiCache Cache Node
![Page 68: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/68.jpg)
AMI Deployment Method - Deploying
Netflix – Asgard
– Open Source tool
– Released in 2012
– “web-based tool for managing
cloud-based applications and
infrastructure.
– Helps do Blue/Green Deploys
– Capable of much more!
![Page 69: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/69.jpg)
But how do we do all this quickly and
easily many times a day?
![Page 70: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/70.jpg)
We need robots
![Page 71: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/71.jpg)
![Page 72: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/72.jpg)
We need robots
![Page 73: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/73.jpg)
We need robots
Amazon SWF
![Page 74: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/74.jpg)
Automating the Process with Robots
Amazon Simple Workflow (SWF)
• Orchestration tool across your infrastructure
• Use it as a middle layer to pass messages and setup tasks to be completed
• Break down individual tasks into different workers
• You define logic between workers
• Anything that can be scripted, can be made into a worker task
• Built in retries, timeouts, logging
• Low cost, reliability, and scalability built in
Amazon SWF
Deciders Workers
YOUR CODE = &
![Page 75: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/75.jpg)
Automating the Process with Robots
Amazon Simple Workflow (SWF)
• Orchestration tool across your infrastructure
• Use it as a middle layer to pass messages and setup tasks to be completed
• Break down individual tasks into different workers
• You define logic between workers
• Anything that can be scripted, can be made into a worker task
• Built in retries, timeouts, logging
• Low cost, reliability, and scalability built in
Amazon SWF
Deciders Workers
YOUR ROBOTS = &
![Page 76: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/76.jpg)
Automating the Process
Workers:
• Bundling code into an RPM – WORKER
• Making a new AMI with this RPM – WORKER
• Deploying a new CloudFormation stack with this RPM – WORKER
• Swapping DNS over to our new stack – WORKER
• Copy AMI across to another region for DR – WORKER
• Clean up old AMIs – WORKER
You get the picture.
Workers
![Page 77: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/77.jpg)
YOUR CODE = ROBOTS
![Page 78: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/78.jpg)
YOUR CODE = ROBOTS
![Page 79: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/79.jpg)
Our Development &Test Infrastructure
Developers &
Operations Internal
Git CI Server
Pre-commit Hook
Testing Environment Subnet
CI Workers
Dev Environment VPC Subnet
DEV WEB ELB Dev Stack
Tier 1
Dev Stack Tier 2
Dev MySQL DB Instance
DEV APP ELB
VPN TUNNEL
VPN facing VPC Subnet
Internet Gateway VPN
Endpoint
Dev Admin Instance
NAT Instance
Amazon S3
Amazon DynamoDB
Amazon SQS
Amazon CloudFront
Amazon Route 53
Amazon SWF
3. Deploy RPM to Dev Environment
2. Build an RPM
1. After CI run kicks off SWF
Execution
Decider Determines next step
![Page 80: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/80.jpg)
Our code has arrived at its destination
![Page 81: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/81.jpg)
Our code has arrived at its destination
But what now?
![Page 82: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/82.jpg)
Monitoring/Logging Infrastructure
• Need to know what’s going on
• Spend the time required to do this well
• Share access to these tools with whole team
• Track every single resource that you can
• Alert on services, their availability, response times
• Make use of different cost models for different parts of this stack
• Try to keep log and other monitoring data for as long a possible
– 6 months? 1 year? Multiple years?
![Page 83: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/83.jpg)
Monitoring/Logging Infrastructure
Tools:
• Logging – Logstash
• Check out Kibana!
– Graylog2
– Syslog-ng/rsyslog/syslog
• Metrics – CloudWatch
– Ganglia
– Graphite
• Monitoring – Nagios
– Munin
– Sensu
![Page 84: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/84.jpg)
![Page 85: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/85.jpg)
HOST LEVEL
METRICS
AGGREGATE LEVEL
METRICS
LOG ANALYSIS BUILD METRICS
![Page 86: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/86.jpg)
AWS Marketplace can help AWS Online Software Store • Customer can find, research, buy software
• Simple pricing, aligns with EC2 usage model
• Launch in minutes
• Marketplace billing integrated into your AWS
account
• 600+ products across 23 categories
Learn more at: http://aws.amazon.com/marketplace
Developer Tool Categories Include • Bug Tracking
• Monitoring
• Source Control
• Testing
![Page 87: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/87.jpg)
Continuous Integration & Deployment on AWS
• Treat infrastructure as code.
• Automate the testing/deploy process end to end.
• Make sure environments mimic each other as closely as possible.
• Use repeatable patterns between environments at a different scale.
• Use different cost models where it makes sense.
• Simplify and streamline the deploy process.
• Let AWS services handle control flows.
• Track everything (instance metrics, application metrics, logs).
![Page 88: Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS](https://reader031.fdocuments.us/reader031/viewer/2022020122/554bd71fb4c905ac708b502d/html5/thumbnails/88.jpg)
Thanks for listening!