DevOps on AWS: Deep Dive on Infrastructure as Code
-
Upload
amazon-web-services -
Category
Engineering
-
view
1.714 -
download
2
Transcript of DevOps on AWS: Deep Dive on Infrastructure as Code
![Page 1: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Alex Corley – Solutions ArchitectPublic Sector - State and Local Government
June 2016
Infrastructure as Code
Best Practices on AWS
![Page 2: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/2.jpg)
Learning objectives
• Understand Infrastructure as Code
• Understand the AWS services that help you manage
your infrastructure as code
• Discover best practices for managing your AWS
infrastructure, host configuration, and applications
![Page 3: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/3.jpg)
Background
Moving to the cloud and AWS allows you to provision and
manage infrastructure in new ways:
• Infrastructure can be provisioned in seconds
• Scale can be achieved without complicated capacity
planning
• APIs let you interact with infrastructure using languages
typically used in applications
![Page 4: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/4.jpg)
What is Infrastructure as Code?
A practice in which traditional infrastructure management
techniques are supplemented by or replaced with code-
based tools and software development techniques
![Page 5: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/5.jpg)
Infrastructure as Code workflow
CodeVersion
Control
Code
ReviewIntegrate Deploy
![Page 6: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/6.jpg)
Infrastructure as Code workflow
CodeVersion
Control
Code
ReviewIntegrate Deploy
Text EditorGit/SVN/
Perforce
Review
Tools
Syntax
Validation
Tools
AWS
Services
![Page 7: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/7.jpg)
Infrastructure as Code workflow
“It’s all software”
CodeVersion
Control
Code
ReviewIntegrate Deploy
Text EditorGit/SVN/
Perforce
Review
Tools
Syntax
Validation
Tools
AWS
Services
![Page 8: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/8.jpg)
Application Configuration
AWS Resources
Infrastructure as Code workflow
Operating System and Host Configuration
![Page 9: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/9.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
![Page 10: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/10.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
Infrastructure Resource
Management
![Page 11: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/11.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
Infrastructure Resource
Management
Host Configuration
Management
![Page 12: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/12.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
Infrastructure Resource
Management
Host Configuration
ManagementApplication Deployment
![Page 13: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/13.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
AWS CloudFormation
AWS OpsWorks
AWS CodeDeploy
![Page 14: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/14.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
AWS CloudFormation
AWS OpsWorks
AWS CodeDeploy
Amazon Virtual Private
Cloud (Amazon VPC)
Amazon Elastic Compute
Cloud (Amazon EC2)
AWS Identity and Access
Management (IAM)
Amazon Relational Database
Service (Amazon RDS)
Amazon Simple Storage
Service (Amazon S3)
AWS CodePipeline
…
Microsoft Windows Registry
Linux networking
OpenSSH
LDAP
Active Directory domain
registration
Centralized logging
System metrics
Deployment agents
Host monitoring
…
Application dependencies
Application configuration
Service registration
Management scripts
Database credentials
…
![Page 15: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/15.jpg)
allOfThis == $Code
![Page 16: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/16.jpg)
AWS CloudFormation
• Create templates that describe
and model AWS infrastructure
• CloudFormation then provisions
AWS resources based on
dependency needs
• Perform version control on,
replicate, and update the
templates like app code
• Integrates with development,
CI/CD, management tools
• No additional charge to use
![Page 17: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/17.jpg)
Benefits
Templated resource
provisioning
Infrastructure
as code
Declarative
and flexible
Easy to use
![Page 18: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/18.jpg)
CloudFormation concepts and technology
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Framework
Stack creation
Stack updates
Error detection and rollback
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Template CloudFormation Stack
![Page 19: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/19.jpg)
Anatomy of a CloudFormation template: JSON
Plain text
Perfect for version control
Can be validated
![Page 20: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/20.jpg)
Anatomy of a CloudFormation template: JSON{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment
"}, ”Prod"]},
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}
![Page 21: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/21.jpg)
Anatomy of a CloudFormation template: JSON
Parameters
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
Mappings
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
Conditionals
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]},
},
Resources
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
Outputs
Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}
Headers
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.",
![Page 22: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/22.jpg)
Description of what your stack does, contains, and so on
Provision time values that add structured flexibility and customization
Predefined conditional case statements
Conditional values set through evaluations of passed references
AWS resource definitions
Resulting attributes of stack resource creation
Headers
Parameters
Mappings
Conditionals
Resources
Outputs
Template components
![Page 23: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/23.jpg)
Template example
Templates (in action):
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, {"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]},
![Page 24: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/24.jpg)
Template example
Templates (in action):
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, {"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]},
“AWSRegionVirt2AMI” map
![Page 25: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/25.jpg)
Template example
Templates (in action):
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" },
“AWSRegionVirt2AMI” map
{"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]},
“AWSInstanceType2Virt” map
![Page 26: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/26.jpg)
Template example
Templates (in action):
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" },
“AWSRegionVirt2AMI” map
{"Fn::FindInMap": ["AWSInstanceType2Virt",
“AWSInstanceType2Virt” map
{ "Ref" : "myInstanceType" }, "Virt"]} ]},
“myInstanceType” parameter
![Page 27: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/27.jpg)
Template example
Templates (in action):
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI",
“AWSRegionVirt2AMI” map
{"Fn::FindInMap": ["AWSInstanceType2Virt",
“AWSInstanceType2Virt” map
{ "Ref" : "myInstanceType" }, "Virt"]} ]},
“myInstanceType” parameter
{ "Ref" : "AWS::Region" },
AWS::Region pseudo parameter
![Page 28: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/28.jpg)
Template example
"myInstanceType" : {
"Type" : "String",
"Default" : "t2.large",
"AllowedValues" : ["t2.micro", "t2.small", "t2.medium", "t2.large"],
"Description" : "Instance type for instances created, must be in the t2 family."
}
"AWSInstanceType2Virt": {
"t2.micro": {"Virt": "HVM"},
"t2.small": {"Virt": "HVM"},
"t2.medium": {"Virt": "HVM"},
"t2.large": {"Virt": "HVM"},
}
"AWSRegionVirt2AMI": {
"us-east-1": {
"PVM": "ami-50842d38",
"HVM": "ami-08842d60"
},
"us-west-2": {
"PVM": "ami-af86c69f",
"HVM": "ami-8786c6b7"
},
"us-west-1": {
"PVM": "ami-c7a8a182",
"HVM": "ami-cfa8a18a"
}
}
Parameters: Mappings: Mappings:
![Page 29: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/29.jpg)
Bootstrapping applications and handling updates
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
"#!/bin/bash -ex","\n",
"yum -y install gcc-c++ make","\n",
"yum -y install mysql-devel sqlite-devel","\n",
"yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","\n",
"gem install --no-ri --no-rdoc rails","\n",
"gem install --no-ri --no-rdoc mysql","\n",
"gem install --no-ri --no-rdoc sqlite3","\n",
"rails new myapp","\n",
"cd myapp","\n",
"rails server -d","\n"]]}}
}
}
Option 1: Use Amazon EC2 UserData, which is available as a property of
AWS::EC2::Instance resources
![Page 30: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/30.jpg)
cfn-init
cfn-hup
Bootstrapping applications and handling updates
Option 2: CloudFormation provides
helper scripts for deployment within
your EC2 instances
Metadata key—
AWS::CloudFormation::Init
The cfn-init helper script reads this
metadata key and installs the
packages listed in this key (for
example, httpd, mysql, and php); cfn-
init also retrieves and expands files
listed as sourcesEC2
CloudFormationcfn-signal
cfn-get-
metadata
![Page 31: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/31.jpg)
Bootstrapping example
"Metadata": {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
},
"sources" : {
},
"commands" : {
},
"files" : {
},
"services" : {
},
"users" : {
},
"groups" : {
}
}
}
Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances:
![Page 32: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/32.jpg)
Bootstrapping example
“WebAppHost" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS:CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc" : [],
"gcc-c++" : [],
"make" : [],
"automake" : [],
Install packages with the native package management tool:
![Page 33: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/33.jpg)
Manage a wide range of AWS services and resources
• Amazon EC2
• Amazon EC2 Container Service
• Amazon EC2 Container Registry
• Amazon EC2 Simple Systems Manager
• AWS Lambda (including event sources)
• AWS Elastic Beanstalk
• Auto Scaling (including Spot fleet)
• Amazon VPC and Managed NAT Gateway
• Elastic Load Balancing
• Amazon Route 53
• Amazon CloudFront
• AWS WAF
• Amazon S3
• Amazon RDS
• Amazon Redshift
• Amazon DynamoDB
• Amazon ElastiCache
• Amazon RDS (including Amazon Aurora)
• Amazon Elastic MapReduce
• Amazon Elasticsearch Service
• AWS Data Pipeline
• AWS Identity and Access Management (including
managed policies)
• AWS Directory Service (Amazon Simple AD) / Microsoft
Active Directory
• Amazon Kinesis
• Amazon SNS
• Amazon SQS
• AWS CloudTrail
• Amazon CloudWatch
• AWS Config
• AWS Key Management Service
• AWS OpsWorks
• AWS CodeDeploy
• AWS CodePipeline
• Amazon WorkSpaces
• Amazon GameLift
AWS resource support is always growing. See the most up-to-date list here.
![Page 34: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/34.jpg)
Template file
defining stack
• The entire infrastructure can
be represented in a
CloudFormation template
Many stacks and environments from one template
![Page 35: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/35.jpg)
Template file
defining stack
• The entire infrastructure can
be represented in a
CloudFormation template
• Use the version control
system of your choice to
store and track changes to
this template
Git
Perforce
SVN
…
Many stacks and environments from one template
![Page 36: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/36.jpg)
Template file
defining stack
• The entire infrastructure can
be represented in a
CloudFormation template
• Use the version control
system of your choice to
store and track changes to
this template
• Build out multiple
environments, such as for
development, test,
production, and even disaster
recovery, using the same
template
Git
Perforce
SVN
…
Dev
Test
Prod
Many stacks and environments from one template
![Page 37: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/37.jpg)
Infrastructure as Code with CloudFormation
Versioning
You track changes within your code
Do it with your infrastructure:
• What is changing?
• Who made that change?
• When was it made?
• Why was it made?(Is it tied to a ticket or bug or project system?)
![Page 38: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/38.jpg)
Testing your CloudFormation templates
Testing your template:
• Validate by using API or AWS Command Line Interface (CLI)
• $ aws cloudformation validate-template—confirm
CloudFormation syntax
• Use something like JSONLint (http://jsonlint.com/) to find
JSON issues like missing commas or brackets
• Throw this into your testing and/or continuous integration
pipelines
![Page 39: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/39.jpg)
Visualizing your CloudFormation templates
• AWS
CloudFormation
Designer
• Visualize template
resources
• Modify template
with drag and drop
gestures
• Customize sample
templates
![Page 40: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/40.jpg)
Deploying your CloudFormation templates
Deploy and update by using console, API, or CLI
OR
• aws cloudformation create-stack --stack-name
myteststack --template-body
file:////home//local//test//sampletemplate.json --
parameters
ParameterKey=string,ParameterValue=string
![Page 41: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/41.jpg)
But what do we do once your
resources are provisioned and
running?
![Page 42: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/42.jpg)
Your infrastructure needs ongoing management
• Updates or patches?
• New software?
• New configurations?
• New code deployments?
• Pool-specific changes?
• Environment-specific changes?
• Run commands across all hosts?
• Be on top of all running resources?
![Page 43: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/43.jpg)
Ongoing management requires proper tooling
Some common challenges:
• Changing a vhost configuration on every web server across
multiple environments (development, staging, production)
• Installing a package on certain hosts to test out newer versions
• Changing the LDAP configuration on every running Amazon EC2
Linux host when the hosts exist across 25 different CloudFormation
templates
![Page 44: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/44.jpg)
We need a tool to interact with
each host that we manage and
that makes it easier to
configure these hosts
![Page 45: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/45.jpg)
AWS OpsWorks
• Configuration management service
for automating operational tasks
using Chef
• Model, control, and automate
applications of nearly any scale and
complexity
• Manage Linux and Microsoft
Windows environments
• Supports both AWS and on-
premises servers
• Launched in 2013
![Page 46: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/46.jpg)
AWS OpsWorks concepts
A stack represents
the cloud
infrastructure and
applications that
you want to manage
together
A layer defines how
to set up and
configure a set of
instances and
related resources
You decide how to
scale: manually,
with 24/7 instances,
or automatically,
with load-based or
time-based
instances
Then deploy your
app to specific
instances and
customize the
deployment with
Chef recipes
![Page 47: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/47.jpg)
AWS OpsWorks concepts: instance lifecycle
Set up Configure Deploy Undeploy Shut down
An agent on each instance understands a
set of commands that are triggered by
OpsWorks. The agent when triggered runs
Chef.
![Page 48: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/48.jpg)
OpsWorks agent communication
1. The EC2 instance connects with the
OpsWorks service to send keepalive/
heartbeat and receive lifecycle events
2. OpsWorks sends a lifecycle event with a
pointer to the configuration JSON
(metadata, recipes) in an S3 bucket
3. The agent downloads configuration
JSON
4. The agent pulls cookbooks and other
build assets from your repository
5. The agent executes the recipe
6. The agent uploads the Chef log
7. The agent reports Chef run status
EC2
instanceOpsWorks
service
“Deploy App”
Your
repository,
for example
GitHub
![Page 49: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/49.jpg)
How OpsWorks bootstraps EC2 instances
The EC2 instance is started by using an IAM role
• UserData passed with instance private key, OpsWorks public key
• The instance downloads and installs the OpsWorks agent
The agent connects to the instance service, gets run info
• Authenticates the instance using the instance’s IAM role
• Picks up configuration JSON from the OpsWorks instance queue
• Decrypts and verifies the message, runs Chef recipes
• Uploads Chef log, returns Chef run status
The agent then polls the instance service for more messages
![Page 50: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/50.jpg)
AWS OpsWorks + Chef
OpsWorks uses Chef to configure the software on the
instance
OpsWorks provides many Chef Server functions to users
• Associates cookbooks with instances
• Dynamic metadata describes each registered node in the
infrastructure
Supports "push" command and control client runs
Supports community cookbooks
![Page 51: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/51.jpg)
Working with Chef and OpsWorks
Similar to CloudFormation templates and application code:
• Mixture of JSON and a Ruby DSL
• Tools exist to do linting and syntax checking
• Versioning
• Built in cookbook versioning
• Some manual/processes scripted abilities
• But still can use source control for versioning
• Use with continuous integration systems like
CloudFormation templates and the rest of your code
![Page 52: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/52.jpg)
Working with Chef and OpsWorks
Basics:
• Nodes
• Roles
• Cookbooks
• Recipes
• Attributes
• Data bags
• Environments
![Page 53: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/53.jpg)
Host configuration management with Chef
package "ntp" do
action :install
end
service "ntpd" do
supports :status => true, :restart => true, :reload => true
action [ :enable, :start ]
end
cookbook_file "/etc/ntp.conf" do
source "ntp.conf"
owner "root"
group "root"
mode 00644
# Restart ntp.conf if /etc/ntp.conf changes
notifies :restart, resources(:service => "ntpd")
Endgroup "ganglia" do
gid 499end
user "ganglia" dohome "/var/lib/ganglia"shell "/sbin/nologin"uid 499gid "ganglia"
end
directory "/etc/ganglia" doaction :create
end
Examples:
![Page 54: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/54.jpg)
Host configuration management with Chef
template "/etc/ganglia/gmond.conf" do
source "gmond.conf.erb"
owner "root"
group "root"
mode 00644
notifies :restart, resources(:service => "gmond")
variables(
:gmetad_name => node[:ganglia][:gmetad_name],
:cluster_name => node[:ganglia][:cluster_name]
)
end
cron "all-gmetrics" do
command "for FILE in `ls /opt/bin/gmetric-*`; do command $FILE; done >/dev/null 2>&1"
end
Examples:
![Page 55: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/55.jpg)
Custom JSON{
"opsworks": {
"data_bags": {
"myapp": {
"mysql": {
"username": "default-user",
"password": "default-pass"
}
}
}
}
}
Host configuration management with Chef
Recipemything = data_bag_item("myapp", "mysql")
Chef::Log.info("username:
#{mything['username']}")
![Page 56: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/56.jpg)
AWS OpsWorks
![Page 57: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/57.jpg)
Deploying applications
![Page 58: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/58.jpg)
Automates code deployments to any instance
Handles the complexity of updating your
applications
Use it to avoid downtime during application
deployment
Deploy to Amazon EC2 or on-premise servers,
in any language and on any operating system
Integrates with third-party tools and AWS
services
AWS CodeDeploy
![Page 59: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/59.jpg)
AWS CodeDeploy concepts
Application
Revision #1
Revision #2
Revision #3
What to deploy?
Revision #1
How to deploy?
Instance
Instance
Instance
Deployment group
Auto Scaling group
Where to deploy?
![Page 60: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/60.jpg)
How it works: package app with appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
permissions:
- object: /var/www/html
pattern: “*.html”
owner: root
group: root
mode: 755
hooks:
ApplicationStop:
- location: scripts/deregister_from_elb.sh
BeforeInstall:
- location: scripts/install_dependencies.sh
ApplicationStart:
- location: scripts/start_httpd.sh
ValidateService:
- location: scripts/test_site.sh
- location: scripts/register_with_elb.sh
![Page 61: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/61.jpg)
How it works: package app with appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html
• Send application files to one
directory and configuration files to
another
• Set specific permissions on specific
directories and files
• Remove or add instance to Elastic
Load Balancing
• Install dependency packages
• Start Apache
• Confirm successful deploy
• More!
permissions:- object: /var/www/html
pattern: “*.html”owner: rootgroup: rootmode: 755
hooks:ApplicationStop:- location: scripts/deregister_from_elb.sh
BeforeInstall:- location: scripts/install_dependencies.sh
ApplicationStart:- location: scripts/start_httpd.sh
ValidateService:- location: scripts/test_site.sh- location: scripts/register_with_elb.sh
![Page 62: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/62.jpg)
How it works: Specify targets
Group instances by:
• Auto Scaling group
• Amazon EC2 tag
• On-premises tag
Development deployment group
AgentAgent Agent
Production deployment group
AgentAgent Agent
AgentAgent Agent
![Page 63: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/63.jpg)
How it works: Deploy
• AWS CLI and SDKs
• AWS Management Console
• AWS CodePipeline and CI/CD partners
• Amazon S3, GitHub
aws deploy create-deployment \
--application-name MyApp \
--deployment-group-name TargetGroup \
--s3-location bucket=MyBucket,key=MyApp.zip
![Page 64: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/64.jpg)
v2 v1 v1 v1 v1 v1 v1 v1
v2 v2 v1 v1 v1 v1 v1 v1
v2 v2 v2 v2 v1 v1 v1 v1
v2 v2 v2 v2 v2 v2 v2 v2
One at a timeMinimum healthy hosts = 99%
[Custom]Minimum healthy hosts = 75%
Half at a timeMinimum healthy hosts = 50%
All at onceMinimum healthy hosts = 0
Choose your deployment configuration
![Page 65: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/65.jpg)
Summary
![Page 66: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/66.jpg)
Summary
• Create, update, and manage AWS resources and their configuration
and properties with CloudFormation
• You can configure OpsWorks and CodeDeploy by using
CloudFormation
• Use OpsWorks for ongoing tweaks to software and configuration of
host-based applications and the operating system
• You can configure and deploy CodeDeploy’s agent with
OpsWorks
• Use CodeDeploy to deploy your applications and their configurations
![Page 67: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/67.jpg)
Best practices
• Your CloudFormation templates and Chef cookbooks should
go in separate repositories
• Include the appspec.yml file and related scripts in your
application’s code repositories
• Every commit should cause an execution of your continuous
delivery pipeline to lint, validate, and/or test
• Use each related service’s CLI, console, and APIs to update or
deploy as necessary
![Page 68: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/68.jpg)
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
AWS CloudFormation
AWS OpsWorks
AWS CodeDeploy
Amazon Virtual Private
Cloud (Amazon VPC)
Amazon Elastic Compute
Cloud (Amazon EC2)
AWS Identity and Access
Management (IAM)
Amazon Relational Database
Service (Amazon RDS)
Amazon Simple Storage
Service (Amazon S3)
AWS CodePipeline
…
Microsoft Windows Registry
Linux networking
OpenSSH
LDAP
Active Directory domain
registration
Centralized logging
System metrics
Deployment agents
Host monitoring
…
Application dependencies
Application configuration
Service registration
Management scripts
Database credentials
…
![Page 69: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/69.jpg)
allOfThis == $Code
![Page 70: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/70.jpg)
Learn more
• AWS CloudFormation
• https://aws.amazon.com/cloudformation/
• https://aws.amazon.com/documentation/cloudformation/
• https://aws.amazon.com/cloudformation/aws-cloudformation-templates/
• AWS OpsWorks
• https://aws.amazon.com/opsworks/
• https://aws.amazon.com/documentation/opsworks/
• https://github.com/aws/opsworks-cookbooks
• AWS CodeDeploy
• https://aws.amazon.com/codedeploy/
• https://aws.amazon.com/documentation/codedeploy/
• https://github.com/awslabs/aws-codedeploy-samples
![Page 71: DevOps on AWS: Deep Dive on Infrastructure as Code](https://reader031.fdocuments.us/reader031/viewer/2022020213/58f9adb0760da3da068b9ae9/html5/thumbnails/71.jpg)
Thank you!