Auto Scaling using AWS

Harish Ganesan

Co founder & CTO

8KMileswww.twitter.com/harish11g

http://www.linkedin.com/in/harishganesan

• Need for Auto Scaling

• How AWS Auto scaling can help us ?

• How to configure an Auto scaling policy in AWS

• Things to remember before Scaling out and down

• Understand the Intricacies while integrating Auto scaling with other Amazon Web Services

• Risks involved in AWS Auto Scaling

Introduction

EC2 : Elastic Compute Cloud

EBS : Elastic Block Storage

S3 : Simple Storage Service

SQS : Simple Queue Service

ELB : Elastic Load Balancing

AMI : Amazon Machine Images

AZ : Amazon Availability Zones ( Data centers)

VPC : Amazon Virtual Private Cloud

RDS : Amazon Relational Database Service

Key AWS terms used

Scale out : Achieving scalability by increasing thenumber of Amazon EC2 instances

Scale up : Achieving scalability by resizing thecapacity( Compute, Memory and EBS) of existingAmazon EC2 instances

Scale down : Decreasing the number of AmazonEC2 instances or the configuration for existing EC2instances

Key Scaling terms

Need for Auto Scaling

Upfront Capacity Investment

Source: AWS

Actual Demand vs Fixed Capacity

Source: AWS

Problem 1: Wasted Capacity

Source: AWS

Problem 2 : Lost Customers

Source: AWS

Need for Auto Scaling

Source: Internet

AWS Auto Scaling allows us to scale our AmazonEC2 capacity out or down automatically accordingto the load patterns . Example :• We can expand the number of Amazon EC2 instances from 1 to

100+ automatically during load peaks

• We can reduce the number of Amazon EC2 instances from 100+ to1 automatically during load valleys

What is AWS Auto Scaling ?

AWS Auto Scaling

Source: AWS

We can closely align our Infrastructure

with our load requirements and save

costs

Load Volatility Scenarios

Candidates for Auto Scaling (1) C

om

pu

te

Time

• Applications which has following Load Volatility characteristics like

– Predictable bursts of request load

– A Pattern in Spikes and lows

– “On” and “Off” loads

– Seasonal trends

Predictable Bursts1

Candidates for Auto Scaling (2) C

om

pu

te

Time

• Applications which has following Load Volatility characteristics like

– Un Predictable bursts of request load

– No specific pattern in Spikes and lows

– Highly elastic ,unexpected and scalable demands

Un Predictable Bursts2

Candidates for Auto Scaling (3) C

om

pu

te

Time

• Applications which has following Load Volatility characteristics like

– Constantly growing

– Successful and growing very fast

– Complex lead time for deployment

Constantly Growing3

What Amazon Auto Scaling can do ?

• Handle all the 3 load scenarios (Candidates)

• Scale out Amazon EC2 instances seamlessly andautomatically when demand increases

• Scale down unwanted Amazon EC2 instancesautomatically and save money when demandsubsides

• Decide the scaling based on AWS CloudWatchmetrics

• Auto Scale your Web servers(Amazon EC2) incombination with AWS Elastic Load Balancing

How much does it Cost ?

Cost for using AWS Auto Scaling service = 0 $

Value= PRICELESS

Note : AWS Auto scaling needs Amazon CloudWatch monitoring service to function . Amazon CloudWatch is billed on usage basis.

Some AWS Auto scaling Concepts

Auto Scaling group : Logical grouping of multipleAmazon EC2 instances for easy scaling andManagement

Health Check: Calls to check on the health status ofeach Amazon EC2 instance in an Auto Scaling group

Launch Configuration: Captures the parametersnecessary to create new EC2 instances in Auto Scalingmode

Some AWS Auto scaling Concepts

Triggers: A CloudWatch alarm and an Auto Scalingpolicy that describes the actions when the alarmthreshold is crossed . Two Triggers – Scaling out andScaling down needs to be created

Policy : Set of instructions for Auto Scaling that tells the service how to respond to AWS CloudWatch alarm messages

Notable parameters on Amazon Auto Scaling

• The percentage of allocated EC2 compute units that are currently in use on the instanceCPUUtilization

• Filters and responds with data that we request about all the Amazon EC2 instance part of the Auto scaling groupAutoScalingGroupName

• Time taken between a request and the corresponding response as seen by the AWS Elastic Load BalancerLatency

• The number (sum) of requests processed by the AWS Elastic Load BalancerRequestCount

• The number (average) of healthy instances behind the AWS Elastic Load balancerHealthyHostCount

• The number (average) of unhealthy instances behind the AWS Elastic Load BalancerunHealthyHostCount

Steps to configure Amazon Auto Scaling with AWS Elastic Load Balancing

Step 1: Configuring AWS Auto Scaling with AWS ELB

elb-create-lb my-load-balancer --headers --listener "lb-port=80,instance-port=8080,protocol=HTTP" --availability-zones us-east-1c

The load balancer port

App server port to which requests needs to be forwarded

Add a name to your load balancer

Step 2: Create a launch configuration

as-create-launch-config my-lconfig --image-id ami-e3826c8a --instance-type m1.small --key my-key-pair --group my-security-group

Amazon EC2 Instance Size

Key pair / Security group settings for the Amazon EC2 instances

Name your launch configuration

Amazon Machine image(AMI) to be launched during scaling

Step 3: Create an AWS Auto Scale Group

as-create-auto-scaling-group my-as-group --availability-zones us-east-1c –launch-configuration my-lconfig --max-size 11 --min-size 3 --cooldown 180 --load-balancers my-load-balancer

Minimum/Maximum number of Amazon EC2 instances maintained by Auto Scale

Load balancer name in which the new Amazon EC2 instances launched will be attached

Name your Auto scale group

Availability Zone in which the Auto scaled Amazon EC2 instances will be launched

Step 4: Configure the Auto scaling Triggers

as-create-or-update-trigger my-as-trigger --auto-scaling-group my-as-group --namespace "AWS/EC2" --measure CPUUtilization --statistic Average --dimensions "AutoScalingGroupName= my-as-group " --period 60 --lower-threshold 20 --upper-threshold 80 --lower-breach-increment"=-2" --upper-breach-increment 4 --breach-duration 180

Lower CPU Limit is 20% and Upper CPU Limit is 80%

Measure the average CPU of the Auto Scale Group

Scale out by 4 Amazon EC2 instances.Scale down by 2 Amazon EC2 instances

Applying Amazon Auto Scaling for 3 different load scenarios

Auto Scaling Policy : Predictable BurstsC

om

pu

te

Time

• Configure Time based Auto Scaling plan

• PutScheduledUpdateGroupAction - specify the date and time of execution, minimum, maximum, and desired Instance size of capacity

Predictable Bursts1

% as-put-scheduled-update-group-action my-group –name “My-scale-up” -- time “2011-04-05T02:00:00Z” -min 10 -max 50

Auto Scaling Policy : Un predictable BurstsC

om

pu

te

Time

• Configure On demand Auto Scaling policy

• Create policies for both Scaling out and Scaling down scenarios

• Auto Scaling will respond to changing conditions dynamically

Un Predictable Bursts2

%as-create-auto-scaling-group as-group --availability-zones us-east-1c –launch-configuration my-lconfig --max-size 20 --min-size 2 --cooldown180 --load-balancers my-load-balancer

Auto Scaling Policy : Constantly Growing C

om

pu

te

Time

• Periodically monitor the load requirements and manually configure the Auto scaling parameters

• Increase the minimum fixed capacity gradually

• Leverage On-Demand Amazon EC2 instances

Constantly Growing3

%as-create-auto-scaling-group my-as-group --availability-zones us-east-1a –launch-configuration my-lconfig --max-size 4 --min-size 2 --cooldown 360 --load-balancers my-load-balancer

Important points to remember before Scaling out / down using AWS Auto scaling

While Scaling out

• Very frequent (minutes) scaling out activity is notcost effective

• Scaling out with smaller instance type units closelyaligns our capacity with our load requirements

• We can create a maximum of 125 scheduled actionsper Auto Scaling group. This allows scaling fourtimes a day for a 31-day month for each AutoScaling group

• We can configure a scheduled Scaling action for upto a month in the future

While Scaling down

• Amazon Auto scaling does not close the AmazonEC2 instances in the same of order of launch

• To avoid data inconsistency architects should ensurethe following points on Amazon EC2 servers ofAuto scale group

• Log files generated by Auto Scaled Amazon EC2 aremoved to Amazon S3 periodically

• Data files uploaded in the Web/App EC2 instances aremoved to S3

• Sessions of Web/App layer and Application Cache layersare replicated to a centralized /distributed resource

Tips ,Pointers and Notes:

While integrating Amazon Auto Scaling with other AWS components

Amazon Auto scaling and Amazon Machine Images (AMI’s)

Amazon Auto Scaling and AMI’s

• Amazon Auto Scaling can launch new EC2 instancesfrom S3 backed AMI’s and EBS backed AMI’s

• AMI’s should be present on the Same Region wherethe EC2 instances will be launched in Auto scalegroup

• AMI’s need not be present on the Same AZ wherethe EC2 instances will be launched in Auto scalegroup

AWS Security Groups

S3 Backed AMI’s for Auto Scaling

AZ : us-east-1a

Amazon Elastic Load Balancer

1

3

EC2 Instances

Amazon Auto Scaling

2

4

Web requests are sent to the Amazon ELB

1

AWS ELB transfers the requests to Amazon EC2 instances launched in us-east-1A availability zone

2

Amazon EC2 instances are configured to be part of AWS Auto Scale group

3

AWS Auto Scaling launches the new Amazon EC2 instances from the S3 backed AMI’s

4

AWS Security Groups

EBS Backed AMI’s for Auto Scaling

AZ : us-east-1b

Amazon Elastic Load Balancer

1

3

EC2 Instances

Amazon Auto Scaling

2

4

Web requests are sent to the Amazon ELB

1

AWS ELB transfers the requests to Amazon EC2 instances launched in us-east-1B availability zone

2

Amazon EC2 instances are configured to be part of AWS Auto Scale group

3

AWS Auto Scaling launches the new Amazon EC2 instances from the EBS backed AMI’s

4

EBS

EBS

EBS

Amazon Auto Scaling and AMI’s

• EBS backed AMI’s are faster to launch compared toS3 backed AMI’s

• EBS backed AMI’s (Windows) launch in 30+ seconds(approx)

• S3 backed AMI’s (Windows) launch in 4 minutes (approx)

• Sometimes it is better to have EC2 instanceslaunched faster in Auto scaling when overall CPUthreshold is breached

• EBS backed AMI’s option in AWS Auto scaling maynot be cost effective compared to S3 backed AMI’s

AWS Auto scaling and Amazon EC2 instances

Amazon Auto Scaling and Amazon EC2

• A single Amazon Auto Scaling group can launch onlyone type of EC2 instances

• Example , We cannot configure Amazon Auto scalinggroup to launch combinations of both m1.small andm1.large instance types

• We can combine Reserved Instances (RI) and On-Demand EC2 Instances in our Scaling architecture

• Amazon Auto scaling cannot be used to scaleAmazon RDS instances

Amazon Auto scaling and AWS regions/AZ’s

Amazon Security Groups

Design 1: Amazon Auto Scaling across AZ’s

CLOUDWATCH

AZ: US-EAST-1A

Amazon Elastic Load balancer

HTTP requests are redirected to Amazon EC2 instances

configured across the AWS availability zones in US East

Amazon EC2

instances

Amazon Auto Scaling

Amazon Security Groups

CLOUDWATCH

AZ:US-EAST-1B

Amazon EC2

instances

Amazon Auto Scaling

S3

Backed

AMI

S3

Backed

AMI

Amazon Security Groups

Design 2: Amazon Auto Scaling across AWS Regions

CLOUDWATCH

AWS USA East Region

Amazon Elastic Load balancer

Amazon EC2

instances

Amazon Auto Scaling

Amazon Security Groups

CLOUDWATCH

AWS Europe Region

Amazon EC2

instances

Amazon Auto Scaling

S3

Backed

AMI

S3

Backed

AMI

Architecture Not Possible

Amazon Auto Scaling ,AWS Regions and AZ’s

• AWS Auto Scaling groups can work across multipleAWS Availability Zones inside same Region

• If an AWS Availability Zone becomes unavailable,Auto Scaling will automatically redistributeapplications to a different Availability Zone

• AWS Auto Scaling groups cannot work acrossRegions

• Example , we cannot configure a same AWS Auto scalinggroup to distribute and create Amazon EC2 instances inboth USA and Europe

Amazon Auto scaling and AWS Elastic Load balancer

AWS Security Groups

Case 1: AWS Auto Scaling with AWS Elastic Load balancing

AZ : us-east-1a

Amazon Elastic Load Balancer

1

3

EC2 Instances

Amazon Auto Scaling

2

4

This Architecture is suitable for Web Layer only . AWS ELB is attached to the AWS Auto Scaling group

1

AWS ELB acts as the gateway and transfers the HTTP requests it receives to Amazon EC2 instances in round robin schedule

2

Amazon EC2 instances are configured to be part of AWS Auto Scale group

3

AWS Auto Scaling launches the new Amazon EC2 instances

4

AWS Auto Scaling ,AWS CloudWatch and AWS Elastic Load Balancing works in union

5

S3 backed AMI

Case 2: Amazon Auto Scaling without AWS ELB

4

EC2 Instances

Amazon Auto Scaling

Reference Auto scaling Architecture in AWS not using AWS ELB

1

Messages needed for processing are put in the AWS SQS using message producer clients

2

Master Amazon EC2 instance node controls the scale up / scale down of the processing nodes using AWS Auto Scaling programmatically

3

Auto scaled Amazon EC2 instance processing nodes picks the messages from Amazon SQS

4

S3 backed AMI

SQS

Amazon EC2

instances of

Auto scale group

reads messages

from AWS SQS

and processes

them

Amazon EC2

instances are auto

scaled

programmatically

depending upon

SQS Load

Master EC2 instance node

controls the Auto scaling EC2

instance group (scale out/down)

activities programmatically

Master EC2 Node

2

3

Amazon Auto Scaling and Amazon ELB

• Auto Scaling supports, but does not require, AWSElastic Load Balancing

• We can add Elastic Load Balancing to our AutoScaling group and use Elastic Load Balancing metrics(request latency or request count) to scale ourapplication

• During the Scaling down process, Amazon AutoScaling instructs the AWS load balancer to removethe Amazon EC2 instance from the load balancinggroup first

Amazon Auto scaling and Amazon Elastic IP

Amazon Auto Scaling and Amazon Elastic IP

• Every new Amazon EC2 instances launched byAmazon Auto scaling has a Private and Public IPaddress

• Configuring Elastic IP for Amazon EC2 instanceslaunched by Amazon Auto Scaling is not a costeffective option

• We need to pay Amazon for unused Elastic IP’s after theScale down process

• Combining Elastic IP & Auto scaling should be analyzedfully before applying in specific use cases

Amazon Auto scaling and Monitoring

Amazon Auto Scaling and Monitoring

• Many traditional infrastructure monitoring systemsrely on Agents to be installed on Amazon EC2instances and get registered with Centralizedmonitoring server for receiving alerts

• Since the Amazon EC2 servers are created andterminated dynamically by AWS Auto scaling, it willcause problems like

• When new EC2 instances are created by Auto scaling themonitoring agents installed in them cannot auto registerwith the Centralized monitoring server

Amazon Auto Scaling and Monitoring

• When EC2 instances are killed by Auto scaling , thecentralized monitoring server will misunderstand thisaction as instance non-availability and createsunnecessary noise

• Hence before integrating monitoring systems intoour AWS Auto Scaled Infrastructure we shouldensure that it can monitor dynamically spawnedAmazon EC2 instances

Amazon Auto Scaling and AWS CloudWatch

• AWS CloudWatch can monitor Auto scaled EC2instances

• AWS CloudWatch should be configured “detailedmonitoring” option for the collection of AutoScaling group metrics

• Configure Auto Scaling with group metrics to sendaggregated data to Amazon CloudWatch everyminute

• Auto scaling takes appropriate Scale out /downactions based on the alarms it receives fromCloudWatch

Amazon Auto scaling and Launch Configurations

Auto Scaling and Launch Configurations

• The maximum number of launch configurations perAWS account is 100

• If we modify our launch configuration, AWS AutoScaling will not apply our new settings to existingAmazon EC2 instances

• We can attach only one launch configuration to anAWS Auto Scaling group at a time

• When AWS Auto Scaling wants to scale down, it firstterminates instances that have an older launchconfiguration

Amazon Auto scaling and Elastic Beanstalk

Amazon Auto Scaling and Elastic Beanstalk

• Amazon Elastic Beanstalk is a deployment tool usedto automatically handle the infrastructure detailslike capacity provisioning, load balancing, auto-scaling, and application health monitoring

• We cannot leverage Elastic bean stalk based Autoscaling in following scenarios

– If our system demands Non Java applications to beconfigured on Auto scaling mode

– If our application contains Java based Backgroundprograms (not a web app ) and they need auto scalingfacility

Amazon Auto scaling and Amazon VPC

Amazon Auto Scaling and Amazon VPC

• Amazon Auto Scaling can scale out /down AmazonEC2 instances inside a Amazon VPC network

• AWS Elastic Load balancer cannot be tied toAmazon Auto Scaling inside Amazon VPC

• Amazon Auto scaling is restricted only to USA eastAZ-1a and Amazon EC2 EU (Ireland) in the AmazonVPC network

AWS Auto Scaling illustration

Following slides illustrates a visual demo of AWS AutoScaling case

• Image 1: Amazon EC2 Servers at initial Stage

• Image 2: Load Increases : Amazon EC2 Servers are ramped up using AWS AutoScaling

• Image 3: Load Decreases

• Image 4: Amazon EC2 Servers are dynamically scaled down using AWS AutoScaling

The below screenshots represent a custom product developed by 8KMiles using AWS

AutoScaling

Image 1 : Initial Stage

Screenshot represents a custom product developed by 8KMiles

Image 2 : Load Increases

Image 3 : Load Decreases

Screenshot represents a custom product developed by 8KMiles

Image 4 : Scaling down the EC2 instances

Risks involved in AWS Auto Scaling

Risk 1: AWS Auto Scaling takes between 30 – 180seconds sometimes to launch a new instance(s) .This intermediate time may cause impairedperformance for our customers

Risk 2: AWS Auto Scaling cannot differentiatebetween valid (vs) malicious traffic , it can scale outservers even for malicious traffic

Auto Scaling is an Art

• Auto Scaling will not work for every situation andevery application

• Just by deploying our application in AWS , does notmean our application will leverage Auto Scaling andscale out automatically

• To leverage Auto scaling effectively in-deptharchitectural expertise is needed in both Applicationand Infrastructure front

Auto Scaling is an Art

• In some cases Auto Scaling is an on-going(monitorand fine tune based on trends/patterns) and not anone time activity

• Badly configured Auto Scaling parameters will

– increase the cost of infrastructure

– cause revenue leakage rather than reducing them

– create unnecessary excess capacity

How do I leverage Amazon Auto Scaling effectively ?

Cloud Architecture Consulting

Cloud Migration & Implementation

Cloud Application Development

Leave it to the experts , we will

handle this

Cloud Adoption Strategy

“Let's get the job done”

• Amazon Systems Integrator and Solution Developer

• Migrated 350+ servers for start-ups, small businesses and Enterprises

• Prior expertise in architecting Auto scaling based solutions

• In-depth understanding of Cloud infrastructure services

Why 8KMiles ?

8KMiles in Media

Contact Us

“All you need is an idea and the cloud will execute it for you.” (Structure 2010 event) - Dr Werner Vogels , CTO of Amazon on 8KMiles

For more details on how 8KMiles Cloud Consulting can help

your business , contact

cloud@8kmiles.com

http://cloud.8kmiles.com

http://cloudblog.8kmiles.com

http://www.8kmiles.com