Building cloud-enabled apps How to take advantage of the AWS scalability & elasticity

Andreas Chatzakis – Solutions Architect @achatzakis

AWS for Startups

London, 12th September 2013

Agenda

• Intro: AWS foundation

• Scalability: from a single server to multiple instances

• Autoscaling: pay only for what you use

• Database: scaling relational or NoSQL DBs

• Agility: not servers => software components

Intro

Utility computing

On demand Pay as you go

Uniform Available

On demand Pay as you go

Uniform Available

Utility computing

Utility computing

Compute

Storage

Security Scaling

Database

Networking Monitoring

Messaging

Workflow

DNS

Load Balancing

Backup CDN

On demand Pay as you go

Uniform Available

Utility computing

US-WEST (Oregon) EU-WEST (Ireland)

ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC

(Sydney)

Regions

ASIA PAC

(Singapore)

US-WEST (Oregon) EU-WEST (Ireland)

ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC

(Sydney)

Availability Zones

ASIA PAC

(Singapore)

Compute Storage

AWS Global Infrastructure

Database

App Services

Deployment & Administration

Networking

Amazon

CloudWatch AWS IAM AWS

CloudFormation

Amazon Elastic

Beanstalk AWS

Data

Pipeline

AWS

OpsWorks

Amazon

CloudSearch Amazon SQS Amazon

SNS

Amazon

Elastic

Transcoder

Amazon SWF Amazon SES

Amazon

DynamoDB

Amazon RDS

Amazon

ElastiCache

Amazon

RedShift

AWS Storage

Gateway

Amazon S3

Amazon

Glacier

Amazon

CloudFront Amazon

EC2

Amazon

EMR Amazon

VPC

Amazon

Route 53 AWS

Direct

Connect

Scaling up as you grow

Day One, User One:

• A single EC2 Instance

– With full stack on this host • Web App

• Database

• Management

• etc

• A single Elastic IP

• Route53 for DNS

EC2

Instance

Elastic IP

Amazon

Route 53 User

Scaling Vertically

• Simplest approach

• Can now leverage PIOPs

• High I/O instances

• High Memory instances

• High CPU instances

• High storage instances

• Easy to change instance sizes

• Will hit an endpoint eventually

hi1.4xlarge

m2.4xlarge

m1.small

Day One, User One:

• We could potentially get to a few hundred to a few thousand depending on application complexity and traffic

• Manual failover

• No redundancy

• Too many eggs in one basket

EC2

Instance

Elastic IP

Amazon

Route 53 User

Day One, User One:

• We could potentially get to a few hundred to a few thousand depending on application complexity and traffic

• Manual failover

• No redundancy

• Too many eggs in one basket

EC2

Instance

Elastic IP

Amazon

Route 53 User

Day Two, User >1:

First lets separate out

our single host into

more than one.

• Web

• Database – Make use of a database

service?

Web

Instance

Database

Instance

Elastic IP

Amazon

Route 53 User

User >100:

First lets separate out

our single host into

more than one.

• Web

• Database – Use RDS to make your life

easier

Web

Instance

Elastic IP

RDS DB

Instance

Amazon

Route 53 User

User > 1000:

Next let’s address our

lack of failover and

redundancy issues: • Elastic Load Balancer

• Another Web instance

– In another Availability Zone

• Enable RDS Multi-AZ

Web

Instance

RDS DB Instance

Active (Multi-AZ)

Availability Zone Availability Zone

Web

Instance

RDS DB Instance

Standby (Multi-AZ)

Elastic Load

Balancer

Amazon

Route 53 User

• Create highly scalable applications

• Distribute load across EC2 instances

in multiple availability zones Feature Details

Available Load balance across instances in multiple Availability Zones

Health checks Automatically checks health of instances and takes them in or out of service

Session stickiness Route requests to the same instance

Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support

Monitoring Publishes metrics to CloudWatch

Elastic Load

Balancer

Elastic Load Balancing

Scaling this horizontally

and vertically here will

get us pretty far ( 10s-100s of thousands )

User >10ks-100ks:

RDS DB Instance

Active (Multi-AZ)

Availability Zone Availability Zone

RDS DB Instance

Standby (Multi-AZ)

Elastic Load

Balancer

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Amazon

Route 53 User

This will take us pretty far

honestly, but we care about

performance and efficiency,

so lets clean this up a bit

Shift some load around:

Let’s lighten the load on

our web and database

instances:

• Move static content from web

servers to S3 and CloudFront

• Move session/state and DB

caching to ElastiCache or

DynamoDB

Web

Instance

RDS DB Instance

Active (Multi-AZ)

Availability Zone

Elastic Load

Balancer

Amazon S3

Amazon

Cloudfront

Amazon

Route 53 User

ElastiCache

DynamoDB

Working with S3 - Amazon Simple Storage Service

• Object based storage for the web

• 11 9s of durability

• Good for things like:

– Static assets ( css, js, images,

videos )

– Backups

– Logs

– Ingest of files for processing

• “Infinitely scalable”

• Serves static content

• Supports fine grained permissions

• Ties in well with CloudFront & EMR

• Logging for S3/CloudFront/Billing

• Supports Encryption (transit/rest)

• Reduced Redundancy 1/3 cheaper

• Glacier for long term storage

• Objects lifecycle configuration

• Automatic partitioning (prefix

choice can maximize throughput)

ElastiCache

• Hosted Memcached

– Speaks same protocol as open source memcached

• Scale from one to many nodes

• Self healing ( replaces dead instance )

• Very fast ( single digit ms speeds usually)

• Local to a single AZ

– So need to run different clusters across different Azs

• Data is only in memory, so not persistent

• Use AWS’s Auto Discovery client to simplify clusters

growing and shrinking without affecting your application

Auto-Scaling!

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

Typical weekly traffic to Amazon.com

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

Typical weekly traffic to Amazon.com

Provisioned capacity

November traffic to Amazon.com

November

November traffic to Amazon.com

Provisioned capacity

November

November traffic to Amazon.com

76% waste

24%

Provisioned capacity

November

November traffic to Amazon.com

November

Decompose into small,

loosely coupled, stateless

building blocks

Prerequisite

Having decomposed into

small, loosely coupled,

stateless building blocks

You can now Scale out with ease

Having done that…

Having decomposed into

small, loosely coupled,

stateless building blocks

We can also Scale back with ease

Having done that…

Trigger auto-scaling

policy

Automatic resizing of compute

clusters based on demand

Feature Details

Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs.

Integrated to Amazon CloudWatch

Use metrics gathered by CloudWatch to drive scaling.

Instance types Run Auto Scaling for On-Demand and Spot Instances. Compatible with VPC.

as-create-auto-scaling-group MyGroup

--launch-configuration MyConfig

--availability-zones us-east-1a

--min-size 4

--max-size 200

Auto-Scaling

Amazon

CloudWatch

Scaling the Database

Self-Managed Fully-Managed

Database Server

on Amazon EC2

Your choice of

database running on

Amazon EC2

Bring Your Own

License (BYOL)

Amazon

DynamoDB

Managed NoSQL

database service

using SSD storage

Seamless

scalability Zero

administration

Amazon RDS

Microsoft SQL,

Oracle or MySQL as

a managed service

Flexible licensing

BYOL or License

Included

Amazon

Redshift

Massively parallel,

petabyte-scale,

data warehouse

service

Fast, powerful and

easy to scale

Database Options

But how do I choose

what DB technology I

need? SQL? NoSQL?

For most use-cases SQL is good fit

• Relational DBs: Established and well worn technology

• Lots of existing code, communities, books,

background, tools, skilled people etc

• Flexible query model, data integrity

• Can actually scale - especially for read heavy apps

Scaling Relational DBs

• Read Replicas (Master – Slave)

• Replication lag

• Splitting Reads and Writes

– Writes => master

– Reads with tolerance to stale data => read replica (slave)

– Reads with need for most recent data => master

• Master is a SPOF

– Try to decouple / introduce fault tolerance to your application

NoSQL data stores

• Distributed systems, can scale up reads but also writes

– Sharding + Replicas

• Compromises: No JOINs, limited querying flexibility vs

Relational

– Denormalize data, schema dictated by app queries

• Eventual consistency vs strong consistency

DynamoDB

• Provisioned throughput NoSQL

database

• Fast, predictable performance

• Fully distributed, fault tolerant

architecture

• Considerations for non-uniform

data

Feature Details

Provisioned throughput

Dial up or down provisioned read/write capacity.

Predictable performance

Average single digit millisecond latencies from SSD-backed infrastructure.

Strong consistency

Be sure you are reading the most up to date values.

Fault tolerant Data replicated across Availability Zones.

Monitoring Integrated to CloudWatch.

Secure Integrates with AWS Identity and Access Management (IAM).

Elastic MapReduce

Integrates with Elastic MapReduce for complex analytics on large datasets.

Being Agile with

continuous deployment

Idea Product

Least friction

Highest quality

Idea Product

Shortest time

Highest accuracy

Idea Product

Failing fast when you need to

Idea Product

Product

Product Freedom to pivot if you have to

How? deploy often

…on a platform built for iteration

Everything that was a constrained resource

is now a programmable resource

Auto Scaling

Group

V1

Elastic

Load

Balancer

Amazon

Relational

Database Service

(RDS)

Auto Scaling

Group

V1

Auto Scaling

Group

V2

Elastic

Load

Balancer

Amazon

Relational

Database Service

(RDS)

Auto Scaling

Group

V1

Auto Scaling

Group

V2

Elastic

Load

Balancer

Amazon

Relational

Database Service

(RDS)

Auto Scaling

Group

V1

Auto Scaling

Group

V2

Elastic

Load

Balancer

Amazon

Relational

Database Service

(RDS)

Auto Scaling

Group

V2

Elastic

Load

Balancer

Amazon

Relational

Database Service

(RDS)

Get smart

Achieve a lot even

if team is small

Offload undifferentiated

heavy lifting

AWS Cloud-Based

Infrastructure

Your Business

More Time to Focus on Your Business

Configuring Your Cloud Assets

70%

30% 70%

On-Premise Infrastructure

30%

Managing All of the “Undifferentiated Heavy Lifting”

Focus on your core business

Relational Database Service

DynamoDB

SimpleDB

Databases Middleware Frameworks Analytics

Simple Queuing Service

Simple Notification Service

Simple Workflow

Elastic MapReduce

Simple Email Service

CloudSearch ElastiCache

CloudWatch

User >1mil+:

RDS DB Instance

Active (Multi-AZ)

Availability Zone

Elastic Load

Balancer

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

Web

Instance

Web

Instance

Web

Instance

Web

Instance

Amazon

Route 53 User

Amazon S3

Amazon

Cloudfront

DynamoDB

Amazon SQS

ElastiCache

Worker

Instance

Worker

Instance

Amazon

CloudWatch

Internal App

Instance

Internal App

Instance Amazon SES

A Summary

• Make your application stateless – local

file system only for temp files

• Use Autoscaling in Multiple AZs

• Make use of self scaling services ( ELB,

S3, DynamoDB, SQS, SWF, SES, etc )

• Select data layer based on real needs

• Leverage managed/low touch services

What is Cloud Computing http://aws.amazon.com/what-is-cloud-computing

Architecture Center http://aws.amazon.com/architecture

Articles http://aws.amazon.com/articles

Documentation http://aws.amazon.com/documentation

QUESTIONS?