How to Scale to Millions of Users with AWS

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Ken Payne, AWS Solutions Architecture

Jean Meyer, CEO at Once Dating AG

7th July 2016

Scaling the Platform for your Startup

Why are you here?

Building the technology platform for your startup

You want to prepare for success

Learn about design patterns & scalability

A pragmatic approach for startups

Priorities for startups

Racing within a window of opportunity

Small team with no legacy

Focus on solving a problem

Avoid over-engineering & re-engineering

Reduce risk of failure when you go viral

A scalable architecture

Can support growth in users, traffic, data size

Without practical limits

Without a drop in performance

Seamlessly - just by adding more resources

Efficiently - in terms of cost per user

Day 1 – Dev & Private Beta

Single host

THE s erver(e.g . Apac he,

MyS Q L)

Elas tic IP

www.example.c om

Amazon Route 53DNS s ervic e

S erver Imag e (AMI)

Day 2 - Public Beta

We need a bigger server

Add larger & faster storage (EBS)

Use the right instance type

Easy to change instance sizes

Not our long term strategy

Will hit an endpoint eventually

No fault toleranceX1 instances offer 1,952 GiB of DDR4

based memory, powered by four Intel®

Xeon® E7 8880 v3 (Haswell)

processors that offer 128 vCPUs

Separating web and DB

More capacity

Scale each tier individually

Tailor instance for each tier

• Instance type

• Storage

Security

• Security groups

• DB in a private VPC subnet

But how do I choose what

DB technology I need?

SQL? NoSQL?

Start with a Relational DB

SQL is versatile & feature-rich

Lots of existing code, tools, knowledge

Clear patterns to scalability (for read-heavy apps)

Reality: eventually you will have a polyglot data layer

• There will be workloads where NoSQL is a better fit

• Combination of both Relational and NoSQL

• Use the right tool for each workload

Key Insight: Relational Databases are Complex

Our experience running Amazon.com taught us that

relational databases can be a pain to manage and operate

with high availability

Poorly managed relational databases are a leading cause

of lost sleep and downtime in the IT world!

Especially for startups with small teams

Relational Databases

MySQL, Aurora, PostgreSQL, Oracle, SQL Server

Fully managed; Zero AdminAmazon

RDS

Aurora

Improving Efficiency

Offload static content

Amazon S3: highly available hosting that scales

• Static files (JavaScript, CSS, images)

• User uploads

S3 URLs – serve directly from S3

Let the web server focus on dynamic content

Amazon CloudFrontWorldwide network of edge locations

Cache on the edge

• Reduce latency

• Reduce load on origin servers

• Static and dynamic content

• Even few seconds caching of popular content can have huge impact

Connection optimizations

• Optimize transfer route

• Reuse connections

• Benefits even non cachable content

C loudFront

CloudFront for static & dynamic content

AmazonRoute 53

EC 2 instanc e(s)

S 3 buc ket

S tatic C ontent

Dynamic C ontent

css/*

js/*

Images/*

Default(*)

C loudFrontDistribution

Database CachingFaster response from RAM

Reduce load on database

Application server

1. If data in cache,

return result

2. If not in cache,

read from DBRDS database

Amazon ElastiC ache

3. And store

in cache

“Lazy Loading”

Amazon ElastiCache: in-memory cache

Simple to Deploy

Managed

• Automatically replaces failed nodes

• Patch management

Elastic

CompatibleElastiCache

Day 3 – Paying Customers

High Availability

Availability Zone a

RDS DB ins tanc e

Web s erver

S 3 buc ket fors tatic as s ets

www.example.c om

Amazon Route 53DNS servic e

Amazon C loudFront

Elas tiC ac henode 1

High Availability

Availability Zone a

RDS DB ins tanc e

Availability Zone b

Web s erver

Web s erver


www.example.c om


Amazon C loudFront


High Availability

Availability Zone a

RDS DB ins tanc e

Availability Zone b

www.example.c om


Elas tic LoadBalanc ing

Web s erver

Web s erver


Amazon C loudFront


Elastic Load Balancing

Managed Load Balancing Service

Fault tolerant

Health Checks

Distributes traffic across AZs

Elastic – automatically scales its capacity

High Availability

Availability Zone a

RDS DB ins tanc e

Availability Zone b

www.example.c om



Web s erver

Web s erver



Amazon C loudFront

High Availability

Availability Zone a

RDS DB ins tanc e

Availability Zone b

www.example.c om



Web s erver

Web s erver

RDS DBs tandby



Amazon C loudFront

Data layer HA

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



Web s erver

Web s erver

RDS DBs tandby

User Sessions?Problem: Often stored on local disk

(not shared)

Quick-Fix: ELB Session stickiness

Solution: DynamoDB


Web s erver

Web s erver

Logged in Logged out

Amazon DynamoDB

Managed document and key-value store

Simple to launch and scale

• To millions of IOPS

• Both reads and writes

Consistent, fast performance

Durable: perfect for storage of session data

https://github.com/aws/aws-dynamodb-session-tomcat

http://docs.aws.amazon.com/aws-sdk-php/guide/latest/feature-dynamodb-session-handler.html

Day 4 – Let’s go viral!

Replace guesswork with elastic IT

Startups pre-AWS

Demand

Unhappy Customers

Waste $$$

Traditional

Capacity

Capacity

Demand

AWS Cloud

Scaling the web tier

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



Web s erver

Web s erver

RDS DBs tandby


Scaling the web tier

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



Web s erver

Web s erver

RDS DBs tandby


Web s erver

Web s erver

Automatic resizing of compute

clusters based on demand

Feature Details

Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs.

Integrated to Amazon CloudWatch

Use metrics gathered by CloudWatch to drive scaling.

Instance types Run Auto Scaling for on-demand and Spot Instances. Compatible with VPC.

aws autoscaling create-auto-scaling-group

--auto-scaling-group-name MyGroup

--launch-configuration-name MyConfig

--min-size 4

--max-size 200--availability-zones us-west-2c, us-west-2b

Auto Scaling Trigger auto-scaling policy

Amazon

CloudWatch

Decompose into small,

loosely coupled, stateless

building blocks

Prerequisite

What does this mean in practice?

Only store transient data on local disk

Needs to persist beyond a single http request?

• Then store it elsewhere

User uploads

User Sessions

Amazon S3

AWS DynamoDB

Application Data

Amazon RDS

Having decomposed into

small, loosely coupled,

stateless building blocks

You can now Scale out with ease

Having done that…

Having decomposed into

small, loosely coupled,

stateless building blocks

We can also Scale back with ease

Having done that…

Take the shortcut

While this architecture is simple you still need to deal

with:

• Configuration details

• Deploying code to multiple instances

• Maintaining multiple environments (Dev, Test, Prod)

• Maintain different versions of the application

Solution: Use AWS Elastic Beanstalk

AWS Elastic Beanstalk (EB)

Easily deploy, monitor, and scale three-tier web

applications and services.

Infrastructure provisioned and managed by EB

You maintain control.

Preconfigured application containers

Easily customizable.

Support for these platforms:

Loose coupling with SQS

Tight coupling

• Place asynchronous tasks into Amazon SQS• SQS – buffer that protects backend systems• Process at own pace• Respond quickly to end users

S Q S

G et Mes s ag e

Bac k End EC 2 Ins tanc e

Put Mes s ag e

Front End EC 2 Ins tanc e

Day 5 – Add Features

Mobile

Push

NotificationsMobile

AnalyticsCognito

Cognito

Sync

Analytics

KinesisData

PipelineRedShift EMR

Your Applications

AWS Global Infrastructure

Network

VPCDirect

ConnectRoute 53

Storage

EBS S3 Glacier CloudFront

Database

DynamoDBRDS ElastiCache

Deployment & Management

Elastic

BeanstalkOpsWorks

Cloud

Formation

Code

Deploy

Code

Pipeline

Code

Commit

Security & Administration

CloudWatch ConfigCloud

TrailIAM Directory KMS

Application

SQS SWFApp

Stream

Elastic

TranscoderSES

Cloud

Search

SNS

Enterprise Applications

WorkSpaces WorkMail WorkDocs

Compute

EC2 ELBAuto

ScalingLambdaECS

Stay focused as you scale your team

AWSCloud-Based

Infrastructure

YourBusiness

More Time to Focus onYour Business

Configuring Your Cloud Assets

70%

30%70%

On-PremiseInfrastructure

30%

Managing All of the “Undifferentiated Heavy Lifting”

Day 6 – Growing Fast

Scaling Relational DBs

Increase RDS instance specs• Larger instance type

• More storage / more PIOPS

Read Replicas (Master – Slave)• Scale out beyond capacity of single DB instance

• Available in Amazon RDS for MySQL, PostgreSQL and Amazon Aurora

• Replication lag

• Writes => master

• Reads with tolerance to stale data => read replica (slave)

• Reads with need for most recent data => master

Scaling the DB

Web s erver

Web s erver

Web s erver

Web s erver

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



RDS DBs tandby


Scaling the DB

Web s erver

Web s erver

Web s erver

Web s erver

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



RDS DBs tandby


RDS read replic a

Scaling the DB

Web s erver

Web s erver

Web s erver

Web s erver

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om



RDS DBs tandby


RDS read replic a

RDS read replic a

What if your app is write-heavy?

Challenge: You will eventually hit the write throughput or

storage limit of the master node

Solutions:

Federation (splitting into multiple DBs based on function)

Sharding (splitting one data set up across multiple hosts)

Database federation• Split up databases by

function/purpose

• Harder to do cross-function

queries

• Essentially delaying the need for

something like sharding/NoSQL

until much further down the line

• Won’t help with single huge

functions/tables

Forums DB

Users DB

Products

DB

Sharded horizontal scaling

• More complex at the

application layer

• ORM support can help

• No practical limit on

scalability

• Operation

complexity/sophistication

• Shard by function or key

space

• RDBMS or NoSQL

User ShardID

002345 A

002346 B

002347 C

002348 B

002349 A

Shard C

Shard B

Shard A

NoSQL data stores

Trade query & integrity features of Relational DBs for

• More flexible data model

• Horizontal scalability & predictable performance

DynamoDB

Provisioned read/write performance per table

Massive and Seamless Scale

Distributed system that can scale both reads and writes

• Sharding + Replicas

Automatic partitioning:

• Data set size growth

• Provisioned capacity increases table

Amazon Route 53DNS servic eNo limit

Availability Zone a

RDS DB ins tanc e


Availability Zone b


www.example.c om


RDS DBstandby


RDS read replic a

RDS read replic a

DynamoDB

RDS read replic a


RDS read replic a

ElastiC ac henode 1

C loudS earc hLambdaS ES S Q S

Day 7 – Rest

Day 8 – Go Serverless!

Continuous

scaling

No servers to

manage

Never pay for idle

– no cold servers

(only happy

accountants)

Benefits of AWS Lambda

Amazon API Gateway: Serverless APIs

Internet

Mobile apps

Websites

Services

AWS Lambda

functions

AWS

API Gateway

cache

Endpoints on

Amazon EC2

Any other publicly

accessible endpointAmazon

CloudWatch

Amazon

CloudFrontAmazon

API Gateway

Summary

A quick review

Keep it simple and stateless

Make use of managed self-scaling services

Multi-AZ and AutoScale your EC2 infrastructure

Use the right DB for each workload

Cache data at multiple levels

Simplify operations with deployment tools

Next steps?

READ!

aws.amazon.com/documentation

aws.amazon.com/architecture

aws.amazon.com/start-ups

ASK FOR HELP!

forums.aws.amazon.com

aws.amazon.com/support

How to Scale to Millions of Users with AWS

Technology

Transcript of How to Scale to Millions of Users with AWS