AWS Summit 2013 Tel Aviv Oct 16 – Tel Aviv, Israel

Carlos Conde – Sr. Mgr. Solutions Architecture

Thomas Metschke – Technical Program Manager AWS OpsWorks

BACK-END USE CASES

PRESENTATION

TIER (FRONT-END)

APPLICATION

TIER (BACK-END)

INFORMATION

TIER (DATABASES)

PRESENTATION

TIER (FRONT-END)

APPLICATION

TIER (BACK-END)

INFORMATION

TIER (DATABASES)

CONTENT DELIVERY

NETWORK

AMAZON

CLOUDFRONT

WEB APPLICATION

FRONT-END AWS ELASTIC

BEANSTALK

AMAZON

DYNAMODB

DATA STORE

AMAZON S3

VIDEO FILES

STORE

#1WORKFLOW

SPAM?

CHECK

VIDEO

TOO

LONG?

CROP

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

INDEX

& NOTIFY

STOP REJECT

SPAM?

CHECK

VIDEO

TOO

LONG?

CROP

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

INDEX

& NOTIFY

STOP REJECT

SPAM?

CHECK

VIDEO

TOO

LONG?

CROP

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

INDEX

& NOTIFY

STOP REJECT

TASKS

DECISIONS

HISTORY

TASKS

DECISIONS

HISTORY

STATELESS !

STATELESS SCALES

HORIZONTALLY

AMAZON SWF ENABLES RESILIENT, SCALABLE,

DISTRIBUTED WORKFLOWS

AMAZON SWF

SERVICE

SWF DECIDER

SWF WORKER

DECIDERS COORDINATION LOGIC

1. Poll for work on a decision list Long polling

2. Evaluate workflow execution history SWF sends full history in JSON format

3. Return decision to Amazon SWF Usually scheduling another task

WORKERS EXECUTION LOGIC

1. Poll for work on a specific task list Long polling

2. Execute works, send heartbeats SWF sends input data from deciders

3. Return success / failure Detailed data can be provided to deciders

AMAZON SWF TRACKS:

Execution Time to start, time to finish, …

Time to finish for overall workflow

Timeouts controlled for each of these (and more)

Heartbeats for long-running activities (optional)

Decider is informed of timeouts Schedule retries, “mitigation” strategies or cleanup tasks

NO NEW LANGUAGE

TO LEARN

YOUR CODE IS YOUR WORKFLOW LANGUAGE

AMAZON SWF MAINTAINS STATE

ALL HORIZONTAL SCALING

PATTERNS APPLY

#2TRANSCODING

AMAZON

ELASTIC TRANSCODER SCALABLE, EASY TO USE AND A COST EFFECTIVE

SD $0.015 / MINUTE Resolution of less than 720p

HD $0.030 / MINUTE Resolution of more than 720p

Transcode up to 20 minutes of content each month for free.

AMAZON ELASTIC

TRANSCODER

#3SEARCH

AMAZON

CLOUDSEARCH ENABLES RESILIENT, SCALABLE,

DISTRIBUTED WORKFLOWS

select *

from articles

where content

like ‘%cloud%’

Relevance &

Ranking

Faceting

Field

Search

Range Search

+

search-mydomain.us-east-1.cloudsearch.amazonaws.com

doc-mydomain.us-east-1.cloudsearch.amazonaws.com

Endpoints

+

HTTP POST author=carlos;

date=2013-02-07;

comment=text;

SCALABLE SEARCH

Small Instance Partition 1

Copy 1

Time: 1800h

<80% CPU

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

>80% CPU

Small Instance Partition 1

Copy 2

Time: 2000h

Small Instance Partition 1

Copy 1

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

>80% CPU

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 3

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Time: 2200h

Small Instance Partition 1

Copy 1

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 3

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2 <30% CPU

Small Instance Partition 1

Copy 1

Time: 0000h

Small Instance Partition 1

Copy 1

Cost savings

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 3

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 1

Traditional required capacity

Small Instance Partition 1

Copy 1

Cost savings

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 3

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 1

Traditional required capacity

Elastic Capacity

Small Instance Partition 1

Copy 1

Cost savings

Elastic Search

Time

Requests

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 3

Small Instance Partition 1

Copy 1

Small Instance Partition 1

Copy 2

Small Instance Partition 1

Copy 1

Traditional required capacity

Elastic Capacity

Savings Savings

Cloud Search

Search Instance Partition 1

Copy 1

Search Instance Partition 1

Copy 2

Search Instance Partition 1

Copy n

Traffic Request

volume & complexity

Data Document quantity & size

Search Instance Partition 1

Copy 1

Search Instance Partition 2

Copy 1

Search Instance Partition n

Copy 1

Cloud Search

Search Instance Partition 1

Copy 1

Search Instance Partition 2

Copy 1

Search Instance Partition n

Copy 1

Search Instance Partition 1

Copy 2

Search Instance Partition 2

Copy 2

Search Instance Partition n

Copy 2

Search Instance Partition 1

Copy n

Search Instance Partition 2

Copy n

Search Instance Partition n

Copy n

Traffic Request

volume & complexity

Data Document quantity & size Cloud

Search

AMAZON

CLOUDSEARCH DOCUMENT

ENDPOINT

SEARCH

ENDPOINT

#4NOTIFICATIONS

AMAZON SES BULK AND TRANSACTIONAL

EMAIL-SENDING SERVICE

$0.10 PER 1000 EMAILS

+ $0.12 per GB of attachments

AMAZON SNS MOBILE PUSH NOTIFICATIONS

#5ENDPOINT

DIRECT API CALLS TO AWS

iOS Android

// Create Amazon S3 Client

AmazonS3Client *s3 = [[AmazonS3Client alloc]

initWithAccessKey:ACCESS_KEY_ID withSecretKey:SECRET_KEY];

// Create Amazon S3 Client

AmazonS3Client s3 = new AmazonS3Client(

new BasicAWSCredentials( ACCESS_KEY_ID, SECRET_KEY ) );

// Put an Object into a Bucket

S3PutObjectRequest *por = [[S3PutObjectRequest alloc]

initWithKey:VIDEO_NAME inBucket:VIDEO_BUCKET];

por.contentType = @“mpeg4";

por.data = video;

[s3 putObject:por];

// Put an Object into a Bucket

PutObjectRequest por =

new PutObjectRequest(VIDEO_BUCKET,

VIDEO_NAME,

new java.io.File( video ) );

s3.putObject( por );

// Get an Object from a Bucket

S3GetObjectRequest *gor = [[S3GetObjectRequest alloc]

initWithKey:VIDEO_NAME withBucket:VIDEO_BUCKET];

S3GetObjectResponse *response = [s3 getObject:gor];

// Get an Object from a Bucket

S3Object data = s3.getObject(VIDEO_BUCKET, VIDEO_NAME);

IAM STS SECURITY TOKEN SERVICE

IAM STS

FOCUS ON YOUR

CORE BUSINESS

AWS Summit 2013 Tel Aviv Oct 16 – Tel Aviv, Israel

Thomas Metschke – Technical Program Manager AWS OpsWorks

AWS OPSWORKS