©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

Aserverlessdatapipeline JulienSimon,PrincipalTechnicalEvangelist,AWS

julsimon@amazon.fr@julsimon

WernerVogels,CTO,Amazon.comAWSre:Invent2015

Managedservices+

AWSLambda=

Serverlessarchitecture

AWSLambda

•  DeploypurefuncJonsinJava,PythonandNode.js

•  WorksnicelywithAWSmanagedservices:AmazonS3,AmazonDynamoDB,etc.

•  Buildevent-drivenapplicaJons•  BuildRESTfulAPIsinconjuncJonwithAmazonAPIGateway•  Payasyougo:numberofrequests+execuJonJme(100msslots)

Anotherwaytoputit…

TimWagner,GeneralManager,AWSLambdaServerlessconference,NYC,May2016

AdRoll h^ps://www.youtube.com/watch?v=JFfvD2cw2IEh^ps://aws.amazon.com/fr/dynamodb/adtech/

60billionReal-TimeBiddingeventsdaily

Buildingaserverlessdatapipeline

Lambda

DynamoDB

Kinesis Firehose

API Gateway

HTTP POST /prod/logger writeTo

Kinesis

DynamoDBToFirehose

S3

eventTable

DynamoDB streams

bucket

EMR, Redshift,

… firehoseToS3

Kinesis Streams Lambda Lambda

KinesisToDynamoDB

Web apps

CLI:createDynamoDBtable

aws dynamodb create-table \--table-name eventTable \--attribute-definitions \AttributeName=userId,AttributeType=N \AttributeName=timestamp,AttributeType=N \--key-schema \AttributeName=userId,KeyType=HASH \AttributeName=timestamp,KeyType=RANGE \--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \--stream-specification StreamEnabled=true,StreamViewType=NEW_IMAGE

eventTable

DynamoDB

CLI:createLambdafuncIonaws lambda create-function \--function-name DynamoDBToFirehose \--role DYNAMODBTOFIREHOSE_ROLE_ARN \--zip-file fileb://DynamoDBToFirehose.zip \--handler DynamoDBToFirehose.lambda_handler \--runtime python2.7 \--memory-size 128 \--description "Write DynamoDB stream to Kinesis Firehose"

aws lambda create-event-source-mapping \--function-name DynamoDBToFirehose \--event-source DYNAMODB_STREAM_ARN \--batch-size 10 \--starting-position TRIM_HORIZON

Lambda

DynamoDBToFirehose

eventTable

DynamoDB streams

DynamoDB

CLI:createKinesisFirehosestreamaws firehose create-delivery-stream \--delivery-stream-name firehoseToS3 \--s3-destination-configuration \RoleARN=FIREHOSETOS3_ROLE_ARN, \BucketARN="arn:aws:s3:::jsimon-public", \Prefix="firehose", \BufferingHints=\{SizeInMBs=1,IntervalInSeconds=60\}, \CompressionFormat="GZIP", \EncryptionConfiguration={NoEncryptionConfig="NoEncryption"}

Kinesis Firehose

firehoseToS3

Lambda

DynamoDBToFirehose

eventTable

DynamoDB streams

DynamoDB

Buildingaserverlessdatapipeline

Lambda

DynamoDB

Kinesis Firehose

API Gateway

HTTP POST /prod/logger writeTo

Kinesis

DynamoDBToFirehose

S3

eventTable

DynamoDB streams

bucket

EMR, Redshift,

… firehoseToS3

Kinesis Streams Lambda Lambda

KinesisToDynamoDB

Web apps

Lines of code: 16 Number of servers: zero Performance & scalability: maximum

ReadyforsometesIng?

hMp://api.julien.org

UpcomingbookonAWSLambda

Wri^enbyAWSTechnicalEvangelistDaniloPocciaEarlyreleaseavailableat:h^ps://www.manning.com/books/aws-lambda-in-acJon

Togofurther

h^p://aws.amazon.com/lambda/h^ps://aws.amazon.com/blogs/compute/AWSUserGroupTurkeyh^p://www.meetup.com/AWS-User-Group-Turkey/Qwiklabs:h^ps://qwiklabs.comServerlessframework:h^ps://github.com/serverless/

Teşekkür ederim J

JulienSimon,PrincipalTechnicalEvangelist,AWSjulsimon@amazon.fr

@julsimon