Build reactive systems on lambda

+ =

BUILDING REACTIVE SYSTEMS WITH

LAMBDA

hi, my name is Yan Cui@theburningmonk

“as a Software Engineer, what is my job?”

“to write code, of course!”- Me, circa 2006

“to provide value to users & stakeholders”

- Me, 2016

- Dan North

“lead time to someone saying thank you is the only reputation

metric that matters.”

THEBALANCE

COMPETING

FORCES

performance

security

scalabilitysimplicity

cost

team

time to market

“Programmers know the benefits of everything and the tradeoffs of nothing.”

- Rich Hickey

https://www.infoq.com/presentations/Simple-Made-Easy

https://www.infoq.com/presentations/Simple-Made-Easy

context is the king

ONLY ONE ZLATAN

security

complexity OUTSIDE the code

deployment

load balancing

caching

monitoring

config management

https://www.infoq.com/presentations/complexity-simplicity-esb

centralised logging

elastic scalingsetup server

https://www.infoq.com/presentations/complexity-simplicity-esb

Leaky Abstraction

AWS re:invent 2014https://www.youtube.com/watch?v=9eHoyUVo-yg

https://www.youtube.com/watch?v=9eHoyUVo-yg

AWS re:invent 2014https://www.youtube.com/watch?v=9eHoyUVo-yg

FAAS(Function-As-A-Service)

https://www.youtube.com/watch?v=9eHoyUVo-yg

THERE IS NO SERVER

cost saving

charged by invocations

not paying for idle servers

elastic scaling

hi guys,come back at 10 to find

out who won!

everyone came back at exactly 10pm!

everyone came back at exactly 10pm!

Lambda scaled instantly

these spikes happen to us

ALL THE TIME

minimise undifferentiated

heavy-lifting

“what about containers?”

important, butinvisible subsystems

economy of scale

NoOps!!

reactive

simple, fast deployment

first Lambda function went live

(team size : 4)

(team size : 7)

155 prod deployments!

- Dan North



easy to get started

now, we stand at 130 Lambda functions in prod, with 170 in dev

fuelling the Yubl platform evolution

My Beautiful Monolith

“What would a good

architecture for us look like?”

small, incremental deployments

fast deployments

no downtime

no lock-step deployments

features deployed independently

features loosely-coupled through messages

minimise cost for unused resources

minimise ops effort

oh, and do all these whilst continue

to deliver value, thanks :-)

love, your users

completely rebuilt search

Legacy Monolith Amazon Kinesis Amazon Lambda

Amazon CloudSearch


Amazon CloudSearchAmazon API Gateway Amazon Lambda

analytics pipeline


Google BigQuery


Google BigQuery

1 developer, 2 daysdesign production

(his 1st serverless project)


Google BigQuery“nothing ever got done

this fast at Skype!”

- Chris Twamley

- Dan North



Rebuiltwith Lambda

and many more…

“JavaScript doesn’t scale well with complexity”

- People on the Internet

“limit complexity, and JavaScript isn’t so bad”

- Me

TESTING

amzn.to/29Lxuzu

http://amzn.to/29Lxuzu

Level of Testing

1.Unitdo our objects do the right thing?are they easy to work with?

Level of Testing

1.Unit2.Integrationdoes our code work against code we can’t change?

handler

handler

test by invoking the handler

Level of Testing

1.Unit2.Integration3.Acceptancedoes the whole system work?

Level of Testing

unit

integration

acceptance

Level of Testing

unit

integration

acceptance

can do all 3 with Lambda

“…We find that tests that mock external libraries often need to be complex to get the code into the right state for the functionality we need to exercise.

The mess in such tests is telling us that the design isn’t right but, instead of fixing the problem by improving the code, we have to carry the extra complexity in both code and test…”

Don’t Mock Types You Can’t Change

“…The second risk is that we have to be sure that the behaviour we stub or mock matches what the external library will actually do…

Even if we get it right once, we have to make sure that the tests remain valid when we upgrade the libraries…”

Don’t Mock Types You Can’t Change

Don’t Mock Types You Can’t ChangeServices

“…Wherever possible, an acceptance test should exercise the system end-to-end without directly calling its internal code.

An end-to-end test interacts with the system only from the outside: through its interface…”

Testing End-to-End



Test Input



Test Input

Validate

“…We prefer to have the end-to-end tests exercise both the system and the process by which it’s built and deployed…

This sounds like a lot of effort (it is), but has to be done anyway repeatedly during the software’s lifetime…”

Testing End-to-End

Jenkins build config deploys and tests

unit + integration tests

deploy

acceptance tests

build.sh allows repeatable builds on both local & CI

LOGGING

2016-07-12T12:24:37.571Z 994f18f9-482b-11e6-8668-53e4eab441ae GOT is off air, what do I do now?

2016-07-12T12:24:37.571Z 994f18f9-482b-11e6-8668-53e4eab441ae GOT is off air, what do I do now?

UTC Timestamp API Gateway Request Id

your log message

organised by Function + Version

LOG OVERLOAD

centralise your logs

CloudWatch Logs AWS Lambda

LogStash ElasticSearch



AWS Elasticsearch



AWS Elasticsearch

Elastic Cloud



AWS Elasticsearch

Elastic Cloud

?

correlation IDs

MONITORING

nowhere to install monitoring agents

no time for background tasks

• Invocation Count• Error Count• Latency (avg, min, max, sum)• Throttling• Granular to the minute• Support custom metrics

• Same basic metrics• Better dashboard• Support custom metrics

https://www.datadoghq.com/blog/monitoring-lambda-functions-datadog/

https://www.datadoghq.com/blog/monitoring-lambda-functions-datadog/

Do It Yourself

PRO TIPset up dashboards

PRO TIPdon’t forget to set

up alarms

PRO TIPadd application-level

metrics

KEEP WARM

functions are unloaded if idle for a while

noticeable cold start time(package size matters)

CloudWatch Event AWS Lambda


ping

ping

ping

ping


ping

ping

ping

ping

HEALTH CHECKS?

KNOW YOUR LIMITS

max 50 MB deployment package size

max 50 MB deployment package sizemax 75 GB total deployment package size*

* limit is per AWS region

Janitor Monkey

Janitor Lambda

max 5 mins execution time

max 6 MB request payload size*

max 6 MB response payload size

* for a request-response event type

default max 100 concurrent executions*

* soft-limit, can be raised via support ticket

looking ahead

.Net core?SQS support?

MULTI-CLOUD FUTURE?

IBM OpenWhisk

Amazon Lambda Azure Web Functions

Google Cloud Functions

competition

faster innovation lower prices

@theburningmonk

@theburningmonktheburningmonk.comgithub.com/theburningmonk

@theburningmonk

@theburningmonktheburningmonk.comgithub.com/theburningmonk

slides at http://bit.ly/2ecqEql

http://bit.ly/2ecqEql

Build reactive systems on lambda

Technology

Transcript of Build reactive systems on lambda