Cloud Native: Designing Change-tolerant Software

@cdavisafc

CLOUD-NATIVEDESIGNING CHANGE-TOLERANT SOFTWARECornelia Davis, Sr. Director of Technology, Pivotal

@cdavisafc

TEXT

A Seattle book store deploys code, on average,

every second

@cdavisafc

TEXT

A Seattle technology company hosts applications for thousands

of major corporations

@cdavisafc

TEXTMajor Outage on 20 September 2015

Airbnb, Nest, IMDB and many more experienced downtime

… Including Netflix

Outage lasted 5+ hours

“brief availability blip” experienced a

@cdavisafc

TEXT

ME?

Developer (wasn’t Ops)

Web architectures for >10 years

Cloud-native for 5 years

Cloud Foundry for 5 years

More recently

Discount code 39% off!: ctwintitx

MICROSERVICES

It is not that simple

Mental Model

@cdavisafc

▸ Cloud-Native App (yup, the microservices)

APP APP

APP

APP

APP

@cdavisafc


▸ Cloud-Native Services APP APP

APP

APP

APP

@cdavisafc


▸ Cloud-Native Services (yup, some are apps)

SERVICE APP

SERVICE APP

SERVICE APP

SERVICE APP

APP

@cdavisafc


▸ Cloud-Native Services (yup, some are apps … and other are other types of services)

SERVICE APP

SERVICE APP

SERVICE APP

SERVICE APP

DATA DATA

DATA

APP

@cdavisafc



▸ Cloud-Native Data (a distributed data fabric)

SERVICE APP

SERVICE APP

SERVICE APP

SERVICE APP

DATA DATA

DATA

?

APP

@cdavisafc CLOUD-NATIVE SOFTWARE



▸ Cloud-Native Data (a distributed data fabric)

▸ The Cloud-Native Collective (implicit and explicit connections)

APP

SERVICE APP

SERVICE APP

SERVICE APP

SERVICE APP

DATA DATA

DATA

?

@cdavisafc

AGENDA (THE REST OF IT)

▸ Cloud-Native App

▸ Cloud-Native Services

▸ Cloud-Native Data

The Cloud-Native Collective

CLOUD-NATIVE APP

@cdavisafc

CLOUD-NATIVE APP

SCALE HORIZONTALLY

APP APP

APP

APP

APP

@cdavisafc

CLOUD-NATIVE APP

APP APP

APP

APP

APP

APP APP

APP

APP

APP

APP APP

APP

APP

APPSCALE HORIZONTALLY

APP APP

APP

APP

APP

@cdavisafc

CLOUD-NATIVE APP

APPAPP

APPAPPAPPAPPAPP

APPAPPAPPAPPAPP

APP APPAPP

APP

APP APPAPP

APP

APP APP

APP

APP


APP APP

APP

APP

APP

@cdavisafc

CLOUD-NATIVE APP

APPAPP

APPAPPAPPAPPAPP

APPAPPAPPAPPAPP

APP APPAPP

APP

APP APPAPP

APP

APP APP

APP

APP


▸ Need an abstraction to treat set of instances as one logical unit

▸ Load Balancing - DNS? Something else?

▸ Dynamic!!!

▸ IP Addresses of instances always changing

APP APP

APP

APP

APP

ROUT

ER

ROUT

ER

@cdavisafc

CLOUD-NATIVE APP

STATELESSNESS

ROUT

ER

APP (INSTANCE 1)

ValidTokens: [UserToken]

/login

token=UserToken

token=UserToken

@cdavisafc

CLOUD-NATIVE APP

STATELESSNESS

ROUT

ER

APP (INSTANCE 1)


/login

token=UserToken

token=UserToken

APP (INSTANCE 2)

ValidTokens: []

@cdavisafc

CLOUD-NATIVE APP

STATELESSNESS

ROUT

ER

APP (INSTANCE 1)


/login

token=UserToken

token=UserToken

APP (INSTANCE 2)Unauthorized ValidTokens: []

token=UserToken

@cdavisafc

CLOUD-NATIVE APP

STATELESSNESS

▸ Do NOT use sticky sessions!

▸ Eventually (and sooner than you think!) the instance you are sticky to will disappear

▸ Do make your instances completely stateless

ROUT

ER

APP (INSTANCE 1)


/login

token=UserToken

token=UserToken

APP (INSTANCE 2)Unauthorized ValidTokens: []

token=UserToken

@cdavisafc

CLOUD-NATIVE APP

STATELESSNESS

▸ Do NOT use sticky sessions!

▸ Eventually (and sooner than you think!) the instance you are sticky to will disappear

▸ Do make your instances completely stateless

▸ Externalize state

ROUT

ER

APP (INSTANCE 1)


token=UserToken

token=UserToken

APP (INSTANCE 2)

token=UserToken

STATE STORE

@cdavisafc

CLOUD-NATIVE APP

APP CONFIGURATION

▸ Property Files are the abstraction used to externalize configuration

▸ They do not hold actual values!

▸ Values coming from the environment are supplied via environment variables

▸ Other configuration is controlled just like source code and injected

RUNTIME CONTEXT WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES

PROPERTY FILE(S) DEFINE THE CONFIGURATION

PARAMETERS AND DEFAULT VALUES

hostIp= ${CF_INSTANCE_IP} specialization= ${SPECIALIZATION:

Science}

APP CODE THAT REFERENCES THE CONFIGURATION PARAMETERS

DEFINED IN THE PROPERTY FILE

public class HelloController { @Value("${hostIp}") private String hostIp; @Value("${specialization}") private String specialization; ... }

SYSTEM PROPERTIES PROVIDED BY THE RUNTIME ENVIRONMENT, VIA

ENVIRONMENT VARIABLES

CF_INSTANCE_IP=10.10.148.29

@cdavisafc

CLOUD-NATIVE APP

APP CONFIGURATION

▸ Other configuration is controlled just like source code and injected

RUNTIME CONTEXT WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES

PROPERTY FILE(S) DEFINE THE CONFIGURATION

PARAMETERS AND DEFAULT VALUES

hostIp= ${CF_INSTANCE_IP} specialization= ${SPECIALIZATION:

Science}

APP CODE THAT REFERENCES THE CONFIGURATION PARAMETERS

DEFINED IN THE PROPERTY FILE

public class HelloController { @Value("${hostIp}") private String hostIp; @Value("${specialization}") private String specialization; ... }

SYSTEM PROPERTIES PROVIDED BY THE RUNTIME ENVIRONMENT, VIA

ENVIRONMENT VARIABLES

CF_INSTANCE_IP=10.10.148.29

APP PROPERTIES STORED AND VERSIONED IN A

REPOSITORY SUCH AS GIT

specialization =Sports

CONFIG SERVER ROLE IS TO INJECT MANAGED APP

PROPERTIES INTO THE APP

@cdavisafc

CLOUD-NATIVE APP

APPLICATION LIFECYCLE

▸ Application lifecycle events have rippling effects through the collective

INVOICE APP ordersvcIP =10.24.1.35

ORDER SERVICE APP 10.24.1.3510.24.63.116

@cdavisafc

CLOUD-NATIVE APP

APPLICATION LIFECYCLE

▸ Application lifecycle events have rippling effects through the collective

▸ The application must broadcast lifecycle change events

▸ (Note: I strongly recommend you use a framework to help you with this!)

▸ And an app must be able to absorb that configuration at the right time (run time?)

INVOICE APP ordersvcIP =10.24.1.35

ORDER SERVICE APP

Some “magic” happens

10.24.63.116

Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …

I’ll adapt to these changes! Orders Service new IP: 10.24.63.116

CLOUD-NATIVE SERVICE

@cdavisafc


LET’S TALK ABOUT AUTONOMY

A major benefit of cloud-native architectures is independent:

▸ Application Scaling

▸ Team Scaling

▸ Development Cycles

▸ Experimentation

▸ Resilience

@cdavisafc


VERSIONED SERVICES

▸ Use Semantic Versioning

▸ major.minor.patch

▸ Use Blue/Green deploys

▸ Deployments needn’t be replacements

▸ Powerful lever in making deployments a non-event

SERVICE APP

SERVICE APP

APP

SERVICE APP

APP

… AND PARALLEL DEPLOYS

@cdavisafc


VERSIONED SERVICES

▸ Use Semantic Versioning

▸ major.minor.patch

▸ Use Blue/Green deploys

▸ Deployments needn’t be replacements

▸ Powerful lever in making deployments a non-event

SERVICE APP

SERVICE APP

APP

SERVICE APP

APP

… AND PARALLEL DEPLOYS

SERVICE APP

1.0.0 2.0.0

@cdavisafc


SERVICE DISCOVERY

▸ Let’s dig into the “magic” (it’s not really magic)

INVOICE APP orderSvcCoords...

ORDER SERVICE APP 10.24.63.116



Some “magic” happens

@cdavisafc


SERVICE DISCOVERY

▸ Dynamic Router maintains routing table

▸ Messaging pattern to decouple service from router

▸ Must handle lost broadcast messages

▸ Request pass through router INVOICE APP orderSvcCoords =orderSvc.example.com



ROUTER

OrderService: 10.24.63.116, … CustomerService: 10.24.3.1, …

@cdavisafc


SERVICE DISCOVERY




▸ Request pass through router INVOICE APP orderSvcCoords =10.24.1.13



SERVICE DISCOVERY SERVER


ORDER SERVICE APP

10.24.1.13


@cdavisafc


SERVICE DISCOVERY




▸ Request pass through router INVOICE APP orderSvcCoords =10.24.63.116



SERVICE DISCOVERY SERVER


ORDER SERVICE APP

10.24.1.13


@cdavisafc


DISTRIBUTED TRACING

@cdavisafc


@cdavisafc


DISTRIBUTED TRACING

▸ App must look for and leave breadcrumbs

▸ Use a framework to help you do this!!!

▸ Tools can then read and correlate logs

SERVICE APP

SERVICE APPAPP

SERVICE APP

SERVICE APP

SERVICE APP

CLOUD-NATIVE DATA

@cdavisafc

CLOUD-NATIVE DATA

WE ARE LOOSELY COUPLED!

@cdavisafc

CLOUD-NATIVE DATA

OR ARE WE?

@cdavisafc

CLOUD-NATIVE DATA

THIS IS NOT CLOUD-NATIVE DATA

@cdavisafc

CLOUD-NATIVE DATA

DATA APIS (SERVICES)

▸ Microservices do not access data layer directly

▸ Except for those that implement the data API

▸ A surface area to:

▸ Implement access control

▸ Implement throttling

▸ Perform logging

▸ Other policies…

@cdavisafc

CLOUD-NATIVE DATA

VERSIONED DATA APIS (SERVICES)

▸ We are already familiar with versioned microservices…

V1 V2

@cdavisafc

CLOUD-NATIVE DATA

VERSIONED DATA APIS (SERVICES)

▸ We are already familiar with versioned microservices…

… andV1 V2

PARALLEL DEPLOYS

@cdavisafc

CLOUD-NATIVE DATA

ANTI-PATTERN: DATA APIS THAT JUST PROXY

@cdavisafc

CLOUD-NATIVE DATA

EVERY MICROSERVICE NEEDS A CACHE

@cdavisafc

CLOUD-NATIVE DATA

“Caching at Netflix: The Hidden Microservice” https://www.youtube.com/watch?v=Rzdxgx3RC0Q

https://www.youtube.com/watch?v=Rzdxgx3RC0Q

@cdavisafc

CLOUD-NATIVE DATA

DATABASE PER MICROSERVICES

@cdavisafc

CLOUD-NATIVE DATA

DATABASE PER MICROSERVICES

▸ Supports Polyglot persistence

▸ Independent availability, backup/restore, access patterns, etc.

@cdavisafc

CLOUD-NATIVE DATA

EVENT LOG AS A SYSTEM OF RECORD

Unified LogSource of Record

@cdavisafc

CLOUD-NATIVE

▸ Cloud-native Apps

▸ Scale out

▸ Statelessness

▸ Externalize configuration

▸ Implications from changes in the application lifecycle

▸ Services

▸ Versioned services (autonomy)

▸ Service Discovery

▸ Distributed tracing

▸ Data

▸ Breaking the Data monolith

▸ Data APIs

▸ Caching

▸ Polyglot Persistence

▸ Event sourcing

THANK YOU

Discount code 39% off!: ctwintitx

Cloud Native: Designing Change-tolerant Software

Technology

Transcript of Cloud Native: Designing Change-tolerant Software