@crichardson

Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsonchris@chrisrichardson.net http://plainoldobjects.com

@crichardson

Presentation goalHow decomposing applications

improves deployability and scalability

and simplifies the adoption of new

technologies

@crichardson

About Chris

@crichardson

(About Chris)

@crichardson

About Chris()

@crichardson

About Chris

@crichardson

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson

About Chris

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

How do services communicate?

API gateway design

Refactoring the monolith

@crichardson

Let’s imagine you are building an online store

@crichardson

Tomcat

Traditional web application architecture

Browser

WAR

MySQL Database

Review Service

Product InfoService

Recommendation Service

StoreFrontUI

developtest

deploy

Simple to

Apache/Load

balancer

scale

JSFSpring MVC

SpringHibernate

Order Service

@crichardson

But there are problems with a monolithic architecture

@crichardson

Users expect a rich, dynamic and interactive

experience

@crichardson

Browser

WAR

StoreFrontUI

Model

View Controller

Presentation layer evolution....

HTML / HTTP

Old style UI architecture isn’t good enough

@crichardson

Browser - desktop and mobile Web application

RESTfulEndpoints

Model

View Controller

...Presentation layer evolution

JSON-REST

HTML 5 - JavaScript

Native mobile clientIoS or Android

Event publisher

Events

Static content

No elaborate, server-side web framework required

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component

Interrupts long running background (e.g. Quartz) jobs

Increases risk of failure

Fear of change

Updates will happen less often - really long QA cycles

e.g. Makes A/B testing UI really difficult

Eggs in one basket

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Shipping team

Accounting Engineering

Obstacle to scaling development

E-commerce application

@crichardson

WAR

Review service

Product Info service

Recommendation service

StoreFrontUI

Reviews team

Product Info team

Recommendations team

UI team

Obstacle to scaling development

Order serviceOrders team

@crichardson

Lots of coordination and communication required

Obstacle to scaling development

I want to update the UI

But the backend is not working

yet!

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

How do services communicate?

API gateway design

Refactoring the monolith

@crichardson

@crichardson

The scale cube

X axis - horizontal duplication

Z axis

- data

partit

ioning

Y axis - functional

decomposition

Scale b

y split

ting s

imilar

thing

s

Scale by splitting

different things

@crichardson

Y-axis scaling - application level

WAR

ReviewService

Product InfoService

RecommendationService

StoreFrontUI

OrderService

@crichardson

Y-axis scaling - application level

Store front web application

reviews web application

recommendations web application

Apply X axis cloning and/or Z axis partitioning to each service

product info web application

ReviewService

Product InfoService

RecommendationService

StoreFrontUI

OrderService

orders web application

@crichardson

Partitioning strategies...

Partition by verb, e.g. shipping service

Partition by noun, e.g. inventory service

Single Responsibility Principle

Unix utilities - do one focussed thing well

@crichardson

Partitioning strategies

Too few

Drawbacks of the monolithic architecture

Too many - a.k.a. Nano-service anti-pattern

Runtime overhead

Potential risk of excessive network hops

Potentially difficult to understand system

Something of an art

@crichardson

Example micro-serviceclass RegistrationSmsServlet extends RegistrationSmsScalatraStack {

post("/") { val phoneNumber = request.getParameter("From") val registrationUrl = System.getenv("REGISTRATION_URL") + "?phoneNumber=" + encodeForUrl(phoneNumber) <Response> <Sms>To complete registration please go to {registrationUrl}</Sms> </Response> } }

For more on micro-services see @fgeorge52

@crichardson

Real world examples

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

@crichardson

There are drawbacks

@crichardson

Complexity

See Steve Yegge’s Google Platforms Rant re Amazon.com

@crichardson

Multiple databases &

Transaction management

@crichardson

Implementing features that span multiple services

@crichardson

When to use it?In the beginning: •You don’t need it •It will slow you down

Later on:•You need it•Refactoring is painful

@crichardson

But there are many benefits

@crichardson

Smaller, simpler apps

Easier to understand and develop

Reduced startup time - important for GAE

Less jar/classpath hell

Who needs OSGI?

@crichardson

Scales development: develop, deploy and scale each service independently

@crichardson

Improves fault isolation

@crichardson

Eliminates long-term commitment to a single technology stack

Modular, polyglot, multi-framework applications

@crichardson

Two levels of architectureSystem-level

ServicesInter-service glue: interfaces and communication mechanisms

Slow changing

Service-level

Internal architecture of each serviceEach service could use a different technology stack

Pick the best tool for the jobRapidly evolving

@crichardson

Easily try other technologies

... and fail safely

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

How do services communicate?

API gateway design

Refactoring the monolith

@crichardson

Inter-service communication options

Synchronous HTTP ⇔ asynchronous AMQP

Formats: JSON, XML, Protocol Buffers, Thrift, ...

Asynchronous is preferredJSON is fashionable but binary format

is more efficient

@crichardson

StoreFrontUI

Product Infoservice

Recommendationservice

Review service

RabbitMQ(Message Broker)

Asynchronous message-based communication

Order service

Pros and cons of messagingPros

Decouples client from server

Message broker buffers messages

Supports a variety of communication patterns

Cons

Additional complexity of message broker

Request/reply-style communication is more complex

Spring IntegrationProvides the building blocks for a pipes and filters architecture

Enables development of application components that are

loosely coupled

insulated from messaging infrastructure

Messaging defined declaratively

@crichardson

Order Service

Messaging Gateway

Channel Service Activator

Shipping service

Development time: same JVM@Servicepublic class OrderServiceImpl {

@Autowiredprivate ShippingService shippingService;

public void placeOrder() { String orderId = generateOrderId(); … shippingService.shipOrder(orderId);}

}

@Servicepublic class ShippingServiceImpl {

public void shipOrder(String orderId) { ....}

}

shipOrder()

@crichardson

Order Service

Messaging Gateway

Channel Service Activator

Shipping service

AMQP

RabbitMQ

AMQP Channel

Test and production: distributed

Application code is unchanged

@crichardson

Synchronous REST

Recommendationservice

StoreFrontUI

Product Infoservice

Review service

REST

... Order service

Pros and cons of RESTPros

Simple and familiar

Request/reply is easy

Firewall friendly

No intermediate broker

Cons

Only supports request/reply

Server must be available

Client needs to know URL(s) of server(s)

@crichardson

Spring MVC makes REST easy@Controllerpublic class AccountController {

@Autowired private MoneyTransferService moneyTransferService; @RequestMapping(value = "/accounts/{accountId}", method = RequestMethod.GET) @ResponseBody public AccountInfo getAccount(@PathVariable String accountId) { Account account = moneyTransferService.findAccountByid(accountId); return makeAccountInfo(account); }

@RequestMapping(value = "/accounts", method = RequestMethod.POST) @ResponseStatus( HttpStatus.CREATED ) public void createAccount(@RequestBody AccountInfo accountInfo, UriComponentsBuilder builder, HttpServletResponse response) { ... }

URL matching &

destructuringobject ⇒XML/JSON

XML/JSON ⇒

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

How do services communicate?

API gateway design

Refactoring the monolith

@crichardson

Use an API gatewayBrowser

Model

View Controller

HTML 5 - JavaScript

Product Infoservice

RecommendationService

Reviewservice

REST

REST

AMQP

APIGateway

Model

View Controller

Native App

Single entry point

Client specific APIs

Protocol translation

@crichardson

Netflix API Gateway

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Device specific end points

@crichardson

Developing an API gateway

Java EE web technologies

Netty-based technology stack

Non-Java options: e.g. Node.JS

@crichardson

API gateway design issues

@crichardson

The need for parallelism

ProductDetails

API

Product Info

Recommendations

Reviews

getProductDetails()

getRecomendations()

getReviews()

Call in parallel

get ProductDetails

@crichardson

Futures are a great concurrency abstraction

An object that will contain the result of a concurrent computationhttp://en.wikipedia.org/wiki/Futures_and_promises

Future<Integer> result = executorService.submit(new Callable<Integer>() {... });

Java has basic futures. We can do much better....

@crichardson

Better: Futures with callbacks

val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }

Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures

@crichardson

Even better: Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }

val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))

val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))

def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))

Scala Futures

Transforms Future

Combines two futures

Transforms list of futures to a future containing a list

@crichardson

Composing concurrent requests using Scala Futures

class ProductDetailsService @Autowired()(....) {

def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)

for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }

}

Asynchronously creates a Future containing result

@crichardson

Handling partial failures

ProductDetails

Controller

Product Info

Recommendations

Reviews

getProductDetails()

getRecomendations()

getReviews()

get ProductDetails X

@crichardson

About Netflix> 1B API calls/day

1 API call ⇒ average 6 service calls

Fault tolerance is essential

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

How to run out of threads

Tomcat

Execute thread pool

HTTP Request

Thread 1

Thread 2

Thread 3

Thread n

Service A Service B

If service B is down then thread

will be blocked

XXXXX

Eventually all threads will be blocked

@crichardson

Their approachNetwork timeouts and retries

Invoke remote services via a bounded thread pool

Use the Circuit Breaker pattern

On failure:

return default/cached data

return error to caller

https://github.com/Netflix/Hystrix

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

How do services communicate?

API gateway design

Refactoring the monolith

@crichardson

How do you decompose your 10 M LOC monolithic

application?

@crichardson

Strategy #1: Stop digging

@crichardson

New functionality = service

Monolith ServiceAnti-corruption layer

Glue code

Pristine

@crichardson

Sounds simple but...

Dependencies between monolith and service

e.g. Common entities

Building the anti-corruption layer can be challenging

Must replicate data between systems

...

@crichardson

Strategy #2: extract services

@crichardson

Module ⇒ service ...

WAR

Module

@crichardson

... Module ⇒ service

WAR ServiceAnti-corruption layer

@crichardson

What to extract?

Have the ideal partitioned architecture in mind:

Partitioned by verb or noun

Start with troublesome modules:

Frequently updated

Stateful components that prevent clustering

Conflicting resource requirements

@crichardson

Service

Domain model 2

Monolith

Untangling dependencies

API A API B API C

Domain model 1

A C

B

X

Z

Y

Trouble!

@crichardson

Useful idea: Bounded contextDifferent services have a different view of a domain object, e.g.

User Management = complex view of user

Rest of application: User = PK + ACL + Name

Different services can have a different domain model

Services exchange messages to synchronize data

@crichardson

Customer management

Untangling orders and customers

Order management

Order Service

placeOrder()

Customer Service

availableCredit()updateCustomer()

Customer

creditLimit...

has ordersbelongs toOrder

total

Invariant:sum(order.total) <= creditLimit

available credit= creditLimit - sum(order.total)Trouble!

@crichardson

Customer management

Replicating the credit limit

Order management

Order Service

placeOrder()

Customer

creditLimit...

Order

total

Customer’

creditLimit

CreditLimitChangedEvent

sum(order.total) <= creditLimit

Customer Service

updateCustomer()

Simplified

@crichardson

Customer management

Maintaining the openOrderTotal

Order management

Order Service

placeOrder()

Customer Service

availableCredit()

Customer

creditLimit

...

Order

customerIdtotal

= creditLimit - openOrderTotal

OpenOrderTotalUpdated

openOrderTotal

@crichardson

Refactoring a monolith is not easy

BUT

the alternative is far worse

@crichardson

Summary

@crichardson

Monolithic applications are simple to develop and deploy

BUT have significant drawbacks

@crichardson

Apply the scale cube

Modular, polyglot, and scalable applications

Services developed, deployed and scaled independently

@crichardson

Use a modular, polyglot architecture

Browser

Model

View Controller

HTML 5 - JavaScript

Product Infoservice

RecommendationService

Reviewservice

REST

REST

AMQP

Front-end server

Model

View Controller

Native App

APIGateway

Event delivery

@crichardson

Questions?

@crichardson chris@chrisrichardson.net

http://plainoldobjects.com