Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability
-
Upload
chris-richardson -
Category
Technology
-
view
2.481 -
download
1
description
Transcript of Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability
@crichardson
Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected] http://plainoldobjects.com
@crichardson
Presentation goalHow decomposing applications
improves deployability and scalability
and simplifies the adoption of new
technologies
@crichardson
About Chris
@crichardson
(About Chris)
@crichardson
About Chris()
@crichardson
About Chris
@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson
About Chris
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
API gateway design
Refactoring the monolith
@crichardson
Let’s imagine you are building an online store
@crichardson
Tomcat
Traditional web application architecture
Browser
WAR
MySQL Database
Review Service
Product InfoService
Recommendation Service
StoreFrontUI
developtest
deploy
Simple to
Apache/Load
balancer
scale
JSFSpring MVC
SpringHibernate
Order Service
@crichardson
But there are problems with a monolithic architecture
@crichardson
Users expect a rich, dynamic and interactive
experience
@crichardson
Browser
WAR
StoreFrontUI
Model
View Controller
Presentation layer evolution....
HTML / HTTP
Old style UI architecture isn’t good enough
@crichardson
Browser - desktop and mobile Web application
RESTfulEndpoints
Model
View Controller
...Presentation layer evolution
JSON-REST
HTML 5 - JavaScript
Native mobile clientIoS or Android
Event publisher
Events
Static content
No elaborate, server-side web framework required
@crichardson
Intimidates developers
@crichardson
Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in one basket
@crichardson
Overloads your IDE and container
Slows down development
@crichardson
Shipping team
Accounting Engineering
Obstacle to scaling development
E-commerce application
@crichardson
WAR
Review service
Product Info service
Recommendation service
StoreFrontUI
Reviews team
Product Info team
Recommendations team
UI team
Obstacle to scaling development
Order serviceOrders team
@crichardson
Lots of coordination and communication required
Obstacle to scaling development
I want to update the UI
But the backend is not working
yet!
@crichardson
Requires long-term commitment to a technology stack
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
API gateway design
Refactoring the monolith
@crichardson
@crichardson
The scale cube
X axis - horizontal duplication
Z axis
- data
partit
ioning
Y axis - functional
decomposition
Scale b
y split
ting s
imilar
thing
s
Scale by splitting
different things
@crichardson
Y-axis scaling - application level
WAR
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
@crichardson
Y-axis scaling - application level
Store front web application
reviews web application
recommendations web application
Apply X axis cloning and/or Z axis partitioning to each service
product info web application
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
orders web application
@crichardson
Partitioning strategies...
Partition by verb, e.g. shipping service
Partition by noun, e.g. inventory service
Single Responsibility Principle
Unix utilities - do one focussed thing well
@crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art
@crichardson
Example micro-serviceclass RegistrationSmsServlet extends RegistrationSmsScalatraStack {
post("/") { val phoneNumber = request.getParameter("From") val registrationUrl = System.getenv("REGISTRATION_URL") + "?phoneNumber=" + encodeForUrl(phoneNumber) <Response> <Sms>To complete registration please go to {registrationUrl}</Sms> </Response> } }
For more on micro-services see @fgeorge52
@crichardson
Real world examples
http://highscalability.com/amazon-architecture
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
@crichardson
There are drawbacks
@crichardson
Complexity
See Steve Yegge’s Google Platforms Rant re Amazon.com
@crichardson
Multiple databases &
Transaction management
@crichardson
Implementing features that span multiple services
@crichardson
When to use it?In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
@crichardson
But there are many benefits
@crichardson
Smaller, simpler apps
Easier to understand and develop
Reduced startup time - important for GAE
Less jar/classpath hell
Who needs OSGI?
@crichardson
Scales development: develop, deploy and scale each service independently
@crichardson
Improves fault isolation
@crichardson
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-framework applications
@crichardson
Two levels of architectureSystem-level
ServicesInter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stack
Pick the best tool for the jobRapidly evolving
@crichardson
Easily try other technologies
... and fail safely
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
API gateway design
Refactoring the monolith
@crichardson
Inter-service communication options
Synchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Asynchronous is preferredJSON is fashionable but binary format
is more efficient
@crichardson
StoreFrontUI
Product Infoservice
Recommendationservice
Review service
RabbitMQ(Message Broker)
Asynchronous message-based communication
Order service
Pros and cons of messagingPros
Decouples client from server
Message broker buffers messages
Supports a variety of communication patterns
Cons
Additional complexity of message broker
Request/reply-style communication is more complex
Spring IntegrationProvides the building blocks for a pipes and filters architecture
Enables development of application components that are
loosely coupled
insulated from messaging infrastructure
Messaging defined declaratively
@crichardson
Order Service
Messaging Gateway
Channel Service Activator
Shipping service
Development time: same JVM@Servicepublic class OrderServiceImpl {
@Autowiredprivate ShippingService shippingService;
public void placeOrder() { String orderId = generateOrderId(); … shippingService.shipOrder(orderId);}
}
@Servicepublic class ShippingServiceImpl {
public void shipOrder(String orderId) { ....}
}
shipOrder()
@crichardson
Order Service
Messaging Gateway
Channel Service Activator
Shipping service
AMQP
RabbitMQ
AMQP Channel
Test and production: distributed
Application code is unchanged
@crichardson
Synchronous REST
Recommendationservice
StoreFrontUI
Product Infoservice
Review service
REST
... Order service
Pros and cons of RESTPros
Simple and familiar
Request/reply is easy
Firewall friendly
No intermediate broker
Cons
Only supports request/reply
Server must be available
Client needs to know URL(s) of server(s)
@crichardson
Spring MVC makes REST easy@Controllerpublic class AccountController {
@Autowired private MoneyTransferService moneyTransferService; @RequestMapping(value = "/accounts/{accountId}", method = RequestMethod.GET) @ResponseBody public AccountInfo getAccount(@PathVariable String accountId) { Account account = moneyTransferService.findAccountByid(accountId); return makeAccountInfo(account); }
@RequestMapping(value = "/accounts", method = RequestMethod.POST) @ResponseStatus( HttpStatus.CREATED ) public void createAccount(@RequestBody AccountInfo accountInfo, UriComponentsBuilder builder, HttpServletResponse response) { ... }
URL matching &
destructuringobject ⇒XML/JSON
XML/JSON ⇒
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
API gateway design
Refactoring the monolith
@crichardson
Use an API gatewayBrowser
Model
View Controller
HTML 5 - JavaScript
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
APIGateway
Model
View Controller
Native App
Single entry point
Client specific APIs
Protocol translation
@crichardson
Netflix API Gateway
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific end points
@crichardson
Developing an API gateway
Java EE web technologies
Netty-based technology stack
Non-Java options: e.g. Node.JS
@crichardson
API gateway design issues
@crichardson
The need for parallelism
ProductDetails
API
Product Info
Recommendations
Reviews
getProductDetails()
getRecomendations()
getReviews()
Call in parallel
get ProductDetails
@crichardson
Futures are a great concurrency abstraction
An object that will contain the result of a concurrent computationhttp://en.wikipedia.org/wiki/Futures_and_promises
Future<Integer> result = executorService.submit(new Callable<Integer>() {... });
Java has basic futures. We can do much better....
@crichardson
Better: Futures with callbacks
val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }
Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures
@crichardson
Even better: Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))
Scala Futures
Transforms Future
Combines two futures
Transforms list of futures to a future containing a list
@crichardson
Composing concurrent requests using Scala Futures
class ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }
}
Asynchronously creates a Future containing result
@crichardson
Handling partial failures
ProductDetails
Controller
Product Info
Recommendations
Reviews
getProductDetails()
getRecomendations()
getReviews()
get ProductDetails X
@crichardson
About Netflix> 1B API calls/day
1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson
How to run out of threads
Tomcat
Execute thread pool
HTTP Request
Thread 1
Thread 2
Thread 3
Thread n
Service A Service B
If service B is down then thread
will be blocked
XXXXX
Eventually all threads will be blocked
@crichardson
Their approachNetwork timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
https://github.com/Netflix/Hystrix
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
API gateway design
Refactoring the monolith
@crichardson
How do you decompose your 10 M LOC monolithic
application?
@crichardson
Strategy #1: Stop digging
@crichardson
New functionality = service
Monolith ServiceAnti-corruption layer
Glue code
Pristine
@crichardson
Sounds simple but...
Dependencies between monolith and service
e.g. Common entities
Building the anti-corruption layer can be challenging
Must replicate data between systems
...
@crichardson
Strategy #2: extract services
@crichardson
Module ⇒ service ...
WAR
Module
@crichardson
... Module ⇒ service
WAR ServiceAnti-corruption layer
@crichardson
What to extract?
Have the ideal partitioned architecture in mind:
Partitioned by verb or noun
Start with troublesome modules:
Frequently updated
Stateful components that prevent clustering
Conflicting resource requirements
@crichardson
Service
Domain model 2
Monolith
Untangling dependencies
API A API B API C
Domain model 1
A C
B
X
Z
Y
Trouble!
@crichardson
Useful idea: Bounded contextDifferent services have a different view of a domain object, e.g.
User Management = complex view of user
Rest of application: User = PK + ACL + Name
Different services can have a different domain model
Services exchange messages to synchronize data
@crichardson
Customer management
Untangling orders and customers
Order management
Order Service
placeOrder()
Customer Service
availableCredit()updateCustomer()
Customer
creditLimit...
has ordersbelongs toOrder
total
Invariant:sum(order.total) <= creditLimit
available credit= creditLimit - sum(order.total)Trouble!
@crichardson
Customer management
Replicating the credit limit
Order management
Order Service
placeOrder()
Customer
creditLimit...
Order
total
Customer’
creditLimit
CreditLimitChangedEvent
sum(order.total) <= creditLimit
Customer Service
updateCustomer()
Simplified
@crichardson
Customer management
Maintaining the openOrderTotal
Order management
Order Service
placeOrder()
Customer Service
availableCredit()
Customer
creditLimit
...
Order
customerIdtotal
= creditLimit - openOrderTotal
OpenOrderTotalUpdated
openOrderTotal
@crichardson
Refactoring a monolith is not easy
BUT
the alternative is far worse
@crichardson
Summary
@crichardson
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
@crichardson
Apply the scale cube
Modular, polyglot, and scalable applications
Services developed, deployed and scaled independently
@crichardson
Use a modular, polyglot architecture
Browser
Model
View Controller
HTML 5 - JavaScript
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
Front-end server
Model
View Controller
Native App
APIGateway
Event delivery