Post on 08-May-2015
description
Decomposing applications for deployability and scalability
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@crichardsoncrichardson@vmware.com
1
Presentation goal
2
How decomposing applications improves
deployability and scalability and
How Cloud Foundry helps
About Chris
3
(About Chris)
4
About Chris()
5
About Chris
6
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
7
vmc push About-Chris
8
Developer Advocate for CloudFoundry.com
Signup at http://cloudfoundry.com
Promo code: Gluecon12
9
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Let’s imagine you are building an e-commerce application
10
Tomcat
Traditional web application architecture
11
Browser
WAR
MySQL Database
developtestdeployscale
ShippingService
AccountingService
InventoryService
StoreFrontUI
Simple to
Apache
But there are problems
12
Users expect a rich, dynamic and interactive experience on mobile devices and desktop
13
Java Web Application
Browser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure
Fear of change
Updates will happen less often e.g. Makes A/B testing UI really difficult 14
Large application
Slow IDESlow start times for web container
15
Scaling development
16
!=Scalable development
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Forces teams to synchronize development efforts Teams need to coordinate updates
Long-term commitment to a single technology stack Let’s suppose you go with the JVM•Some polyglot support•Many (but not all) JVM languages interoperate easily• e.g. Scala, Groovy modules within a Java application•But you can’t use non-JVM languages
Application depends on a particular:•Framework - e.g. Spring or Java EE•Container - e.g. Tomcat
Switching technology stack ⇒ touches entire application•Painful•Rarely done
17
18
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
3 dimensions to scaling
19
X axis - horizontal duplicationZ ax
is - d
ata pa
rtitio
ning
Y axis - functionaldecomposition
Scale by cloning
Scale
by sp
litting
simila
r thin
gsScale by splitting different things
Y axis scaling - functional partitioning Splits monolithic application into a set of services Each service implements a set of related functionality Partitioning schemes:•Partition functionality by noun or by verb•Single Responsibility Principle•Unix utilities - do one focussed thing well
20
ShippingService
AccountingService
billing web application
shipping web application
InventoryService
inventory web application
21
Y axis scaling - application level
Store Front
Store front web application
Apply X axis cloning and/or Z axis partitioning to each service
Real world examples
22
http://highscalability.com/amazon-architecture
Between 100-150 services are accessed to build a page.
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
There are drawbacks Complexity: •Architectural•Development•Deployment•Application management
Deciding when to use it• In the beginning: you don’t need it and it will slow you down•When you do need it: refactoring existing code is painful
23
See Steve Yegge’s Google Platforms Rant re Amazon.com
But there are many benefits Scales development: focussed two pizza devops teams Deploy services independently Scale services independently Improves fault isolation Enforces well defined interfaces between components Eliminates long-term commitment to a single technology
stack
24
Modular, polyglot applications
Two levels of architecture System level•Defines the inter-service glue: interfaces and
communication mechanisms•Slow changing
Service level•Defines the internal architecture of each service•Far fewer constraints on technology•Each service could use a different technology stack•Rapid evolving
25
Versus
If services are small... Regularly rewrite using a better technology stack Pick the best developers rather than best <pick a
language> developers ⇒ polyglot culture Adapt system to changing requirements and better
technology without a total rewrite
26
Fred George “Developer Anarchy”
Moreover: you are not the same you ... Cell lifetimes:• hours - some white blood cells• days - stomach lining cells• years - bone cells• lifetime - brain cells
50 to 70 billion of your cells die each day Yet you (the system) remains intact
27
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
Can we build software systems with these
characteristics?
Too much technical debt ⇒ component death?
28
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Inter-service communication options Multiple collaborating services ⇒ need a communication
mechanism Many choices:•Synchronous ⇔ asynchronous•Transports: HTTP, AMQP, ...•Formats: JSON, XML, Protocol Buffers, Thrift, ...•Even via the database
Distributed application ⇒ error handling strategies
29
Asynchronous is preferred
JSON is fashionable but binary format is more efficient
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
WidgetInventoryService
wgrus-inventory.war
InventoryService
wgrus-inventory.war
MySQL
30
RabbitMQ(Message Broker)
Asynchronous message-based communication
Benefits and drawbacks Benefits•Decouples caller from server•Caller unaware of server’s coordinates (URL)•Message broker buffers message when server is down/slow
Drawbacks•Additional complexity of message broker •RPC using messaging is more complex
31
Writing code that calls services
32
Composable futures Problem:•Service A needs to call services B and C and then D•Makes sense to call B and C parallel•Yet most concurrency APIs are low-level, error-prone etc
Solution:•Use Akka composable futures = really nice abstraction
33
val futureB = callB()val futureC = callC()
val futureD = for { b <- futureB.mapTo[SomeType] c <- futureC.mapTo[SomeType] d <- callD(b, c) } yield d
val result = Await.result(futureD, 1 second). asInstanceOf[SomeOtherType]
http://doc.akka.io/docs/akka/2.0.1/scala/futures.htmlhttp://en.wikipedia.org/wiki/Futures_and_promises
Two calls execute in parallel
And then invokes D
Get the result of D
Spring Integration
Builds on Spring framework
Implements EAI patterns = high-level of abstraction for building message based applications
Provides the building blocks for a pipes and filters architecture
• Pipes = message channels
• Filters = endpoints that filter, transform, route messages and integrate with external messaging infrastructure
Enables development of components that are
• loosely coupled
• insulated from underlying messaging infrastructure
34
Handling failure Errors happen (especially in distributed systems) Use timeouts and retries•Never wait forever•Some errors are transient so retry
Use per-dependency bounded thread pool with bounded queue• Limits number of outstanding requests•Fails fast if service is slow or down
Use circuit breaker•High rate of errors ⇒ stop calling temporarily •Avoids calling service that has issues
On failure•Returned cached or default data• Invoke custom error handler
35http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
36
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Modular application
Choice of presentation layer technology+
Redeploy UI frequently/independently
37
NodeJS is the fashionable technology
38
Many JavaScript client frameworks have a NodeJS counterparte.g. socket.io
NodeJS isn’t the only game in town
39
JVM-based http://vertx.io/
A modern web application
40
Browser
Service 1
Service 2
...
HTML 5Application
Socket.ioclient
Events
RESTful WS
Server Application
Socket.ioserver
Node JS
NodeJS - using RESTful WS and AMQP
41
Node JS
Service
RabbitMQ Service
REST
AMQP AMQP
RESTRequests
Eventssocket.io
Updating the UI is easy
42
Update the UI independently of rest of system Easily run A/B tests Enables fast iteration of the UI
http://theleanstartup.com/principles
But coordination with backend changes required Let’s imagine that you are deploying an advanced search
feature:•Enhancements to search service•Enhancements to UI
Before•Deploy new war
Now:•Some coordination required•Deploy updated backend service•Deploy updated NodeJS and browser code•Enable feature using feature switch
43
44
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Traditional tools: monolithic applications
45
Developing modular apps is more difficult Many more moving parts to manage•Platform services: SQL, NoSQL, RabbitMQ•Application services: your code
Who is going to setup the environments:• the developer sandbox?• ...•QA environments?
46
But Cloud Foundry helps...
Applica'on Service Interface
Data Services
Other Services
Msg Services
Easy polyglot application deployment and service provisioning
vFabric Postgres
vFabric RabbitMQTM
Additional partners services …
OSS community
Private Clouds
PublicClouds
MicroClouds
Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK
$ vmc services......
=========== Provisioned Services ============
+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
Multi-application manifest - part 1--- applications: inventory/target: name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis
49
Path to application
Required platform services
Multi-application manifest - part 2 store/target: name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq
50
Path to application
Required platform services
One command to create platform services and deploy application
$ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OKStaging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (5K): OK Push Status: OKStaging Application 'store': OK Starting Application 'store': ... 51
vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications
Micro Cloud Foundry: new developer sandbox
52
Open source Platform as a Service project
App Instances Services
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OKBinding Service [mysql1]: OKStaging Application: OK Starting Application: OK Getting tunnel connection info: OK
Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Running JUnit test with Caldecott
55
Configure your test code to use port + connection info
Summary Monolithic applications are simple to develop and deploy
BUT applying the scale cube
Decomposes your application into services Enables scaling for transactions and data volumes Tackles application complexity Enables scaling for development Enables frequent, independent deployments Make it easy to leverage other technologies
AND
Cloud Foundry simplifies the development and deployment of modular, service-based applications
56
Questions?
@crichardson crichardson@vmware.com
www.cloudfoundry.com promo code Gluecon12