20100907 fuse-community-evening-adrian-trenaman-no-logo

Progress Education

Architectures, best practices anduse-cases for ActiveMQ,ServiceMix, Camel & CXF

The pragmatic architect speaks outFUSE Community Evening, Rotterdam, 09/2010

Adrian TrenamanSenior Principal Solutions Architect, Progress Software

[email protected]

http://trenaman.blogspot.com

FUSE Community Evening Rotterdam - September 2010© 2010 Progress Software Corporation. All rights reserved.2

Agenda

A little bit about what the FUSE family (ActiveMQ, ServiceMix,CXF, Camel) really is:• Conceptual architecture• Standards and technologies• Deployment and scaling• Experience on the road

Some tips and best practices• A non-exhaustive list!

How you can be successful with FUSE• As architects• As developers• As project managers


What is FUSE?

Frequent, certified, supported releases of Apache Software• FUSE Message Broker - Apache ActiveMQ• FUSE ESB - Apache ServiceMix• FUSE Services Framework - Apache CXF• FUSE Mediation Router - Apache Camel

Unique selling points of FUSE (in the open source world):• The right license: ASL (no GPL, no LGPL, no attribution, …)• The right company: FuseSource (backed by Progress Software Corp.)

provides enterprise-level support for FUSE, backed by Apachecommitters.

• The right financial model: subscription-based, not license based.

WHAT? You mean it’s notabout the technology???

SOA Fundamentals© 2010 Progress Software Corporation. All rights reserved.4

ServiceMix 4 - architecture, standards & technologies


Apache ServiceMix 4 - conceptual architecture

JVM-based runtime container forintegration and SOA.• EIP-style integration flows• SOAP web services• REST web services• Business processes• Reliable messaging• Business Logic

<<jvm>>:ServiceMix4

IntegrationFlows

Web Services

RESTfulServices

BusinessProcesses

Reliable Messaging

Business Logic


Apache ServiceMix 4 - conceptual architecture(cont’)

ServiceMix provides a uniformapproach for common cross-functional concerns• Logging• Lifecycle and deployment• Configuration• Versioning & Dependency Mgmt• Management• Security• Transactions

<<jvm>>:ServiceMix4

IntegrationFlows

Web Services

RESTfulServices

BusinessProcesses

Reliable Messaging

Business Logic

Logging Lifecycle /Deployment Configuration

Versioning Management Security


Apache ServiceMix 4 - standards and technologies

<<jvm>>:ServiceMix4

IntegrationFlows

Web Services

RESTfulServices

BusinessProcesses

Reliable Messaging

Business Logic

Logging Lifecycle /Deployment Configuration

Versioning Management Security

EIP (Apache Camel) orJBI (ServiceMix 3)

BPEL (Apache ODE)

WSDL, SOAP,XML, XSD, JAX-WS, JSON,(Apache CXF)

Java, JVM-basedlanguages

JMS (ApacheActiveMQ)

OSGi Config AdminLog4J, Slf4J,commons, Javalogging, …

OSGiSSH, JAAS,HTTPS, TLS, …

JMX, web-console, ssh


Modular deployment with bundles and features

You can deploy almost anythinginto ServiceMix 4• War, Jar, bundle, spring, …

Prefer OSGi bundles for yourrouting / integration / businesslogic• More modular design, explicit

versioning, classpath control.• Can share classes or objects

(OSGi services)• Dynamic wiring of OSGi services

allows live hot deployment ofpatches without impacting yourproduction deployment.

Use the ‘feature’ mechanism togroup and co-deploy bundles.

<<jvm>>:ServiceMix4

a:bundle

b:bundle c:bundle

f1 f2

x:bundle

y:bundle

common

smx:> features:addUrl file:my-features.xml

smx:> features:install f1

my-features.xml


Getting into production…

Deployment from Maven repository is ideal…• … but remember, admins don’t like running internet-happy developer

tools like Maven in production!• Be prepared to deliver artifacts via .zip or .tar.gz - there are

features-based Maven plugins to help!

MavenDEV

SYSTEST UAT PROD

.tar.gz


Scaling up, scaling out…

‘to understand ServiceMix, first you must understandActiveMQ’


FUSE Message Broker (Apache ActiveMQ)

A high performance, reliable messaging fabric, supportingJMS, C, .Net, and other frameworks.

Clients connect to thebroker usingconnectors withsimple URIs

File & JDBC-basedmessage storessupported.

… all based on aflexible Spring-based core.

Topics and queuescreateddynamically.

Networkconnectors controlhow the brokerinteracts withother brokers fordiscovery,replication,failover, clusteringand store &forward.


ActiveMQ: networked brokers

Brokers use network connectors to share consumerinformation and make routing decisions using “store-and-forward”• JMS clients use failover URLs or auto-discovery to connect to a live

broker.

c1:Consumer c2:Consumer

p:Producer<<jvm>>

frodo:ActiveMQ<<jvm>>

gandalf:ActiveMQ

Foo:Foo:

m

1: Producer sendsa message m tothe broker.

m

2: Broker‘frodo’ decidesto routemessage viabroker‘gandalf’.

m 3: messageis delivered.


Aside: networked brokers allow you to createlocal broker clusters and implement wide-scale cross-

geography architectures.


ActiveMQ replication, clustering & failover

<<jvm>>samwise:ActiveMQ

Master statereplicated toslave usingnetworkconnector.

<<jvm>>frodo:ActiveMQ


Brokerscompete forfile system ordatabaselock.


Broker replication Shared message store


Master

Networks of master-slave pairs

A clustered, highly available approach

<<jvm>>merry:ActiveMQ


<<jvm>>gandalf:ActiveMQ

<<jvm>>saruman:ActiveMQ


<<jvm>>pip:ActiveMQ

Slave


… the thing is, it’s largely the same scaling model inServiceMix


Master

ServiceMix 4: networks of master-slave pairs

Differs from AMQ only in that slaves can be ‘partially active’

<<jvm>>merry: SMX4

<<jvm>>frodo:SMX4

<<jvm>>gandalf:SMX4

<<jvm>>saruman: SMX4

<<jvm>>samwise: SMX4

<<jvm>>pip: SMX4

Slave


Master

ServiceMix 4: embedded brokers

Embedded brokers speed up in-VM traffic, and facilitatelocation transparency via a ‘messaging fabric’

<<jvm>>merry: SMX4

<<jvm>>frodo:SMX4

<<jvm>>gandalf:SMX4

<<jvm>>saruman: SMX4

<<jvm>>samwise: SMX4

<<jvm>>pip: SMX4

Slave

ActiveMQ ActiveMQ ActiveMQ

ActiveMQ ActiveMQ ActiveMQ


‘The unbearable chattiness of brokers’

or

‘too many brokers spoiled the pot’


Breaking out the broker

Broker ‘meta-traffic’ will increaseas more brokers are added.• Sharing info on what consumers

are listening to what destinations.

If you have many SMX instances,then it might be more sensible todeploy your AMQ infrastructureseparately.• This also keeps things

conceptually simple.• We like simple.

Master

<<jvm>>merry: SMX4

<<jvm>>frodo:SMX4

<<jvm>>gandalf:SMX4

<<jvm>>saruman:

SMX4

<<jvm>>samwise:

SMX4

<<jvm>>pip: SMX4

Slave

<<jvm>>master:AMQ

<<jvm>>slave: AMQ

SOA Fundamentals© 2010 Progress Software Corporation. All rights reserved.21

Respecting and supporting our JBI heritage


JBI - fundamentals

<<jvm>>:SMX

<<component>>:FTP

NMR

<<component>>:EIP

<<component>>:SAXON

<<component>>:JMS

ServiceMix acts as acontainer for“components”,communicating witheach other using theXML-based NormalizedMessage Router.

Use well-knowncomponents like JMS,HTTP, CXF, Bean,FTP, FILE, or writeyour own.


JBI fundamentals (cont’)

<<jvm>>:SMX

<<component>>:FTP

NMR

<<component>>:EIP

<<component>>:SAXON

<<component>>:JMS

:Transformer

:Pipeline

:Producer

:Poller

Build an application byconfiguring and wiringendpoints as SUs,combining them intoSAs that can bedeployed atomically.

Endpoints areconfigured using xbean(Spring) configuration;deployment artifactsare created usingmaven plugins.


JBI fundamentals: packaging

<<jar>>

<<zip>>

.class

xbean.xmljbi.xml

resourcesftp-input-su

<<zip>>

.class

xbean.xmljbi.xml

resourceseip-su

<<zip>>

.class

xbean.xmljbi.xml

resourcesjms-output-su

<<zip>>

.class

xbean.xmljbi.xml

resourcesxslt-transformer-su

Each component isspecialized using a SU.

In ServiceMix, theendpoint(s) are defined inan xbean.xml file.

Maven plugins are used togenerate jbi.xml file

SUs can optionally containsupport classes andresources such as WSDL& XSD

SUs are bundled togetherinto an SA to be deployedatomically.


<<jvm>>:SMX

NMR

<<jvm>>:SMX

NMR

JBI support in ServiceMix 4

Full support for JBI 1.0 Use JBI without the packaging!

• Spring + JBI components.

Use NMR without canonical XMLpayload• Great for bundle-to-bundle traffic.

Use Interceptors to monitorendpoints and messageexchanges.

New clustering architecture:• Clustered consumers write to

NMR which uses a single JMSqueue.

• Providers listen on queue usingmessage selectors

• ActiveMQ network connectorsprovide location transparency.

:FilePoller

:JMSProvider


Riding camels…


FUSE Mediation Router (Apache Camel)

Camel provides an embedded DSL (in Java & Spring) forimplementing enterprise integration patterns.• The DSL uses URIs to define endpoints which are combined by form

integration flows.

from("activemq:topic:Quotes”) .filter().xpath("/quote/product = ‘widget’"). to("mqseries:WidgetQuotes") .filter().xpath("/quote/product = ‘gadget’"). to("mqseries:GadgetQuotes");


Riding the camel…

Camel describes your integration flow in one place, using anintuitive language.• … a big improvement on the forest of SA’s and SU’s of ServiceMix 3.

Camel supports 50+ endpoint technologies, and 35+ EIPs

Supports transactional SEDA architectures

Deploy anywhere (standalone, JEE, Tomcat, ActiveMQ,SMX, Spring)


CXF: a service by any other name would smell assweet…


Implementservices usingJAX-WS or JAX-RS

@WebServiceClass MyImpl implementsInvoicing { public void process( Invoice inv) { }}

Invoice inv = new Invoice();

invoiceSvc.process(inv);

FUSE Services Framework (Apache CXF)

Flexibly create & deploy code-first or WSDL-first Java webservices.

:MyImpl:Consumer

Separation of concerns

XSD

WSDL

Choose the payload (XML,SOAP, JSON) or transport(HTTP, JMS) declarativelywith no impact on code.


Cool things about FUSE Service Framework

Code RESTful services using JAX-RS

Generate client-side JavaScript code on the fly for browser-based WS clients:• http://localhost:9000/InvoicingService?js

Deploy anywhere:• J2EE: WebLogic, WebSphere, Tomcat, Jboss• OSGi: Servicemix 4• JBI: ServiceMix 3• SpringFramework• Lightweight FUSE Spring Container• Standalone: public static void main(…) { }


Tips and best practices…


ActiveMQ - tips and best practices

Keep it simple: follow one of these well-known deploymentpatterns:• Lone Broker• Master Slave with shared data-store• Cluster of Master Slave Pairs• Cross Data Centre• Cross Geography

Management• Prefer secure JMX over web console, and use a professional console

like FUSE HQ or Nagios.• Don’t just leave management to the administrators / operations team.

Continually test and validate performance.

Understand and plan your storage requirements


ActiveMQ - tips and best practices (cont’)

Understand consequences of sync vs async messaging

Understand consequences of persistent versus non-persistent messaging• 20,000 mps, 2,000 mps, 200 mps

Understand impact of ‘fsync’ (now default in KahaDB)

Use JMS transactions to minimize your disk syncs andincrease performance!

If you’re going to have a DLQ, use it.• Who is listening for errors? What are they going to do about it?

- Camel: route message to DB and send email?- Camel: read message and initiate business process via WS?- Camel: route message to custom error handling application?


Camel - tips and best practices

Give your camel context an id - makes JMX easier.

Give your route an id with routeId() - make JMXmanagement easier

Use the Spring init-method and destroy-methodattributes to cleanly start and stop your routes.

Parameterize your routes - no hard-coding of URIs.

Never assume that your route will work with all components.• Each producer component may creates its own message content - do

you have the right converters in place?

A route should span no more than one screen of text

Add threading judiciously, as you need it.


Camel - tips and best practices (cont’)

Consider explicit techniques vs. dynamic techniques• Camel does a lot of stuff dynamically (e.g. converters) - will this

confuse your development team?

If deploying into FUSE ESB, deploy regularly to iron out OSGiissues.• Don’t leave it all to the last minute.


CXF - tips and best practices

Use WSDL first

Think about where your WSDL is going to come from (filesystem, HTTP, auto-generated)

Think about security early

Make your endpoints manageable via JMX

Get your configuration out of Spring, and into properties filesvia property placeholders

Be mindful of 'first invocation' lazy initialization of JAX-Bcontexts: first invocation is slow!


FUSE ESB - tips and best practices

Understand cost/benefit of deploying ActiveMQ in ServiceMix.

Deploy only what you need.

Use the OSGi Config Admin service - seamless way to getconfiguration via property placeholders, and allows dynamicupdates of configuration

Consider what 'master-slave' means for routes, web services,and RESTful services

Consider OSGi Blueprint Services instead of Spring 3.0

Use features for manage deployment of your bundles

Ensure that features are packaged for 'maven-less'deployment in production


Summing up…


Adopting FUSE

Java, Maven, Spring - if you don’t have these skills, beprepared to get them.

Invest in technical steering - don’t go in the wrong direction

Nominate FUSE expert(s) on your team - then let themspread the love.

Know yourself - If you池e not source-friendly, then getprofessional help.

In the source, there is truth - don’t be afraid of the truth.


Parting words from a battle-hardened consultant…

Don’t confuse enterprise integration with workflow or BPM• Use the right tool for the job at hand.

Respect the {n|cr}appy path above the happy path.

Validate performance early, and keep validating it.

Prefer Camel; use JBI as an integration point.

Never forget your customer. Give them visibility and control.• Thing about management early.

20100907 fuse-community-evening-adrian-trenaman-no-logo

Technology

Transcript of 20100907 fuse-community-evening-adrian-trenaman-no-logo