Post on 22-Sep-2020
ESB HOWTOFrom Software Selection
toMission-Critical Deployment
Eugene CiuranaDirector, Systems Infrastructure
LeapFrog Enterprises, USA
About Eugene Ciurana
• 15+ years of experience building mission-critical, high-availability systems infrastructure
• 10+ years of Java work
• Engaged by the largest companies in the world
• Retail
• Finance
• Oil industry
• Background ranges from industrial robotics to on-line retail systems
Do You Have an Integration Problem?• If you are in a mid-sized to large enterprise, you have many
applications that must inter-operate
• Custom
• Third-party
• Legacy
• Integration is point-to-point
• Applications communicate over dedicated links
• File exchange, database, API calls, RPC
• Hard and expensive to respond quickly to business demand
Do You Have an Integration Problem?• Multiple tiers
• Each tier is implemented with different technology
• Coexistence of legacy and new systems
• Different skills for each tier
• Multiple physical locations
• Multiple protocols?
• Ad hoc interfaces between systems?
• Data and process consolidation
• Point-to-point integration is a huge problem!
Point-to-Point Integration Problems
• Every problem becomes a new project
• Different systems may support different interoperability protocols
• JMS
• SOAP
• REST
• BPEL
• etc.
SOAP
• Creates vendor lock-in or artificial dependencies between systems
Point-to-Point Integration Problems• Point-to-Point
integration is more expensive
• Requires more regression testing
• Consumes more resources
JMS
JMS
JMS
FTP
SOAP
REST
Application
A
Application
B
ApplicationC
Application
D
Application
E
Application
F
Integration Through Services• SOA = Services-Oriented Architecture
• Collection of services that communicate with one another
• No dependencies on other services
• Self-contained
• Messaging: mechanism for communication between two or more services
• Real-time, asynchronous, synchronous
• May occur over different transports
• HTTP, FTP, JMS, RMI, CORBA, etc.
Integration Through Services
• Web services are the current mechanism for data exchange between applications
• Shorthand: web services are a tangible implementation of SOA concepts
• Some people think that SOAP == web services
• That’s a very limited view of the universe
• Web services may be implemented in a variety of ways:
• HTTP: SOAP, REST
• JMS: synchronous and asynchronous queueing
Can you think of others?
ESB Evolution• First... came the queueing architectures
• Message queueing (MQ, Sonic MQ, JMS) was not enough
• Service consumers and providers must implement it
• Expensive to maintain two different interoperability layers for the same service (e.g. JMS, REST)
• Messaging buses evolved to accommodate other protocols
• Different approaches by different vendors/dev groups
• ESB ::= Enterprise Service Bus
• Queuing + messaging + support for multiple protocols
ESB Evolution• Enterprise Service Bus concept: David Chapelle, Progress
Software
• Mule ESB: Ross Mason, real-world implementation in parallel
• Open-source, battle-tested... it works!
• Commercial ESBs
• Open-source ESBs
• Commercial and open-source ESBs all claim to do the same thing... Mule is often selected because it delivers on the promises made by commercial vendors and is more mature and robust than other open-source offerings
Use Commercial or Open-Source?
• Motivation: commercial vendors want to lock you in
• Their ESBs work great with other products in their catalogue, not so great or not at all with third-party software
• Motivation: open-source seamless integration
• Many are immature or marred by philosophical baggage
• This presentation is about Mule ESB: open-source, great support organization behind it, active volunteer community, and an enterprise track record
Geeking Out... Let’s Do ESB!
• What you need to know:
• Java programming (intermediate to advanced)
• Basic concepts about messaging
• Web services, HTTP, or JMS
• “Enterprise Integration Patterns”
• Gregor Hohpe, Bobby Wolf, et. al.
• What you’ll need:
• Java 5
• Mule 1.4.1 or later
Getting Started• Get Mule:
• http://mule.codehaus.org
• tar xzf mule-1.4.2.tar.gz
• ln -s ./mule-1.4.2 mule (sym links are useful)
• Environment:
• export MULE_HOME=~/development/mule
• export PATH=“$PATH:$MULE_HOME/bin”
• export JAVA_HOME=/usr/local/java
• Create a work directory and go there:
• mkdir myproject; cd myproject
You’re readyto
Mule!
Sample Project• Need to create an OpenLaszlo component from a content
management system
• Day Software’s Communiqué (CQ)
• Building OpenLaszlo integration into CQ is expensive and time-consuming
• Time critical: develop an OpenLaszlo compiler module and integrate this with CQ using Mule
• Total development and integration time: 3 days
• Components are fully reusable!
• Implemented as services
Architecture Overview
Mule ESB
CMS AuthoringRIA
ComponentEngine
Customer
Master
Single Sign-On System
Active
Directory
CMS Publishing
E-Commerce Site
HostedMicro-site
Internet
Mule Environment• Mule relies on $MULE_HOME to find its libraries
• Three very important directories under Mule:
• lib/mule - core Mule libraries
• lib/opt - third-party libraries such as Axis, Apache Commons, dom4j, log4j, Jetty, JUnit, etc. - almost everything you need is here!
• lib/user - this is where you’ll store your .jar(s) with your own components
• OpenLaszlo doesn’t ship with Mule, so in lib/opt:• lrwxr-xr-x lps-4.0.3.jar -> /opt/LPS/lps-4.0.3/Server/WEB-INF/lib/lps-4.0.3.jar
Mule In Action• Applications are at the endpoints
• Configuration describes endpoints, channels, connectors, transformers, and routers
• Service objects provide functionality to “tie” things together
• Written by Mule, third-parties, or your dev team
Application Channel
Transformers
Message
Sender
Connector
Inbound
Router Service
ObjectComponent
Outbound
Router
Transformers
Message
Receiver
Connector
Channel Application
Mule
SWFRendereringEngine
Defining Models
Communiquésource files
list
FileRetrieverComponent
SourceFiles
Repository
File system
SWFCompiler
BuiltFiles
Repository
SWFRendereringEngine
FileRetrieverComponent
SWFCompiler
SourceFiles
Repository
BuiltFiles
Repository
File system
Communiquésource files
list
Defining Models<model name=“SWFRenderingEngine”>
<endpoint name=“LPSDrop” value=“http://localhost/lpsdrop">
<mule-descriptor name= “FileRetrievalService” implementation=“com.pkg.FileRetriever”>
Complete file at:http://ciurana.eu/JavaZone2007/swf-engine.xml
A Mule model maps to a composite of components andendpoints.
Defining Endpoints• Endpoints are the interfaces between the Mule and the
applications
• They are abstractions of the underlying messaging system
• An endpoint can abstract any network transport, a database, memory channels, a file system, etc.
<endpoint-identifiers> <endpoint-identifier name="LPSDrop" value=“http://localhost:8016/lpsdrop” /> <endpoint-identifier name="SourceFiles" value="file://./sources" /> <endpoint-identifier name="CompiledFiles" value="file://pubserver/triggers" /> <endpoint-identifier name="ServiceMonitor" value="stream://System.out" /></endpoint-identifiers>
Transformers
• Data may come through an inbound endpoint in a format different than the one expected by the outbound endpoint or the service object
• You can transform the data as it comes into the component or after it leaves
• You may apply transformations to the message itself, the message envelope, or both
• Example: lpsdrop listens for a method POST with a payload in the form: fileCount=0&file1=/path/to/whatever.gif&... - FileRetriever expects a HashMap with these data
Transformers
public class POSTToHashMap extends AbstractTransformer { // Private members and methods . .
// Public methods public Object transform(Object anObject) { HashMap payload = null; if (this.isValidPOST(anObject)) payload = this.extractDataFrom(anObject);
return payload; } // transform}
fileCount=2&file1=x.gif&file2=prog.lzx
Various operations to convert the POST datato a HashMap.
http://ciurana.eu/JavaZone2007/POSTToHashMap.java
+Object transform(Object source);
+String toString();
AbstractTransformer
+Object transform(Object source);
POSTToHashMap
Transformers public Object transform(Object anObject);
Mule transformers resolve these calls through configuration. As a developer/implementer, you must know the return type expected by the component and specify it as part of the configuration.
<transformers> <transformer name="POSTToHashMap" className="com.leapfrog.transformers.POSTToHashMap" returnClass="java.util.HashMap" /></transformers>
Notice that you must specify the return type for the transformation.
Handling Application Messages• You may extend an application’s capabilities through the use
of a service component
• Called Universal Message Objects or UMOs in older versions of Mule - the documentation still refers to them this way
• A service component is a POJO/bean that runs under Mule
• It executes business logic on incoming messages from any destination
• Messages can be dispatched to another endpoint
• There is no Mule-specific code in them
• Configure the bean using the Mule XML file
Implementing Business LogicSWFRendereringEngine
FileRetrieverComponent
SWFCompiler
SourceFiles
Repository
BuiltFiles
Repository
File system
Communiquésource files
list
SWFCompiler
Implementing Business Logic
<<component>>
Build
Files
SWFBuilder<< component>>
OpenLaszlo +setArgList()
+run()
+status()
<<mixin>>
Compiler
<<component>>
Source
Files
Mule
FileConnector
Implementing Business Logicpublic class SWFCompiler extends Service { . . public Object service(UMOEventContext eventContext) { String sourceFileName = this.sourceDir+"/"+sourceFile; byte[] byLaszloSource; if (this.validateEnvironment()) try { byLaszloSource = eventContext.getMessageAsBytes(); this.dumpBytesToFile(byLaszloSource, sourceFileName); this.compile(sourceFile, objectFile); this.packageSWF(); } catch (Exception e) { log.error(e.toString()); } else log.error("Invalid compiler environment.");
return this.fileSWFAsBytes(); } // service
http://ciurana.eu/JavaZone2007/SWFCompiler.java
Implementing Business Logicpublic class SWFCompiler extends Service { . . public Object service(UMOEventContext eventContext) { String sourceFileName = this.sourceDir+"/"+sourceFile; byte[] byLaszloSource; if (this.validateEnvironment()) try { byLaszloSource = eventContext.getMessageAsBytes(); this.dumpBytesToFile(byLaszloSource, sourceFileName); this.compile(sourceFile, objectFile); this.packageSWF(); } catch (Exception e) { log.error(e.toString()); } else log.error("Invalid compiler environment.");
return this.fileSWFAsBytes(); } // service
http://ciurana.eu/JavaZone2007/SWFCompiler.java
This is just a POJO - Service is a superclass that we defined
Message can be Object, String, etc. UMOEventContext always works
Application business logic - do
what you need.
Return an object... send this to an endpoint?
public class SWFCompiler extends Service {
public Object service(UMOEventContext eventContext) {
byLaszloSource = eventContext.getMessageAsBytes(); this.dumpBytesToFile(byLaszloSource, sourceFileName); this.compile(sourceFile, objectFile); this.packageSWF();
return this.fileSWFAsBytes();
<mule-descriptor name="SWFCompilationService" implementation="com.leapfrog.services.SWFCompiler"> <inbound-router> <endpoint address="SourceFiles"> <filter pattern="source_file.lzx" className="org.mule.providers.file.filters.FilenameWildcardFilter"/> </endpoint> </inbound-router>
<properties> <property name="sourceDir" value="./sources" /> <property name="objectDir" value="./triggers" /> <property name="objectFile" value="animation.swf" /> <property name="cacheDir" value="./cache" /> <property name="LPSHome" value="/Users/eciurana/ESB/lps-4.0.3" /> </properties></mule-descriptor>
Implementing Business Logic
private String sourceDir; // in SWFCompiler
public void setSourceDir(String sourceDir) { this.sourceDir = sourceDir;} // setSourceDir
Deploying Mule
• Your components are ready and you configured the models, endpoints, routers, and transformers - what’s next?
• Deploy Mule embedded in another application
• Saves you from writing message, transport code
• Deploy Mule with another framework like Spring
• It becomes another pluggable component
• Deploy Mule stand-alone in a high-performance dedicated deployment
Deploying Mule• The applications are deployed as .jar files to the server
• Satisfy all the package dependenciesSWF Rendering Engine
+SWFEngine-OpenLaszlo
Compilerlzc Main
-MuleCore
VMConnectorFileConnector
SWFBuilder
conf/swf-engine.xml
# OpenLaszloopt/lps-4.0.3.jar
mule/mule-core.jar
user/swfengine.jar
Deploying Mule• Stand-alone server deployment
• Select a fast server, multiple-cores, and lots of memory
• Mule needs to move messages through quickly
• Gigabit connections if set up behind the firewall
• Fail-over: two Mule instances behind a front-end load balancer or router
• Use in-memory caching or JavaSpaces to share state
• Only one instance at a time is active
• Warm server will establish endpoint connections on-demand
Production Deployment
• How many Mule instances in a server?
• Deploy a single instance with large config file?
• Deploy multiple instances, each for a subsystem?
• Multiple JVMs?
• Multiple virtual servers?
• Xen, VMWare
• Easier administration
Mule
Server
Production DeploymentSWFRenderer
jar
mulecore
jar
config.xml
CMS server
<uses>
author
Mule Server
Mule Server
Mule Server
lazlo-runtime
jar
Production Deployment
Complete Sample Architecture
Firewall
External
Device
PC
application
Web
browser thesite.comhttp
Cart,
session
data
Dedicated StoreJava 6, Wicket, Tomcat,
etc.
SOAP REST
SOAP
REST
SOAP REST
Mule ESB
SOAP, REST, JMS, MQ, BPEL, JDBC, caching, in-memory
SOAP REST
REST
Microsite
Java 6
Wicket
CMS
feed
httpInternet
SOAP
REST
SOAP
REST
ERP
SOAP
REST
CMS ContentRepository
JDBC
Customer
Master
SOAP
REST
Crowd Single Sign-On
Active Directory(domain)
Custom
Merchandizing
Tools
SOAP
REST
REST
CRM
Mule ESB
Conclusions• You probably need an ESB for application integration if your
company is from mid- to enterprise-size
• SOA: avoid marketectures, define a sensible architecture that combines open-source and commercial products, leveraging your domain-specific applications as services
• Using Mule open-source ESB
• Easy configuration, integration, expansion and customization
• Deploying in production: don’t put all the eggs in one basket
• Application monitoring: custom and commercial support
Questions&
Answers
Eugene CiuranaDirector, Systems Infrastructure
LeapFrog Enterprises, USAeugenex@leapfrog.com
pr3d4t0r - irc.freenode.net#esb, ##java, #awk
This presentation is hosted athttp://ciurana.eu/JavaZone2007