Fast SOA with Apache Synapse
-
Upload
paul-fremantle -
Category
Technology
-
view
14.093 -
download
0
description
Transcript of Fast SOA with Apache Synapse
Contents• What is Apache Synapse• History of the project• Some examples• Delving into the model• Performance• Connectors, Adaptors• Getting started• Getting involved
What is Apache Synapse?
• Is it?– A highly performant XML Gateway– An Enterprise Service Bus– An HTTP proxy server– A protocol switcher– A way of hosting message manipulation
logic
What is Apache Synapse?• A lightweight Enterprise Services Bus (ESB)
– Available as a WAR file, NT Service, Linux Daemon – Runs as a process with its own Listeners, Tasks and
Senders– Can be deployed standalone or part of a cluster or
distributed network– High performance, asynchronous, streaming design– Can initiate work – scheduled tasks– Supports multiple transports including HTTP, JMS, TCP,
SMTP and (S)FTP– Simple to extend
HTTP8080
HTTPS8443
JMS
SMTP
inflow
log xslt send
outflow
send xslt log
HTTP8080
HTTPS8443
JMS
SMTP
Flows
Synapse startup Using Bouncy castle JAR for Java 1.5Starting Synapse/Java ...Using SYNAPSE_HOME: C:\SYNAPS~1.1\bin\..Using JAVA_HOME: c:\jdkUsing SYNAPSE_XML: -Dsynapse.xml="C:\SYNAPS~1.1\bin\..\repository\conf\synapse.xml"2007-11-12 12:16:58,250 [-] [main] INFO ServerManager Using the Axis2 Repository C:\
SYNAPS~1.1\bin\..\repository2007-11-12 12:17:01,921 [-] [main] INFO SynapseInitializationModule Initializing Synapse at :
Mon Nov 12 12:17:01 GMT 20072007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule Loading
mediator extensions...2007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule
Initializing the Synapse configuration ...2007-11-12 12:17:01,968 [127.0.0.1-pzfdell] [main] INFO XMLConfigurationBuilder Generating
the Synapse configuration model by parsing the XML configuration (some deleted)2007-11-12 12:17:04,359 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOSender HTTP Sender
starting2007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOListener HTTPS Listener
starting on port : 84432007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO ServerManager Starting transport
https on port 84432007-11-12 12:17:05,046 [127.0.0.1-pzfdell] [main] INFO ServerManager Ready for processing
History of the project• August 2005 Kicked off as an incubator
proposal• December 2005 Milestone 1 released• July 2006 Milestone 2 released• December 2006 Synapse 0.90 released• January 2007 Graduates from Incubator• May 2007 Released 1.0• December 2007 Becomes a Top Level
Project• January 2008 Synapse 1.1.1 released
A really simple exampleExposing an existing SOAP endpoint as
XML/HTTP, XML/JMS<definitions> <proxy name=“example" transports=“http jms"> <target> <endpoint> <wsdl uri=“http://remote.com/service?wsdl“ service=“RemoteService" port=“RemotePort"/> </endpoint> <outSequence> <send/>
</outSequence> </target> </proxy> </definitions>
Some other things you can do “out of the box”
• Content-based routing– Direct messages based on properties or XPath expressions
• Convert from existing formats (CSV, Fixed records, etc) to and from XML
• Transform XML – Using XSLT or XQuery
• Validate using XML Schema• Update databases and enhance messages with extra data
– DBLookup and DBReport mediators – any JDBC database• Switch between SOAP and non-SOAP, any transport
– HTTP, HTTPS, SMTP/POP3, XMPP, JMS, TCP, Filesystem, FTP, AMQP
• Add or handle WS-* support including– WS-Security, WS-SecureConversation, WS-Trust, WS-
ReliableMessaging• Load balancing, failover, throttling
Registries and Dynamism
Synapse “Registry”HTTP
extensible
readand cache
notify
“Transactional updates”-existing flows continue to use old config
Also graceful restart -switch off transport cleanly-restart Synapse when all work is done
A more detailed example
NY
VariableFormat records
London
Existingtrade
database
D2243578XX910D6233578YYYY9
<FEED> <CAK>2243578</CAK> <AccNo>3289019</AccNo> <CAT>AK</CAT> …</FEED>
WS-Security (encryptionand digital signature)
MQ
Validation
Overview of the flow<sequence name=“new york”> Flatpack with Variable record config E4X script to reformat XML
Send to London</sequence><sequence name=“london”>
DBreport Filter Send to MQ
</sequence>
Results• Replicated a 3 month project is 3 days
– Included enhancing the Flatpack mediator to support Variable Records
– Added WS-Security over original scenario– 1200 lines of config reduced to less than 300– Much more maintainable – clean separation of
concerns between departments
Google Spreadsheet and CSV
Google.com
CSV
Synapse
HttpPollerJob
CSV to XML Atom
Esper• Event Stream Processing project
– http://esper.codehaus.org
• Allows you to define queries on sets of “events”SELECT AVG(price) FROM
OrderEvent.WIN:TIME(30 sec)
• EsperMediator for Synapse makes it simple to add CEP to your SOA
Ganglia, Quartz, Esper
Synapse
QuartzScheduler
Gangliagmond
telnet 8649
<GANGLIA_XML>GMondPoller
Job
EsperMediator
SELECTAVG(CPU_USER)FROM GANGLIA
urn:gmond
Alertnotification
IMAtom
orJMS
urn:cpu
<definitions xmlns="http://ws.apache.org/ns/synapse"><task class="org.fremantle.gmond.GMonTask“ name=“Gmon”> <trigger interval="5000"/> <property name="hostname" value="localhost"/>
<property name="port" value="8649"/> <property name="to" value="urn:gmond"/> </task>
<in> <filter source="get-property('To')" regex="urn:gmond">
<log/> <class name="org.fremantle.esper.EsperMediator"> <property name="Configuration"
value=“./repository/conf/esper.conf.xml"/> <property name="statement"
value="select avg(cpu_user) from Ganglia.win:length_batch(10) "/> <property name="EventToAddress" value="urn:cpu"/> </class> </filter> <filter source="get-property('To')" regex="urn\:cpu">
<send> <endpoint>
<address uri="http://localhost:9090/notfication"/> </endpoint> </send> </filter> </in></definitions>
Performance
• Non-blocking IO– Apache HTTPCore – http://hc.apache.org/httpcomponents-cor
e/httpcore-nio/index.html
• Streaming XML and Binary– Apache Axiom– http://ws.apache.org/commons/axiom
Non-blocking graphically
Thread2
Incoming req
Socke
t open
Thread1Socke
t open
Requestprocessing
Responseprocessing
Outgoing resp
Outgoing req
Incoming resp
Synapse
This model means:1. Synapse threads never blocked during normal processing2. Number of sockets open >> number of threads
TIM
E
Demonstrating Performance• Synapse by default runs
– 2 listener threads– 2 sender threads– 8 worker threads
• Added a 100ms thread sleep to the server
• Ran 250 concurrent clients for 10000 runs– Simply would not have run without NIO
• Also did a simple test comparing:– 346 bytes in/ 1,170 bytes out– Direct to Axis2: 7.4ms– Via Synapse: 8.1ms – diff = 0.710ms!!
XSLT performance
Extending Synapse• Java Mediators
– Implement a simple Java API• Java Commands
– Pure POJO with mapping into messages using XPath• Scripts
– Using the Apache BSF project you can write scripts in JavaScript, Groovy and Ruby
• Synapse extensions– First class mediators with their own “Domain” XML configuration– Simply drop the JAR into the classpath
• Tasks– Repetitive jobs can be run at set intervals
• Registries– Synapse can dynamically load its configuration from a remote registry
• Transports– Create new connectors to remote systems
Simple Mediator: CSV->XMLpublic boolean mediate(MessageContext mc) {
DataHandler dh = PayloadHelper.getBinaryPayload(mc);BufferedReader br;new BufferedReader(new InputStreamReader(dh.getInputStream()));CSVReader csvReader = new CSVReader(br);
OMFactory fac = OMAbstractFactory.getOMFactory();OMElement el = fac.createOMElement("csv", csvNS);// create element to hold datawhile ((nextLine = csvReader.readNext()) != null) {
rownum++;// add elements to XML
}br.close();PayloadHelper.setXMLPayload(mc, el);return true;
}
Tasks – Quartz integration• Simple repetitive actions• Can also be used to start a long-running activity at
startup• Uses the Quartz Scheduler to run items
– www.opensymphony.com/quartz
• Tasks must implement the Task interfacepackage org.apache.synapse.startup;
public interface Task
{
public abstract void execute();
}
• Tasks may implement the ManagedLifecycle interface• Properties are set by injection (String and XML)
Sample task - MessageInjectorpublic class MessageInjector implements Task, ManagedLifecycle{
public void setTo(String url) { to = url; }public void setMessage(OMElement elem) { message = elem; }public void execute() { MessageContext mc =
synapseEnvironment.createMessageContext(); mc.setTo(new EndpointReference(to)); PayloadHelper.setXMLPayload(mc,
message.cloneOMElement());synapseEnvironment.injectMessage(mc);
}}
Script example<script language="js"><![CDATA[ var rowset = mc.getPayloadXML() var ns = new Namespace("http://..."); var d6 = rowset..ns::row.(@recordname=="D6"); var output = <FEEDS/> for (var i=0; i<d6.length(); i++) { var row = d6[i]; var cak = row.ns::entry.(@name==“CAK").text(); … var feed = <FEED> <CAK>{cak}</CAK> <AccNo>{accno}</AccNo> </FEED>; output.appendChild(feed); } mc.setPayloadXML(output); </script>
Community Site for extensionshttp://esbsite.org
Connectors, Adapters, Other projects
• CSV, Fixed and Variable records – FlatPack, OpenCSV
• FIX transport using QuickFix/J• AMQP transport using Apache QPid• Complex Event Processing based on Esper• Atom using Apache Abdera• Rules using Drools• EDI using Smooks• BPEL using Apache ODE• Hessian Binary XML using Hessian• URLRewriter • IM Mediator MSN, ICQ, Yahoo, Jabber
User quotes"Synapse's simplicity of configuration, ease of
extensibility, and integration were the key factors in our choice. Synapse's speed and stability, non-blocking I/O support, and responsive development community thensealed the decision,"
"Synapse passed all our performance benchmark, loadand memory leak tests with flying colours."
“Thanks for the quick fix. Synapse continues to rock.”
Futures• We continue to add new mediators and transports• Better support for complete HTTP Proxy
– Jointly done with HTTPCore• More work on a non-XML DSL for Synapse• Better support for REST
– DELETE and PUT support• Unifying support for events
– WS-Eventing, Atom, RSS, XMPP• Improved cluster management
– Graceful restart• Improved JMX management
– Better granularity
Getting Started
Getting Involved• Join us
– [email protected]– [email protected]
• Raise a JIRA for an improvement you would like– http://issues.apache.org/jira/browse/SYNAPSE
• Submit a patch• Contribute a mediator or a task definition
Questions?