WSO2 BAM2 Integration with Mule ESBKasun Weranga Gunathilake

Software Engineer – WSO2 BAM

Why server monitoring?

Analyze all business transactions To find a root cause for a failure Prevent outages with rapid problem identification and resolution Monitor performance and availability of all ESBs Understand resource utilization for tuning and capacity planning

Business Activity Monitoring

“The aggregation, analysis, and presentation of real-time information about activities inside organizations and involving customers and partners.” - Gartner

Aggregation

● Capturing data● Data storage● What data to

capture?

Analysis● Data operations● Building KPIs● Operate on large

amounts of historic data or new data

● Building BI

Presentation

● Visualizing KPIs/BI● Custom Dashboards● Visualization tools● Not just dashboards!

BAM Architecture

Things to Monitor

● Server logs● Server resource consumption● Statistics of services in Mule ESB ● Analyzing business transactions ( by inspecting

payload)

Monitor Server logs

● Mule ESB uses log4j for logging.● A Log4j appender can be used to capture log

events.● Write a custom BAM agent to publish log

events from Mule ESB to BAM.

Monitor Server logs

Custom Agents

● Java SDK provided● Asynchronous & non-blocking● Use Thrift for high performance message

throughput● Due to Thrift, different languages can be

supported● Compatible with CEP/BAM

Log4jAppender

● log4j.appender.LOGEVENT=org.wso2.carbon.mule.log.appender.LogEventAppender

log4j.appender.LOGEVENT.receiverUrl=tcp://localhost:7611

log4j.appender.LOGEVENT.authenticationUrl=ssl://localhost:7711

log4j.appender.LOGEVENT.trustStorePassword=wso2carbon

log4j.appender.LOGEVENT.columnList=%-5p,%d,[%t],%c:,%m%n

log4j.appender.LOGEVENT.userName=admin

log4j.appender.LOGEVENT.password=admin

log4j.appender.LOGEVENT.processingLimit=1000

log4j.appender.LOGEVENT.maxTolerableConsecutiveFailure=20

● log4j.rootCategory=INFO, console, LOGEVENT

The Analyzer Engine

● Analyzers are based on Easy to learn, SQL like Hive query language

● Runs locally or delegates to Hadoop cluster● Scalable analytics● Cluster can range from a couple of nodes to 1000s

● Analyzers can be scheduled● Supports Polyglot data architecture

Advantage of Using BAM

● Provide central place to analyze all the log information (ex:- logs of ESB cluster).

● Ability to analyze large amount of historical log events

● Efficient way to analyze the root cause for a failure.

Analyze Mule logs

● select logTime, logType, threadName, category, logMessage, logTimestamp from LogCollector where logtype='WARN' and logTimestamp > unix_timestamp('2013-01-06 11:00:00') order by logTimestamp;

Demonstration..

● Configure Mule ESB to send log events to BAM● Analyze logs using BAM

Monitor Business Transactions

● Capture useful information by inspecting payloads

● A Mule ESB interceptor can be used to capture data from payloads

● Write a custom BAM agent to publish captured data from Mule ESB to BAM

Monitor Business Transactions

Mule Interceptor

● Use Bookstore example comes with Mule● Capture order details using Interceptor

<flow name="OrderService">

<http:inbound-endpoint address="http://0.0.0.0:8777/services/order" exchange-pattern="request-response" doc:name="Public order interface"/>

<cxf:jaxws-service serviceClass="org.mule.example.bookstore.OrderService" />

<custom-interceptor class="org.wso2.carbon.mule.interceptor.BAMInterceptor"/>

<component doc:name="Order Service">

<singleton-object class="org.mule.example.bookstore.OrderServiceImpl"/>

</component>

<async>

<vm:outbound-endpoint path="emailNotification" exchange-pattern="one-way" doc:name="Email Notification"/>

<vm:outbound-endpoint path="dataWarehouse" exchange-pattern="one-way" doc:name="Data Warehouse"/>

</async>

</flow>

Analyze and present information

● Hive queries use to analyze the captured data.● Schedule Hive scripts to do the periodic

summarizations.● Gadget gen tool can be used to generate

gadgets and deploy on dashboard.● Plug 3rd party reporting engines to present

information

Demonstration...

● Capture order details from Mule bookstore sample

● Summarize sales against book titles and authors

● Visualize in dashboard

Monitor Server via JMX

● Server resource consumption (CPU, heap/non-heap memory, threads, etc..)

● Statistics of services deployed in Mule ESB

Monitor Server via JMX

● Enable JMX in Mule ESB (can use <jmx-default-config> )

● Configure JMX agent to pull statistics and store in BAM

– Provide the JMX server (Mule ESB)– Select JMX attributes need to monitor– Configure publisher

● Install JMX statistics toolbox in BAM

Monitor Server via JMX

Demonstration

● Monitor server resources (CPU, Memory)● Monitor statistics of order service in Mule ESB

Real time Monitoring of Mule ESB

● Improve Risk Management by rapid problem identifications

● Detect specific business conditions or patterns to enable instantaneous response.

● To take timely business decisions

Real time Monitoring

● Real time analytics via CEP● Based on Siddhi - A high performance complex

event processing engine developed at WSO2

● Can be seamless integrated with BAM to form a complete analytics solution

Real time Monitoring

Demonstration...

● Notify sudden increase in sales● Notify users about unusual system errors

Summary

● Importance of Monitoring the server● BAM architecture● Monitor server logs● Monitor business transactions● Monitor server via JMX● Realtime monitoring via CEP

Questions?

Useful Links....● BAM Docs

http://docs.wso2.org/wiki/display/BAM201

● Setting up JMX agenthttp://docs.wso2.org/wiki/display/BAM210/Setting+Up+JMX+Agent+for+BAM

● Write a custom data-agent http://wso2.org/library/articles/2012/07/creating-custom-agents-publish-events-bamcep

● CEP docs http://docs.wso2.org/wiki/display/CEP201

● Hive query language https://cwiki.apache.org/confluence/display/Hive/LanguageManual

● Mule docs http://www.mulesoft.org/documentation/display/MULE3USER/Using+Interceptors

http://www.mulesoft.org/documentation/display/MULE3USER/JMX+Management