INNOV-17: How to Build Event Stream Processing (ESP) and Business Activity Monitoring (BAM) into...
-
Upload
darrell-ball -
Category
Documents
-
view
217 -
download
0
Transcript of INNOV-17: How to Build Event Stream Processing (ESP) and Business Activity Monitoring (BAM) into...
INNOV-17: How to Build Event Stream Processing (ESP) and Business Activity Monitoring (BAM) into Your Application
Dr Gareth SmithVP Products
Progress ApamaPrincipal ArchitectProgress Apama
Dr John Bates
© 2006 Progress Software Corporation2INNOV-17: How to Build ESP and BAM into Your Application
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is Event Stream Processing (ESP)?
• What is Business Activity Monitoring (BAM)?
Apama platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
© 2006 Progress Software Corporation3INNOV-17: How to Build ESP and BAM into Your Application
Event Stream Processing: “When 3 credit card authorizations for the same card occur in any 5 second
window, deny the request and check for fraud.”
Event Stream Processing: “When 3 credit card authorizations for the same card occur in any 5 second
window, deny the request and check for fraud.”
Event Stream ProcessingA New Computing Physics
time1 2 3 4 5 6 7 8 9
Static Data Processing: “How many shoes did we sell in our New York
stores last week?”
Static Data Processing: “How many shoes did we sell in our New York
stores last week?”
© 2006 Progress Software Corporation4INNOV-17: How to Build ESP and BAM into Your Application
Event Stream Processing: “When 3 credit card authorizations for the same card occur in any 5 second
window, deny the request and check for fraud.”
Event Stream Processing: “When 3 credit card authorizations for the same card occur in any 5 second
window, deny the request and check for fraud.”
ESP & BAMA New Computing Physics
time1 2 3 4 5 6 7 8 9
Business Activity Monitoring: “Show on a dashboard (a) how many credit card transactions we
have processed today (b) average & peak transactions per minute (c) how many fraudulent transactions per
hour.”
Business Activity Monitoring: “Show on a dashboard (a) how many credit card transactions we
have processed today (b) average & peak transactions per minute (c) how many fraudulent transactions per
hour.”
© 2006 Progress Software Corporation5INNOV-17: How to Build ESP and BAM into Your Application
Example Event Driven ApplicationsMonitor, Analyze, and Act on Business Conditions in Real Time
Application Event Examples Event-Driven Analysis & Action
Customer Service
Customer requests, CRM records
Alert me if an open customer request with priority > 3 did not get any reply event for at least 2 hours
Alert me if an open customer request was reassigned more than 3 times
Supply ChainInventory system, warehouse deliveries, warehouse restock
When (Warehouse 1 capacity < 50%) and (Warehouse 2 capacity < 30%) and
(Warehouse 3 capacity < 25%)Alert the head of operations with the average capacity in each of the warehouses
RFID/Supply Chain
RFID reader detecting EPC codes, goods delivery schedule
When pallet is loaded onto the wrong truck alert warehouse manager
Automated Trading
Stock tick & quote data
Within any 20 second window, when HP rises by more than 2%, but IBM doesn’t, buy IBM
Transportation Smartcard ID readWhen the rider travels more than 5 times within zone 1, charge at single-ride rate for subsequent trips
© 2006 Progress Software Corporation6INNOV-17: How to Build ESP and BAM into Your Application
What do events look like?
Events are typed messages describing a change within a system• Example:
– NewInvoice(customer, product, amount)– StockTick(symbol, price, volume)– TruckLocation(truck, driver, cargo, x,y,z)
Events are analogous to a new entry in a database table or a Sonic message
Detecting patterns in individual events or a correlation of several events over time can indicate an opportunity or threat to the business
© 2006 Progress Software Corporation7INNOV-17: How to Build ESP and BAM into Your Application
Business Benefits of ESP
Monitor, Analyze, ACT• The ability to take an action immediately when a business
scenario is detected– Enables your application to respond instantly to opportunity or
threat
It’s easy to add powerful capabilities to your existing application• You already have the events flowing in your system – you
just need to send them into Apama
Add real-time rules to your application without continually changing your code• You don’t have to go on extending your application• Once the events are flowing into Apama, simply add or
modify a rule in Apama• Rules can be defined by business or technical users
– Codeless and code based development
© 2006 Progress Software Corporation8INNOV-17: How to Build ESP and BAM into Your Application
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is Event Stream Processing (ESP)?
• What is Business Activity Monitoring (BAM)?
Apama Platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
© 2006 Progress Software Corporation9INNOV-17: How to Build ESP and BAM into Your Application
Apama Product Suite
Event Store
Dashboards
Integration Adapter FrameworkIntegration Adapter Framework
Developer StudioApama
IDEDashboard
StudioScenario Modeler
Research Studio
Event Manager
© 2006 Progress Software Corporation10INNOV-17: How to Build ESP and BAM into Your Application
MonitorScriptVirtual MachineMonitorScript
Virtual Machine
JavaVirtual Machine
JavaVirtual Machine
EventsEvents EventsEvents
C/C++C/C++
Composite event
sequencer
Composite event
sequencer
Match Notification
Event Registration
Scenarios
Multi-dimensional
event matcher
Multi-dimensional
event matcher
Event Input
Queue
Event Input
Queue
Inside the Event Manager
Event Output Queue
Event Output Queue
Feedback
© 2006 Progress Software Corporation11INNOV-17: How to Build ESP and BAM into Your Application
Monitorscript
monitor RingtoneServiceMonitor {
SMS-Request request;
action onload {
on all SMS-Request (service=“Ringtone"):request {
on SMS-Request (user = request.user) within(60.0) {
emit UpdateBilling(request.user, “Double Discount”);}
} }}
event SMS-Request { string user; string service; location loc;}
© 2006 Progress Software Corporation12INNOV-17: How to Build ESP and BAM into Your Application
JMON: Monitors in Java
import com.apama.jmon.*;
public class BillRequestService implements Monitor, MatchListener {
public void onLoad() {
EventExpression eventexpr = new EventExpression(“SMS-Request(service=\“Billing\”)”);
eventexpr.addMatchListener(this); }
public void match(MatchEvent event) {
System.out.println(“Billing Information Requested”); }}
© 2006 Progress Software Corporation13INNOV-17: How to Build ESP and BAM into Your Application
Apama IDE
© 2006 Progress Software Corporation14INNOV-17: How to Build ESP and BAM into Your Application
The Apama Scenario Modeler Enables Graphical Construction of ESP Scenarios
Intuitive visual user interface designed for business analysts
Express time-based real-time rules with a high level development tool
“SmartBlocks” encapsulate pre-packaged modules made available to non-programmers.
Each scenario, or group of rules, represents a “pattern” which can be adjusted by business users to specify conditions to monitor, analyze and act on.
© 2006 Progress Software Corporation15INNOV-17: How to Build ESP and BAM into Your Application
Apama Dashboard StudioCreate Customized Dashboards for Your Real-Time Business Processes
Select from a palette of graphical objects. Each object can be laid out in a graphical dashboard and bound to Apama events
Users can customize the look and feel of all widgets, and specify which Apama event scenarios to visualize.
• Real-time variables and analytics can be visualized using graphs, charts, tables etc.
• Enables event-driven Apama logic to be visualized in real-time
• Deployment options from standalone dashboard to thin client portal
© 2006 Progress Software Corporation16INNOV-17: How to Build ESP and BAM into Your Application
Apama
Real-Time Event Processing
Event Store - Capture
• Event Store captures all events in time series order
• Event Store is also able to capture derived events, e.g. Moving Average, from Event Manager
• Event Store can capture all output from Event Manager as an audit trail
EventStore
Historical Event Processing
Apama Dashboards
© 2006 Progress Software Corporation17INNOV-17: How to Build ESP and BAM into Your Application
Apama Research Studio Business Intelligence
Event Store – Replay & Backtesting
• Enables “pre-flight testing” of Apama scenarios with historical data sequences
•Output of runs can be captured in Event Store and charted
•Also enables “digital forensics” – using recorded information to tune future performance
Apama Dashboards
Apama
Real-Time Event Processing
EventStore
Historical Event Processing
© 2006 Progress Software Corporation18INNOV-17: How to Build ESP and BAM into Your Application
Integration Adapter Framework
Purpose of an adapter• Receive an event stream
or create an event stream– Intercept transactions– Use database triggers– Subscribe to Sonic
messages
• Normalize to Apama event format for efficient analysis & response
• Convert Apama events back to actionable messages OpenEdgeSonic Bus
Normalization Normalization
Native Sonic Messages
Transport Integration
APIPublish/ subscribe
Transport Integration
Event Manager
Mapping
Apama Events
Mapping
© 2006 Progress Software Corporation19INNOV-17: How to Build ESP and BAM into Your Application
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is Event Stream Processing (ESP)?
• What is Business Activity Monitoring (BAM)?
Apama Platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
© 2006 Progress Software Corporation20INNOV-17: How to Build ESP and BAM into Your Application
Building an Example Application
Case Study: Revenue Assurance• Ensuring that all service requests are both fulfilled and
billed correctly and within a timely manner• Monitor via complex KPIs and present an graphical
overview of the status• Alert relevant staff with specific problems• Undertake Autonomous Actions: modify the Service
QoS settings to resolve problems
Same underlying principles are valid for tracking & billing in Supply Chain, ERP, Logistics etc.
© 2006 Progress Software Corporation21INNOV-17: How to Build ESP and BAM into Your Application
A (very) simplified revenue chain
Switch (SMSC, MSC, SIP server)
ApplicationServer(s)
BillingUser’s mobile
1 2
3
45
Processa) User requests a service (1 + 2)b) Service is fulfilled (4 + 5)c) Request is Billed (3)
Problem• Any message (1-5) may be lost, corrupt or delayed• Depends on the throughput and the QoS
Parameters (Complex-KPIs) that we calculate:• Revenue leakage: requests that are not billed• Overcharging: requests that are billed but not fulfilled
© 2006 Progress Software Corporation22INNOV-17: How to Build ESP and BAM into Your Application
Scenario
Smart blocks
Our revenue assurance demo package
1 2
3
45
ReconciliationValue Added
ServiceBilling
- Calculate parameters- Apply treshold and raise alarms- Take corrective actions (e.g. adjust QoS)
Dashboard- Display what’s going on- Monitor the alarms- Notify leakages that can’t be fixed
Switch (SMSC, MSC, SIP server)
ApplicationServer(s)
BillingUser’s mobile
© 2006 Progress Software Corporation23INNOV-17: How to Build ESP and BAM into Your Application
Demonstration
© 2006 Progress Software Corporation24INNOV-17: How to Build ESP and BAM into Your Application
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is Event Stream Processing (ESP)?
• What is Business Activity Monitoring (BAM)?
Apama Platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
© 2006 Progress Software Corporation25INNOV-17: How to Build ESP and BAM into Your Application
Conclusions
ESP & BAM enables• Rules to be defined external to the application to monitor,
alert & respond to key business opportunities & threats• Vizualization of key business events on real-time
dashboards
In many cases events are already flowing through existing applications
Apama enables• Easy connection to existing applications• Graphical ESP rules definition for rapid development• Graphical dashboard creation
© 2006 Progress Software Corporation26INNOV-17: How to Build ESP and BAM into Your Application
Questions?
© 2006 Progress Software Corporation27INNOV-17: How to Build ESP and BAM into Your Application
Thank you foryour time
© 2006 Progress Software Corporation28INNOV-17: How to Build ESP and BAM into Your Application