IBM Haifa Research Lab – Event Processing © 2010 IBM Corporation Event processing – past,...
-
Upload
drusilla-johnson -
Category
Documents
-
view
214 -
download
1
Transcript of IBM Haifa Research Lab – Event Processing © 2010 IBM Corporation Event processing – past,...
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation
Event processing – past, present, future
VLDB 2010 Tutorial, Singapore, September 15th, 2010
Opher Etzion ([email protected])
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation2
An example to kick-off this tutorial:
Act: Passenger has been re-routed to another destination – send the luggage
Act: Bag has reached to the wrong aircraft
Notify: Bag has been checked but did not reach the ULD within 20 minutes
Notify: Bag has been checked but did not reach the connecting flight
THE LUGGAGE PERSPECTIVE:
Across the 24 largest airlines more than 5.6 million bags went missing in 2006, this is an average of 15.7 bags per 1,000 travelers. 15% of the bags are never found.
BBC News, April 4, 2007
Event processing can help here..
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation3
Another example: premature baby monitoring
Patient is hooked up to multiple monitors (in hospital or at home) - the physician can set up event- based rules on multiple measurements and patient’s history when to send an alert and to whom:
DefinedPattern
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation4
Outline of this tutorial
What is behind event processing ? How is it related to other computing terms? Where are its roots?
Event processing – architecture, building blocks
The present: State of the practice in event processing – languages, implementation issues, challenges in implementing event processing applications
The future: Trends and research challenges
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation5
What is behind event processing ? How is it related to other computing terms? Where are its roots?
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation6
What is “event processing” anyway?
or
Event processing is a form of computing that performs operations on events
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation7
In computing we ptocessed events since early days
Network and System Management
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation8
Emerging technologies in enterprise computing(Gartner Hype Cycle, Summer 2009)
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation9
What’s new?The analog: moving from files to DBMS
In recent years – architectures, abstractions, and dedicated commercial products emerge to support functionality that wastraditionally carried out within regular programming. For some applications it is an improvement in TCO; for others is breaking the cost-effectiveness barrier.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation10
What is an event
There are various definitions of events
Event is an occurrence within a particular system ordomain; the word event has double meaning: the real-worldOccurrence as well as its computerized representation
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation11
In daily life we often react to events..
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation12
Sometime we even react to the occurrence of multiple events
I closedthe deal with the Canadians
We closed two huge deals in a single day,It is a good opportunity to send all the team toSome fun time in Singapore
I closed the deal with the Australians
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation13
EventPatterns
Pattern detection is one of the notable functions ofevent processing
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation14
What we actually want to react to are – situation
TOLL VILOATOR FRUSTRATED
CUSTOMER
Sometimes the situation is determined by detecting thatsome pattern occurred in theFlowing events.
Toll violation Frustrated customer
Sometimes the events can approximate or indicate withsome certainty that the situation has occurred
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation15
Event processing is being used for various reasons
IBM Haifa Research Lab – Event Processing
IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
DetectDecide
Respond
EP Solution Segments – Business Value
Real-Time Operational
Information Dissemination
Observation
Predictive ProcessingActive Diagnostics
Reactions to events are done as part of business transactions – achieving low latency decisions, and quick reaction to threats and opportunities
Getting the
right
information
in the right
granularity
to the right
person at
the right
time
Diagnose
problems
based on
symptoms and
resolve them
Quick observation into exceptional business behavior and notification to the appropriate people.
Mitigate or
eliminate
predicted events
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation16
Event Driven Architecture
Event driven architecture: asynchronous, decoupled; each component is autonomic.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation17
Ancestor: active databases
On event
When condition
Do action
With coupling mode
Composite events were inherited to event processing
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation18
Ancestor: Data Stream management system
Source: Ankur Jain’s website
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation19
Ancestor: Temporal databases
There is a substantial temporal nature to event processing. Recently – also spatial and spatio-temporal functions are being added
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation20
Ancestor: Messaging – pub/sub middleware
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation21
Ancestor: Discrete event simulation
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation22
Ancestor: Network and system management
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation23
Event processing – architecture, building blocks
24
Flower StoreVan Driver
Ranking and Reporting
System
Bid Request
Delivery Bid
Assignments,
Bid alerts, Assign Alerts
Control System
GPS Location
Location Service
Location
Driver’s Guild
Ranking and reports
Delivery
confirmation
Pick Up confirmation
Ranked drivers / automatic assignment
Bid System
StorePreferences
Delivery Request
Assignment System
Manual Assignment Assignment
Assignments,
Pick Up Alert
Delivery Alert
Fast Flower Delivery
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation25
Event processing network
Event Producer 1
Event Consumer 1
Event Consumer 2
Event Producer 2
Event Consumer 3
Agent 2
Channel
Agent 1
State
Agent 3
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation26
Example of EPN – part of the FFD example
Automatic assignment
Driver
Bid routing
Store reference
Manual assignment preparationBid Request
channel Bid enrichment
Assignment manager
Alerts channel
Assignment channel
Assignment request channel
Driver status
Driver enrichment
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation27
Event Processing AgentContext
EventChannel
EventConsumer
EventType
EventProducer
GlobalState
The seven
Building blocks
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation28
Event type definition
Header Payload Open content
System defined event attributes
Attributes specific to the event type
Additional free format data included in the event instance
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation29
Producer – State Observer in workflows
State observerPush:
Instrumentation points;
Pull:
Query the state
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation34
Consumer - Performance monitoring dashboard
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation36
Producer and consumer - Sixth sense
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation38
Event Processing Agent
Filter Transform Detect Pattern
Translate Aggregate Split Compose
Enrich Project
Event Processing Agents
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation39
The EPA picture
Output
Not selected
Instanceselection
Context expression
Pattern detect EPA
Relevance filtering
Input terminal filter expression
Relevant event types
DerivationDerivation expression
MatchingPattern signature:
Pattern typePattern parametersRelevant event typesPattern policies
Pattern matching set
Participant events
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation40
Filter EPAA filter EPA is an EPA that performs filtering only, and has no matching or derivation steps, so it does not transform the input event.
Filtering
Filter EPA
Filtered Out
Filtered In
Non-Filterable
Principal filter expression
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation41
Transform EPA sub types
Translate Compose
Aggregate Enrich
Split
Project
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation42
Sample of pattern types
all pattern is satisfied when the relevant event set contains at least one instance of each event type in the participant set
any pattern is satisfied if the relevant event set contains an instance of any of the event types in the participant set
absence pattern is satisfied when there are no relevant events
relative N highest values pattern is satisfied by the events which have the N highest value of a specific attribute over all the relevant events, where N is an argument
value average pattern is satisfied when the value of a specific attribute, averaged over all the relevant events, satisfies the value average threshold assertion.
always pattern is satisfied when all the relevant events satisfy the always pattern assertion
sequence pattern is satisfied when the relevant event set contains at least one event instance for each event type in the participant set, and the order of the event instances is identical to the order of the event types in the participant set.
increasing pattern is satisfied by an attribute A if for all the relevant
events, e1 << e2 e1.A < e2.A
relative max distance pattern is satisfied when the maximal distance between any two relevant events satisfies the max threshold assertion
moving toward pattern is satisfied when for any pair of relevant events
e1, e2 we have e1 << e2 the location of e2 is closer to a certain object then the location of e1.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation43
Pattern detection example
Pattern name: Manual Assignment Preparation
Pattern Type: relative N highest
Context: Bid Interval
Relevant event types: Delivery Bid
Pattern parameter: N = 5; value = Ranking
Cardinality: Single deferred
Find the five highest bids within the bid interval
Taken from the Fast Flower Delivery use case
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation44
Why do we need policies - A simple example: heavy trading scenario
Given: A stream of events of a single topic, about the activity in the stock market for a certain stock.
An event is produced every 10 minutes when there is trade in the stock.
Each event consists of: quote (current stock-quote), volume (an accumulated volume of traded events within these 10 minutes).
A selection specification: “trigger an automatic trade program if the volume exceeds 300,000 3 times within an hour; pass as an argument the last quote and the sum of the 3 volume values”.
Event-Id Time-Stamp Quote Volume E1 9:00 33.23 E2 9:10 33.04 320,000 E3 9:20 33.11 280,000 E4 9:30 33.01 400,000 E5 9:40 32.90 315,000 E6 9:50 33.04 320,000 E7 10:00 33.20 303,000 E8 10:10 33.33 219,000 E9 10:20 33.11 301,000 E10 10:40 33.00 210,000 E11 10:50 32.78 400,000 E12 11:00 32.70 176,000
How many times the trade programming is triggered ;Which arguments are used in each triggering?
Why defining patterns is not that easy? Because we need to tune up the semantics
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation45
Pattern policies Evaluation policy—This determines when the matching sets are produced
Deferred
Cardinality policy—This determines how many matching sets are produced within a single context partition
Repeated type policy—This determines what happens if the matching step encounters multiple events of the same type
Every - Override – First – Last – with max/min value of…
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation46
Pattern policies – cont.
Consumption policy—This specifies what happens to a participant event after it has been included in a matching set
Consume Reuse Bounded reuse
Order policy—This specifies how temporal order is defined
By occurrence time – by detection time – by stream position – by attribute
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation47
Our entire culture is context sensitive
In the play “The Tea house of the August Moon” one of the characters says: Pornography question of geography
•This says that in different geographical contexts people view things differently
•Furthermore, the syntax of the language (no verbs) is typical to the way that the people of Okinawa are talking
When hearing concert people are not talking,
eating, and keep their mobile phone on “silent”.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation48
Context has three distinct roles (which may be combined)
Partition the incoming events
The events that relate to each customer are processed separately
Grouping events together
Different processing forDifferent context partitions
Determining the processing
Grouping together events that happened in the same hour at the same location
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation49
Context Definition
A context is a named specification of conditions that groups eventinstances so that they can be processed in a related way. It assignseach event instance to one or more context partitions.
A context may have one or more context dimensions.
A context is a named specification of conditions that groups eventinstances so that they can be processed in a related way. It assignseach event instance to one or more context partitions.
A context may have one or more context dimensions.
TemporalTemporal
SpatialSpatial
State OrientedState Oriented
Segmentation OrientedSegmentation Oriented
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation50
Context Types Examples
SpatialSpatial
State OrientedState Oriented
TemporalTemporal
ContextContext
“Every day between 08:00and 10:00 AM”
“A week after borrowing a disk”
“A time window bounded byTradingDayStart andTradingDayEnd events”
“Every day between 08:00and 10:00 AM”
“A week after borrowing a disk”
“A time window bounded byTradingDayStart andTradingDayEnd events”
“3 miles from the trafficaccident location”
“Within an authorized zone ina manufactory”
“3 miles from the trafficaccident location”
“Within an authorized zone ina manufactory”
“All Children 2-5 years old”“All platinum customers”“All Children 2-5 years old”“All platinum customers”
“Airport security level is red”“Weather is stormy”“Airport security level is red”“Weather is stormy”
Segmentation OrientedSegmentation Oriented
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation51
Context Types
Fixed locationFixed location
Entity distance locationEntity distance location
Event distance locationEvent distance location
SpatialSpatial
State OrientedState Oriented
Fixed intervalFixed interval
Event intervalEvent interval
Sliding fixed intervalSliding fixed interval
Sliding event intervalSliding event interval
TemporalTemporal
Segmentation OrientedSegmentation Oriented
ContextContext
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation52
The present: State of the practice in event processing – languages, implementation issues, challenges in implementing event processing applications
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation53
An ObservationThe Babylon Tower symbolizes the tendencyOf humanity to talk in multiple languages.
The Event Processing area is no different: most languages in the industry really followthe hammer and nails syndrome – and extended existing approaches• imperative script language• SQL extensions• Extension of inference rule language
The epts language analysis workgroup is aimed to understand the various stylesAnd extract common functions that can be used to define what is an event processing language; this tutorial is an interim report
It does not seem that we’ll succeed to settleIn the near future around a single programming style
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation54
Existing Styles for EP languages (samples)
InferenceRules
Stateoriented
Agent Oriented
Imperative/Script Based
RuleCore
Oracle
Aleri Streambase
Esper
EventZero
TIBCO
WBE
Apama
Spade
AMiT
Netcool Impact
* - if we add simple and mediated event processing the picture is even more diversified
ECARules
SQL extension
Coral8
Agent LogicStarview
XChangeEQProva
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation58
CCL – Pattern Matching
RFID monitoring application
Checks if a tag has been seen by readers A and B, then C, but not D, within a 10 second window.
Insert into StreamAlerts Select StreamA.id From StreamA a, StreamB b, StreamC c, StreamD d Matching [10 seconds: a && b, c, !d] On a.id = b.id = c.id = d.id
Insert into StreamAlerts Select StreamA.id From StreamA a, StreamB b, StreamC c, StreamD d Matching [10 seconds: a && b, c, !d] On a.id = b.id = c.id = d.id
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation59
Microsoft Streaminsights
var topfive = (from window in inputStream.Snapshot() from e in window
orderby e.f ascending, e.i descending select e).Take(5);
var avgCount = from v in inputStream group v by v.i % 4 into eachGroup
from window in eachGroup.Snapshot() select new { avgNumber = window.Avg(e => e.number) };
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation60
Esper EPL – FFD Example
/** Not delivered up after 10 mins (600 secs) of the request target delivery time*/insert into AlertW(requestId, message, driver, timestamp)select a.requestId, "not delivered", a.driver, current_timestamp()from pattern[
every a=Assignment (timer:interval(600 + (a.deliveryTime-current_timestamp)/1000) and
not DeliveryConfirmation(requestId = a.requestId) and
not NoOneToReceiveMSG(requestId = a.requestId))];
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation61
ruleCore - Reakt Event stream view - a unique context of events
a view contains a window into the inbound stream of events andcontains commonly only semantically related events
Situation - an interesting combination of multiple events as they occur over time
An item with an RFID tag being picked up from the shelf and then moving past the checkout without being paid for
Rule - an active event processing entity reacting to specific combinations of inbound events over time
Action - the last part of a rule's evaluation in response to a detected situation
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation62
Amit Terminology
Situation
Conditions
Lifespane5e5 e8e8
Initiator Terminator
Event SelectionEvent Selection ActionsActionsOperationOperation
Keys
e1e1
e3e3
Input events
OperatorJoiningCountingTemporalAbsenceAggregation
Notifications
Messages
Definition updates
User plug-ins
e2e2
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation67
Non functional properties - scalability
scalability in the volume of processed events
Scalability in the quantity of agents
Scalability in the quantity of producers
scalability in the quantity of consumers
scalability in the quantity of context partitions
Scalability in context state size
Scalability in the complexity of computation
scalability in the processor environment
Scalability is the capability of a system to adapt readily to a greater or lesser intensity of use, volume ,or demand while still meeting its business objectives .
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation68
Non functional requirements - securityEnsuring only authorized parties are allowed to be event producers
of event consumersEnsuring that incoming events are filtered so that authorized
producers cannot introduce invalid events, or events that they are not entitled to publish
Ensuring that consumers only receive information to which they are entitled. In some cases a consumer might be entitled to see some of the attributes of an event but not others.
Ensuring that unauthorized parties cannot add new EPAs to the system, or make modifications to the EPN itself (in systems where dynamic EPN modification is supported)
Keeping auditable logs of events received and processed, or other activities performed by the system.
Ensuring that all databases and data communications links used by the system are secure.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation69
Performance objectives
Objective
Number
Objective Name Objective metrics
1 MAX input
throughput
Maximize the quantity of input events processed by a
certain system or sub-system within a given time period
2 MAX output
throughput
Maximize the quantity of derived events produced by a
certain system or sub-system within a given time period
3 MIN average
latency
Minimize the average time it takes to process an event
and all its consequences in a certain system or sub-
system
4 Min maximal
latency
Minimize the maximal time it takes to process an event
and all its consequences in a certain system or sub-
system
5 Latency
leveling
Minimize the variance of processing times for a single
event or a collection of events in a certain system or sub-
system
6 Real-time
constraints
Minimize the deviation in latency, from a given value,
for the processing of an event and all its consequences in
a certain system or sub-system.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation70
Optimizations
Optimizations related to EPA assignment: partition, parallelism, distribution and load shedding.
Optimization related to the execution process: scheduling, routing optimizations and load shedding
Optimizations related to the coding of specific EPAs: code optimization, state management.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation71
Putting derived events in order
Auction Context Interval timeB
id event
Auctionentry
Anomaly 1
Bid
events
Auction
entries
Anomaly 2
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation72
Ordering in a distributed environment - possible issues
The occurrence time of an event is accurate, but the event arrives out-of-order and processing that should have included the event might already been executed.
Neither the occurrence time nor detection time can be trusted, so the order of events cannot be accurately determined.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation73
Solutions:
Buffering techniques: Assumptions:
Events are reported by the producers as soon as they occur;
The delay in reporting events to the system is relatively small, and can be bounded by a time-out offset;
Events arriving after this time-out can be ignored.
Principles: Let be the time-out offset,
according to the assumption it is safe to assume that at any time-point t, all events whose occurrence time is earlier than t - have already arrived.
Each event whose occurrence time is To is then kept in the buffer until To+, at which time the buffer can be sorted by occurrence time, and then events can be processed in this sorted order.
Retrospective compensation:
Find out all EPAs that have already sent derived events which would have been affected by the "out-of-order" event if it had arrived at the right time.
Retract all the derived events that should not have been emitted in their current form.
Replay the original events with the late one inserted in its correct place in the sequence so that the correct derived events are generated.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation74
Inexact event processing uncertainty whether an event actually occurred inexact content in the event payload inexact matching between derived events and the situations they purport to
describe
Sourcemalfunction
Malicioussource
Projection oftemporal
anomalies
Imprecise source
Sampling orestimate
Propagation of
inexactness
Uncertainevent
Inexact event content
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation75
False positives and false negatives
False positive situation detection refers to cases in which an event representing a situation was emitted by an event processing system, but the situation did not occur in reality.
False negative situation detection refers to cases in which a situation occurred in reality, but the event representing this situation was not emitted by an event processing system
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation76
Retraction
Delivery Request issued
Bid Request issued
Assignment issued
Pick-up occurred
Delivery occurred
Order Cancellation
Cancel bid preparation
Cancel bid and assignm
ent process
Notify assigned driver about
cancellation
Undoable
May still be cancelled w
ith some
penalty
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation77
The future: Trends and research challenges
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation78
Trend I: Going from narrow to wide
Border security radiation detection
Mobile asset geofence
Logistic and scheduling
Unauthorized use of heavy machinery
Hospital patient and asset tracking
Activity monitoring for taxing and fraud detection
Intelligent CRM in banking
EDA and asynchronous BPM in retail
Situation awareness in energy utilities
Situation awareness in airlines
Reduce cost in injection therapy
Next generation navigation
Real-time management of hazardous materials
Finding anomalies in point of sales in retail stores
Elderly behavior monitoring
Some recently reported applications (EPTS use-cases WG)
Source: ebizQ Event processing market pulse
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation79
Trend I: Going from narrow to wide
Taking event processing outside enterprise computing:
Robotics
Bio-Informatics
Socio-technical systems
HomeAutomation
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation80
Trend II: Going from monolithic to diversified
Variety of functions
Variety of Quality of Service requirements:
Variety of platforms
“One size fits all” will not work – Instead a collection of building blocksthat can fit together
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation81
Trend III: Going from proprietary to standard-based – standard directions
The current situation: Babylon tower: variety of languages, event representation…
Serves as enabler to achieving other trends and general maturity
The shift of vendors from start-up dominant to bigger companies makes the atmosphere more friendly towards standards.
Areas forStandards:ModelingEvent representationInteroperabilityLanguages PIM
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation82
Trend IV: Going from programmer centered to semi-technical person centered
Source: ebizQ Event processing market pulse
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation83
Trend V: Going from stand-alone toembedded
Packaged applications
Business Activity Monitoring
Sensor Platform
Middleware and platforms
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation84
Trend VI: Going from reactive to proactive
TRAFFIC JAM
End of game
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation85
Emerging directions: Four directions to observe
Multiple platforms – same look and feel
The engineering of constructing EP applications
Adding intelligence toEvent processing
Tailor-made optimizations
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation86
Emerging direction I: multiple platforms – same look and feel
ApplianceAppliance Stream Platform
Stream Platform
Cloud computingPlatform
Cloud computingPlatform
ESB / Messaging Platform
ESB / Messaging Platform
EmbeddedEmbedded
Virtual Event Processing Platform
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation87
Emerging direction II: Tailor-made optimizations
Local optimizations: each EPA will be optimized for its own purpose / assumptions / QoS indicators – average/worst case latency, input/output throughput …
Global optimization: scheduling, load balancing, assignment…
EPA EPA
EPA
EPA
Producer
Producer
EPA
Consumer
Consumer
Local optimizations
Global optimizations
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation88
Emerging direction III: Event processing software engineering
Best practices
MethodologiesModeling
& meta-modeling
DesignPatterns
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation89
Emerging direction IV: Intelligent event processing
Offline and continuous mining of meaningful patterns in event histories
Inexact event processing – handling inexact events and also false positives and false negatives
Causality – a key for proactive, but also vital for provenance
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation90
Summary
Event processinghas emerged fromsome academic disciplines
The state of the Practice is the 1st generation of products – mainlyengineering based
Going to the nextphase – many challenges that require collaborationof research andindustry
Already attracted coverage of analysts and all major software vendors