Oracle Reference Architecture · that are tailored and enhanced for industry verticals. Each...

Oracle Reference ArchitectureEvent Driven Architecture Foundation

Release 3.0.1

E17785-03

September 2010

ORA Event Driven Architecture (EDA) Foundation, Release 3.0.1

E17785-03

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

Primary Author: Anbu Krishnaswamy

Contributing Author: Stephen G. Bennett, Robin Smith, Alex Alves, Eric Hsiao, Payal Srivastava

Contributor: Dave Chappelle, Bob Hensle, Mark Wilkins, Patrick McLaughlin, Sanjay Baldwa, Lloyd Williams

Warranty Disclaimer

THIS DOCUMENT AND ALL INFORMATION PROVIDED HEREIN (THE "INFORMATION") IS PROVIDED ON AN "AS IS" BASIS AND FOR GENERAL INFORMATION PURPOSES ONLY. ORACLE EXPRESSLY DISCLAIMS ALL WARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. ORACLE MAKES NO WARRANTY THAT THE INFORMATION IS ERROR-FREE, ACCURATE OR RELIABLE. ORACLE RESERVES THE RIGHT TO MAKE CHANGES OR UPDATES AT ANY TIME WITHOUT NOTICE.

As individual requirements are dependent upon a number of factors and may vary significantly, you should perform your own tests and evaluations when making technology infrastructure decisions. This document is not part of your license agreement nor can it be incorporated into any contractual agreement with Oracle Corporation or its affiliates. If you find any errors, please report them to us in writing.

Third Party Content, Products, and Services Disclaimer

This document may provide information on content, products, and Services from third parties. Oracle is not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and Services. Oracle will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or Services.

Limitation of Liability

IN NO EVENT SHALL ORACLE BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES, OR DAMAGES FOR LOSS OF PROFITS, REVENUE, DATA OR USE, INCURRED BY YOU OR ANY THIRD PARTY, WHETHER IN AN ACTION IN CONTRACT OR TORT, ARISING FROM YOUR ACCESS TO, OR USE OF, THIS DOCUMENT OR THE INFORMATION.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

Contents

Preface ................................................................................................................................................................. ix

Introduction to IT Strategies from Oracle (ITSO) ................................................................................... ixDocument Purpose and Scope................................................................................................................... xAudience....................................................................................................................................................... xiDocument Structure .................................................................................................................................... xiHow to Use This Document...................................................................................................................... xiiConventions ................................................................................................................................................ xii

1 Introduction

1.1 Real-Time Enterprise (RTE)....................................................................................................... 1-11.2 EDA Key Drivers ........................................................................................................................ 1-21.3 Application of Event Driven Architecture .............................................................................. 1-31.4 Events and Event Driven Architecture - An Example........................................................... 1-31.4.1 Airlines Example.................................................................................................................. 1-31.4.2 Financial Trading Example ................................................................................................ 1-4

2 EDA Conceptual View

2.1 EDA Definitions .......................................................................................................................... 2-12.1.1 Event ...................................................................................................................................... 2-12.1.2 Event Driven Architecture (EDA) ..................................................................................... 2-12.2 Event Types.................................................................................................................................. 2-12.3 Event Processing ......................................................................................................................... 2-32.3.1 Simple Event Processing (SEP) .......................................................................................... 2-32.3.2 Event Stream Processing (ESP) .......................................................................................... 2-32.3.3 Complex Event Processing (CEP)...................................................................................... 2-32.4 Event Repository (aka Event Catalog) ..................................................................................... 2-42.5 Event Processing Network (EPN)............................................................................................. 2-52.6 Event Delivery Network (EDN) ............................................................................................... 2-52.7 Other Terms and Concepts........................................................................................................ 2-62.7.1 Continuous Query Language (CQL)................................................................................. 2-62.7.2 Sliding Windows.................................................................................................................. 2-72.7.2.1 Count-based sliding window ..................................................................................... 2-72.7.2.2 Time-based sliding window ....................................................................................... 2-72.7.3 Batched Windows................................................................................................................ 2-72.7.3.1 Count-based batched window.................................................................................... 2-7

iii

2.7.3.2 Time-based batched window...................................................................................... 2-72.7.4 Situation Awareness............................................................................................................ 2-72.7.5 Event Channels .................................................................................................................... 2-82.7.6 Event Producer or Event Source........................................................................................ 2-82.7.7 Event Consumer or Event Sink.......................................................................................... 2-82.7.8 Event Generator ................................................................................................................... 2-82.7.9 Event Sensors........................................................................................................................ 2-82.8 Event Lifecycle............................................................................................................................. 2-82.9 EDA High Level Conceptual View ....................................................................................... 2-102.9.1 Producers ........................................................................................................................... 2-112.9.2 External Feeds/Events ..................................................................................................... 2-112.9.3 Detection and Capture ..................................................................................................... 2-112.9.4 Processing .......................................................................................................................... 2-122.9.5 Contextual/Historical Data............................................................................................. 2-122.9.6 Distribution........................................................................................................................ 2-122.9.7 Consumers ......................................................................................................................... 2-122.10 EDA Capabilities...................................................................................................................... 2-122.10.1 Engineering........................................................................................................................ 2-132.10.1.1 Event Repository ....................................................................................................... 2-132.10.1.2 Metadata Management ............................................................................................. 2-142.10.1.3 Event Search and Discovery .................................................................................... 2-142.10.1.4 Usage Tracking .......................................................................................................... 2-142.10.1.5 Modeling..................................................................................................................... 2-142.10.1.6 Testing ......................................................................................................................... 2-142.10.2 Event Generation and Capture ....................................................................................... 2-142.10.2.1 Sensing ........................................................................................................................ 2-142.10.2.2 Generation .................................................................................................................. 2-142.10.2.3 Adaptation.................................................................................................................. 2-152.10.3 Event Processing ............................................................................................................... 2-152.10.3.1 Correlation.................................................................................................................. 2-152.10.3.2 Filtering ....................................................................................................................... 2-152.10.3.3 Enrichment ................................................................................................................. 2-152.10.3.4 Splitting....................................................................................................................... 2-152.10.3.5 Batching ...................................................................................................................... 2-152.10.3.6 Prediction.................................................................................................................... 2-152.10.3.7 Lineage ........................................................................................................................ 2-162.10.3.8 Pattern Recognition ................................................................................................... 2-162.10.3.9 Aggregation................................................................................................................ 2-172.10.3.10 Routing........................................................................................................................ 2-172.10.3.11 Transformation .......................................................................................................... 2-172.10.3.12 Query Language ........................................................................................................ 2-172.10.3.13 Recording.................................................................................................................... 2-172.10.3.14 Playback ...................................................................................................................... 2-172.10.3.15 Language Extension .................................................................................................. 2-182.10.4 Event Distribution and Response................................................................................... 2-182.10.4.1 Processing Networks................................................................................................. 2-182.10.4.2 Caching ....................................................................................................................... 2-18

iv

2.10.4.3 Channels ..................................................................................................................... 2-182.10.4.4 Subscription................................................................................................................ 2-182.10.4.5 Broadcast..................................................................................................................... 2-192.10.4.6 Reception .................................................................................................................... 2-192.10.5 Event Monitoring and Management.............................................................................. 2-192.10.5.1 Reporting .................................................................................................................... 2-192.10.5.2 Alerts ........................................................................................................................... 2-192.10.5.3 Analytics ..................................................................................................................... 2-192.10.5.4 Dashboards................................................................................................................. 2-192.10.5.5 Exception Management ............................................................................................ 2-192.10.5.6 Business Process Control.......................................................................................... 2-192.11 Event Meta-model.................................................................................................................... 2-202.12 EDA Architecture Principles .................................................................................................. 2-222.12.1 Loose Coupling ................................................................................................................. 2-222.12.2 Standards Based................................................................................................................ 2-232.12.3 Discoverable ...................................................................................................................... 2-232.12.4 Interoperable ..................................................................................................................... 2-232.12.5 Granularity ........................................................................................................................ 2-242.12.6 Business Event Focus ....................................................................................................... 2-242.12.7 Extensibility ....................................................................................................................... 2-242.12.8 Low latency, High Volume ............................................................................................. 2-242.12.9 Security............................................................................................................................... 2-25

3 EDA Standards and Technologies

3.1 The Bayeux Specification ........................................................................................................... 3-13.2 OSGi .............................................................................................................................................. 3-23.3 Java Messaging Service (JMS) ................................................................................................... 3-23.4 JSR 190: Event Tracking API for Java Micro Edition ............................................................. 3-23.5 WS-Notification........................................................................................................................... 3-23.5.1 WS-BaseNotification............................................................................................................ 3-23.5.2 WS-BrokeredNotification ................................................................................................... 3-33.5.3 WS-Topics ............................................................................................................................. 3-33.6 WS-Eventing ................................................................................................................................ 3-33.7 EPC IS ........................................................................................................................................... 3-33.8 Java Management eXtension (JMX).......................................................................................... 3-43.9 Data Distribution Service (DDS) Specification by OMG....................................................... 3-43.10 Extensible Messaging and Presence Protocol (XMPP) .......................................................... 3-4

4 EDA/Core ORA Interlock

4.1 EDA, SOA, BPM, and BAM....................................................................................................... 4-14.2 EDA and Security........................................................................................................................ 4-34.2.1 Authentication...................................................................................................................... 4-34.2.2 Authorization ....................................................................................................................... 4-44.2.3 Confidentiality ..................................................................................................................... 4-44.3 EDA and ORA Engineering ...................................................................................................... 4-44.4 EDA and ORA Integration ........................................................................................................ 4-5

v

5 Summary

Glossary

A Further Reading and References

A.1 Related Documents.................................................................................................................... A-1A.1.1 Suggested Pre-reading ....................................................................................................... A-1A.2 Other Resources and References.............................................................................................. A-1A.3 EDA Standards ........................................................................................................................... A-2

vi

viii

List of Figures

1–1 EDA Example - Airlines............................................................................................................. 1-41–2 Trading System Example ........................................................................................................... 1-52–1 Event Types.................................................................................................................................. 2-22–2 Processing domains .................................................................................................................... 2-42–3 Event Processing Network (Sample)........................................................................................ 2-52–4 Event Delivery Network (EDN)................................................................................................ 2-62–5 Event Lifecycle............................................................................................................................. 2-92–6 EDA Simplified Conceptual View......................................................................................... 2-102–7 High Level Conceptual View ................................................................................................. 2-102–8 EDA Capabilities...................................................................................................................... 2-132–9 EDA Pattern Recognition Example ....................................................................................... 2-162–10 Sample Event Structure........................................................................................................... 2-202–11 EDA Relationships................................................................................................................... 2-212–12 Event Meta-model - Design-time........................................................................................... 2-212–13 Event Meta-model - Runtime ................................................................................................. 2-224–1 EDA, SOA, BPM and BAM........................................................................................................ 4-24–2 Event-Driven Services ................................................................................................................ 4-24–3 Events Produced by a Service ................................................................................................... 4-3

Preface

Oracle Reference Architecture is a product-agnostic reference architecture based on architecture principles and best practices that are widely applicable and that can be implemented using a wide variety of products and technologies. Oracle Reference Architecture (ORA) does not include any implementation artifacts for the prescribed architecture. ORA addresses the building of a modern, consistent IT architecture while minimizing the risk of product incompatibilities and obsolescence.

The ORA EDA documents present the ORA architectural concepts from the perspective of EDA, highlighting the specific details of EDA as an elaboration of the ORA core concepts with respect to this technological approach. This ORA EDA perspective comprises two documents:

■ EDA Foundation: primarily a reference architecture for EDA, including principles, standards, definition of EDA, EDA concepts, and its relationship to ORA

■ EDA Infrastructure: relates the EDA capabilities, as defined by the reference architecture, to the Oracle infrastructure and provides a number of architecture views to help the architects and developers focusing on EDA.

This document is part of a series of documents that describe the Oracle Reference Architecture (ORA) Event Driven Architecture (EDA) strategy. This "EDA Foundation" document provides the underlying architectural definitions of associated EDA concepts. Contained within this document are background and definitions of EDA along with detailed definitions of Eventing concepts. The EDA Foundation document presents important basic concepts of EDA that are fundamental to building applications for an enterprise EDA environment.

A conceptual architectural framework is provided as the foundation for ORA describing a layered Events model and supporting infrastructure. The main focus of this document is to address the challenges faced by architects attempting EDA, by explaining the underpinning architectural concepts that distinguish EDA from architectural approaches.

Introduction to IT Strategies from Oracle (ITSO)IT Strategies from Oracle (ITSO) is a series of documentation and supporting collateral designed to enable organizations to develop an architecture-centric approach to enterprise-class IT initiatives. ITSO presents successful technology strategies and solution designs by defining universally adopted architecture concepts, principles, guidelines, standards, and patterns.

ix

ITSO is made up of three primary elements:

■ Oracle Reference Architecture (ORA) defines a detailed and consistent architecture for developing and integrating solutions based on Oracle technologies. The reference architecture offers architecture principles and guidance based on recommendations from technical experts across Oracle. It covers a broad spectrum of concerns pertaining to technology architecture, including middleware, database, hardware, processes, and services.

■ Enterprise Technology Strategies (ETS) offer valuable guidance on the adoption of horizontal technologies for the enterprise. They explain how to successfully execute on a strategy by addressing concerns pertaining to architecture, technology, engineering, strategy, and governance. An organization can use this material to measure their maturity, develop their strategy, and achieve greater levels of success and adoption. In addition, each ETS extends the Oracle Reference Architecture by adding the unique capabilities and components provided by that particular technology. It offers a horizontal technology-based perspective of ORA.

■ Enterprise Solution Designs (ESD) provide extensive architecture perspectives that are tailored and enhanced for industry verticals. Each solution design is built on a customization of ORA that includes specific capabilities, processes, services, and applications for that industry. Oracle products are mapped to the solution designs in order to illustrate where they fit within the overall industry architecture.

ORA EDA Foundation, along with ORA EDA Infrastructure, extend the Oracle Reference Architecture. They are part of a series of documents that comprise the EDA Enterprise Technology Strategy, which is included in the IT Strategies from Oracle collection.

Please consult the ITSO web site for a complete listing of SOA and ORA documents as well as other materials in the ITSO series.

Document Purpose and ScopeBusiness landscape in today's environment is very dynamic and demanding. Businesses can no longer be operating in reactive mode. If they want to maintain and increase their competitive advantage, they must become very agile and proactive. Business events must be acted upon in a timely manner and business decisions must be made in real time. Situation awareness provides businesses competitive advantage that others may lack.

x

http://www.oracle.com/goto/itstrategies

Traditional "pull" based paradigms are not best suited to handle this need alone. They need to be complemented with "push" based technologies to create this niche. Event Driven Architecture (EDA) is one such approach that nicely complements the existing enterprise technologies to provide the infrastructure needed to support the real-time, event-based push architectures.

The other important driver for EDA is the need to analyze complex relationships between business events and identify business opportunities, threats, and anomalies that need to be addressed. Business users need this insight in time to improve the quality of the decisions they make and to stay nimble in business.

Today, probes and sensors are deployed in everything from IT networks to enterprise software systems and physical world devices (through RFID readers, barcode scanners, manufacturing equipment sensors, and others). As these systems continue to proliferate, they generate events at a growing rate. Significant improvements in operational business decisions await those organizations that can capture and process these events into meaningful business insight. Analyzing and interpreting millions of minor events is beyond human capability. EDA is the solution for this problem of handling a high volume of events.

The underlying ORA core concepts are not reproduced in this perspective document except where it is necessary to show EDA-specific views.

As an architectural foundation, the EDA perspective includes a conceptual architecture definition, architectural principles for EDA, and an outline of associated standards and technologies. It lays the foundation for the next document in the series, the EDA Infrastructure.

The EDA Foundation document does not attempt to offer a methodology for the implementation of an EDA strategy, although it is expected that it will be used in conjunction with such a methodology as the foundation for the architectural approach.

AudienceThis document is intended for enterprise architects, application architects, project managers and developers. The material is designed for a technical audience that is interested in learning about the intricacies of Event Driven Architecture and how to design and build enterprise class EDA infrastructure.

Document StructureThis document is organized into chapters that introduce EDA concepts, standards, principles and ORA interlock.

Chapter 1, "Introduction" - provides an introduction to Event Driven Architecture (EDA) and describes the drivers and benefits of EDA.

Chapter 2, "EDA Conceptual View" - provides a conceptual model for ORA Event Driven Architecture and describes the capabilities required for EDA infrastructure.

Chapter 3, "EDA Standards and Technologies" - describes the standards and technologies related to EDA.

Chapter 4, "EDA/Core ORA Interlock" - describes how EDA relates to the rest of the elements in the Oracle Reference Architecture.

Chapter 5, "Summary" - summarizes the ORA EDA Foundation document.

"Glossary" - provides a list of terms specific to this document. ORA master glossary provides the terms that are common across the documents.

xi

Chapter A, "Further Reading and References" - provides a list of documents and reading resources for further information and reference.

How to Use This DocumentThis document is designed to be read from beginning to end. Those that are already familiar with EDA may wish to skip the introduction chapters and proceed with the conceptual view.

ConventionsThe following typeface conventions are used in this document:

Convention Meaning

boldface text Boldface type in text indicates a term defined in the text, the ITSO Master Glossary, or in both locations.

italic text Italics type in text indicates the name of a document or external reference.

underline text Underline text indicates a hypertext link.

xii

1
1Introduction
With any new technology, it is important to get the concepts right before diving deeply into the architecture. This is especially true for evolving technologies that are still in the hype cycle. These technologies in general haven't matured and most companies consider them risky to adopt.

Although EDA has been around a while, at least conceptually, it has picked up steam in recent years. Several new products and technologies around EDA have evolved in the recent past and some of these are driven by the development in other related technologies like Service Oriented Architecture (SOA) and Business Process Management (BPM).

Most companies do not view EDA in isolation; rather, they implement EDA in association with other technologies that complement EDA. For example, many companies consider EDA as an extension to SOA to add "event driven" capabilities to their SOA infrastructure. Event Driven SOA (ED-SOA), as it is called, is the extended SOA that includes the integration of EDA and SOA so that events can be produced by SOA Services and SOA Services can be triggered through event based mechanisms. Oracle Reference Architecture uses composition as a method to combine these separate ETSs.

This chapter explores the definition, concepts, and terminology behind EDA and discusses the business benefits of the technology.

1.1 Real-Time Enterprise (RTE)Many consider Event Driven Architecture (EDA) as a pure technology play. Some misunderstand EDA as a messaging solution. However, EDA is one of the key technology enablers of agile enterprises that strive to become Real-Time Enterprises. Along with technology strategies such as SOA and BPM, EDA provides the ability to organizations to deliver business information to the business users in real-time and allows them to react and respond in a timely fashion. This effectively reduces the insight to action gap and helps companies to run more competitive than their peers.

Gartner defines RTE as “The Real-Time Enterprise (RTE) is an enterprise that competes by using up-to-date information to progressively remove delays to the management and execution of its critical business processes.” Gartner also asserts that Real-Time Enterprise is event-driven and they should expand their use of event-oriented design, message-oriented middleware, and publish-subscribe communication. The use of Complex Event Processing and Business Activity Monitoring (BAM) remove unnecessary delays in the business process and ensure that right information is delivered to the right people at the right time.

Finger and Bellini define RTE in their book "The Real-Time Enterprise" as, "The real-time enterprise is a customer-driven organization that executes and adapts its

Introduction 1-1

EDA Key Drivers

mission-critical business processes using a sense and respond infrastructure that spans people, companies and computers to provide information that is timely enough to make effective decisions and act, and where a late answer is a wrong answer." This definition clearly leads to the fact that EDA is an enabler of a much more important mission of an organization, which is to deliver value to customers and partners by sensing and responding to their needs in a fast and timely manner.

1.2 EDA Key DriversEDA has several drivers, some of which are business oriented and others that are technology oriented. The following list summarizes the key business and technology drivers.

■ Business Agility is one of the key drivers of RTE and EDA. In order to stay nimble, today's businesses need to quickly identify opportunities, threats, and anomalies, and react to them appropriately in a fast and efficient manner. An immediate response is required in many cases to maintain the competitive edge of the business. EDA provides fast and time sensitive insight information about the business activities to act promptly. This is particularly helpful in the areas of fraud detection, risk management, compliance, real-time Business Intelligence (BI), and Business Activity Monitoring (BAM).

■ Mergers and Acquisitions (M&A) are a very common trend in this economy where consolidation plays a key role in the strategy of businesses. EDA can make M&A easier on the technology side as loosely coupled systems integrate faster and painlessly.

■ EDA allows businesses to derive "situational awareness" from the analysis of events by constructing an actionable view of the business using events from within and around the business. This means that risks and opportunities are surfaced in real-time. EDA can enable automated responses to certain business conditions, e.g. out of stock condition or delayed flights. It can tie into BPM to trigger programmed responses to conditions as they arise.

■ EDA makes partner integration flexible. Business to Business (B2B) integrations can make use of the EDA infrastructure on both sides of the partner network to provision services faster and easier.

■ Some industries require very fast, low latency event processing applications to support their business requirements and to fulfill the client SLAs.

■ There is a need to support multiple subscribers as many parties in the organization are interested in the same events. They may take different actions based on their function but they need to know when the event occurs and the nature of it.

■ More and more enterprise architectures are adopting loosely coupled architectures. Loose coupling enables IT systems to be flexible supporting business agility. EDA enables loose coupling by separating producers and consumers. This means that consumers can be added on the fly without affecting the existing setup.

■ EDA enables the use of events as a communication mechanism between SOA services. It expands the classic SOA pattern of request/response to include sense/respond (i.e. apply SOA to a broader set of applications).

■ From an IT standpoint, the need to create a potential foundation for a more flexible systems/network management platform by deriving situational awareness from application and system level events is another driver for EDA.

1-2 ORA Event Driven Architecture (EDA) Foundation

Events and Event Driven Architecture - An Example

1.3 Application of Event Driven ArchitectureEDA is applicable almost in any industry but certain industries benefit more so than others. The industries with the following characteristics would benefit from the capabilities added by EDA.

■ Some industries deal with large volume of business events. These events may be consumer generated or device generated. An example would be RFID sensor events generated from the warehouse of a large distributor.

■ Some applications require fast, low latency response times. For example, financial services and the field of algorithmic trading require fast response times. EDA can be applied for such use cases.

■ The need for correlating events in real-time as they happen is a common requirement in many industries. This may require complex pattern matching capabilities combined with time window processing. Fraud detection is an example where data from various disparate sources need to be correlated to detect possible fraud scenarios.

■ The growth in social computing and enterprise mash-ups have created unprecedented demand for real-time geospatial applications. EDA has proved to be an useful technology for implementing the real-time geospatial applications.

1.4 Events and Event Driven Architecture - An ExampleThis section describes two examples to demonstrate the concept of events and EDA. The first example illustrates a scenario in the airlines industry. The second one illustrates a financial trading example.

1.4.1 Airlines ExampleFigure 1–1 shows an example of Event Driven Architecture (EDA) in the airlines industry. It shows various functions of an airline including reservations, ticketing, flight operations, gate operations, and baggage handling.

Introduction 1-3


Figure 1–1 EDA Example - Airlines

The business activities that happen within each of the functions result in business events such as check-in, flight-open, or passenger-on. For large airlines, the number of events on a busy day could be overwhelming. Some of these events are internally produced and distributed, while some are generated directly by equipment such as flight gears or gate scanners. For example, the flight-push-back and flight-wheels-up events shown in the diagram are generated by the flight movement.

The EDA infrastructure filters, consolidates, and routes the events appropriately. It also integrates with the enterprise systems by providing the information in a loosely coupled manner in a format they can handle.

Business Activity Monitoring systems and business processes can receive the events and take appropriate actions to react to certain events or to remedy certain situations. For example, a flight-delay event may further trigger a passenger notification event to notify the passengers and other subscribers.

It is also common to have external events feed into the EDA infrastructure for further aggregation. For example, an external agency like Federal Aviation Authority (FAA) or Federal Bureau of Investigation (FBI) may send terrorist alerts that need to be cross-checked with the ticketing and gate operation events. The EDA infrastructure can correlate these events in real-time and notify the business users of any business exceptions.

1.4.2 Financial Trading ExampleEDA has several uses in the financial industry. The example shown in Figure 1–2 illustrates a trading scenario that may be applicable to instruments like stocks and options.



Figure 1–2 Trading System Example

Several subsystems participate in this use case either by publishing events, receiving events, or both. Trading systems could be very complex, but this example simplifies the scenario by showing only the key events and interactions of the trade execution use case.

Trading systems have always demanded the highest possible performance to fulfill orders faster than the competition to ensure accurate execution of orders. Failing to execute orders in a timely fashion may cost financial institutions millions of dollars in lost revenue.

Institutions generally subscribe for quotes or rates from the market providers. These quotes are usually high volume and fast moving. EDA helps to process such high volume of events to filter out the significant changes that may affect the trading decisions.

Most internal applications are interested in the status of orders. They subscribe to the order placement, cancellation, and change events that are published by the Order Management system. The trading applications are very much inter-dependent and need to exchange information in real-time. Failure to do so may cause inconsistency and customer dissatisfaction. For example, a cancelled order should be executed and vice versa.

Some changes to the account may affect trading decisions. For example, a negative balance on a non-margin account may lead to cancellation of all pending orders. EDA allows the account changes to be dynamically propagated to the interested systems, thus allowing such decisions to be made in a timely manner.

Trader console allows traders to manually control trade execution by cancelling trades, pricing manually, and turning the automatic trading on or off.

Trade engines receive all interested events and make the decision so as to execute or cancel the order. This is a very important step, as several downstream systems are interested in when the trade happens. Post trade, audit, and settlement systems need to take appropriate auditing, settlement, and other actions as soon as the trade is executed. EDA can distribute such events to the appropriate systems for further processing.

Although this simple example looked at only a few of the possible events in a trading system, in real world the generated events are much more complex and

Introduction 1-5


inter-dependent. The patterns are identified using EDA and are used in areas such as fraud detection, compliance, and risk management.


2
2EDA Conceptual View
This section presents the event lifecycle, terms, and concepts related to EDA.

2.1 EDA Definitions

2.1.1 EventThere are several different ways an event is defined in the industry but a widely recognized definition is provided below.

An Event is a change in any state that indicates a business opportunity, threat, anomaly, or just a notification. It is something that happens inside or outside the business that should be noted or acted upon. Events may refer to the specification or the instance of the occurrence.

Events may be business-centric or technology-related. Technical events (or raw events) are much more fine grained and are typically used in system monitoring and management. The focus of EDA is generally business-centric events that may be defined at various levels of granularity. In some cases, the events may not appear business-centric but in truth, they may just be business events at a granular level signifying the occurrence of a business activity (e.g. movement of a good or change of a stock quote).

2.1.2 Event Driven Architecture (EDA)Extending the definition of an Event to Event Driven Architecture, EDA is defined as a style of loosely coupled architecture that enables Real-Time Enterprise (RTE) through the identification of, and reaction to, the business opportunities, threats and anomalies. EDA facilitates production, detection, processing, and consumption of the business events. EDA defines the organization of components that interact through the exchange of events.

2.2 Event TypesEvent type denotes the classification of events. Events may be classified using a number of common criteria. For example, based on the focus of the event, they can be classified as business events or technical events. This section looks at the semantic classification of events.

With respect to EDA, events are classified into the following categories.

■ Ordinary Events

■ Notable Events

EDA Conceptual View 2-1

Event Types

■ Stream Events

■ Transactional Events

Figure 2–1 Event Types

Figure 2–1 shows these event types and how they relate to the event volume and significance. Highly significant events must be transported and processed reliably using reliable messaging and High Availability techniques. Insignificant events are in general dispensable and can be implemented using in-memory messaging for speed and cost reasons.

The table below summarizes the description and examples of each of these event types.

Event Type Description Example

Ordinary Events Common non-critical events that are typically generated intermittently.

Information alerts, non-critical business events

Notable Events Events that are important and need to be acted upon. These events typically occur at low volume.

Flight take off, customer service request, new application submission

Stream Events Continuous stream of events that may not be significant individually but may be part of a critical pattern.

Stock quotes, RFID scans, heartbeats


Event Processing

2.3 Event ProcessingEvents at source are rarely actionable. This means that they are not significant enough for the business to react and take a meaningful action. This is especially true for streaming and high volume events. These events need to be further processed to derive actionable events. Event processing is the manipulation of events through the application of business rules to identify opportunities, threats, and anomalies that need to be acted upon.

2.3.1 Simple Event Processing (SEP)Simple Event Processing (SEP) is the processing of ordinary or notable events that may initiate further downstream actions. SEP can be achieved with simple queuing implementations.

2.3.2 Event Stream Processing (ESP)Event Streams are continuous input events that often occur in high-volume. They are generally time ordered and do not end. They are almost impossible to process or analyze in real-time with traditional relational database management systems.

Event Stream Processing (ESP) is the processing of linearly ordered sequence of events to derive notable events and stream to interested subscribers. ESP provides a new data management infrastructure to support and analyze streams in real-time.

For example, stock quotes may be streamed at a frequent rate but the consumer may only be interested when the stock price changes. ESP can identify the stock price change and raise a notable event to the consumer. Similarly there are situations where duplications need to be eliminated in a flood of events like in RFID streams. ESP can be applied in the scenario for eliminating duplicates.

2.3.3 Complex Event Processing (CEP)Complex Event Processing is the processing of a confluence of events over a time window for complex patterns and correlations that may infer business opportunities, threats, and anomalies. CEP is a technique for analyzing streams of event data in real-time, improving situational awareness and enabling immediate response to the changing market conditions.

Forrester defines CEP as "Software infrastructure that can detect patterns of events (and expected events that didn't occur) by filtering, correlating, contextualizing, and analyzing data captured from disparate live data sources to respond as defined using the platform's development tools."

CEP requires complex, high performance infrastructure that supports pattern recognition, time window processing, and continuous query language.

Figure 2–2 shows the CEP domain by volume of events and relativity between the individual events. The relativity could be causal, temporal, spatial, or others. CEP is a

Transactional Events Important events generated through business transactions. These could be high volume events but may not necessarily have a correlating pattern.

Retail sales events, stock trades, flight ticket bookings

Event Type Description Example


Event Repository (aka Event Catalog)

great choice when the relativity is high regardless of the volume of events. Also SEP, ESP, and CEP can be applied to events of any significance. This is why event significance is not shown as a dimension in this diagram. CEP can process really high volume of events within - what is referred to as - the "event cloud". Event cloud contains events from multiple unrelated streams of events.

Figure 2–2 Processing domains

Low volume events that are not really related to each other are a good fit for Simple Event Processing. These events have a low correlation ratio and may not require advanced event processing capabilities like pattern recognition.

CEP is sometimes referred to as Business Event Processing (BEP) to show the focus on the business level events.

Streaming events are typically in the high volume category. Event Stream Processing (ESP) is a more simplified form of CEP that can be used to process such stream events.

2.4 Event Repository (aka Event Catalog)In medium to large enterprises, the specification of events should be maintained in a common repository for visibility, sharing and reuse. Just like any other technology, governance is as important if not more so with EDA.

In order to avoid potential "event sprawl", where multiple groups duplicate the specification and production of events, an enterprise scoped event repository should be maintained. The event repository is where business descriptions, specifications, processing rules, inter-relationships, and consumption information of events are stored and managed.


Event Delivery Network (EDN)

2.5 Event Processing Network (EPN)EDA is generally composed of several processing steps intermingled with user logic. Events are filtered and manipulated through a series of processing steps that derive useful and actionable sub-events. This kind of processing requires the components of EDA such as the event processors and channels to be connected in a network fashion.

Figure 2–3 Event Processing Network (Sample)

Event Processing Network (EPN) is the application model in which modular network of event processors are connected through adapters, channels, streams, caches, and listeners to perform a sequence of business process steps based on the processing outcome of individual nodes. Figure 2–3 illustrates an Event Processing Network.

An EPN can be arranged into different forms for different purposes. For example, a simple "producer -> processor -> consumer, consumer" would be used for routing, whereas a "producer -> processor -> processor -> consumer" configuration would be used for data aggregation.

Event Processing Networks are discussed further in the ORA EDA Infrastructure document.

2.6 Event Delivery Network (EDN)Event Delivery Network (EDN) provides an abstract mediation layer for event publishing and delivery. The idea of EDN is to decrease the coupling between the producers and consumers while improving developer productivity by providing an infrastructure that supports declarative development of event-based components. Figure 2–4 illustrates the concept of EDN.


Other Terms and Concepts

Figure 2–4 Event Delivery Network (EDN)

EDN makes it easy for the producers to define events and publish them. It also makes it intuitive for the consumers to subscribe and receive events. EDN provides the following capabilities:

■ Event Definition: Events are defined using a neutral language such as Event Definition Language (EDL) and are catalogued and distributed to interested parties.

■ Publish-subscribe abstraction: The details of publish-subscribe are totally abstracted from the producers and consumers.

■ Routing: Events are routed to the interested parties using the defined routing rules.

■ Declarative specification: Most rules are specified using declarative languages to avoid any development efforts.

■ Rich semantics: Rich subscription semantics allow flexibility in specifying the rules. For example, XPath may be supported as a language to develop the rules, providing the flexibility of XML and the power of XPath query language.

■ Subscription granularity: Subscriptions may be refined granularly using name spaces, event names or content based rules.

■ Implementation abstraction: The event delivery implementation mechanism is totally abstracted from the users of the EDN. For example, the EDN may use JMS implementation or Database implementation but that detail is transparent to the producers and consumers.

2.7 Other Terms and ConceptsThis section describes some of the key terms and concepts related to EDA.

2.7.1 Continuous Query Language (CQL)The idea of event processing is to apply certain pattern matching rules and business rules to the events to derive notable events from a bunch of ordinary events. This requires an expression language that is suitable for capturing the essence of the


Other Terms and Concepts

processing rules. The expression language must support advanced features like specification of sliding time window and joins across streams.

Continuous Query Language (CQL) is a query language to process the continuous stream of event data over a defined time range.

2.7.2 Sliding WindowsWith streams, it is necessary to define the boundaries to meaningfully process the events. Sliding windows offer a way to limit the events to be processed at any given time. There are two types of sliding windows: count-based and time-based.

2.7.2.1 Count-based sliding windowCount-based sliding window instructs the engine to only keep the specified number of events for a stream. The events kept for processing are the latest events from the stream. If the list is full, any new event would replace the oldest event in the list.

2.7.2.2 Time-based sliding windowTime-based sliding window is a sliding time range specified to define the scope of events to be processed. A time-based sliding window is a moving window extending to the specified time interval into the past based on the system time. Time-based sliding windows enable us to limit the number of events considered by a query, as do count-based sliding windows.

2.7.3 Batched WindowsBoth count-based and time-based windows may be batched. With batching, the window doesn’t slide but instead new windows are created for every defined batch.

2.7.3.1 Count-based batched windowA count-based window may be batched based on the specified number. With this approach, a new batched window is created for every specified number of events.

2.7.3.2 Time-based batched windowThe time-based batch window buffers events and releases them every specified time interval in one update. Time-based batch windows control the evaluation of events, as does the length batch window.

2.7.4 Situation AwarenessA widely accepted definition of Situation Awareness is by Endsley, who defines it as, "the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future".

Situation awareness involves being aware of what is happening around to understand how information, events, and actions will impact the goals and objectives, both in the present and in the near future. It is the ability to receive real-time intelligence to assess the current state so that the business can react accordingly.

EDA, combined with Business Activity Monitoring is a perfect solution for situation awareness. These technologies provide the visibility and intelligence to be able to monitor, analyze, and react to business events.


Event Lifecycle

2.7.5 Event ChannelsEvent channels or streams are logical conduits used for the distribution of events on both inbound and outbound sides of the processing. Channels are one of the primary mechanisms that enable loose coupling in EDA. Channels are the medium for propagating events. Within a channel, the events might be formed as streams or as clouds.

Some also use the term "Relation". Relation is a special type of channel that distributes contextual information from the local relational databases and other sources primarily on the inbound side of the processing.

2.7.6 Event Producer or Event SourceThe system or component that is responsible for the definition and production of an event is called the Event Producer or Event Source. ORA documentation uses the term Event Producer or simply Producer to refer to this element.

2.7.7 Event Consumer or Event SinkThe system or component that subscribes and consumes the event is called the Event Consumer or Event Sink. ORA documentation uses the term Event Producer or simply Producer to refer to this element. Event consumers react to the event and may further create and propagate other derived events.

2.7.8 Event GeneratorEvent Generator is a component that raises events when specified conditions are met. For example, a File Event Generator generates events when a file is created or modified in the file system. Event Generators differ from Event Producers in such a way that they are just intermediary components that generate an event based on a specified criteria. They don’t define or own the Event specification.

2.7.9 Event SensorsEvent Sensors are devices that detect a change in state (e.g. movement of goods) and propagate events. Event Sensors are used in several applications such as Warehouses and Point of Sale systems.

2.8 Event LifecycleFigure 2–5 illustrates the event lifecycle from production to reaction. The primary states of the event lifecycle are:

■ Generate: Events are created.

■ Detect and Capture: Events are sensed and captured.

■ Process: Events are processed in this stage.

■ Store: Events are stored for making them available to interested subscribers.

■ Distribute: Events are distributed to the interested parties.

■ Monitor: Events are monitored and analyzed.

■ React: Appropriate action is taken to handle the event.


Event Lifecycle

Figure 2–5 Event Lifecycle

Each of these lifecycle stages requires certain architecture capabilities to be able to accomplish the objectives of that stage. The table below summarizes the objectives, architecture capabilities and logical components for each of these stages. The capabilities are discussed later in this document and the logical components are discussed further in the ORA EDA Infrastructure document.

Objectives Capabilities Components

Generate Produce Event creation, sensing, propagation

Triggers, Event Generators, Sensor Devices

Capture Sense, detect Detection, harvest, adapt

Readers, Adapters, Edge Servers

Process Filter, manipulate, derive

Filter, split, enrich, aggregate, correlate, pattern recognition, transform, CQL/EQL support

Processor, Router, Pattern Analyzer

Store Availability and Performance

Cache management, access management

Cache Manager, Data Grid

Distribute Disseminate, Loose coupling

Subscription, broadcast, event networks, delivery

Channels, Streams, Queues, Topics

React Perform, resolve, correction

Event reception and response, sinking

Listeners, Reactors

Monitor Real time situational awareness

Sensing, reception, correlation, reporting, alerts, analytics

Listeners, Dashboard, Event Engine, Analytics Engine, Report Manager


EDA High Level Conceptual View

2.9 EDA High Level Conceptual ViewThe conceptual architecture highlights the core capabilities and the primary relationships that belong specifically to an EDA system, unencumbered by implementation details.

Figure 2–6 shows a rather oversimplified view of EDA. The basic concept is that a large number of inbound events and messages are captured and processed to produce a smaller number of outbound events that are more meaningful to the business.

Figure 2–6 EDA Simplified Conceptual View

Figure 2–7 takes the concept to the next level by showing more details. It shows the high level conceptual elements and how they participate in an Event Driven Architecture.

Figure 2–7 High Level Conceptual View

The conceptual elements shown in the figure are explained below.


EDA High Level Conceptual View

2.9.1 ProducersEvents may be produced by a variety of sources in a wide variety of formats. One of the underlying principles of EDA is loose coupling. This means that the event producers are completely decoupled from the potential consumers. Event producers should make no assumptions about the consumers but ensure that the events are easily consumable by using widely adopted standards for message definition and propagation.

Event producers play the key role of translating the human actions and other happenings to technology representations by raising the events that are consumable by the systems. Examples of event producers include:

■ Geospatial event producers: An important application of the EDA is in the area of Geospatial technologies. Geospatial producers generate positional information using GPS-enabled sensors. These are used in several industries including transportation, logistics, shipping, and emergency services such as fire services.

■ Sensor devices: Sensor devices are hardware that sense a physical change or read physical data (e.g. bar code) and transmit them as events.

■ RFID readers: RFID readers generate edge events by reading RFID tags. RFID readers may generate duplicate readings depending on the frequency of the scans, hence these events need further processing for elimination of duplicates.

■ Business Solutions: Business Solutions include a spectrum of enterprise assets including business processes, SOA Services, applications, event generators, batch processes, and databases. These business solutions may generate events in the course of their processing.

■ Internal Feeds: Internal feeds are data broadcast for the consumption of internal systems. These include alerts, heartbeat messages, and internal information updates (e.g. HR updates).

2.9.2 External Feeds/EventsEDA allows key external events to be integrated in real-time. This is important for most businesses to stay competitive and respond to market reactions. For example, airline companies need to know when their competitors change ticket pricing in order to keep up with the market. This allows them to maximize the profit and limit the losses. Another example is the market data feed received by financial service institutions.

External Feeds and Events include various types of external data streams and market events that may impact or influence the business. These events offer vital information that need to be integrated with the internal business systems. Examples of these events include stock quotes and flight status.

2.9.3 Detection and CaptureEvents need to be sensed and captured before being delivered to the processing components. At the point of origination, the events may not be in a form suitable for the consumption of the event processing systems. The event detection infrastructure must be able to sense, read, and convert the event to a format understandable by the event processor. The event detection and capture technology must be unintrusive to the producing systems in order to eliminate any impact to them.


EDA Capabilities

2.9.4 ProcessingA critical element of the EDA conceptual model is the Event Processing. Inbound events are filtered and manipulated to derive useful and actionable business events in this stage. Event Processing is rules-driven. These rules, or queries, are used for processing the inbound stream of events, eventually generating the outbound stream of events. Generally, the number of outbound events is much lower than that of the inbound events due to elimination and aggregation.

2.9.5 Contextual/Historical DataWhen processing the inbound events, it is sometimes necessary to also use the contextual or historical information from the internal sources like databases to derive and enrich the outbound events. For example, in the financial industry, the credit card processing events may be combined with the problem cards information from the internal sources to generate fraud detection events.

2.9.6 DistributionAlthough the outbound events may be consumed by the event consumers directly, in order to make them available to all potential consumers and to improve the runtime performance, the outbound events may be stored in a distributed cache and delivered on demand.

2.9.7 ConsumersEvent Consumers are the subscribers of the events that are interested in specific events. Consumers react to the events of interest by performing predefined actions. Examples of consumers include:

■ Analytics: Analytical systems consume the events for analysis and business intelligence. They may further route the events to data warehouses for historical analysis. A specific area worth mentioning is the field of real-time predictive analytics. EDA enables predictive analytics to be done at real-time by combining the event intelligence with the historical perspective from data mining.

■ Geospatial Applications: Geospatial applications consume the positional events and use them to provide valuable information to the users. For example, a Fire Services Management geospatial application may use mash-up techniques to show the location of a fire accident and the nearest fire trucks and connect to the dispatch application to manage the situation.

■ Monitoring: Monitoring systems such as dashboards consume the event and present them to the business users in a user friendly manner.

■ Solutions: Solutions include a spectrum of enterprise assets including business processes, SOA Services, applications, and databases. Business solutions consume events for state updates or further processing.

■ Feeds: Feed generators may consume outbound event streams to publish them internally or externally.

2.10 EDA CapabilitiesFigure 2–8 shows the key capabilities of an Event Driven Architecture. These capabilities are organized by the following categories.

■ Engineering


EDA Capabilities

■ Event Generation and Capture

■ Event Processing

■ Event Distribution and Response

■ Event Monitoring

Figure 2–8 EDA Capabilities

2.10.1 EngineeringRigor and discipline in the engineering process is required for the development of quality software solutions. Event driven solution development is no exception from the general rule. Successful implementation of EDA requires engineering infrastructure that supports modeling, design, development, testing, and governance of EDA solutions. The engineering capabilities required are summarized below.

2.10.1.1 Event RepositoryAs events and event processing components are defined and developed, they must be catalogued and maintained in a central repository. The repository plays a major role in governance and reuse. The absence of such a central repository would lead to "event sprawl", where similar events are defined by multiple groups within the enterprise adding to the cost and complexity. Events themselves are simply the definitions of the data structures that are passed across. The event processing logic is encapsulated in the form of event processing components and it is important to capture them as well. These event processing components should be reused as well to lower the cost and improve the agility.


EDA Capabilities

2.10.1.2 Metadata ManagementThe event repository must capture the event metadata. Event metadata describes the events and event dependencies. The dependencies are not limited to just event types but also span to other solutions like business processes, Services etc. Event metadata enables impact analysis and design-time governance. For example, when the definition of an event changes, the consumers could be identified and notified so that the changes take place appropriately on the consumer side.

2.10.1.3 Event Search and DiscoveryThe value of EDA is the free flow of events across the enterprise backbone for fast and dynamic processing. Having just this runtime backbone is not sufficient for the success of EDA. In order for the potential consumers to develop solutions that use these events, they must be harvested and made discoverable through common search mechanisms. The Engineering infrastructure should provide the capabilities to search and discover the events and event specifications.

2.10.1.4 Usage TrackingTracking the usage of events produces key statistics that help assess the value of the event components. The usage information also helps with the prioritization of EDA solution development.

2.10.1.5 ModelingEvent modeling covers the specification of events that include the event definition, state transitions, content structure, and element definitions.

2.10.1.6 TestingEDA engineering should include testing infrastructure to test the functional and non-functional aspects of the event processing components. High availability and scalability of EDA solutions must be tested before deploying them in production.

2.10.2 Event Generation and CaptureThe main event generation and capture related capabilities are

■ Event Sensing

■ Event Generation

■ Event Adaptation

2.10.2.1 SensingThe first step in the event lifecycle is to detect a state change of object or condition. Sensing is the act of monitoring and reading the physical conditions in order to convert them to business events understandable by systems. Event Sensors are hardware devices that provide this capability at a physical level. Sensing is also required on the systems side to detect hardware and software state changes. Examples of this include file system changes, email notifications, and system alerts.

2.10.2.2 GenerationEDA also requires the capability to generate events. Generation includes creation and publishing of events. Physical events are converted an object representations that are transportable.


EDA Capabilities

2.10.2.3 AdaptationThe events generated are almost always different from those expected by the event processors. The event types must be converted to a canonical form that can further be processed. There may also be differences in the communication protocols that need to be mediated. Adaptation is the capability to mediate the differences in event types and communication protocols.

2.10.3 Event ProcessingEvent processing is a critical step in the event lifecycle. A mature EDA would require most of the capabilities discussed below but not all of them will be required in every use case.

2.10.3.1 CorrelationThe events generated by the producers may not mean much individually until they are correlated with the other events or contextual information. EDA should have the ability to resolve complex correlations between events that may be causal, temporal or spatial.

2.10.3.2 FilteringEvent filtering eliminates events that are trivial or irrelevant. The filtering criteria could be based on time factor or event attributes. An example of the former is heartbeat events sampled every minute. Another example is discarding all stock trades with a stock price of $20 or below.

2.10.3.3 EnrichmentEvents are sometimes enriched with additional data obtained from other events or from the local data sources. Computations or algorithms may need to be applied to certain fields to enrich the events. For example, an incomplete or short postal address (e.g. 5-digit zip code) in the event may be verified against the postal service database and expanded with the full address and 9-digit zip code.

2.10.3.4 SplittingTypically the number of outbound events is a lot less than the inbound events since they are filtered in the process. However, there are situations where it is necessary to split an event into multiple events. This is particularly useful when an event is overloaded with logically unrelated information that may not be useful together. Also the potential consumers for these different segments of information may be different. In such cases, the event content may be split into multiple events so that the consumers can subscribe to the appropriate event type.

2.10.3.5 BatchingEvent Batching is the grouping of events by a given criteria. It is useful when a stream of events need to be grouped and delivered as a batch together at the same time.

2.10.3.6 PredictionEvent Prediction is a powerful capability that enables real-time predictive analytics. Predictive analytics is an area of statistical analysis that deals with extracting information from data and using it to predict future trends and behavior patterns. The core of predictive analytics relies on capturing relationships between explanatory variables and the predicted variables from past occurrences, and exploiting it to predict future outcomes.


EDA Capabilities

EDA allows real-time predictive analytics by combining the events with the historical information from Data Mines and Data Warehouses.

2.10.3.7 LineageEvent Lineage or Causality allows the drill-down of events to identify their origins and causes. Synthesized Events are derived by applying a sequence of criteria to an inbound event stream. Event Lineage allows them to be traced back to the original events.

2.10.3.8 Pattern RecognitionEDA can look for complex patterns and generate events when they are detected. In order to do it, the following capabilities need to be supported by the EDA infrastructure.

■ Specification of the pattern to look for

■ Identification of the pattern in a stream of events

■ Generation of composite event that signifies the occurrence of the pattern

The event streams may contain a variety of patterns that may include:

■ Non-event detection: Detecting when something does not happen.

■ New event detection: Recognizing when a new event occurs or a value changes to a new one. An example is detecting the new price when the price of a product changes.

■ Old event detection: When an event changes a value, the old value is detected in this case. In the previous example, the old price is derived when the price of the product changes.

■ All pattern: All pattern is where all values from a given set are detected in a stream.

■ W pattern: A double dip in the value of the events causes W pattern.

An example of pattern recognition is identifying a "W" pattern in stock trading. Figure 2–9 illustrates this concept where two occurrences of the "W" pattern are detected.

Figure 2–9 EDA Pattern Recognition Example

Figure 10 - EDA Pattern Recognition Example


EDA Capabilities

2.10.3.9 AggregationAggregation allows events to be consolidated. Aggregation may be useful in a number of scenarios including the following:

■ Events may be duplicated due to network or application issues. Sometimes the Quality of Service (QoS) requirements may lead to deliberate duplication of events. Heartbeat events are usually repeated periodically. Aggregation allows these events to be aggregated and sent as a master event.

■ Some events may be tightly related to other events and in order to be useful, they may need to be combined by adding fields or by arithmetically aggregating. For example, a shipping system effectively combines address change events and order events into request for shipment to a specific address.

2.10.3.10 RoutingEvent Routing is an important capability in enterprise deployments. Events may need to be routed to the appropriate destination through the infrastructure. In a federated configuration, events may be routed to the sub-networks. Routing may be based on the event header or event content.

2.10.3.11 TransformationThe format of events produced is sometimes incompatible with the consuming systems. The events can be converted to a canonical model through transformation. For example, the producing system may send first name and last name separately but the consumer might expect the full name. Transformation allows the first and last names in the event to be combined into a full name before the consumer receives it.

2.10.3.12 Query LanguageEvent processing rules are coded in the form of a query language. The query language allows the rules to be captured using a SQL-like expression language. Event Stream Processing and Complex Event Processing require the ability to define a sliding time window or time range to process the events. The query language processing should be fast and efficient as most event driven applications require low latency response times.

2.10.3.13 RecordingEvents typically happen at a very fast pace and real-time monitoring may not always reveal the necessary information for troubleshooting. For testing and troubleshooting, the events flowing through an Event Processing Network should be recorded in an event repository for future playback. Recording should be flexible enough to record events by producer, by consumer or by channel. This allows isolated testing and troubleshooting.

2.10.3.14 PlaybackIf all interaction in a system occurs through events one can recreate the system state from scratch simply by replaying all events. The stream of events recorded can be played back using the event playback capability. Playback should also allow flexibility in terms of which event streams are to be picked up for analysis. The selection may be based on the producer, consumer, or the channels. Troubleshooting usually happens in a non-production environment. So it should be possible to playback the events in an environment other than the one they were captured in.


EDA Capabilities

2.10.3.15 Language ExtensionIn order to accommodate the changes on business and technology, EDA should be extensible. One way to extend the capabilities of the EDA is to be able to extend the language to suit the needs of the business. For example, a business that adds map-based features to its customers might want to add geospatial extensions to the expression language. With language extensions, new expressions to support the use case may be added in a non-intrusive way.

2.10.4 Event Distribution and ResponseOnce the events are processed, the outbound stream is a set of events that are in general important to the business. In an enterprise EDA, several business solutions may be interested in receiving these notable events. So, these events need to be distributed to the interested consumers for further processing and response.

EDA utilizes core integration mechanisms (e.g. HTTP, JMS, etc.) to implement the response portion of "sense and respond". EDA may leverage existing assets (e.g. SOA Services) or integrated components as the "handler" of the response.

This section highlights the capabilities related to event distribution and response.

2.10.4.1 Processing NetworksEvent processing is hardly a one-step process. Real world use cases require the events to be processed a number of times applying the concepts of filtering, correlation, aggregation etc. with different input combinations. Processing networks allow the EDA components to be connected together to achieve the same.

2.10.4.2 CachingOn the outbound side, the events may be distributed directly to the interested consumers but consumers may not be connected all the time. Also the processing components should be decoupled from the distribution components to improve processing and delivery performance. In order to make the outbound events available for the consumers, events may be cached and distributed. A cache is a temporary storage area for events, created exclusively to improve the overall availability and performance.

2.10.4.3 ChannelsEvents are distributed through named channels both on the inbound side as well as the outbound side. Channels not only allow the consumers to be decoupled from the event processor but they also allow the events to be logically separated for processing and consumption. Channels may be "typed" to handle a specific type of event. Channel typing improves performance by allowing consumers to receive only the types of events that are of interest.

2.10.4.4 SubscriptionConsumers must be able to subscribe to the events that are of interest. Subscription may include filtering criteria to limit the events received from a channel. Consumers may also want to receive the events that were published when they were inactive. This is achieved through Durable Subscription. Durable Subscription is an important capability that enables the consumers to ensure that they don’t miss any important events.


EDA Capabilities

2.10.4.5 BroadcastEvent broadcast allows events to be distributed over a public channel for anyone to consume at any time. Receiving events through broadcast does not require a subscription. The events are picked up as they are delivered to an open public channel.

2.10.4.6 ReceptionThe consumers should have the capability to observe or listen to the events and receive them when they arrive. This is typically implemented through event listeners.

2.10.5 Event Monitoring and ManagementIn an EDA, two different types of monitoring are quite common:

■ System monitoring: The EDA components are monitored and managed for healthy operation. Event volume, frequency and any bottlenecks in the systems may be detected and diagnosed.

■ Business Activity Monitoring: The event content may signify business criticality. The content is examined for key monitoring parameters and displayed visually to the user for further actions.

2.10.5.1 ReportingThe event data is assembled and formatted for live and on-demand reports. Reporting should support a number of visual formats including charts, graphs, columns, KPIs, and spreadsheets. Reports are created initially based on the initial snapshot but should be refreshed continually as the new events arrive for up-to-the-second information delivery. The end users should be able to customize the reports to suit their needs.

2.10.5.2 AlertsSome events and event patterns require the users to be alerted through email, text, dashboard, or other means. The monitoring system should allow configuration and delivery of alerts.

2.10.5.3 AnalyticsAnalytics provides real-time business intelligence. The event data is analyzed against current and historical data to provide business insight and trends to the end users.

2.10.5.4 DashboardsDashboards display the information to the users in a business-friendly format using visual tools and widgets. The architecture should allow fast and easy development of dashboards and integration of dashboard components.

2.10.5.5 Exception ManagementThis capability allows business exceptions and SLA deviations to be monitored. In some cases it may be necessary to manage the exceptions when they occur by taking a certain sequence of steps.

2.10.5.6 Business Process ControlBusiness Activity Monitoring should allow basic process control to alter the business processes based on the monitoring information. This requires process integration with the monitoring infrastructure to initiate or change the sub-processes.


Event Meta-model

2.11 Event Meta-modelSuccessful EDA deployments require standardized structure to be used for events across the enterprise. Using a standard structure makes event integration easy and intuitive. Figure 2–10 illustrates a sample event structure that shows the typical fields. Events generally have two major parts, a header and a body. The header has metadata such as name, type, timestamp, and routing information. Some of this information may be used for event routing and filtering. The event body contains the payload.

Figure 2–10 Sample Event Structure

Note that the event structure does not specify the operation that a consumer must perform upon receiving the event. This decreases the coupling between producers and consumers of events. The producer simply sends a message reporting that an event has happened. The logic to decide what to do about the event is handled purely by the consumers. This gives the flexibility of changing or adding event consumers without modifying sources. By contrast, most request/reply paradigms including SOA Services require a method name on which the service requestor and service provider must agree; so both requestor and provider must be changed in lockstep. The enhanced plug and play nature of EDA systems is a key advantage over common request/reply systems.

Figure 2–11 illustrates EDA relationships that show the major participants in an EDA and the relationships between them. Event producer produces the event and uses the channels to send the events. Event consumer consumes the event through the channel. Event may be classified into ordinary, notable, stream or transactional event.

Stream and Relation are types of channels. Streams are generally referred to the inbound channels that provide the events to be processed. By contrast, Relations are the channels that provide the contextual information from local databases and other sources. The information from these channels are joined and processed for producing outbound events.

Events are described by event specification that contains key information including event elements, preconditions and dependencies. Events are catalogued in the event repository using the event specification.


Event Meta-model

Figure 2–11 EDA Relationships

Figure 2–12 shows a basic design-time event meta-model. Loosely coupled nature of the EDA drives the need for independent development lifecycle for the producers and consumers. This makes governance of event-driven application development challenging. The producers typically do not have any knowledge of the consumers. They define the event specification that the consumers can use to understand the event and event structure. An event repository is used as the infrastructure to harvest events that can be searched and discovered by the consumers. When modeling event flow, the developers design the event interface based on the event specification.

Figure 2–12 Event Meta-model - Design-time

Event specification is a detailed description of the events that serves as a "loose agreement" between the producer and consumer. It includes the following.

■ Name: Name of the event

■ Description: Describes the event

■ Type: Type of the event.


EDA Architecture Principles

■ Preconditions: describes the conditions that need to be met in order for the event to occur.

■ Event Structure: defines the structure of the body of the event.

■ Security: specifies the security requirements that may include authentication and access control information.

■ Implications: describes the significance and business impact of the event.

Figure 2–13 illustrates the runtime event meta-model. The producer publishes the event through the channel and the consumer subscribes to the event and receives it through the channel. Processor applies the event processing rules to the incoming events received from the inbound channels, producing the output events that are distributed through the outbound channels.

Figure 2–13 Event Meta-model - Runtime

Stream and Relation are the types of the channel, as distinguished earlier. The events are further classified into Ordinary, Notable, Stream Event and Transactional.

2.12 EDA Architecture PrinciplesThe following section contains a list of sample architecture principles that pertain to the Event Driven Architecture.

2.12.1 Loose Coupling

Principle Loose Coupling

Statement Event driven systems must be loosely coupled.



2.12.2 Standards Based

2.12.3 Discoverable

2.12.4 Interoperable

Rationale Loose coupling reduces dependency, improves flexibility and promotes agility

Implications ■ Producers and consumers should have no knowledge of each other. Producers can generate events without knowing the identities of the consumers. Conversely, consumers can receive events without knowing the identities of the producers.

■ Event specification and interface should be well defined.

■ Assumptions between systems must be eliminated or reduced.

■ Channels must provide a level of indirection and anonymity.

Principle Standards based

Statement Event driven systems must be standards based.

Rationale Need for loose coupling drives the need for standards. Standards based design promotes interoperability and portability across vendor implementations.

Implications Message structure definitions and EDA channel interfaces should use open standards vs. proprietary formats and protocols.

Principle Discoverable

Statement Business events must be discoverable.

Rationale Promotes integrity and reuse. Avoids "event sprawl".

Implications ■ Document and catalog event definitions and dependencies.

■ Provide a mechanism to search and discover the events.

■ Multiple systems should not be producing the same instance of the Event; rather the Event should be distributed by the owning system and made available to the interested systems.

■ The architecture should be built to eliminate any duplicate event instances.

Principle Interoperable

Statement EDA must be designed to support interoperability and must complement existing technologies.

Rationale Promotes business modularity and interoperability. Integrates better with the enterprise systems.



2.12.5 Granularity

2.12.6 Business Event Focus

2.12.7 Extensibility

2.12.8 Low latency, High Volume

Implications ■ Infrastructure to support event-Service collaboration.

■ Events designed to trigger Services and Services designed to receive or generate events.

■ Interoperability with Business Processes enables events to initiate business processes and vice versa.

Principle Granularity

Statement Capture and process events at the most granular level.

Rationale Improves flexibility in usage and increases the possibility of derived or aggregate events.

Implications ■ Stream or complex event processing needs to be able to correlate granular events.

■ Additional processing may be required in some cases to determine the events of interest.

Principle Business Events Focus

Statement EDA must focus on supporting and handling business events.

Rationale Focus on business events ensures that real-time business visibility and agility are achieved.

Implications ■ Solution design to support business event subscription, consumption and processing.

■ EDA infrastructure to provide the capabilities to route and transform business events.

Principle Extensibility

Statement The architecture must be modular and must support extensibility.

Rationale Allows additional capabilities to be added with minimal change to the system.

Implications ■ Adapter based design to extend support for additional protocols.

■ Extension cartridge based design to enable plug-and-play extensions to the language processor and other components.

Principle Low latency, High Volume



2.12.9 Security

Statement Event Driven Architecture must enable low latency processing and be able to support processing of high volume of events.

Rationale Real-time visibility and business intelligence requires high volume of events to be processed fast to be able to make in-time business decisions.

Implications ■ Components to support high performance requirements

■ In-memory processing for fast response

Principle Security

Statement Events that are sensitive must only be made available to the appropriate systems/users.

Rationale Events may contain confidential or highly secure information. This kind of business sensitive information must be duly protected.

Implications ■ Secure access to the channels that transmit business sensitive information.

■ Integrate the EDA components to the corporate security systems.

■ Encrypt the event data as needed.

■ Require consumers to provide credentials to access sensitive event information.


3
3EDA Standards and Technologies
This section discusses the industries standards and technology related to Event Driven Architecture. Although the concept of Event Driven Architecture has existed for some time, certain specialties of EDA like Complex Event Processing have evolved in the recent past. Most standards related to messaging, SOA, and BPM are also germane to EDA.

An organization involved in developing event processing standards is the Event Processing Technical Society (EPTS). EPTS defines its mission as "To promote understanding and advancement of Event Processing technologies, to assist in the development of Standards to ensure long-term growth, and to provide a cooperative and inclusive environment for communication and learning".

This section highlights some of the standards directly or indirectly related to EDA.

3.1 The Bayeux SpecificationThe publish-subscribe servers in most CEP technologies are based on the Bayeux protocol proposed by the cometd project (see http://cometd.com/). The Bayeux protocol defines a contract between the client and the server for communicating with asynchronous messages over HTTP. The primary purpose of Bayeux is to support responsive bidirectional interactions between web clients, for example using AJAX, and the web server.

Bayeux is a protocol for transporting asynchronous messages (primarily over HTTP), with low latency between a web server and a web client. The messages are routed via named channels and can be delivered:

■ server to client

■ client to server

■ client to client (via the server)

By default, publish-subscribe routing semantics are applied to the channels. Delivery of asynchronous messages from the server to a web client is often described as server-push. Bayeux seeks to reduce the complexity of developing web applications by allowing developers to more easily interoperate, to solve common message distribution and routing problems, and to provide mechanisms for incremental improvements and extensions.

EDA Standards and Technologies 3-1

OSGi

3.2 OSGiThe OSGi Service Platform specification delivers an open, common architecture for service providers, developers, software vendors, gateway operators, and equipment vendors to develop, deploy and manage services in a coordinated fashion.

OSGi technology is the dynamic module system for Java. The OSGi Service Platform provides functionality to Java that makes Java the premier environment for software integration and thus for development. Java provides the portability that is required to support products on many different platforms. The OSGi technology provides the standardized primitives that allow applications to be constructed from small, reusable, and collaborative components. These components can be composed into an application and deployed.

3.3 Java Messaging Service (JMS)EDA heavily uses messaging for transporting events. The Java Message Service (JMS) defines the standard for reliable enterprise messaging. Enterprise messaging, often also referred to as Messaging Oriented Middleware (MOM), is universally recognized as an essential tool for building enterprise applications. By combining Java technology with enterprise messaging, the JMS API provides a powerful tool for solving enterprise computing problems.

Enterprise messaging provides a reliable, flexible service for the asynchronous exchange of critical business data and events throughout an enterprise. The JMS API adds to this a common API and provider framework that enables the development of portable, message based applications in the Java programming language.

The JMS API improves programmer productivity by defining a common set of messaging concepts and programming strategies that will be supported by all JMS technology-compliant messaging systems.

3.4 JSR 190: Event Tracking API for Java Micro EditionThis specification defines an optional code package that will standardize the tracking of application events on a mobile device and the submission of these event records to an event-tracking server via a standard protocol.

3.5 WS-NotificationThe purpose of the Web Services Notification is to define a set of specifications that standardize the way Web Services interact using "Notifications" or "Events". They form the foundation for Event Driven Architectures built using Web Services.

These standards play a key role in the implementation of Event Driven SOA. The components of this specification are:

■ WS-BaseNotification

■ WS-BrokeredNotification

■ WS-Topics

3.5.1 WS-BaseNotificationThis is the base specification on which all the other specifications in the family depend. It defines the normative Web Services interfaces for two of the important roles in the notification pattern, namely the NotificationProducer and NotificationConsumer roles.


EPC IS

This specification includes standard message exchanges to be implemented by service providers that wish to act in these roles, along with operational requirements expected of them.

3.5.2 WS-BrokeredNotificationWS-BrokeredNotification defines the Web services interface for the NotificationBroker. A NotificationBroker is an intermediary which, among other things, allows publication of messages from entities that are not themselves service providers. It includes standard message exchanges to be implemented by NotificationBroker service providers along with operational requirements expected of service providers and requestors that participate in brokered notifications.

3.5.3 WS-TopicsWS-Topics defines a mechanism to organize and categorize items of interest for subscription known as "Topics." These are used in conjunction with the notification mechanisms defined in WS-BaseNotification. WS-Topics defines three topic expression dialects that can be used as subscription expressions in subscribe request messages and other parts of the WS-Notification system. It further specifies an XML model for describing metadata associated with topics.

3.6 WS-EventingWeb Services often want to receive messages when events occur in other Services and applications. A mechanism for registering interest is needed because the set of Web Services interested in receiving such messages is often unknown in advance or will change over time. This specification defines a protocol for one Web Service (called a "subscriber") to register interest (called a "subscription") with another Web Service (called an "event producer") in receiving messages about events (called "notifications" or "event messages"). The subscriber may manage the subscription by interacting with a Web Service (called the "subscription manager") designated by the event producer.

To improve robustness, a subscription may be leased by an event producer to a subscriber, and the subscription expires over time. The subscription manager provides the ability for the subscriber to renew or cancel the subscription before it expires.

There are many mechanisms by which event producers may deliver events to the event consumers. This specification provides an extensible way for subscribers to identify the delivery mechanism they prefer. While asynchronous, pushed delivery is defined here, the intent is that there should be no limitation or restriction on the delivery mechanisms capable of being supported by this specification.

3.7 EPC ISThe main focus of the EPC Global group is to create both a worldwide standard for RFID and the use of the Internet to share data via the EPCglobal Network.

EPC Information Services (EPCIS) is an EPC global standard designed to enable Electronic Product Code (EPC) related data sharing within and across enterprises. This data sharing is aimed at enabling participants in the EPCglobal Network to obtain a common view of the disposition of EPC-bearing objects within a business context. EPCIS standard can be used by any application in any industry. However, the standard supports extensibility so end users and industry groups can address specific application and industry use cases through industry and custom extensions or companion specifications.


Java Management eXtension (JMX)

3.8 Java Management eXtension (JMX)Java implementations of EDA use JMX for monitoring and managing the infrastructure. Java Management Extensions (JMX) is a specification for monitoring and managing Java resources such as applications, JVM, and JEE resources. It enables a standard generic management system to monitor applications; raise notifications when the application needs attention; and change the state of an application to remedy problems. Because JMX is dynamic, it can be used to monitor and manage resources as they are created, installed, and implemented.

3.9 Data Distribution Service (DDS) Specification by OMGData Distribution Service (DDS) is a specification of a publish-subscribe middleware for distributed systems created in response to the need to standardize a data-centric publish-subscribe programming model for distributed systems.

DDS is networking middleware that simplifies complex network programming. It implements a publish/subscribe model for sending and receiving data, events, and commands among the nodes. Nodes that are producing information (publishers) create "topics" (e.g., temperature, location, pressure) and publish "samples." DDS takes care of delivering the sample to all subscribers that declare an interest in that topic.

DDS handles all the transfer chores: message addressing, data marshalling and demarshalling (so subscribers can be on different platforms than the publisher), delivery, flow control, retries, etc. Any node can be a publisher, subscriber, or both simultaneously.

The DDS publish-subscribe model virtually eliminates complex network programming for distributed applications.

DDS supports mechanisms that go beyond the basic publish-subscribe model. The key benefit is that applications that use DDS for their communications are entirely decoupled. Very little design time has to be spent on how to handle their mutual interactions. In particular, the applications never need information about the other participating applications, including their existence or locations. DDS automatically handles all aspects of message delivery, without requiring any intervention from the user applications, including determining who should receive the messages, where recipients are located, and what happens if messages cannot be delivered.

This is made possible by the fact that DDS allows the user to specify Quality of Service (QoS) parameters as a way to configure automatic discovery mechanisms and specify the behavior used when sending and receiving messages. The mechanisms are configured upfront and require no further effort on the user's part. By exchanging messages in a completely anonymous manner, DDS greatly simplifies distributed application design and encourages modular, well-structured programs.

3.10 Extensible Messaging and Presence Protocol (XMPP)The Extensible Messaging and Presence Protocol (XMPP) is an open technology for real-time communication, which powers a wide range of applications including instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.

The core technologies covered by XMPP include:

■ The base XML streaming layer

■ Channel encryption using Transport Layer Security (TLS)


Extensible Messaging and Presence Protocol (XMPP)

■ Strong authentication using the Simple Authentication and Security Layer (SASL)

■ Use of UTF-8 for complete Unicode support, including fully internationalized addresses

■ Built-in information about network availability ("presence")

■ Presence subscriptions for two-way authorization

■ Presence-enabled contact lists ("rosters")


Extensible Messaging and Presence Protocol (XMPP)


4
4EDA/Core ORA Interlock
This section discusses how EDA perspective interlocks with the core ORA elements.

EDA can be implemented using different types of middleware technologies. Some of these are discussed in the core ORA documents such as ORA Foundation Infrastructure document. Messaging systems and SOA infrastructure can be greatly leveraged for EDA implementations.

Messaging technologies such as queuing and publish-subscribe particularly provide features helpful for improving the integrity and efficiency of EDA processes. Event producers and consumers are decoupled in time through the use of message queuing. An event producer can send an event and disconnect at any time because the messaging system can store the event until the consumer is ready to receive it. Messaging systems offer different Qualities of Service (QoS) such as best effort delivery, exactly once delivery, at least once delivery, or at most once delivery.

EDA does not require the use of queuing or publish-subscribe messaging system, although they are often useful. Conversely, an application that uses queuing or publish-subscribe is not necessarily implementing Event Driven Architecture style.

4.1 EDA, SOA, BPM, and BAMEDA is generally not executed as a standalone enterprise initiative. It is usually part of a larger enterprise initiative that may include a multitude of technologies including SOA and BPM. EDA is also looked at as an enabling technology or strategy for Business Activity Monitoring (BAM). EDA and all these other technologies complement each other. Figure 4–1 captures the essence of the relationship between these technologies.

EDA/Core ORA Interlock 4-1

EDA, SOA, BPM, and BAM

Figure 4–1 EDA, SOA, BPM and BAM

EDA, as discussed before, handles business opportunities, threats, and anomalies. It provides situation awareness and visibility into business operations. EDA adds real-time processing and response capabilities to the other technology architectures that would otherwise be difficult to implement.

Business Process Management (BPM) is primarily about business flow, process automation, and coordination of business functions. BPM can definitely benefit from EDA through event integration. Automated business processes may generate events and participate in the EDA. Conversely, business processes may be initiated by the occurrence of an event.

Service Oriented Architecture (SOA) enables encapsulation of business functionality as reusable Services that can be invoked from EDA, BAM, or BPM (not shown in the diagram). The open and modular nature of SOA Services enables them to be invoked through events as shown in Figure 4–2.

Figure 4–2 Event-Driven Services

SOA and EDA share components of the architecture that enables events generated by SOA Services to be integrated with the event processing systems. For example, an Enterprise Service Bus (ESB) can mediate Services as well as Events, enabling a tighter and efficient integration between these technologies. This concept is further explored in the ORA EDA Infrastructure document. Figure 4–3 illustrates this simple concept of events produced by SOA Services.


EDA and Security

Figure 4–3 Events Produced by a Service

Business Activity Monitoring (BAM) provides real-time visibility into business processes. As explained before, BAM plays a key role in monitoring KPIs and SLAs, and managing exceptions. BAM brings the concept of Business Intelligence (BI) from data warehouses to SOA, BPM, and EDA through real-time dashboards and analytics. BAM can integrate with SOA and BPM directly or through the EDA infrastructure. The bidirectional arrow between BAM and other technologies illustrates that there is two-way interaction between them. For example, BAM not only receives monitoring events from BPM but also initiates BPM sub-processes.

4.2 EDA and SecuritySecurity is an important concern for any type of technology. EDA is no exception from this rule. The fact that the event producers are completely decoupled from the event consumers makes it a little more challenging. Event may carry sensitive information that must be transmitted securely and must be delivered only to trusted recipients. The core ORA Security document describes the general security concepts that are touched upon in this section.

EDA should provide a variety of mechanisms to protect the event resources such as data and event streams, event configuration, user name and password data, security policy information, remote credentials, and network traffic. The types of security needs in EDA include topics such as authentication, authorization, and confidentiality. EDA should integrate with the existing security systems such as LDAPs, security services, and entitlement solutions.

4.2.1 AuthenticationAuthentication is the process of verifying that a user or resource consumer is who he/she claims to be. This is generally accomplished by providing an identifier along with a password, token, or signature that is unique to the user and trusted to be secret.

In an EDA, the event producers and event consumers must be appropriately authenticated before they can publish or receive sensitive events. This means that the channels and streams that the producers and consumers access should be capable of providing secured access.

EDA infrastructure may provide the basic security infrastructure to manage the user credentials or may plug-in to the existing security infrastructure to use the enterprise security credentials. For larger enterprise implementation, it is recommended to use the enterprise user base.


EDA and ORA Engineering

4.2.2 AuthorizationAuthorization is the process of granting or denying requests by a party (e.g. client or user) to perform actions on a resource. It is generally performed by comparing rights granted to the user with actions the user is attempting to perform. Access decisions have been divided into three topics including coarse-grained, fine-grained, and data field level access control.

Given the potentially large number of users of a system, access privileges are generally not assigned at the user level. Instead, users are assigned to groups (mimicking the organizational structure of a company), or roles (defined based on job functions that users perform), or some combination of the two. Access privileges are then assigned to groups and/or roles.

The typical roles that are used in an EDA infrastructure for development and operational purposes may include but not limited to:

■ Operator: Operations role that has read-only access to event server resources

■ Monitor: Has operator privileges as well as permission to enable/disable diagnostic functions, such as creating a diagnostic profile and recording events (then playing them back.)

■ Deployer: Has permission to deploy, undeploy, update, suspend, and resume any deployed application.

■ Business User: Has permission to update the event processing rules associated with the processor of a deployed application.

■ Administrator: Has all administrative privileges including user and security management.

4.2.3 ConfidentialityConfidentiality refers to the ability to protect data from being seen by entities it is not intended for, both while stored and in transit. Sensitive data housed within a completely secure, locked down, computing environment may be considered safe enough without the need for extra security measures. However, most data eventually must travel beyond the boundaries of locked down environments and must be protected through some means. This problem has been solved through encryption, which scrambles the data so it becomes unrecognizable to everyone except those with the keys to decrypt it.

EDA may use transport layer encryption and message level encryption to ensure confidentiality of event messages. Sensitive events fields may be encrypted at the message level using symmetric or asymmetric encryption. Producers, consumers, and other intermediaries use transport layer security to transmit events securely.

It should be noted that the strength of security inversely affects the performance of the EDA solution. So the level of security must be chosen to meet both the security needs and the performance requirements. Some low latency applications may need to compromise on security in order to meet the service level agreements. This doesn’t mean that the EDA solution will be run without security but the security management may be pushed to the perimeter to be handled by the external components and appliances.

4.3 EDA and ORA EngineeringThe underlying core principle of ORA Engineering is asset sharing and enterprise


EDA and ORA Integration

development through an integrated asset management approach. The ORA Engineering document discussed the concept of Asset-centric engineering and the conceptual elements of Engineering related disciplines including Integrated Development, Quality Management, Deployment, and Asset Management. All those concepts are very much applicable to EDA.

Section 2.10 of this document described the conceptual view that showed the Engineering related capabilities of EDA infrastructure. It lists Event Repository, Metadata Management, Search and Discovery, and Usage Tracking as core capabilities of EDA Engineering. This aligns with the Asset Management described in the ORA Engineering.

The Engineering principles discussed in the ORA Engineering document apply to EDA as well.

4.4 EDA and ORA IntegrationThe ORA Integration document describes the core concepts of service-oriented integration and explains why service-oriented integration is different from how integration has traditionally been done.

The primary goal of service-oriented integration is to better leverage existing systems within the IT environment by applying service-oriented principles. Ultimately, the goal is to enable the assembly of composite applications, with little or no custom coding, that include capabilities sourced from existing systems. Composite applications are applications that pull together data, functionality, and process from multiple existing sources to solve a business problem or create new business value.

EDA can be seen as an integral part of an ORA integration layer that connects SOA Services, Business Processes and Data sources. From the perspective of ORA integration, EDA can be thought of as a mediation layer that provides capabilities such as message transformation, routing, and protocol mediation. A simplistic view of EDA is the integration of Event Producers and Event Consumers through a sequence of steps that filter out unwanted data and aggregate or derive useful data.


EDA and ORA Integration


5
5Summary
Event Driven Architecture (EDA) introduces the capabilities to build loosely coupled architectures that support the real-time processing, monitoring, and response needs of the business. Businesses gain competitive edge by staying on top of opportunities, threats and anomalies that may affect their business. The style of EDA is different from what architects and developers have been used to, but the addition of this push-based technology extends the architectural capabilities of the enterprise to support the needs of agile businesses.

The concepts, terms, and standards discussed in this document help the stakeholders build a common understanding of Event Driven Architecture. The principles and conceptual view of EDA discussed in this document lay the foundation for ORA EDA Infrastructure document.

Summary 5-1

Glossary

Please refer to the Oracle Reference Architecture (ORA) master glossary for common glossary definitions.

Geospatial

Geospatial is a term widely used to describe the combination of spatial software and analytical methods with terrestrial or geographic datasets. The term is often used in conjunction with geographic information systems and geomatics.

Real-Time Enterprise (RTE)

Gartner defines RTE as “The Real-Time Enterprise (RTE) is an enterprise that competes by using up-to-date information to progressively remove delays to the management and execution of its critical business processes.”

Glossary-1

Real-Time Enterprise (RTE)

Glossary-2

A
AFurther Reading and References
The IT Strategies From Oracle series contains a number of documents that offer insight and guidance on many aspects of technology.

ORA EDA Foundation is part of a series of documents that comprise the Oracle Reference Architecture, which is included in the IT Strategies from Oracle (ITSO) collection. Although this document was written as a standalone document and contains sufficient background information to introduce the topic of EDA, it assumes the reader is familiar with supporting concepts.

In particular, the following documents may be of interest:

A.1 Related DocumentsOracle Reference Architecture (ORA) consists of a set of documents, each targeting a specific aspect of the Oracle computing environment. This document covers aspects of Event Driven Architecture.

The reference architecture material also includes the following related documents:

ORA Monitoring and Management - This document provides a reference architecture for designing a management and monitoring framework to address the needs for the modern IT environment.

ORA Security - The ORA Security document describes important aspects of security including identify, role, and entitlement management, authentication, authorization, and auditing (AAA), and transport, message, and data security.

A.1.1 Suggested Pre-readingThe following documents are suggested pre-reading for those that would like to more fully understand the concepts this document builds upon.

■ ITSO Overview - provides an understanding of the scope and documentation approach of ITSO and ORA.

A.2 Other Resources and ReferencesIn addition, the following materials and sources of information relevant to EDA may be useful:

■ Michelson, Brenda, Patricia Seybold Group - Event Driven Architecture Overview

■ Complex Event Processing in Real World - Oracle White paper (search http://www.oracle.com)

Further Reading and References A-1

http://www.oracle.com

EDA Standards

■ http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/index.html - Oracle Complex Event Processing (CEP) documentation

■ http://en.wikipedia.org - Wikipedia

■ http://java.sun.com/javaee/ - Java Platform Enterprise Edition

■ http://www.oracle.com/technetwork/middleware/repository/overview/index.html - Oracle Enterprise Repository documentation

■ http://www.oracle.com/technetwork/middleware/bam/overview/index.html - Oracle Business Activity Monitoring

■ Chandy, Mani and Schulte, Roy - What is Event Driven Architecture (EDA) and Why does it matter?

■ http://complexevents.com/ - Complex Event Processing - Applications, products, research, and developments in event processing

■ Bellini, Joseph, and Peter Fingar. The Real-time Enterprise: Competing on Time Using the Revolutionary Business SEx Machine. Tampa, FL: Meghan-Kiffer, 2004. Print

■ Taylor, Hugh. Event-driven Architecture: How SOA Enables the Real-time Enterprise. Upper Saddle River, NJ: Addison-Wesley, 2009. Print

■ Hugos, Michael. Building the Real-time Enterprise: an Executive Briefing. Hoboken, NJ: Wiley, 2005. Print

■ Luckham, David C. The Power of Events: an Introduction to Complex Event Processing in Distributed Enterprise Systems. Boston [etc.: Addison-Wesley, 2007. Print

A.3 EDA Standards■ http://soa.omg.org/SOA-docs/EDA-Standards.htm - EDA Standards

■ http://docs.oasis-open.org/wsn/wsn-ws_base_notification-1.3-spec-os.pdf - WS-BaseNotification

■ http://jcp.org/en/jsr/detail?id=190 - JSR 190: Event Tracking API for J2ME

■ http://www.omg.org/technology/documents/formal/event_service.htm - OMG Event Service Specification

■ http://svn.cometd.org/trunk/bayeux/bayeux.html - The Bayeux Specification

■ http://www.ep-ts.com/ - Event Processing Technical Society

A-2 ORA Event Driven Architecture (EDA) Foundation

http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/index.html

http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/index.html

http://en.wikipedia.org

http://java.sun.com/javaee/

http://www.oracle.com/technetwork/middleware/repository/overview/index.html

http://www.oracle.com/technetwork/middleware/repository/overview/index.html

http://www.oracle.com/technetwork/middleware/bam/overview/index.html

http://www.oracle.com/technetwork/middleware/bam/overview/index.html

http://complexevents.com/

http://soa.omg.org/SOA-docs/EDA-Standards.htm

http://docs.oasis-open.org/wsn/wsn-ws_base_notification-1.3-spec-os.pdf

http://docs.oasis-open.org/wsn/wsn-ws_base_notification-1.3-spec-os.pdf

http://jcp.org/en/jsr/detail?id=190

http://www.omg.org/technology/documents/formal/event_service.htm

http://www.omg.org/technology/documents/formal/event_service.htm

http://svn.cometd.org/trunk/bayeux/bayeux.html

http://www.ep-ts.com/

Oracle Reference Architecture · that are tailored and enhanced for industry verticals. Each...

Documents

Transcript of Oracle Reference Architecture · that are tailored and enhanced for industry verticals. Each...