Oracle EDA Solution Platform - doag.org · Alert users to bottlenecks & solutions ... Oracle Event...

<Insert Picture Here>

Oracle EDA Solution Platform


Agenda• Event Processing• Event-Driven Architecture Suite• Product Deep Dive

• Oracle Complex Event Processing• Demonstration – Mission Critical, HA –

Emergency Services Solution• Q/A

Event-Driven Applications

Financial Services

Transportation & Logistics

Public Sector & Military

Manufacturing

Utilities & Insurance

Telecommunications & Services

Algorithmic trading

Asset management

Distributed order orchestration

‘Negative Working Capital’ inventory management

Grid Infrastructure ManagementReponses to calamities – earthquake, flooding

• Proximity/Location Tracking• Intrusion detection systems• Military asset allocation

Need to support one or more of:• High volume• Continuous streaming• Sub-millisecond latency• Disparate sources • Time window processing• Complex pattern matching

Copyright © 2006, Oracle. All rights reserved.5

Processors• Set of queries applied

to the streams Listeners

• Handle triggers raised by the processors

Events• Implemented as

JavaBean or Map

Enriched Streams

• From any source: data streams, web services, Java, Database

Adapters

• Translate external events/data into java objects for processing

Focus on Event-Driven Architecture (EDA)Lightweight, Low Latency, Extreme High Throughput, and Java-based Application Container

Data/Msg.Feeds Listener/SINK:

User Code(Plain Java)

Data Feed Adapters

ProcessEvents(CEP)

Incoming Data Streams1,000,000+ Messages/Sec

Aggregate, correlate, filter dataCan Handle Unlimited Queries

Resulting Data fed to vast business opportunities

with Java languageLatency

Microsecond Latency (Avg.)

EDA Java Application Container

Oracle Complex Event Processor

Average Latency vs. Injection Rate (10 Connections)

0

10

20

30

40

50

60

70

80

0 200000 400000 600000 800000 1000000 1200000

Injection Rate (events/second)

Aver

age

Late

ncy

(mic

rose

cond

s)Distribution of Latency Values

20,136,515

3,054,550

131,5980

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

0 - 99 100 - 199 >=200

Latency Range (microseconds)

# of

Out

put E

vent

s

Event ProcessorOracle Complex Event Processing (CEP)

Streams Continuous input, often in high-volume Time ordered Does not end Impossible to process / analyze in real-

time with traditional relational database systems

BA BOEING D 77.575 800 20080305 10:03:02:78

DO DUPONT D 41.575 3000 20080305 10:03:04:12

AA ALCOA INC D 20.125 1000 20080305 10:03:01:55

AXP AMER EXPRESS CO D 45.875 500 20080305 10:03:02:10

BA BOEING D 77.575 800 20080305 10:03:02:78

C CITIGROUP D 34.125 2000 20080305 10:03:03:05

CAT CATERPILLAR D 22.5 600 20080305 10:03:03:46

DO DUPONT D 41.575 3000 20080305 10:03:04:12

AA ALCOA INC D 20.125 1000 20080305 10:03:01:55


BA BOEING D 77.575 800 20080305 10:03:02:78

C CITIGROUP D 34.125 2000 20080305 10:03:03:05


DO DUPONT D 41.575 3000 20080305 10:03:04:12

AA ALCOA INC D 20.125 1000 20080305 10:03:01:55


BA BOEING D 77.575 800 20080305 10:03:02:78

C CITIGROUP D 34.125 2000 20080305 10:03:03:05


DO DUPONT D 41.575 3000 20080305 10:03:04:12

AA ALCOA INC D 20.125 1000 20080305 10:03:01:55


BA BOEING D 77.575 800 20080305 10:03:02:78

Event Processing provides a new data management infrastructure to support and analyze Streams in real-time

Oracle Complex Event ProcessingIn-Memory, Continuous Queries

Event Processing Output Filtering

New stream filtered for specific criteria, e.g. stock price > $22

Correlation & Aggregation Scrolling, time-based window metrics, e.g.

average # of stock trades in the last hour Pattern Matching

Notification of detected event patterns, e.g. price changes A, B and C occurred within 15 minute window


DO DUPONT D 41.575 3000 20080305 10:03:04:12

AA ALCOA INC D 20.125 1000 20080305 10:03:01:55


BA BOEING D 77.575 800 20080305 10:03:02:78

EP

Runs In-Memory (not in Database)***Logic is defined through Continuous Queries on the data

Example: Stock Trading “W” Pattern

1 9 12 19

minutes

XY W

Z

Monitor business processes & services in real-time Key Performance Indicators (KPIs) Service-Level Agreements (SLAs)

Analyze events as they occur Correlate events & KPIs Identify trends as they emerge Alert users to bottlenecks & solutions

Act on current conditions Event-driven alerts Real-time dashboards BPEL processes & web services integration

Business Impact of Events, Oracle BAMMeaningful, Event-driven Intelligence for End-Users

Lightweight Server for Event Processing Financial Services: Signal Generation Demonstration

Data/Msg.Feeds Listener/SINK:

User Code(Plain Java)

Data Feed Adapters

ProcessEvents(CEP)

EDA Java Application Container

Load Generator

-AMEX symbols- Level 2 mkt data (520 bytes)

Delivering Event-Driven SOAExample: Baggage Handling

“Situational Awareness”

Ticketing Reservations

bag-trackingbag-in-containercontainer-on-plane

Gate OperationsFlight Operations

Baggage Handling

missing-bagflight-delaystatistics

BAM

flight-openflight-boardingflight-closedflight-push-backflight-wheels-up

BPM/ BPEL

passenger-onpassenger-off

lost-bag

Event Server

WLRT

CEP

check-inbag-check

SOA Message

Event

Legend

Service Bus

Edge Device (RFID)

Event-Driven Architecture

Complex Event Processor

B PM P roces s es

event

JMSclosed loop

Applic a tions

event

Databas es

S treams

event

event

M es s ag ing

Event Sources

BAM Dashboards

BAM Alerts

BPM/BPEL Processes

Response Services

C oherenc e R ea l Time L is teners

event• Event sourcing• Continuous query language to

define event• Correlation• Sequencing• Window of time evaluation

• Pattern matching• Event publishing• Real time Java infrastructure

• Real Time Systems Management & Fault mitigation• Border Security, Transportation Logistics• Realtime Slot Machine Usage “persuasion”• Online Fraud Detection• Smart Meters• Roadways Toll Management Systems• Intelligent Cable Set-Top Management• Algorithmic Trading, Smart Order Routing• Real Time Manufacturing Production line monitoring• Emergency Services Real Time Location Proximity Tracking

Many CEP Customer Use Cases

Event-Driven Architecture CustomersSelect Customer List

Utilities & Service Industries/Others

Finance/ Banking

Public Sector

Telecom High Tech

Travel/ Transport

FXall

http://www.tdameritrade.com/welcome1.html

http://www.sprint.com/index.html

http://www.cisco.com/en/US/hmpgs/index.html

http://www.dell.com/content/default.aspx?c=us&l=en&s=gen

Evolution of Oracle Event-Driven Architectures

• 1992 Database: Oracle DML Triggers• 1997 Database: Oracle Advanced Queuing• 2002 Database: Oracle Data Streams• 2004 Middleware: Oracle Business Activity Monitoring• 2006 Middleware: Event-Driven Architecture Suite• 2008 Middleware: BEA Web Logic Event Server

• Oracle CEP 10gR3 Released October 2008• Coming Soon: Oracle Event Processing 11g

• Oracle Business Activity Monitoring (BAM), java edition• Oracle Complex Event Processor

Oracle EDA SuiteComplete Suite for Industrial Event Processing

Routing Transform DataQuality

ETL &Replication

Data IntegratorData IntegratorOracle Service BusOracle Service Bus

Mediation

REAL-TIMEVISIBILITY

& PROCESSING

AlertsBusiness Monitoring

BAMBAM

Event FeedsData Streams

Java-based EventServer

CEPCEP

SocketAdaptersAdapters

DB FileCONNECTIVITY

ROUTING & DATA

SERVICES

Business Rules

J2EE Application Server(Oracle AS, WebLogic, WebSphere, JBoss)

Industry SpecificJMS

CoherenceJRockit VM JRockit RTENTERPRISE PERFORMANCE & SCALABILITY

Advanced Tooling for Developers, Administrators, IT Management,

Business Users

App Frameworks

POJO Spring (DM) Framework Services

Late

ncy

Anal

yzer

Core Event Infrastructure

Complex Event Processing

Stream Management

Real Time Kernel

Extended Event InfrastructureCluster

ManagementData

Caching HTTP Pub/Sub EngineEvent

Repository

Oracle Complex Event Processing

Oracle Complex Event Processor (11gR1)Lightweight Java Application Server Full environment for building and running Java

applications Services -- Security, Logging, User Mgmt Module Framework - OSGi

High Throughput Hundreds of thousands of events/second

Multiple-choice JVM JRockit or WebLogic Real Time

Unique JVM Tools

Enterprise Web 2.0 & Eclipse-based tooling Server Management & Monitoring (Visualizer)

Visual Query Creation/Manipulation RAD IDE for EDA/CEP Application Development Visual EPN Editing/Dynamic Construction Enhanced EPL Editing Capabilities

Easy-to-Use Development Environment Service Framework – Spring DM, POJO

Event Processing infrastructure CEP Engine Event Processing programming constructs and

services Time-critical streaming Well-known event processing language

Config/Admin Security Logging

Equinox OSGi™ Foundation Services

OSGi OSGi

WebLogic Application GridWebLogic Real Time

JRockit DGC

JRockit

Visualizer

Data Services Framework

Local Cache Programmatic access to local caches Integrate directly within EPN Nodes

Advanced Tooling for Developers, Administrators, IT Management,

Business Users

Core Event Infrastructure

App Frameworks

POJO Spring Services

Enterprise Server Topology High Availability Managed Event Server Cluster

Historical Event Management Store Events for persistence Record/Playback Event activity Report on Event dialogs

Web2.0 Web Client push-based infrastructure Web based clients send and receive

asynchronous messages over HTTP Event server components can subscribe to and

publish to channels

Late

ncy

Ana

lyze

r

Complex Event Processing

Stream Management

Real Time Kernel

Extended Event Infrastructure

ClusterManagement

DataCaching HTTP

Pub/Sub EngineEvent

Repository

Oracle Complex Event Processing

WebLogic Application GridWebLogic Real Time

JRockit DGC

JRockit

Foundation Services

CoherenceConfig/AdminSecurity Logging

Coherence Integration (Clustering) Scale out Distributed Computing Grid

Oracle Complex Event Processor (11gR1)

Oracle Business Activity Monitoring

Continuous Query Language Rich SQL 99 Compliant continuous query

language SQL extensions for Pattern recognition 150+ Built in Maths & Stats functions Strong Integration to Oracle DB TimesTen Integration

Business Activity Monitoring Integration


Oracle CEP Application Model

e

Adapter

Adapter

Processor

Adapter

Data Source

Data Source

Queries<<Source>>

<<Source>>

<<Sink>>

Service1 Service2

<<OSGI Service>> <<OSGI Service>>

Export Import

Event Processing Network (EPN)

OSGI Bundle/Spring Application Context

Queries

Channel

Channel

Channel

Channel

Event->External Format

Event->External Format

External Format->Event

External Format->Event

Event Processing Networks (EPN)

EDA is composed of several processing steps intermingled with user logic.

This arrangement or network of event processing components is called an event processing network (EPN).

Adapter

Adapter

Processor

Listener- POJO

Event Source

Data Source

Query

RuleProcessor

Query

Query

RuleProcessor

Query

RuleProcessor

Query

RuleCache Rule

Processor

QueryListener- ALSB

• Caches, stores, loaders, listeners, UDF are all configured as spring beans

• Cache can be configured as Staging Area• Caches are source(s)/destination(s) for a Processor• Caches are used for:

• Distributing Streaming Data• Enriching events• Staging area for Correlated data• Publishing events to a cache• Fault tolerance• High availability• Scalability

Oracle CEP Caching Programming Model & Usage with Coherence

Continuous Query Language (CQL)

• System and functional capabilities:• High-volume stream processing• Complex relationships (e.g. non-existence, followed by)

• SQL-based• Leverages skill-set• Integrates to relation-data

• Extensions• Semantic differences: deals with streams, as opposed to

tables• Additional clauses for Temporal Conditions

RuleProcessor

Query

Oracle and CQL

• Initiative for a complete “continuous” query language

• Start with SQL ’99 plus “continuous” query extensions• Based on Stanford University research

• Industry standards discussion• Event Processing Technical Society (EPTS)• ANSI SQL• OMG

• Adoption Today• ANSI SQL Standards Proposal for CQL Pattern Matching

• Oracle, IBM, Stanford University• OpenSource Adoption of CQL• Oracle Complex Event Processor (CEP) Release Available in 11g R1

23

RuleProcessor

Query

24

Eclipse plug-ins extend WTP, support OSGI development

Copyright © 2009, Oracle. All rights reserved.

Simplified Administration and Deployment Management, Advanced EDA Event Processing Network Performance TuningDynamic Continuous Query Event Processing Language Injection

Oracle CEP Visualizer : Navigation Techniques


CQL Visual Construction


CQL Query Plan and Real Time monitoring


Oracle CEP Cluster Management


Coherence Cache Management with OCEP

• Included in Oracle CEP install• Works with open hardware x86 hardware• Addresses the key latency issue with Java• Unique non-intrusive tooling to identify latency bottlenecks• Extension of the award winning JRockit JVM

0

50000

100000

150000

200000

250000

2P Opteron 2220 4P Opteron 885 2P Intel X5355

SPEC

jbb2

005

bops

Sun JVMJRockit JVM

CEP Performance Enhanced by JRockitJRockit Real-Time Provides High Throughput, Low Latency JVM

Oracle CEP: A Complete Event-Driven Application Infrastructure

• “Keep up” with streaming event sourcesevent server Oracle CEP

• Support an event model and associated tools that make solutions

• Fast to build• Easy to adapt• High performanceevent processing network

Oracle CEP• Process at high speed/low latency

with determinism real-time JVM JRockit RT

• Ensure that output is highly available and scalable distributed cache Coherence

33

Oracle CEP & Emergency Services Scenario

• Initiate Fire emergency resolution services• Isolate Fire location and define proximity/exclusion zones• Identify Fire resources, best equipped, in nearest vicinity, &

expedite movement

• Monitor in Real Time, Resource Movements• A Dashboard indicating, second by second, location and

zone proximity status• System pro-actively and dynamically responds, averting

dangerous zone transgressions

“999” “We have a fire in downtown Liverpool”

34

• What are “Exclusion” Zones?• Zones are areas where resources “should not” be• Zones are defined as complex shapes (polygons)• Zones can move instantly, due to weather conditions, Fire spreading

• How are Emergency response resources Monitored?• Real time streaming GPS events provide resource location updates

• Resources can be personnel or equipment• Emergency response team requires notification when:

• Resources are “near” or “in” a Zone• Resources have not left a Zone within a defined period

• Notifications have filtering requirements• e.g. not repeating the same condition with a fixed time • Indications of No movement for a specific time

Oracle CEP & Emergency Services Scenario

35

Database and Event-Driven Architecture Emergency Services Scenario

Location Alerts

latitudelongitude

Resource Location

Exclusion zoneresource map

Flex Application

area mappolygon definitions

ESB/ BPEL

personnel-in zonepersonnel-out of zone

Auto provision

Oracle Database

Device (GPS)

Oracle Spatial

Zone Updates

Oracle

JRocket Real-Time

CEP

36

AQ

Zone Definitions

Resource LocationsMatches and Alerts

SQL

ZONE Manager

Oracle CEP Application (EPN)

Dashboard

Implementation

37

Emergency Services Scenario - DEMO

Highlights Emergency Area (ZONES) Continuous monitoring of Resource Movements

Location Proximity CEP Analysis

Critical Proximity Alert Notification to Downstream

38


Q&A

Oracle EDA Solution Platform - doag.org · Alert users to bottlenecks & solutions ... Oracle Event...

Documents

Transcript of Oracle EDA Solution Platform - doag.org · Alert users to bottlenecks & solutions ... Oracle Event...