Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3...

52
- 1 of 52 - SUN MICROSYSTEMS Event Processing with GlassFish ESB Version 1.1 June 17, 2009 Please send feedback to: [email protected]

Transcript of Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3...

Page 1: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 1 of 52 -

SUN MICROSYSTEMSEvent Processing with GlassFish ESB

Version 1.1June 17, 2009

Please send feedback to: [email protected]

Page 2: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 2 of 52 -

Table of Contents

Introduction .................................................................................................................................. 3 Scenario ........................................................................................................................................ 4 Environment ................................................................................................................................. 5 Tutorial Format ............................................................................................................................. 5

Project 0: Preparing Your Environment .................................................................................6

Project 1: Creating an Intelligent Event ProcessorFinish Line ........................................................................................................................... 7Creating the IEP Module ................................................................................................... 8Creating the Intelligent Event Processor .......................................................................... 8Specifying Input Streams ................................................................................................... 10Adding Time-Based Windows .......................................................................................... 13Adding an Event Correlator / Filter ................................................................................. 15Specifying an Output Stream ........................................................................................... 19Validating the Event Processor ........................................................................................ 19Creating the WSDL for the Event Processor ................................................................. 20Exploring the Generated WSDL ...................................................................................... 20

Project 2: Assembling the Composite Application

Creating the Composite Application ................................................................................ 25Exploring the Service Assembly ....................................................................................... 26Deploying the Composite Application .............................................................................. 27

Project 3: Testing the Composite ApplicationCreating Test Cases ......................................................................................................... 28Running the Test Cases ..................................................................................................... 32Viewing the Generated Alert ............................................................................................. 33Adding Timestamp to the Alert ......................................................................................... 33

Project 4: Storing KPI Data in a Database

Adding a Save Stream Output .......................................................................................... 36Exploring the Service Assembly ........................................................................................37Establishing the KPI Database ......................................................................................... 38Logging Into the GlassFish Admin Console .................................................................... 39Establishing a Connection Pool ....................................................................................... 40Defining a JDBC Resource .............................................................................................. 42Reviewing Generated Save Stream Database Table ................................................... 42Running the Test Cases .................................................................................................... 44

Project 5: Exploring Behind the ScenesFinding the IEP Internal Database .................................................................................... 45Watching Data Flow Through Internal IEP Tables ........................................................... 47Mapping Internal IEP Tables to Event Model Operators ................................................ 48

Summary ........................................................................................................................................ 50 References .....................................................................................................................................52

Page 3: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 3 of 52 -

Introduction GlassFish ESB is built atop the Java Business Integration (JBI) architecture as implemented in the OpenESB project. The purpose of this tutorial is to explore one of OpenESB's most exciting JBI components – the Intelligent Event Processor (IEP). Although IEP wasn't formally bundled in GlassFish ESB 2, it is included GlassFish ESB 2.1. IEP is also a part of Java CAPS 6 which, like GlassFish ESB, includes a subset of OpenESB technology.

IEP is a JBI service engine component that provides powerful support for event-driven architecture (EDA). EDA is a software architectural pattern that focuses on producing, detecting, consuming and acting upon events. An event is a significant change in state of something of interest to the business. There are three fundamental event processing styles:

- Simple: events directly related to specific change of state (sensor notes pressure change) - Event stream: events screened to identify important ones (RFID scanning) - Complex (CEP): events analyzed for patterns and inferencing (casual, temporal,

spatial) determines decision-impelling situations

Industry analysts are telling customers and vendors that SOA alone is not enough to solve the complex opportunities that businesses face today. SOA provides a framework for loosely-coupling business services for building composite applications. EDA delivers an orthogonal perspective offering uncoupled event-driven components.

Comparing and contrasting SOA and EDA is a controversial topic. Some see EDA as a subset of SOA. Others are convinced that SOA and EDA are peer technologies.

SOA, illustrated on the left, is often implemented with web services with a synchronous request/response interaction between a service consumer and service provider. In contrast, EDA, shown on the right, deals largely in asynchronous events. The event processor illustrated hereis performing complex event processing (CEP) to accept raw events, determine if something decision-impelling has happened and notify subscribers. A subscriber might be a BPEL business process that needs to execute based on a particular stimulus. In this tutorial, we'll build a simple event processor that shows CEP in action.

Page 4: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 4 of 52 -

Scenario PennyLotto is a fictitious online gambling company run from the Cayman Islands. Fraud is impacting the reputation of the site. A particular type of scam is concerning management right now:

1. People steal credit cards 2. Open Penny Lotto accounts 3. Transfer money from the credit card into their PennyLotto account 4. Make no bets 5. Transfer money from their PennyLotto account to a bank account

Management suspects that PennyLotto has become a money laundering pass through for a stolen credit card ring and wants to be proactive in detecting fraudulent transactions. The PennyLotto IT department is tasked with exploring EDA and CEP to address management concerns. The CIO has hired your consulting team to craft a simple POC that demonstrates the technology.

To keep the POC simple, here is all there is to the specification: The event processor is to watch three types of transactions: 1) credit card to PennyLotto, 2) betting and 3) PennyLotto to bank. Here is the heuristic to detect possible fraud:

if: customer: has transferred money in from credit card within the past 24 hours and has not bet in the past 24 hours and has transferred money out to a bankthen: record name and amount in an alert

In the spirit of “a picture is worth a thousand words,” here is the essence of the POC specification:

Page 5: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 5 of 52 -

Environment You'll need a modern computer with at least 2GB of RAM (the more the merrier) and about1GB of free disk space. Although the JDK and NetBeans work on a broad range ofoperating systems, I'll assume that you are using Windows XP. Since the products are writtenin 100% pure Java, you will likely be successful with another operating system, but theinstructions in this tutorial were tested on a Windows XP (SP3) laptop.

You'll need to have JDK 6 and the latest GlassFish v 2.1 ESB build installed. I last tested this tutorial with the GlassFish ESB v2.1 build available on June 17, 2009. Download the latest JDK from http://java.sun.com/javase/downloads/index.jsp and install it. I had good luck with JDK 6 Update 14. Then, download and install the latest stable GlassFish ESB build from: https://open-esb.dev.java.net/Downloads.html.

I strongly advise you to use the latest GlassFish ESB V2.1 software. It will get us on the same page. In this tutorial, I documented what I experienced using that version of the software. It will help if you have the same basis for your tutorial experience.

Tutorial Format In the pages that follow, you'll see a 3-column format. In the first column, we'll include headings to act as mileposts that to remind you of the purpose of the steps in the section. Hopefully, these headings will also be helpful later as you use this document as a reference. Each heading is listed in the table of contents to help you find a section quickly.

In the second column, we've included detailed, step-by-step guidance on the keystrokes and mouse clicks required to accomplish the tasks. Along the way, you will find illustrations so you can compare what you see on your screen to what we intend.

In the third column, you'll find narrative (in green) that provides background to the steps listedin the second column. I have included some conceptual topics, informal advice and somedeeper technical tips here. I call it the “buddy column” as it lets me “chat” with you about whatyou are doing as if two friends were sitting together at the keyboard. Having these sorts ofcomments in a separate column hopefully keeps the second column more concise whileproviding some deeper learning insights.

Page 6: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

- 6 of 52 -

Project 0: Preparing Your Environment

All you need to work through this tutorial is the GlassFish ESB v2.1 GA build The distribution includes the IEP service engine component and the related NetBeans plug-ins.

As mentioned in the Environment section earlier, I strongly recommend that you download the latest GlassFish ESB v2.1 (rather than v2 or a NetBeans release). In addition to bug fixes, GlassFish ESB v2.1 also introduces support for NetBeans 6.5.1 and GlassFish 2.1. Here is where to find it:

The URL is: https://open-esb.dev.java.net/Downloads.html. I won't provide instructions here on how to install GlassFish ESB. It's pretty easy. Just make sure you have a JDK 6 installed first, download the GlassFish ESB installer and execute the installer. Taking the defaults for the installation wizard prompts typically works great.

Page 7: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Finish Line In the spirit of “start with the end in mind,” here is the event model we will build in this project and assemble and test in subsequent projects:

CC_to_Lotto, Betting_Activity and Lotto_to_Bank are input streams that bring the three types of transactions in.

Create_Alert is the output stream that identifies suspicious customers.

Store_Alert_in_DB adds the alert as KPI (Key Performance Indicator) data to a database table that we might use to drive an executive dashboard display.

In between the inputs and outputs, in Detect_Threat, we implement the logic required to filter out the suspect transactions.

Here's where all the “heavy lifting” is done.

- 7 of 52 -

Project 1: Creating an Intelligent Event Processor

Sun has shown the PennyLotto demonstration at conferences in order to educate and inspire audiences about event-driven architecture. The demo introduces how OpenESB technology supports complex event processing (CEP). The beauty of the demo is that it is simple. There is no elaborate scenario or sophisticated business policy to get in the way of understanding how the NetBeans tooling and the Intelligent Event Processor (IEP) provide Event-Driven Architecture (EDA) facilities.

I thought highly enough of the demo to here in tutorial form. We'll build an event model, assemble it into a composite application and then test it. We'll do it all from scratch in two projects. Then, we'll extend the original demonstration by adding support for storing KPI (Key Performance Indicator) data to a database.

Page 8: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Creating the IEP Module

Creating the Intelligent Event Processor

Start GlassFish ESB.

Let's create an Intelligent Event Processing Module as a NetBeans project.

From the menu bar, select File / New Project / SOA / Intelligent Event Processing Module. Click Next.

For Project Name, enter: MyIEPProject

For Project Location, enter: C:\MyIEPTutorial

Click Finish. The project is created and it's logical structure of the project appears in the Projects panel:

Switch over to the Files panel to see the physical structure that has been created:

Let's create an IEP Processor module inside the MyIEPProject.

Return to the Projects panel.

On the MyIEPProject node (see the note in column 3 if your aren't using GlassFish ESB 2.1), right-click and select New / Intelligent Event Processor.

Use the GlassFish ESB v2.1 item in your Window's Start menu or execute the start _netbeans.bat file in your C:\GlassFishESBv2.1 directory.

Copy and paste much as you can (lie project names and locations) to keep this from becoming a typing exercise.

If the directory does not exist, it will be created for you.

I assume that you are using GlassFish ESB 2.1. If you are using an earlier release (which I don't recommend), right-click on Process Files (not the MyIEPProject node) so that NetBeans automatically creates the iep file in the src directory. If the file doesn't get created here, it isn't visible in the Projects panel, the WSDL will not be generated in the right place and our module can not be added to a composite application.

- 8 of 52 -

Page 9: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

For File Name, enter: myIEPLotto

Click Finish.

Your myIEPLotto.iep empty canvas appears and you see it added to the logical project structure:

Let's use the palette to the event model. First, explore a bit. Notice that the palette is organized into categories each containing multiple operators:

If you don't see myIEPLotto.iep under Processor Files, back up a step and read the column 3 important note there.

If the palette panel is not visible, display it by using the Window / Palette option from the menu bar.

Notice that most of the palette categories are collapsed in this screen shot. This makes it easier to deal with such an extensive palette.

The array of features in the palette is vast. To see a short description of each palette item, use the on-line help facility (Help / Help Topics menu bar option). Search for help on “IEP.” You'll find an “About the IEP Operator Palette and Design View” section that provides an overview of each palette selection.

I also found the “About Operator Types” on-line help section valuable since it succinctly defined each category of operator.

- 9 of 52 -

Page 10: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Specifying Input Streams

From the palette, drag an Input / Stream Input operator onto your canvas and drop it:

Double-click this new StreamInput0 element (the icon, not the label) to display its Property Editor window.

For Name, enter: CC_to_Lotto

Use the Add Attribute button to add a new attribute row (one empty one is already available) with these attributes and values:

● depositorName VARCHAR 50● inflowAmount DOUBLE

Click OK to record your input stream updates.

“Input / Stream Input” means select the Stream Input operator from the Input palette category.

The values correspond to the Attribute Name, Data Type and Size columns.

This will instruct the tooling to add messages and operations to a generated WSDL that allows us to route depositorName and inflowAmout input into the event model.

- 10 of 52 -

Page 11: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

From palette, drag another Input / Stream Input and drop it on the canvas below CC_to_Lotto.

Double-click it and, for Name, enter: Betting_Activity

As you did earlier, add these attributes:

● bettorName VARCHAR 50● betAmount DOUBLE

Select OK.

From palette, drag a third Input / Stream Input and drop it onto the canvas below Betting_Activity.

Again, double-click it and, for Name, enter: Lotto_to_Bank

Add these two attributes and values:

● withdrawerName VARCHAR 50● outflowAmount DOUBLE

That's data for the Attribute Name, Data Type and Size columns respectively. For Double, no size is necessary.

You will again need to use the Add button to add a second row to the attributes table so you can specify outflowAmount.

- 11 of 52 -

Page 12: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Click on the Source control in the upper-right corner of the canvas to view the XML that is being created as you add components to the canvas:

So, much like the GlassFish ESB BPEL Editor creates BPEL XML statements behind the scenes, the modeler here for IEP is creating the XML document that is the basis for runtime processing.

Click on the Design control to return to the graphical canvas view.

Save All your work.

- 12 of 52 -

Page 13: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Adding Time-Based Windows

From the palette, drag a Stream Converter / Time Based Window onto the canvas and drop it to the right of CC_to_Lotto.

Connect it to CC_to_Lotto by clicking and dragging a line between the triangle symbols on each element:

Double-click the TimeBasedWindow0 operator and change the Name: CC_to_Lotto_Trans_in_Last_Day

Edit the Size: 1 day

Click OK.

On-line help defines Time Based Windows as: “Enables definition of relations based on a specified duration.”

This is a moving time window 24 hours wide that restricts the data our event model considers as input for evaluation.

The Attributes should automatically appear. If they don't, you didn't wire CC_to_Lotto to the Time Based Window before you displayed the property editor.

- 13 of 52 -

Page 14: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Drag another Stream Converter / Time Based Window and drop it to the right of Betting_Activity.

Connect it to Betting_Activity.

Double-click it and edit its properties:

● Name: Betting_Trans_in_the_Last_Day● Size: 1 day

Click OK to dismiss the Property Editor.

Click on the Source control again to view the underlying XML.

We got a bit verbose with the “Betting_Trans_in_the_Last_Day” name. Position the edit cursor near the top of the XML and use the Find feature (Ctrl-F) to search for in_the

Change the name in the XML source to read Betting_Trans_in_Last_Day

Click on the Design control to return to the graphical mode and see that the changed label appears. You can edit in either Design or Source modes.

The Source and Design views are kept synchronized.

- 14 of 52 -

Page 15: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Adding an Event Correlator / Filter

The model looks like this so far:

From the palette, drag a Correlation and Filter / Stream Projection and Filter and drop it onto the canvas to the right of Lotto_to_Bank.

Connect it to CC_to_Lotto_in_Last_Day

Connect it to Betting_Activity_in_Last_Day

Connect it to Lotto_to_Bank

The model looks like this:

Remember to wire up to the upstream operator before accessing the property editor. This will assure that the attributes are automatically assigned from the input stream.

According to the online help, a Stream Projection and Filter “enables filtering event or modifying selected event attributes.

Again, we typically “wire up” a new operator first before we explore it's properties in the property editor. That way we can see what attribution is passed in from upstream operators.

Try out the Auto Layout feature to tidy things up:

- 15 of 52 -

Page 16: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Double-click the new StreamProjectionAndFilter0 operator and set Name to: Detect_Threat

Drag Lotto_to_Bank withdrawerName to the empty Expression row. Notice that the columns in the row are automatically populated. Press Enter to confirm the new row contents.

Click the Add Attribute button to get new empty Expression row.

Now drag Lotto_to_Bank outflowAmount to the new Expression row and see that row populated. Again, press Enter to record the addition.

Notice that Lotto_to_Bank has been added to the FROM field since we are referencing elements in it in our select clause:

We will be using some graphical editor features to, essentially, build an SQL statement. You could certainly just type in the syntax in the text boxes or even just edit the XML in Source mode, but I think that you will find the editor helpful.

Notice the Help button. You can click it to display context-sensitive help for the operator you are configuring.

- 16 of 52 -

Page 17: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

With the select and from portions of the SQL statement in place, we need to specify the where clause.

In the WHERE section, enter (copy and paste):EXISTS (SELECT 'x' FROM)

Remove the trailing ) symbol and replace it with a space. Then, drag CC_to_Lotto_in_Last_Day and drop it after FROM

Enter a space and then enter WHERE and a space

Drag and drop CC_to_Lotto_in_Last_Day depositorName after WHERE

Enter = at the end and drag Lotto_to_Bank withdrawerName after that = symbol

Add the closing ) symbol at the end.

Press Enter to add a blank line and include (copy and paste) the following:ANDNOT EXISTS (SELECT 'y' FROM)

Remove the trailing ) symbol and replace it with a space.

If you are experienced with SQL, the where clause will look familiar to work you have done in the past when building SQL Select statements.

Again, you can merely type in the SQL where clause, but the tool makes it easier by letting you drag and drop portions. This eases the typing burden a bit.

You may need to make the panel wider.

- 17 of 52 -

Page 18: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Drag and drop Betting_Activity_in_Last_Day after FROM

Enter a space and then enter WHERE and then a space

Drag Betting_Activity_in_Last_Day bettorName after WHERE

Enter = at the end and drag Lotto_to_Bank withdrawerName after that the = symbol

Add the closing ) symbol at the end.

The expression is finished! Carefully check yours to see if it matches this completed where clause:

To make quick corrections, you can just copy and paste the following into the WHERE text area:

Select OK to close the Property Editor.

Save All your work.

This is the Where clause that we could have just typed in (copy and pasted) in the beginning, but I that it was important to show you the drag and drop features.

- 18 of 52 -

EXISTS (SELECT 'x' FROM CC_to_Lotto_Trans_in_Last_Day WHERE CC_to_Lotto_Trans_in_Last_Day.depositorName=Lotto_to_Bank.withdrawerName)ANDNOT EXISTS (SELECT 'y' FROM Betting_Trans_in_Last_Day WHERE Betting_Trans_in_Last_Day.bettorName=Lotto_to_Bank.withdrawerName)

Page 19: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Specifying an Output Stream

Validating the Event Processor

From the palette, drag an Output / Batched Stream Output and drop it onto the canvas to the right of Detect_Threat.

Connect it to Detect_Threat

Double-click it and set Name: Create_Alert

Click OK to dismiss the Property Editor and your event processor should look like this:

Let's make sure that we have created a good event processor.

Click on the Source control at the upper-left of the modeling canvas to view the XML.

Click on the Check File tool to check the XML syntax.

Click on the Validate File tool to check that the event model definition is complete.

Let's review. We have three inputs coming in: money in, bets and money out. Two inputs are filtered by moving 24 hour windows that will provide only transactions that arrived into our model within the last 24 hours.

The Detect_Threat operator applies some logic, expressed in SQL select syntax. For those transactions that are returned in the answer set for the SQL select statement, the model generates an output as an alert.

Output streams allow you to emit stimuli from your event model. In the WSDL that will be generated, message types and operations will be created for each output steam you add to your model.

- 19 of 52 -

Page 20: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Creating the WSDL for the Event Processor

Exploring Generated WSDL Interface

Save All your work.

Saving is an important point in producing your event processor. At save time is when the “magic” happens. The WSDL document that describes the inputs and outputs for the event model is automatically generated for you.

Notice in the Projects panel that myIEPLotto.wsdl is created for you under MyIEPProject / Processor Files node. It is through this single WSDL that we drive inputs into and get outputs from our event processor.

Let's explore the generated WSDL next. Double-click on myIEPLotto.wsdl to display the generated WSDL document in the WSDL editor:

It was magic for me at first. When I started exploring IEP, I wondered how my event model was going to be invoked and receive event data. I also wondered how it was going to direct the stimuli (output stream) it produces.

Then, it hit me. This event model is wrapped by a WSDL interface, but I didn't create the WSDL. Where did it come from? Then I noticed that when I save my model, the tooling automatically generates it. Neat!

- 20 of 52 -

Page 21: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Notice that message types are created for each input stream and output stream. Two types are created (Msg and Batch_Msg). These show the format of event data coming in and stimulus alerts going out. The data types are not defined in a separate XSD document. You'll see them specified in-line in the WSDL's Types section:

Two port types are created, one for input (the input streams) and one for output (the output stream). For each port type, you'll find that operations have been created for each. For input streams, there is a “regular” operation and a “batch” operation. The first one accepts just one input. The “batch” one accepts a repeating group of data:

Here are the messages. You see a Msg message and a Batch_Msg message for each operation. This makes it easy for you to test by sending in just a single transaction (use Msg) or a batch of transactions (Batch_Msg) as you do your testing.

When we test later, we'll want to send in a bunch of input at one time, so we will use the Batch_Msg version of each message.

I think that “port type” is a confusing term. The WSDL 1.1 specification uses “port type” to mean a named set of abstract operations and the abstract message that are involved. In Java, we use the term “interface” to describe such a construct. The WSDL 2.0 specification apparently thinks our Java term is clearer since WSDL 2.0 has renamed “port type” to “interface.” Until WSDL 2.0 becomes pervasive, we are stuck with “port type” though.

For your testing convenience, the concrete bindings for the operations on all input ports are set to SOAP and the output port operations are all bound to file. That makes it easy for you to use the composite application's test facility to send in SOAP inputs and then peek in the generated file for the output.

If you want your input to come from someplace else or your output to go somewhere else, just change the concrete portion of the WSDL. For example, instead of SOAP for input, you might bind to a JMS for a queue or topic.

- 21 of 52 -

Page 22: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

The concrete portion of the WSDL (Bindings and Services) is key as it specifies how data is to be provided to and generated by the operations . By default, all input ports are bound to SOAP and all output ports are bound to FILE.

This makes it easy to test your event processor by using the built-in test case facility of NetBeans tooling for composite applications. Test cases allow you to send SOAP/HTTP to a specified end point. Having the output going to a file makes it easy to see the results of the test.

See that for all the input stream-related bindings, a “soap:operation” is provided. For the stream output-related binding, you'll see “file:operation” specified.

So, by default, we'll have SOAP inputs coming in and outputs written to the file system.

- 22 of 52 -

Page 23: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

The name of the output file for each output operation is specified in the Bindings section of the concrete portion of the WSDL:

The directory name where the output file will be written is specified in the Services section:

You could edit the file and directory names, but let's leave the defaults in place.

If you don't see the Properties panel,use the Window / Properties menu bar option to display it.

- 23 of 52 -

Page 24: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

The key take-away from this WSDL exploration is that you can easily change the sources of model input and destination output. It is just a matter of modifying the concrete portion of the generated WSDL.

Save All your work.

Let's move on to get ready to test our event model.

As you probably realize, the bindings for SOAP will result in using the HTTP binding component (BC) to get data into our event processor.

The File binding will employ the File BC to output results.

- 24 of 52 -

Page 25: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Creating the Composite Application

Let's assemble a composite application containing our event processor so we can test it.

Create a new composite application by selecting File / New Project / SOA / Composite Application

The wizard appears. Click Next.

For Project Name, enter: MyIEPCompApp

Click Finish.

See that your MyIEPCompApp has been added to the Projects panel:

Now, add your event processor to the composite application by right-clicking on MyIEPCompApp and selecting Add JBI Module.

Select your MyIEPProject and click the Add Project JAR Files button.

Why do we have to create a composite application to test our event processor? Well, we need to assemble the JBI service assembly that formally engages the IEP SE, HTTP BC and File BC at runtime. The specification of a service assembly happens in the NetBeans tooling when we build a composite application and add JBI modules to it.

The Project Location should default to C:\MyIEPTutorial. That's what we want.

The project name is highlighted in red since we haven't added any JBI components to it yet.

- 25 of 52 -

Project 2: Assembling the Composite Application

We did a lot of work in Project 1 building an event processor. I bet you are curious to see if it works. To test it, we'll first need to create a composite application containing our MyIEPProject module.

Page 26: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Exploring the Service Assembly

The Composite Application Service Assembly (CASA) Editor appears and we can see that we have added one module to our composite application. However, no WSDL Ports have been identified so that the module can “talk” to the outside world (via a SOAP web service call or the file system, for example).

Right-click on MyIEPCompApp and select Clean and Build and the CASA Editor will be populated with the WSDL ports. The canvas now shows the content of the JBI service assembly for our composite application.

In the WSDL Ports column, we see that SOAP (HTTP BC) is delivering the inputs and that FILE (File BC) is delivering the outputs.

Save All your work.

Check the on-line help facility using the Help / Help Contents menu bar option and search for “casa”. You'll learn that the Composite Application Service Assembly (CASA) editor is used to assemble a composite application containing JBI modules.

The CASA Editor also provides a visual interface to modify the deployment configuration for a composite application.

The CASA Editor provides a palette from which you can drag and drop WSDL bindings, external Service Units, and connections to a composite application. This allows you to interactively specify WSDL Port configurations and connections between components in the composite application.

Check out the on-line help!

- 26 of 52 -

Page 27: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Deploying the Composite Application

Start GlassFish in the Services panel by selecting the Start option:

After about a minute or so, you'll be able to expand the GlassFish V2 hierarchy to verify that GlassFish is up and running:

This indicates that GlassFish is up and running. Now we can deploy our composite application for testing.

Back in the Projects panel, right-click on MyIEPCompApp and select Deploy.

Before we can deploy our application, we need to startup the JBI framework provided hosted by the GlassFish application server.

If you deploy the composite application before starting GlassFish, that's no problem. GlassFish will be started automatically before deployment begins.

You'll know when deployment is finished when you see the “BUILD SUCCESSFUL” message in the Output panel.

- 27 of 52 -

Page 28: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Creating Test Cases

Let's use the test case facility to build and exercise three test cases to provide the three types of input our event processor accepts:● CC_to_Lotto● Betting_Activity● Lotto_to_Bank

Find the Test node under MyIEPCompApp in the Projects panel, right-click and select New Test Case.

For Test Case Name, enter: CC_To_Lotto_TestCase

Click Next. When prompted for the WSDL document, select myIEPLotto.wsdl under MyIEPProject - Processor Files

Click Next. Select the CC_to_LottoBatch operation under InputPort.

Click Finish.

The test case input XML document appears:

We could use a driver utility that automatically sends lots SOAP XML documents to the web service endpoints for each of the three input streams.

Here, we'll just use the built-in Test Case facility and manually send in three batches of transactions: - credit card to lottery- betting - lottery to bank

See the comment: <!--Zero or more repetitions:-->

This reminds us that the CC_to_LottoBatch operation can accept a repeating group of transactions.

If we would have wanted to just send in a single transaction, we would have selected the CC_to_Lotto operation instead. Then, the test case would have used the CC_to_Lotto_MsgObj instead of the CC_to_LottoBatch_MsgObj we are using here.

- 28 of 52 -

Project 3: Testing the Composite Application

We have a composite application deployed to GlassFish that is ready for testing. Now, we will create test cases for the composite application to feed it the three types of transactions (CC_to_Lotto, Betting_Activity and Lotto_to_Bank). We'll watch for suspicious customers to appear in the Create_Alert.txt file.

Page 29: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Since we selected the “batch” operation, you see by the comment that we can input multiple occurrences of the CC_to_Lotto_MsgObj message.

Delete the existing XML and copy/paste the following so we have a repeating group of test transactions for movement of money from a credit card into a lottery account (CC_to_Lotto):

Save All your work.

Here we have in place a batch of credit card to PennyLotto transactions. You see one transaction for each of our three “players:” Tom, Sally and Joey.

<!--Zero or more repetitions:-->

Copy and paste the entire XML since XML is hard to type flawlessly.

- 29 of 52 -

<soapenv:Envelope xsi:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/ http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:myi="myIEPLotto_iep"> <soapenv:Body> <myi:CC_to_LottoBatch_MsgObj> <!--Zero or more repetitions:--> <CC_to_Lotto_MsgObj> <depositorName>Tom</depositorName> <inflowAmount>100.00</inflowAmount> </CC_to_Lotto_MsgObj> <CC_to_Lotto_MsgObj> <depositorName>Sally</depositorName> <inflowAmount>1100.00</inflowAmount> </CC_to_Lotto_MsgObj> <CC_to_Lotto_MsgObj> <depositorName>Joey</depositorName> <inflowAmount>2100.00</inflowAmount> </CC_to_Lotto_MsgObj> </myi:CC_to_LottoBatch_MsgObj> </soapenv:Body></soapenv:Envelope>

Page 30: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

We also need a test case to supply a batch of betting transactions into our event model. So, again find the Test node under MyIEPCompApp in the Projects panel, right click and select New Test Case.

For Test Case Name, enter: Betting_Activity_TestCase

Click Next. When prompted for the WSDL document, again select myIEPLotto.wsdl under MyIEPProject - Processor Files

Click Next. Select the Betting_ActivityBatch operation under InputPort.

Click Finish.

The test case input XML document appears. Delete the existing XML and copy/paste from below so it looks like this:

Save All your work.

Here you see that we have created a batch of betting transactions and notice that Sally and Joey made bets, but Tom didn't.

More copy and paste.

- 30 of 52 -

<soapenv:Envelope xsi:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/ http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:myi="myIEPLotto_iep"> <soapenv:Body> <myi:Betting_ActivityBatch_MsgObj> <!--Zero or more repetitions:--> <Betting_Activity_MsgObj> <bettorName>Sally</bettorName> <betAmount>25.00</betAmount> </Betting_Activity_MsgObj> <Betting_Activity_MsgObj> <bettorName>Joey</bettorName> <betAmount>45.00</betAmount> </Betting_Activity_MsgObj> </myi:Betting_ActivityBatch_MsgObj> </soapenv:Body></soapenv:Envelope>

Page 31: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

To complete our test suite, we need a batch of transactions that move funds from PennyLotto accounts to external bank accounts. So, once again, find the Test node under MyIEPCompApp in the Projects panel, right click and select New Test Case.

For Test Case Name, enter: Lotto_to_Bank_TestCase

Click Next. When prompted for the WSDL document, select myIEPLotto.wsdl under MyIEPProject - Processor Files

Click Next. Select the Lotto_to_BankBatch operation under InputPort.

Click Finish and delete all the existing XML. Then, use copy/paste to make the test look like this:

Save All your work.

Notice that Tom and Sally moved money out of PennyLotto accounts. Remember that Tom didn't bet! I'm suspicious.

Here's our final copy and paste test case data.

- 31 of 52 -

<soapenv:Envelope xsi:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/ http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:myi="myIEPLotto_iep"> <soapenv:Body> <myi:Lotto_to_BankBatch_MsgObj> <!--Zero or more repetitions:--> <Lotto_to_Bank_MsgObj> <withdrawerName>Tom</withdrawerName> <outflowAmount>100.00</outflowAmount> </Lotto_to_Bank_MsgObj> <Lotto_to_Bank_MsgObj> <withdrawerName>Sally</withdrawerName> <outflowAmount>200.00</outflowAmount> </Lotto_to_Bank_MsgObj> </myi:Lotto_to_BankBatch_MsgObj> </soapenv:Body></soapenv:Envelope>

Page 32: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Running the Test Cases

Now, we'll run each of the test cases to simulate the transactions coming into our event model. Open up Windows Explorer and watch the C:\temp\myIEPLotto_iep directory so you can see when the Create_Alert.txt file is created.

No alert will be generated until you run all three test cases because your Detect_Threat stream projection and filter is evaluating data from all three transaction sources:

Right-click on CC_To_Lotto_TestCase and select Run to send in some funds transfers from credit card into lottery accounts. Watch the build.xml (run,text-single) tabbed panel in the Output window for the “BUILD SUCCESSFUL” message indicating that the test case has run.

Right-click on Betting_ActivityBank_TestCase and select Run to send in your batch of betting transactions. Again, watch the build.xml (run,text-single) tabbed panel in the Output window for messages.

Finally, right-click on Lotto_to_Bank_TestCase and select Run to submit your transactions that move money from lotto accounts to personal bank accounts. Again, watch the build.xml (run,text-single) tabbed panel in the Output window for messages.

Wait about 15 seconds or so for Create_Alert.txt to be created.

Before your first test run, the myIEPLotto_iep directory won't yet exist in temp.

When we have all three transaction types sent in, the Create_Alert.txt output should appear in the C:\temp\myIEPLotto_iep directory. Wait at least 30 seconds before going into troubleshooting mode.

Here's some troubleshooting advice in case you never see the Create_Alert.txt file appear in the temp directory. Try these steps:1. Stop GlassFish (right-

click on GlassFish V2 in the Services panel and select Stop).

2. When GlassFish has stopped, in Windows Explorer, delete the myIEPLotto_iep directory under temp.

3. Start GlassFish (right-click on GlassFish V2 in the Services panel and select Start).

4. Wait about 30 seconds and you should see the myIEPLotto_iep directory created and the Create_Alert.txt appear inside.

This sequence of troubleshooting steps “resets things” to give you a fair chance at a retry.

- 32 of 52 -

Page 33: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Viewing the Generated Alert

Adding Timestamp to the Alert

Open the Create_Alert.txt file in an editor (like Notepad) to see:

Let's check this for accuracy:● CC_To_Lotto_TestCase contains inflow

transactions for Tom, Sally and Joey ● Betting_Activity_TestCase sends in bets for

Sally and Joey● Lotto_to_Bank_TestCase provides outflow

transactions for Tom and Sally

We see that Tom is the only “player” who transferred money from his credit card into PennyLotto, withdrew it, but never bet during the one day time frame. He is a possible culprit! IEP will nab him!!

Let's make a minor enhancement to our model so that the alert output includes a time stamp. That would be important data for further investigation and for plotting trends on an executive dashboard.

To add the time stamp, open your myIEPLotto.iep model in the editing canvas and double-click on the Create_Alert operator. The Batched Stream Output Property Editor window displays:

- 33 of 52 -

<msgns:Create_AlertBatch_MsgObj xmlns:msgns="myIEPLotto_iep"><Create_Alert_MsgObj><withdrawerName>Tom</withdrawerName><outflowAmount>100.0</outflowAmount></Create_Alert_MsgObj></msgns:Create_AlertBatch_MsgObj>

Page 34: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Turn on the time stamp by toggling on the Include Timestamp check box.

Click OK to dismiss the editor window.

Save All your work.

Redeploy the composite application by right-clicking on MyIEPCompApp and selecting Deploy.

Close the editor window where you were editing the Create_Alert.txt file.

Resubmit each of your test cases by right-clicking on each and selecting Run in this order:• CC_To_Lotto_TestCase• Betting_Activity_TestCase• Lotto_to_Bank_TestCase

Wait about 15 seconds and then use your editor to again see the contents of the Create_Alert.txt file.

You'll see:

We see two alerts for Tom. The last one has the <Timestamp> data as we hoped. The first one doesn't.

Why do we have two records in the file? Let's explore.

15 seconds is just an estimate. I watch the Date Modified column in Windows Explorer for the Create_Alert.txt file. When I see the date change, I know that the file has been refreshed.

- 34 of 52 -

<msgns:Create_AlertBatch_MsgObj xmlns:msgns="myIEPLotto_iep"><Create_Alert_MsgObj><withdrawerName>Tom</withdrawerName><outflowAmount>100.0</outflowAmount></Create_Alert_MsgObj></msgns:Create_AlertBatch_MsgObj><msgns:Create_AlertBatch_MsgObj xmlns:msgns="myIEPLotto_iep"><Create_Alert_MsgObj><withdrawerName>Tom</withdrawerName><outflowAmount>100.0</outflowAmount><Timestamp>2009-04-15T10:23:00.839-05:00</Timestamp></Create_Alert_MsgObj></msgns:Create_AlertBatch_MsgObj>

Page 35: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Find myIEPLotto.wsdl under MyIEPProject/Processor Files in the Projects panel and double-click it to display the WSDL document in the WSDL Editor.

See that the multipleRecordsPerFile property on the file:message element is set to true:

So, the Create_Alert.txt file is acting as a log. Each time new data is evaluated and a potential culprit is found, a new alert record is appended to the end of the file. The <Timestamp> field records when the alert was produced.

During testing, if you want to clear the Create_Alert.txt log file for a fresh start, just delete it in Windows Explorer. It will be recreated when you run all three of your test cases again.

It's convenient to be able to clear the output file by just deleting it. You'll see that the IEP does maintain a Create_Alert.txt.lck lock file, instead of using the operating system to lock Create_Alert.txt.

- 35 of 52 -

Page 36: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Adding a Save Stream Output

Here is how our event processor looks so far:

Let's add an operator to save the KPI alert data to a database table.

From the palette, drag an Output / Save Stream Output and drop it onto the canvas to the above Create_Alert.

Connect the new Save Stream Output to the Detect_Threat operator.

Double-click it and set the Name to: Store_Alert_in_DB

For Table Name, enter: Alert

For JNDI Name, enter: jdbc/myIEPTutorialDB

This is how we left it at the end of Project 2.

According to the on-line help, the Save Stream operator is mainly used to output diagnostic data to a database table for subsequent analysis. However, I found it a very convenient way of logging the alert data to the database rather than to the file system,

The JNDI Name is a reference to a JDBC resource well configure in GlassFish a bit later. The name provides the “handle” that the Save Stream Output will use to get to the database table where we'll store our alert KPI data.

- 36 of 52 -

Project 4: Storing KPI Data in a Database

KPIs,or Key Performance Indicators, are being talked about a lot these days. According to Wikipedia, KPIs are “financial and non-financial measures or metrics used to help an organization define and evaluate how successful it is, typically in terms of making progress towards its long-term organizational goals. KPIs can be specified by answering the question, "What is really important to different stakeholders?". KPI data often is displayed in executive dashboard applications to keep stakeholders informed.

It sounds like alerts for possible money laundering transactions is important to one of PennyLotto's key stakeholders – its management team. So, in addition to logging the alerts to a file, we will them to a database table. Having alerts in a database table makes it easier to access the KPI data to drive an executive dashboard display. We'll add the database logging facility here.

Page 37: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Exploring the Service Assembly

Click OK to dismiss the Property Editor and your event processor should look like this:

Save All your work.

Deploy your MyIEPCompApp composite application again by right-clicking on it in the Projects panel and selecting Deploy.

Expand the MyIEPCompApp hierarchy to find the Service Assembly node. Double-click on it to display the service assembly in the CASA Editor:

If you have done other GlassFish ESB work that output data to a database, you might be surprised to see that the service assembly hasn't changed since adding the Store_Alert_in_DB operator. We still see only two WSDL Ports: SOAP and File. The reason it's the same service assembly configuration is that IEP encapsulates the data base operations inside your MyIEPProject. It automatically utilizes whatever underly JBI infrastructure is required to reach the database.

- 37 of 52 -

Page 38: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Establishing the KPI Database

In configuring Store_Alert_in_DB, we mentioned a database table called Alert and a JNDI name of jdbc/myIEPTutorialDB. To support that configuration, we now need to create a database for the table in Java DB and specify a connection pool and JDBC resource in GlassFish.

Let's create the database first. In the Services panel, right-click on the Java DB node and select Create Database. The Create Java DB Database window appears.

For Database Name, enter: MyIEPTutorialDB

For User Name, enter: admin

For Password, enter: admin

Click OK to create the database.

As soon as the database is created, you'll see that a connection node has been added for it:

This is basic Java EE JDBC configuration. To use a database from a Java EE container, you need to first create a connection pool and then, create a JDBC resource that uses the connection pool.

- 38 of 52 -

Page 39: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Logging Into the GlassFish Admin Console

Let's open that new connection. Right-click on it and select Connect.

If prompted for Password, enter: admin

Toggle on the Remember password check box if you don't want to be prompted for the password each time you log on inside NetBeans.

Click OK to login. You can tell you are logged in when you can expand the connection hierarchy:

The database is in place, now we need to add the connection pool and JDBC resource to the GlassFish environment. To do that, we'll use the GlassFish Admin Console.

To invoke the Admin Console, right-click on GlassFish V2 in the Services panel under Servers and select the View Admin Console option. A browser page appears challenging you for GlassFish login credentials.

For User Name, enter: admin

For Password, enter: adminadmin

Click the Login button.

- 39 of 52 -

Page 40: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Establishing a Connec-tion Pool

In the left-side menu hierarchy, expand the Resources / JDBC node to see where our connection pool and JDBC resource are specified:

Click on Connection Pools to view existing pools. Click the New button to create a new one:

For Name, enter: myIEPTutorialPool

For Resource Type, select: javax.sql.DataSource

For Database Vendor, select JavaDB

Click Next.

This is standard Java EE database access. You establish a connection pool that manages connections to the underlying database. Then, you define the data source. In your application you use JDBC to interact with the data source which uses the pool to interact with the database.

Most databases are accessible via JDBC. We'll use Java DB which is Sun's supported version of Apache Derby. It's bundled with NetBeans so it is a convenient choice.

- 40 of 52 -

Page 41: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Scroll down to the Additional Properties section and use the check boxes and the Delete Properties button to delete all but the following five properties. Then, for the remaining five, specify the values as shown:

Click Finish. The connection pool appears in the list:

Click on myIEPTutorialPool so we can test the connection.

The Edit Connection Pool section appears for our connection. Click on the Ping button. If you see the “Ping Succeeded” message, GlassFish can “see” your new database:

For copy and paste convenience, it's:User: adminDatabaseName: MyIEPTutorialDB Password: adminServerName: localhostPortNumber: 1527

The Ping feature lets you get quick verification that the connection pool can communicate successfully with the database.

- 41 of 52 -

Page 42: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Defining a JDBC Resource

Reviewing Generated Save Stream Database Table

Now, we need to configure a JDBC resource that uses our connection pool to access the database.

Click on JDBC Resources to view existing resources. Click the New button to create a new one.

For JNDI Name, enter: jdbc/myIEPTutorialDB

Accept the default Pool Name (myIEPTutorialPool).

Click OK to create the JDBC Resource. It now appears in the list of resources:

Deploy your MyIEPCompApp composite application again by right-clicking on it in the Projects panel and selecting Deploy.

The tooling just did something you might not expect. After deployment is complete, return to the Services panel and right-click on the Tables node under you MyIEPTutorialDB connection to see that the Alert table was created automatically for you:

JNDI is Java Naming and Directory Interface. and it provides Java with a uniform interface to multiple naming and directory services. The JNDI Name provides a “handle” for an application to reach a data source via a connection pool.

You might also notice that IEP has created two JDBC resources it will use behind the scenes: jdbc/iepseDerbyXA and jdbc/iepseDerbyNonXA.

You didn't need to manually create the Alert table. IEP did it for you at deploy time. Notice that it added a couple fields of its own (EMS_SEQID and EMS_TIMESTAMP) to the two you saw in the Save Stream properties (WITHDRAWERNAME and OUTFLOWAMOUT).

- 42 of 52 -

Page 43: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

See that two of the columns are familiar: WITHDRAWERNAME and OUTFLOWAMOUT. These were the attributes of your Store_Alert_in_DB. The were “inherited” via the linkage to Detect_Threat:

View contents of Alert table by right-clicking on the ALERT node under Tables in the Services panel and selecting the View Data option. You see that there is no data in the table yet:

When we run our test cases again, one row will appear for every alert.

NetBeans provides a nice, built-in facility to review the schema of a database and to dig into the contents of a particular table.

- 43 of 52 -

Page 44: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Running the Test Cases

Let's run the three test cases again to see if a row for the culprit Tom appears in our Alert table.

First, let's reset the file log by finding the Create_Alert.txt file in the C:\temp\myIEPLotto_iep directory and deleting it. I used Windows Explorer.

Now, resubmit each of your test cases by right-clicking on each and selecting Run in this order:• CC_To_Lotto_TestCase• Betting_Activity_TestCase• Lotto_to_Bank_TestCase

Wait about 15 seconds and then use your editor to again see the contents of the Create_Alert.txt file.

Look inside the file to see that one alert was generated for Tom.

Return to the database view window you displayed earlier and click the Run SQL control to resend the select statement to display the contents of the ALERT table.

You'll see that the alert for Tom was also added to the database:

Also check the contents of the Create_Alert.txt file to find the alert there.

Success! The alert was written to both outputs just as we hoped.

Troubleshooting Note: If you don't see the file created, you may need to resubmit your three test cases again.

- 44 of 52 -

Page 45: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Finding the IEP Internal Database

By default, the event processor stores its data in Java DB. So, I wanted to peek inside the IEP database. I wanted to open a connection to it, but I didn't know the database name.

To find the name of the database, right-click on the Java DB node under Databases in the Services panel and select Properties. The Database Location shows you the directory you need to investigate to find all of the Java DB databases:

Click Cancel to dismiss the window.

So, in Windows Explorer, look in the C:\GlassFishESBV2.1\glassfish\databases directory.

You will find a directory named iepseDB. Ah ha, that's the database.

With the database name in hand, let's build a connection to it so we can explore the tables that IEP uses. Right-click on the Databases node in the Services panel and select New Connection.

You will also see the MyIEPTutorialDB where your database is stored that you created in the last project.

- 45 of 52 -

Project 5: Exploring Behind the Scenes

No matter what wonderful technology you are working with, it isn't so wonderful when you hit a problem. I am always seeking troubleshooting insight and have had a craving to learn how the event processor stores its data. Being able to peek inside internal tables might help me during those dark times when I need to troubleshoot a problem. Let me show you what I learned.

Page 46: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

For Host, enter: localhost

For Port, enter: 1527

For Database, enter: iepseDB

For User Name, enter: iepseDB

For Password, enter: iepseDB

Toggle on the check box for Remember password so you aren't prompted for the password each time you open the connection in NetBeans.

Click OK. When prompted to Select schema, accept IEPSEDB, the default, and click OK.

The new connection (iepseDB) appears in the Services panel.

Right-click on the new connection and select Connect.

Expand the connection hierarchy to see all the tables in the iepseDB database:

OK, how did I know what User Name and Password to use? Well, I guessed? Better answer is that I Googled for “iepseDB password”.

- 46 of 52 -

Page 47: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Watching Data Flow Through Internal IEP Tables

The interesting tables are the “Q_0_O” tables as this is where our data appears as it flows through our event model. Let's explore that data flow.

First, let's clear these tables. I found that whenever the composite application is deployed, these tables are cleared. So, from the Projects panel, redeploy your MyIEPCompApp composite application.

Now, we'll resubmit each of the test cases and peek into the underlying tables as we go along.

In the Projects panel, run CC_To_Lotto_TestCase.

In the Services view the contents of the Q_0_O0 table. You'll see our test data from our test case appear in the database table:

Now, run Betting_Activity_TestCase.

View the contents of the Q_0_O1 table. You'll see that the event processor has stored the betting test data:

Right-click on the MyIEPCompApp node in the Projects panel and select Deploy.

In the Projects panel, right-click on each test case and select Run.

Right-click on each table and select View Contents.

If you don't see any data, resubmit your test case.

- 47 of 52 -

Page 48: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

Mapping Internal IEP Tables to Event Model Operators

Now, run the Lotto_to_Bank_TestCase test case.

View the contents of the Q_0_O2 table to see the withdrawals test data:

With that third test case submitted, the event processor has all it needs to do the filtering. After pausing for 15 seconds or so, view the contents of the Q_0_O5 table to see the resulting alert for Tom:

Ok, I can hear you asking: “How did you know which table represented which test case and how did you know that the alert was written internally to the Q_0_O5 table?”

Well, the internal tables map 1:1 to the operators on our event model. The number at the end of the “Q_0_O” file name matches the order we added the operators to the event canvas. The picture on the next page illustrates the mappings of internal tables to event model operators.

- 48 of 52 -

Page 49: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

I didn't find the tables for the two Time Based Window operators (Q_0_O3 and Q_0_O4) very interesting.

I must strongly caution you not to change the structure of these tables. Also, since these are internal tables, they are subject to change as the IEP SE evolves.

So, there is a quick behind-the-scenes peek at how the event processor stores its data internally. This insight may help you troubleshoot your event models when you have created a model that is syntactically correct, but doesn't generate the output you expect. You could step across the model and interrogate the underlying table for each operator to see where the flow of data filtering went wrong.

We have explored enough. Proceed to the following section to review what we covered. Then, take a break. You deserve it.

- 49 of 52 -

Page 50: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

● The tutorial started with a brief introduction to the Event-Driven Architecture (EDA) approach and commented on how it relates to SOA.

● We started work on replicating Sun's simple PennyLotto demo. The scenario presented specifications for fraud detection that involved scanning three incoming transaction types and generating an alert when a heuristic determines a possible money laundering situation.

● The IEP tooling provides a rich palette of operations that included categories like input, output, aggregation, correlation/filter, relation operator and sequence (to name just a few). We used a number of these as we defined our event processor.

● We used Input Steam to get our three types of transactions into our model. We specified Time Based Window operators to bind the scope of consideration to 1 day. We did the "real work" in a Correlation and Filter operator where we specified the heuristic in SQL syntax. Finally, we used an Output Stream to produce the alert output.

● After dragging and dropping the appropriate operations from the palette onto the canvas, we configured each. Configuration included identifying data attributes (like depositorName and inflowAmount) that were to be evaluated.

- 50 of 52 -

Summary

We covered a lot of ground in this tutorial. Thanks for hanging in there. Here are the highlights.

Page 51: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

● We saw that our event processor is “wrapped” by a WSDL interface and that the WSDL is automatically created for us when we save the model.

● We explored both the logical and concrete portions of the generated WSDL document so we understood how data was coming in (SOAP) and going out (FILE) for the various operations. These were defaults that could be easily changed to use other binding components like JMS or JDBC, for example.

● We built a composite application and added our event processor module to it. We deployed the application to the JBI runtime that GlassFish hosts.

● The built-in testing facility allowed us to easily send the three transaction types into our event processor as SOAP documents. We checked the file system to see that an alert was generated.

● We extended the basic PennyLotto demo by adding using a Save Steam Output operator to also output our alert to a database table. Storing potential Key Performance Indicator (KPI) data like alerts to a database might form the basis for updates to an executive dashboard application.

● Finally, we took a peek “behind the scenes” to explore how IEP stores its data. We learned that, for our model, there was a 1:1 mapping between operators we had used and underlying database tables. We followed the flow of data into these tables as we ran each of our three test cases. Hopefully this insight will help in troubleshooting event models that don't perform like you thought they should.

On the next page, you'll see a few references to IEP-related documentation, demos and labs.

- 51 of 52 -

Page 52: Event Processing with GlassFish ESB - OpenESBopen-esb.net/files/OpenESB Documents/Papers... · - 3 of 52 - Introduction GlassFish ESB is built atop the Java Business Integration (JBI)

I hope you enjoyed this tutorial experience and I appreciate your interest in GlassFish ESB technology.

Please provide feedback on this tutorial to:[email protected].

- 52 of 52 -

- IEP Component Page: https://open-esb.dev.java.net/IEPSE.html

- IEP Wiki: http://wiki.open-esb.java.net/Wiki.jsp?page=IEPSE

- Java One 2008 PennyLotto Demo: http://www.glassfishwiki.org/jbiwiki/attach/IEPSE/EDA_Pod_Demo.zip

- Java One 2008 IEP Hands-On Lab: http://www.glassfishwiki.org/jbiwiki/attach/IEPSE/5540_IEP.zip

- Insider Trading Detection Demo: http://www.glassfishwiki.org/jbiwiki/attach/IEPSE/InsiderTradingDetection.zip

- Dead Device Detection Demo: http://www.glassfishwiki.org/jbiwiki/attach/IEPSE/DeadDeviceDetection.zip

References

Here are some references that may help you learn more about the Intelligent Event Processor: