Using JMS Queues with Oracle Data Integrator O R A C L E W H I T E P A P E R | N O V E M B E R 2 0 1 5

Using JMS Queues with Oracle Data Integrator

1 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

Table of Contents

Introduction 2

Main Article: Using JMS Queues with Oracle Data Integrator (ODI) 3

Configuring JMS Queues in Oracle WebLogic 12c 5

Configuring JMS Servers 7

Configuring JMS Modules 11

Configuring JMS Queues 16

Configuring Connection Factories 23

Configuring the XML and DTD Files for ODI 31

Configuring JMS Queues in Oracle Data Integrator 35

Configuration #1: JMS Queue Booking 35

Configuration #2: JMS Queue Revenue 35

Configuring JMS Queues in the ODI Topology 36

Configuring ODI Data Models for JMS Queues 42

Adding the ODI JMS Queue Knowledge Modules 49

Configuring an ODI Mapping to Consume JMS Queue Messages 51

Creating an ODI Mapping to Publish Messages to JMS Queues 55

Testing the JMS Queue Configuration 58

How to Add Messages to the JMS Queue 58

Executing the ODI Mapping to Publish Messages to the JMS Queue 66

Conclusion 68

Summary of Additional Resources 70

2 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

.Introduction

This article presents a demo on how to configure and use Java Message Service (JMS) queues with

Oracle Data Integrator (ODI). Two examples of how to use JMS queues with ODI are presented. The

first example demonstrates how to use ODI to consume JMS messages from JMS queues and load

their content into Oracle tables. The second example demonstrates how to use ODI to integrate data

from Oracle tables, transform it, and send it as a JMS message to a JMS queue.

In ODI, the JMS Queue XML technology will be used to demonstrate these two examples. The JMS

messages illustrated in this article are extensible markup language (XML) documents.

This article is divided into three main parts:

The first part shows how to configure two JMS queues using the WebLogic Server 12.1.3.0.0,

which includes creating the JMS servers, the JMS modules, the JMS queues, and the JMS

connection factories. ODI will consume JMS messages from the first JMS queue, and will

send JMS messages to the second JMS queue.

The second part shows how to configure these two JMS queues in ODI. The ODI discussion

includes how to configure the ODI physical data servers, the ODI data models, the ODI

knowledge modules, and the ODI mappings.

The third and final part shows how to test the JMS queues with ODI. JMS messages will be

added into a JMS queue using the WebLogic Console. Then, ODI will consume these JMS

messages, and load their content into an Oracle table. This Oracle table will then be used by

ODI as a source dataset to transform, and send a JMS message to another JMS queue.

This white paper was inspired by Oracle customers in the airline industry; they use JMS queues and

ODI to consume, transform, and send millions of JMS messages every day. Their JMS messages

include passenger bookings, reservations, ticketing, and other airline subject areas.

If you would like to download a free copy of this ODI demo, go to ODI 12.1.3 SmartExport Demo for

JMS Queues, and search for JMS.

https://java.net/projects/oracledi/downloads/directory/ODI/Repository%20Sampleshttps://java.net/projects/oracledi/downloads/directory/ODI/Repository%20Samples

3 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

Main Article: Using JMS Queues with Oracle Data Integrator (ODI)

Oracle defines the Java Message Service (JMS) Application Programming Interface (API) as a messaging standard

that allows application components to create, send, receive, and read messages. It enables distributed

communication that is loosely coupled, reliable, and asynchronous. The JMS API is based on the Java Platform

Enterprise Edition (Java EE).

The JMS API offers several advantages:

Neither the sender nor the receiver needs to know anything about each other; only the message format and

the destination are required. Thus, the JMS API enables distributed communication that is loosely coupled.

The sender and the receiver do not have to be available at the same time in order to communicate. The

sender can send a message to a destination, and the receiver can retrieve it at anytime.

A JMS provider can deliver messages to a client as they arrive; thus, the client does not have to request

messages in order to receive them. This is known as JMS Asynchronous Communication.

The JMS API enables reliable communication; the JMS API can guarantee that a message is delivered once

and only once. Additional levels of reliability are available within the JMS API architecture for applications

that cannot afford to miss or get duplicate messages.

The Oracle WebLogic 12c server is fully compliant with both the Java EE 5.0, and the JMS 1.1 Specification. The

WebLogic JMS architecture includes JMS servers that can host a defined set of JMS modules. JMS modules

contain destinations such as queues and topics - and resources such as connection factories. Client JMS

applications such as Oracle Data Integrator (ODI) can either produce messages to destinations, or consume

messages from destinations.

Figure 1 shows a conceptual representation of how the WebLogic JMS architecture can be used with ODI. ODI can

consume JMS messages from WebLogic JMS destinations. Likewise, ODI can send JMS messages to WebLogic

JMS destinations. In ODI, the content of JMS messages can be loaded into a SQL database and transformed by

any transformation engine, such as the Oracle database or a Hadoop cluster.

http://docs.oracle.com/javaee/6/tutorial/doc/bncdr.html

4 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

WebLogic JMS with ODI

Message Message

WebLogic JMS

Transformation

Engine

Consum

e Publish

ODI

Figure 1: WebLogic JMS with ODI

ODI supports two types of JMS destinations: JMS queues, and JMS topics. For each JMS destination, two types of

messages are supported:

5 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

JMS Messages This type of message contains rows of data known as text payloads and they can be

in the form of text, or binary documents. When JMS messages are processed by ODI, the JMS message is

mapped to a single ODI datastore. Each JMS message is stored as a record in the ODI datastore.

JMS XML Messages This type of message contains a complete XML data structure known as XML

payloads. In ODI, a JMS XML message is mapped to a relational schema known as the ODI XML

schema. The ODI XML schema contains - at a minimum - two ODI datastores representing the XML data

structure: the root element, and the child element.

JMS messages and JMS XML messages contain header fields and message properties such message ID, message

type, and message destination. ODI knowledge modules can filter messages based on header fields or message

properties; thus, reducing the number of messages retrieved from a JMS destination. Also, ODI can filter messages

using datastore columns from the ODI XML schema, but the filtering is applied after the messages are consumed

from the JMS destination.

This article focuses on how to create JMS queues in WebLogic 12c, and how to use ODI to consume and send JMS

XML messages from and to JMS queues, respectively.

For additional information on the JMS API and the ODI JMS technologies, go to the following resources:

Java Messaging Service (JMS) Specifications Introduction to the Java Message Service (JMS) Fusion Middleware Programming JMS for the WebLogic Server Using Java Message Services (JMS) in Oracle Data Integrator (ODI)

Configuring JMS Queues in Oracle WebLogic 12c

This section demonstrates how to configure JMS queues in WebLogic. Two JMS queues will be configured in this

section. The first JMS queue will be used as a source ODI will read and consume JMS XML messages from this

JMS queue. The second JMS queue will be used as a target ODI will send JMS XML messages to this JMS

queue.

Figure 2 illustrates the logical configuration of these two JMS queues:

https://jcp.org/en/jsr/detail?id=343http://docs.oracle.com/javaee/6/tutorial/doc/bncdq.htmlhttp://docs.oracle.com/cd/E24329_01/web.1211/e24387/fund.htm#JMSPG116https://docs.oracle.com/middleware/1212/odi/ODIKM/jms.htm#ODIKM1035

6 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

JMS XML Queues with ODI

WebLogic JMS

Oracle Data Server

ODI PublishConsume

7 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR

Figure 2: JMS XML Queues with ODI

Figure 2 above shows the configuration of two JMS servers: BookingServer, and RevenueServer. The first JMS

server, BookingServer, will host a JMS module called BookingModule. Two JMS resources will be created in this

JMS module: BookingQueue, and BookingConnection. BookingQueue is a JMS queue containing JMS XML

messages about passenger bookings. ODI will consume these JMS XML messages from this queue. The

BookingConnection is a JMS connection factory. A JMS connection factory is an object in WebLogic that

encapsulates connection configuration information. ODI will use a JMS connection factory to create a connecti