ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Oracle Data Integrator 11G New Features Overview
O R A C L E W H I T E P A P E R | M A Y 2 0 1 5
Table of Contents
Executive Overview 5
Oracle Data Integrator 11.1.1.9.0 Release 5
Studio Login Security Enhancements 5
Oracle Data Integrator Asynchronous Web Services 5
Oracle Data Integrator 11.1.1.7.0 Release 5
Feature Highlights: Hot Pluggability 5
IBM WebSphere Support 5
Other Features 6
XML Driver Enhancements 6
SSL Support for Standalone Agent 6
Oracle Enterprise Data Quality Integration 6
Oracle Data Integrator 11.1.1.6.0 Release 6
Feature Highlights: BI and Data Warehousing 6
Shortcuts 6
Tracking Variables and Sequences 6
Global Knowledge Modules 7
Enhanced Session Logging 7
Handling Failed Load Plan Enhancements 7
Enhanced Variable Handling in Load Plans 7
Feature Highlights: Development Lifecycle 8
Smart Export and Import 8
Feature Highlights: Data Quality 9
Oracle Enterprise Data Quality Integration 9
Other Features 9
Groovy Editor 9
Support of Undo and Redo Operations 9
Autocomplete for Text Fields and Lists 9
Version Numbering for Knowledge Modules 9
Oracle Data Integrator 11.1.1.5.0 Release 1
Feature Highlight: SOA and Application Integration 1
Complex File Technology 1
Web Services Enhancements 2
Web Service Container Based Authentication 2
Asynchronous Web Services and Callbacks 2
Full SOAP Support for Outbound Calls 2
Features Highlights: BI and Data Warehousing 2
Load Plans 2
Oracle Business Intelligence Enterprise Edition Data Lineage 4
Other Features 5
Commands on Connect/Disconnect 5
Built-in Technology Additions and Updates 5
Groovy Technology 5
Ordered and Non-Ordered Join Syntax 5
New Method for Setting Task Names 6
Shared Library for WLS Agent 6
Performance Optimization 6
Oracle Data Integrator 11.1.1.3.0 Release 6
Architecture for Enterprise-Scale Deployment 6
High Availability and Scalability 6
Corporate Security Integration 7
Enterprise-Scale E-LT Architecture 7
Simplified Deployment and Unified Administration 8
Simplified Deployment 8
Better Control over Production 9
Oracle Data Integrator Console 9
Enhanced Session Control 10
Enhanced Error Management 10
Design-Time Productivity 11
Design-Time Experience 11
New IDE Based on JDeveloper IDE Framework 11
Redesigned Interface Editor 12
Auto-Fixing 12
Quick-Edit 12
Small (yet Important) Things 13
Java API for Task Automation 13
E-LT Features for Better Performance 13
Datasets and Set-Based Operators 13
Partitioning 14
Lookups 14
Derived Select for Temporary Interfaces 15
Support for Native Sequences 16
Support for Natural Joins 16
Automatic Temporary Index Management 16
Broaden Connectivity 17
Conclusion 17
.
Executive Overview
Oracle Data Integrator (ODI) is a best-of-breed data integration platform focused on fast bulk
data movement and handling complex data transformations.
Oracle Data Integrator is fully integrated with the Oracle technology stack, including Oracle
Database, Exadata Database Machine, Exalogic, Big Data Appliance, WebLogic Server,
Business Intelligence, and Oracle Applications. ODI is the strategic data integration platform
for Oracle.
The 11G version of ODI has pushed the state of the art technology in data integration further
ahead of the rest of the industry. Since the first ODI 11.1.1.3.0 release in July 2010, Oracle
has continued its investments on this strategic data integration platform with the 11.1.1.5.0,
11.1.1.6.0 and 11.1.1.7.0 releases.
This whitepaper describes in detail some of the new features and capabilities offered in the
Oracle Data Integrator 11G platform.
Oracle Data Integrator 11.1.1.9.0 Release
Studio Login Security Enhancements
Oracle Data Integrator Studio login credentials can now be stored in a password-protected wallet. In order
to login to the Oracle Data Integrator Studio, a login profile must be created where the connectivity to the
ODI repository and the user credentials are specified. Oracle Data Integrator can now save these
credentials in either an encrypted login XML file or a password-protected wallet. Depending on your
security requirements you can now choose either of them. This feature is already available in ODI 12c
releases.
Oracle Data Integrator Asynchronous Web Services
The Oracle Data Integrator agent provides web services for invoking Oracle Data Integrator sessions in
both synchronous and asynchronous modes. In the synchronous mode, the web services invocation is
blocked until the session execution finishes. However, in the asynchronous mode the web services call is
not blocked and the caller provides details of the end-point where the response should be sent on session
completion. Oracle Data Integrator can now send responses to such asynchronous invocation to end-
points that are secured with Oracle Web Service Manager (OWSM) policies.
Oracle Data Integrator 11.1.1.7.0 Release
Feature Highlights: Hot Pluggability
A core strategy and a key feature of Oracle Fusion Middleware is hot-pluggability. Built for the
heterogeneous enterprise, Oracle Fusion Middleware consists of modular component software that runs
on a range of popular platforms and interoperates with middleware technologies and business applications
from other software vendors such as IBM, Microsoft, and SAP. The hot-pluggability aims to make
components deployable and operational in non-Oracle environments and on multiple application servers.
IBM WebSphere Support
This ODI release introduces the support of IBM WebSphere Application Server (WAS).
ODI Agents and datasources can now be deployed on Oracle WebLogic Server (WLS) and IBM
WebSphere Application Server (WAS).
Other Features
In addition to the IBM WebSphere support listed above, the following features have been included in this
patch.
XML Driver Enhancements
Full support for XML Schema constructs by using standard Oracle parser:
» New JDBC driver parameter compat_mode to indicate the compatibility with mapping modes such
as the XDK XSD parser.
» New JDBC driver command WRITEMAPPINGFILE to help understand the relational structure that
has been created for the XSD/DTD file. The mapping file contains the element/attribute name to
table/table.column name mapping for each element/attribute.
» The JDBC command CREATE SCHEMA includes now the compat_mode parameter.
SSL Support for Standalone Agent
The Standalone Agent now supports the SSL protocol for secure connectivity. New parameters
added to specify the Standalone Agent server side transport protocol and to configure the SSL
truststore and keystore:
» New environment variables in agent.sh/agent.bat scripts.
» New configuration variables in odiparams.sh/odiparams.bat scripts.
» Java system properties in odiparam.sh/odiparams.bat scripts.
Oracle Enterprise Data Quality Integration
The OdiEnterpriseDataQuality Open Tool introduced in ODI 11.1.1.6 to invoke an Oracle Enterprise Data
Quality Job is now available as a standard ODI tool. The tool includes a new, optional parameter to
connect to MBean domains.
Oracle Data Integrator 11.1.1.6.0 Release
Feature Highlights: BI and Data Warehousing
The following features have been added in this release for customers using Oracle Data Integrator in
Business Intelligence and Data Warehousing projects.
Shortcuts
This ODI release introduces new objects called shortcuts which greatly improve productivity by allowing
end users to express the large commonality that often exists between two different versions of the same
source application, such as same tables and columns, same constraints, and same transformations.
Shortcuts are links to common Oracle Data Integrator objects stored in separate locations and can be
created for datastores, integration interfaces, packages, and procedures. In addition, release tags have
been introduced to manage the materialization of shortcuts based on specific tags.
Tracking Variables and Sequences
Variables and sequences are often used in Oracle Data Integrator processes. Oracle Data Integrator
11.1.1.6 introduces a new feature allowing end users to determine the actual values of variables and
sequences that were used during an executed session. Tracking variables and sequences is extremely
useful for debugging purposes.
Figure 1 - Variable and Sequence Values are displayed in the Operator Navigator
With the variable tracking feature you can also easily determine whether the variable was used in a
mapping or an internal operation such as an Evaluate Variable step.
Global Knowledge Modules
ODI 11.1.1.6 introduces Global Knowledge Modules (KMs) allowing specific KMs to be shared across
multiple projects. In previous versions of Oracle Data Integrator, KMs were specific to Projects and could
only be used within the project into which they were imported. Global KMs are now listed in the Designer
Navigator in the Global Objects accordion.
Enhanced Session Logging
The readability of the execution logs has been improved in this patch for Knowledge Modules and
Procedure steps. The final code for source and target commands is now available in the Operator
Navigator, making it easier to review executions containing several runtime parameters.
Handling Failed Load Plan Enhancements
It is now possible to change the execution status of a failed Load Plan step from Error to Complete on the
Steps tab of the Load Plan run Editor. This particular Load Plan step will then be ignored the next time the
Load Plan run is restarted. This is useful, for example, when the error causing this Load Plan step to fail is
not possible to fix at the moment and you want to execute the rest of the Load Plan regardless of this Load
Plan step status.
Enhanced Variable Handling in Load Plans
Load Plan variables that are not used in a Load Plan can now be hidden to improve the readability of Load
Plans.
Feature Highlights: Development Lifecycle
Oracle Data Integrator has been enhanced to further simplify the management of the development
lifecycle. The following feature has been added in ODI 11.1.1.6:
Smart Export and Import
Exporting and importing Oracle Data Integrator objects between repositories is a common practice when
working with multiple environments such as Development, Quality Assurance and Production. The new
Smart Export and Import feature guides you through this task and provides advanced code management
capabilities.
Figure 2 - Smart Export and Import provides a lightweight and consistent export and import mechanism. Smart Export automatically exports an object with all its object dependencies. It is particularly useful when
you want to move a consistent lightweight set of objects from one repository to another because Oracle
Data Integrator manages all object dependencies automatically while creating a consistent sub-set of the
repository.
The Smart Export and Import feature is a lightweight and consistent export and import mechanism
providing several key features such as:
Automatic and customizable object matching rules between the objects to import and the objects already
present in the repository
A set of actions that can be applied to the object to import when a matching object has been found in the
repository
Proactive issue detection and resolution that suggests a default working solution for every broken link or
conflict detected during the Smart Import
Feature Highlights: Data Quality
Oracle Enterprise Data Quality extends the Data Quality features of Oracle Data Integrator to provide
advanced data governance capabilities. This release integrates Oracle Data Integrator and Oracle
Enterprise Data Quality together.
Oracle Enterprise Data Quality Integration
With the Enterprise Data Quality Open Tool it is now possible to invoke an Oracle Enterprise Data Quality
(Datanomic) Job in a Package. Developers can design a Data Quality process in Oracle Enterprise Data
Quality and then invoke it in a Package in ODI along with the ETL steps.
The Enterprise Data Quality Open Tool is installed using the standard procedure for Open Tools and can
be used in a Package or a Procedure, similarly to the tools provided out of the box in Oracle Data
Integrator.
Other Features In addition to the features listed above, the following features have been included in this patch.
Groovy Editor
This release introduces the Groovy editor. The Groovy editor provides a single environment for creating,
editing, and executing Groovy scripts within the ODI Studio context. It provides all standard features of a
code editor such as syntax highlighting and common code editor commands.
Support of Undo and Redo Operations
It is now possible to undo or redo changes in editors, dialogs, wizards and in the Property Inspector using
the following keyboard shortcuts: CTRL+Z and CTRL+Y.
Autocomplete for Text Fields and Lists
Certain text components and drop down lists in the ODI Studio now support the auto-complete feature,
making end users more productive.
Version Numbering for Knowledge Modules
The version numbering of Knowledge Modules improves the information provided to identify the used
environment:
» It is now possible to determine when a Knowledge Module (KM) has been modified and when it is
not the original Knowledge Module as released by Oracle.
»
» The KM modifications can be tracked by a version number.
»
» It is now possible to find out when a KM has been released with an external component such as a
jar file or a dll file (This is the case for example for the SAP and Hyperion KMs.)
»
» It is possible to indicate whether a given ODI version is compatible with a specific KM version.
1 | ENTER TITLE OF DOCUMENT HERE
Oracle Data Integrator 11.1.1.5.0 Release
Feature Highlight: SOA and Application Integration
Oracle Data Integrator is the only ETL tool that can run entirely within a Service Oriented Architecture. ODI is the
most deeply integrated ETL product for running data integration as a service, accessing web services and writing to
web services as a target. ODI 11.1.1.5 has been further enhanced for facilitating application integration and more
advanced interaction with a SOA stack.
Complex File Technology
Complex file formats (multiple record files) can now be integrated using the new Complex File technology. This
technology leverages a new built-in driver that converts transparently complex file formats into a relational structure
using a Native Schema (nXSD) description file.
Figure 3 - Complex files described with the Native Format Builder can be integrated in Oracle Data Integrator With ODI 11.1.1.5 the nXSD formats may be created manually, or with a graphical tool called the Native Format
Builder Wizard (included with BPEL Process Manager). The Native Format Builder Wizard guides you through the
creation of a native schema file from the following formats shown in Figure 3. A sample data file format for the
selected type must already exist; you cannot create a native schema without one. You can also select to modify an
2 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
existing native schema previously created with this wizard, except for those generated from a Document Type
Definition (DTD) or COBOL copybook.
Web Services Enhancements Web service support in Oracle Data Integrator has been enhanced with the following capabilities:
Web Service Container Based Authentication
When external authentication and container based authentication with Oracle Platform Security Services (OPSS)
are configured, authentication can be passed to the ODI Run-Time Web Services using HTTP basic authentication,
WS-Security headers, SAML tokens and so forth.
Asynchronous Web Services and Callbacks
Oracle Data Integrator batch jobs are long running sessions in a web service context. Long running transactions are
handled using asynchronous web service calls and callbacks. Starting with this release, a scenario or session can
be started using the Run-Time Web Services on a one-way operation. When the session completes, the result of the
execution can trigger an operation on a callback address. This pattern can be used for handling any WSAddressing
compliant long running sessions, started for example with Oracle BPEL.
Full SOAP Support for Outbound Calls
The OdiInvokeWebService tool now supports full-formed SOAP messages as an input, including the SOAP header
and body. Access to the SOAP header improves the ease of interaction with web service security, authentication,
and other application-specific information.
Features Highlights: BI and Data Warehousing
The following features have been added in this patch level for customers using Oracle Data Integrator in Business
Intelligence and Data Warehousing Projects.
Load Plans
Oracle Data Integrator is often used for populating very large data warehouses. In these use cases, it is common to
have thousands of tables being populated using hundreds of scenarios. The execution of these scenarios has to be
organized in such a way that the data throughput from the sources to the target is the most efficient within the batch
window. Load Plans help the user organizing the execution of scenarios in a hierarchy of sequential and parallel
steps for these types of use cases.
Load Plans are new objects introduced in this release to organize at a high level the execution of packages and
scenarios. Load Plans provide features for parallel, sequential, and conditional scenario execution. They also
support advanced restartability and exception handling capabilities.
3 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 4 - Load Plans support parallel, sequential and conditional execution Load Plans can be created and modified in production environments. They can be started, stopped, and restarted
from a command line, from Oracle Data Integrator Studio, Oracle Data Integrator Console or a Web Service
interface. They can also be scheduled using the run-time agent's built-in scheduler or an external scheduler.
Load plans are not substitutes for packages or scenarios. They are used to organize at a higher level the execution
of packages and scenarios. Unlike packages, Load Plans provide native support for parallelism, restartability and
exception handling. Load plans are moved to production as is, whereas packages are moved in the form of
scenarios. Load Plans can be created in Production environments.
The Load Plan instances and Load Plan runs are similar to Sessions. The difference is that when a session is
restarted, the existing session is overwritten by the new execution. The new Load Plan Run does not overwrite the
existing Load Plan Run, it is added after the previous Load Plan Runs for this Load Plan Instance. The Load Plan
Instance cannot be modified at run-time.
A scenario is an atomic runtime component that is given to the production. When in production, the production
administrator can orchestrate the various scenarios using Load Plans in order to optimize the time taken to execute
the ETL jobs in the given timeframe and ease the administration of all the ETL jobs.
For example, the production administrator will:
» Rely on Load Plans to orchestrate the execution of the various Scenarios produced by the ETL developers as part of their own projects and will organize the production globally instead of a per-project approach.
4 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
» Design the load plans in order to handle the parallelism of the scenarios in order to best use the hardware resources
» Design the load plans in order to handle serial steps to avoid overloading the system with too many scenarios at a given time
» Modify the load plans to take into account the real execution times and tune the load on the systems
» Rely on the load plans to restart only what is needed to be restarted in case of failure. This relies on the fact that the scenarios should be designed in order to be fully restartable (see above)
Oracle Business Intelligence Enterprise Edition Data Lineage
Oracle Business Intelligence Enterprise Edition (OBIEE) users need to know the origin of the data displayed on their
reports. When this data is loaded from source systems into the data warehouse using Oracle Data Integrator, it is
possible to use the Oracle Data Integrator Lineage for Oracle Business Intelligence feature to consolidate ODI
metadata with OBIEE and expose this metadata in a report-to-source data lineage dashboard in OBIEE.
Figure 5 - OBIEE Users can click to view the Lineage of the data from their dashboards
An important capability that Oracle has provided is the ability to view detailed metadata lineage information from
directly within the BI Answers dashboards.
5 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
13
Figure 6 - OBIEE Users can see the Lineage details directly in their BI dashboards
Other Features
In addition to the features listed above, the following features have been included in this patch level.
Commands on Connect/Disconnect
It is now possible to define data server commands that will be automatically executed when connections to this data
server are created or closed by ODI components or by a session. Such a command can be used for altering and
pre-configure sessions (for example, for internationalization of messages or forcing implicit conversions), or in multi-
tenant infrastructure where commands must be placed at connection time to access information of a given tenant.
Built-in Technology Additions and Updates
The following technologies used in Oracle Data Integrator have been added and updated to benefit from their new
features:
» Embedded HSQL engine is upgraded to version 2.0. This embedded engine is used for the Memory Engine as well as the XML and LDAP Drivers' built-in storage
» Jython BSF engine is updated to version 2.1
» JAX-WS/JRF is now used as a standard stack for web service calls and processing. Axis is no longer used
Groovy Technology
Groovy is added to the list of scripting engines supported by Oracle Data Integrator for use in knowledge modules
and procedures.
Ordered and Non-Ordered Join Syntax
Technologies can now support both the ordered or non-ordered (database-specific) syntax for joins. The Oracle DB
technology was modified to support both join syntaxes.
6 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
New Method for Setting Task Names
A new setTaskName method is available to update at run-time the name of a task. This API can be used in
knowledge modules and procedures to modify at run time the task name displayed in the log, making it more
readable for production users.
Shared Library for WLS Agent
A new template called Oracle Data Integrator - Agent Libraries includes libraries shared by all the deployed
JEE agent in a domain, and must be deployed before the Oracle Data Integrator - Agent default template
or a generated template.
This shared library facilitates the deployment, patching and maintenance of ODI domains, as it is deployed
once and shared by both the default and user-generated templates.
Performance Optimization
The following optimizations have been made in the design-time and run-time components to improve their
performance:
» Long text storage modified to use CLOBs.
» Agent-Repository network communications reduced at run-time
» Agent JDBC to JDBC loading mechanism reviewed and optimized
Oracle Data Integrator 11.1.1.3.0 Release
Architecture for Enterprise-Scale Deployment
Oracle Data Integrator 11g dramatically changes the vision of data integration in the information system. In fact, it
makes data integration a key service in the information system, providing the backbone infrastructure for Enterprise
Information Management.
The new Oracle Data Integrator platform integrates in the broader Fusion Middleware platform and becomes a key
component of this stack. Oracle Data Integrator 11g provides its run-time components as Java EE applications,
enhanced to fully leverage the capabilities of the Oracle WebLogic Application Server. Oracle Data Integrator
components include exclusive features for Enterprise-Scale Deployments, high availability, scalability, and hardened
security.
High Availability and Scalability
High-Availability (HA) and Scalability is fully supported via clustered deployments for Java EE components. The ODI
components deployed in WebLogic Server benefit from the capabilities of the WLS cluster for scalability, including
JDBC Connection Pooling and Load Balancing.
In addition to the cluster-inherited HA capabilities, the run-time agent also supports a Connection Retry mechanism
to transparently recover sessions running in repositories that are stored in HA-capable database engines such as
Oracle RAC.
7 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 8 - High Availability Configuration for Oracle Data Integrator JEE Components
Corporate Security Integration
Security in Oracle Data Integrator can be hardened and unified with the corporate security stores.
The source and target data server passwords can be externalized in the enterprise credential store. The
externalized passwords are no longer retained in the Oracle Data Integrator Repository.
In addition, the Oracle Data Integrator user can be mapped to users from the corporate identity store (LDAP, Oracle
Internet Directory or Active Directory). With this feature, authentication is delegated to the corporate authentication
service and Single Sign-On (SSO) can be enabled.
With these external password storage and external authentication features, passwords as well as user identities can
be retained in a centralized location meeting any corporate standards for security.
Enterprise-Scale E-LT Architecture
The E-LT architecture mandates the use of lightweight run-agents able to run in strategic locations (for example on a
Data Warehouse server) for optimal performances of the integration processes. Oracle Data Integrator 11g
continues the support of Standalone Agents. Standalone agents support the same level of service as the Java EE
Agent, including corporate security integration and connection retry, yet require a simple Java Machine. Standalone
agents also support the ODI built-in load-balancing feature and can
8 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
be protected using Oracle Process Manager and Notification Server (OPMN).
With this new architecture, every run-time component of Oracle Data Integrator guarantees high scalability, 24/7
availability, and best-in-class security.
Simplified Deployment and Unified Administration
Enterprise-Scale deployments frequently end up being a dilemma between simplicity and efficiency. Indeed,
setting up components in highly available and highly secured configurations is a challenge that not every
administrator is ready to accept. In addition, monitoring and managing such configurations can become a
complex activity.
Simplified Deployment
Oracle Data Integrator simplifies the deployment and administration of the data integration services and
components. Java EE components deploy easily and quickly in an Oracle WebLogic Server using
preconfigured templates or templates that are generated based on the metadata defined in the topology. It
is also possible to create data server definitions for sources and targets in the topology and to deploy these
in few clicks as datasources in an Oracle WebLogic Server.
With these deployment capabilities, creating or extending a domain for data integration is a matter of
minutes.
Figure 9 - Wizard for creating an Agent Template for WLS Unified Administration and Monitoring
9 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
The Data Integration Architecture includes a number of components, including the Java EE and
Standalone Agents as well as the Oracle Data Integrator Console web application. To manage and monitor
these components, Oracle Data Integrator provides a new plug-in that integrates in Enterprise Manager
Fusion Middleware Control.
With this plug-in, the Fusion Middleware administrator can manage from a single screen his data
integration components along with his WebLogic servers, his deployed web applications, or his SOA
composite applications.
This component allows the administrator to monitor the status, metrics, and notifications of Oracle Data
Integrator components, including Repositories, Agents and Oracle Data Integrator Console instances.
Figure 10 - Oracle Data Integrator in Oracle Enterprise Manager
Better Control over Production
Having a large-scale deployment involves frequently hundreds of integration flows and requires extreme tools for
monitoring the production environment.
Oracle Data Integrator enhances the control capabilities available to the production operators by providing unique
features.
Oracle Data Integrator Console
The new Oracle Data Integrator Console replaces and unifies the former 10g Metadata Navigator and Lightweight
Designer components. This web interface for production operations has been rewritten using the ADF-Faces Ajax
Framework for a rich user experience.
10 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Using this console, production users can set up an environment, export and import the repositories, manage run-
time operations, monitor the sessions, diagnose the errors, browse design-time artifacts and generate lineage
reports.
This web interface integrates seamlessly with Oracle Fusion Middleware Control Console and allows Fusion
Middleware administrators to drill down into the details of any Oracle Data Integrator design-time and run-time
object.
Figure 11 - Oracle Data Integrator Console
Enhanced Session Control
Sessions can now be stopped in an immediate mode. This new mode aborts the current operation (for example, a
long-running SQL statement started on a database engine) instead of waiting for its completion before stopping the
session.
In addition to this mechanism, sessions that failed due to a repository or the run-time agent failure are automatically
identified as stale and moved from the Running state to the Error state for operator review.
Enhanced Error Management
Error messages raised by Oracle Data Integrator Components and Sessions have been enhanced to provide
administrators and production operators with precise information for troubleshooting and fixing the status of the
architecture and debugging the sessions.
Enhanced messages cover component lifecycle (agent startup, shutdown, schedule refresh, and so forth) as well as
session lifecycle (incorrect scenario version, load balancing issue, agent not available and so forth). They render
clearer errors starting at the lowest level of the session (steps and task). These error messages are enriched with
11 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
context and infrastructure information to help development or production users quickly find the source and solution
for an issue.
Errors raised by components or sessions are automatically notified to the operation administrator in Enterprise
Manager Fusion Middleware Control.
Design-Time Productivity
Oracle Data Integrator 11g introduces a new JDeveloper-based integrated development environment (IDE) called
the Oracle Data Integrator Studio. This client is entirely redesigned in this release to dramatically increase the
developer’s productivity and make advanced features more accessible.
Design-Time Experience
New IDE Based on JDeveloper IDE Framework
The new Oracle Data Integrator studio is used as a replacement for all Oracle Data Integrator 10g modules
(Designer, Topology, Operator, and Security Manager). All the features of these modules now appear as Navigators
within the Oracle Data Integrator Studio.
This new IDE provide state-of-the-art features for developer productivity, including redesigned and reorganized
editors in addition to enhanced windows, improved docking, and document navigation and management.
12 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 12 - In Oracle Data Integrator Studio, Editing the “Customer” Interface
Redesigned Interface Editor
The Interface Editor, used to create the integration interfaces, has been entirely redesigned to use the JDeveloper
diagramming framework, providing:
» Enhanced diagram look and feel, plus manual or automated diagram layout organization
» Better support of large diagrams via thumbnails, zoom in/out, and bird’s eye
» Small but useful features such as source column drag and drop in the target mapping and column
sort/selection in the mapping
» Reorganized and searchable Property Inspector
Auto-Fixing
When saving an interface or clicking the Error button in the Interface Editor toolbar, the list of all design errors is
displayed with meaningful messages and tips. Automated fixes are suggested and can be applied with a single click.
Quick-Edit
The new Interface Editor includes a new Quick-Edit tab to edit the interface diagram faster. The Quick-Edit Editor
displays the interface components in a tabular form, supports mass-updates, and intuitive and accessible keyboard
navigation.
13 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 13 - Quick Edit view for the “Customer” Interface
Small (yet Important) Things
In addition to these major changes, some small improvements have been made in ODI to make the developer’s life
easier. For example:
» Code Simulation – When running a session at design-time, it is possible to make a code simulation instead
of running a full execution. This simulation mode generates a session simulation report displaying the
generated code for review.
» Reverse-Engineering – When a model is created, the reverse-engineering context is automatically set to the
default context. In addition, when performing a selective reverse-engineering, the system tables are now
hidden from the display.
» Scenario Naming Convention – A user can define a pattern (using for example the object name, folder path,
or project name) for the naming convention that is automatically applied to new scenarios created by this
user.
» Long Object Names – Object names have been extended to support long database object names (128
characters) and repository object labels (400 characters).
Java API for Task Automation
Oracle Data Integrator 11g introduces a new Java API for creating and managing both design-time and run-time
operations.
Using this API, it is for example possible to perform the following operations:
» Create or modify the topology contents
» Set up projects and models
» Create and browse interfaces, procedures, and packages
» Start and manage sessions and monitor their status
This API allows developers to programmatically perform operations without using the user interfaces and
to create their own applications to access the repository content.
E-LT Features for Better Performance
New core features have been built in Oracle Data Integrator Studio around the E-LT architecture and declarative
design approach to support a more productive development process and better performances at run-time.
Datasets and Set-Based Operators
This major enhancement introduces the notion of datasets in interfaces. A dataset represents the data flow coming
from a group of joined and filtered source datastores. Each dataset includes the target mappings for this group of
sources. Several datasets can be merged into the interface target datastore using set-based operators such as
Union and Intersect.
Figure 15 illustrates the use of datasets to merge data from a source file and from an HR system. Each
of these datasets (From File, From HR) is a flow with its own set of source tables, joins, filters and mappings.
14 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 14 - One interface merges with a UNION operator flows from two diagrams These two flows appear in different diagram tabs in the Interface Editor. They are merged using the UNION operator
as shown in Figure 15.
Figure 15 - Two diagram tabs for the datasets in the Interface Editor This feature accelerates the interface design and reduces the number of interfaces needed to merge several data
flows into the same target datastore.
Partitioning
Oracle Data Integrator now supports partitioning features of the data servers. Partitions can be reverse-
engineered using RKMs or manually created in models. When designing an interface, it is possible to define
the partition to address on the sources and target datastores. Oracle Data Integrator code generation
handles the partition usage syntax for each technology that supports this feature.
Lookups
15 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
A wizard is available in the Interface Editor to design a lookup from a driving source datastore to a lookup datastore
or interface. These lookups appear as a compact graphical object in the Sources diagram of the interface. The user
can choose how the lookup is executed: as a Left Outer Join in the FROM clause or as an expression in the
SELECT clause (in-memory lookup with nested loop). This second syntax is frequently more efficient on small
lookup tables.
This feature simplifies the design and readability of interfaces using lookups, and allows optimized execution of
these lookups.
Figure 16 - Lookup Creation Wizard
Derived Select for Temporary Interfaces
When using a temporary interface as a source in another interface, it is possible to avoid persisting the
target datastore of the temporary interface by generating instead a Derived Select (sub-select) statement
corresponding to the execution of this temporary interface. Consequently, the temporary interface no
longer needs to be executed to load the temporary datastore. The code generated for the sub-select can be
optionally customized in an IKM.
16 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 17 - Use Temporary Interface as Derived Table option can be checked when an interface is used as a source This feature eliminates the need for complex packages to handle the execution order of temporary interfaces. It
dramatically simplifies the execution of cascades of temporary interfaces.
Support for Native Sequences
Oracle Data Integrator now provides support for a new type of sequences that directly maps to database-defined
sequences. Such a sequence is selected from a list retrieved from the database when the sequence is created.
Native Sequences are used as regular Oracle Data Integrator sequences, and the code generation automatically
handles technology-specific syntax for sequences.
This feature simplifies the use of native sequences in all expressions and enables cross-referencing on native
sequence usage.
Support for Natural Joins
Oracle Data Integrator now provides support for Natural joins, defined at technology level. A natural join does not
require any join expression to be specified, and is handled by the engine that processes it. This engine matches
automatically columns with the same name.
Automatic Temporary Index Management
When creating joins or filters on source tables, it is possible to have Oracle Data Integrator automatically generate
temporary indexes for optimizing the execution of these joins or filters. The user simply selects the type of index
from a list for the given technology. Knowledge modules automatically generate the code for creating and dropping
the temporary index as needed.
This feature accelerates join and filter execution and enables better performances for integration interfaces.
17 | ORACLE DATA INTEGRATOR 11G NEW FEATURES OVERVIEW
Figure 18 - Two temporary Indexes (Non-Unique and Unique) will be created for optimizing the
execution of the join between SRC_CUSTOMER and AGE_GROUP
Broaden Connectivity
Data Integration is about moving and transforming data from sources to targets in a heterogeneous world. In this
context, hot-pluggability, that is being able to access and integrate any source and any target in the most performing
way, is a constant challenge that Oracle Data Integrator addresses using the knowledge modules architecture.
The Oracle Data Integrator highly flexible knowledge module architecture allowed the release of significant
connectivity improvements in the previous version lifecycles. These improvements included:
» Application Adapters for Oracle E-Business Suite, PeopleSoft, Siebel, JDE EnterpriseOne, SAP ERP, and
SAP BW,
» Performance optimizations for Teradata
» Adapters for Oracle OLAP and Oracle Changed Data Capture.
» Adapters for Hyperion Planning, Financial Management, and Essbase
» Changed Data Capture using Oracle GoldenGate.
» Oracle Date Integrator 11g adds to this long list the following improvements:
» Adapter for sourcing from ADF-View Objects exposed in an Oracle Business Intelligence Enterprise Edition
Server
» Support for Oracle Database Multi-Table Insert
» Support for Teradata Multi-Statements
Of course, the Knowledge Modules have also been enhanced in various manners to support the new productivity
and performance enhancements such as datasets and temporary indexes.
Conclusion
With the Oracle Data Integrator 11G releases Oracle introduced several new enhancements such as a
redesigned ODI Studio user interface, WebLogic support for ODI agents’ deployments, Java API,
Oracle Business Intelligence Enterprise Edition Data Lineage, Load Plans, Web Services
Improvements, Smart Export and Import, Global Knowledge Modules or the integration with Oracle
Enterprise Data Quality.
The ODI 11G releases continue to improve Oracle’s strategic Data Integration platform while
preserving the key product differentiators: Declarative Design, Knowledge Modules, Hot-Pluggability,
and E-LT.
Oracle Corporation, World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065, USA
Worldwide Inquiries
Phone: +1.650.506.7000
Fax: +1.650.506.7200
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.0115 White Paper Title May 2015 Author: Oracle
C O N N E C T W I T H U S
blogs.oracle.com/oracle
facebook.com/oracle
twitter.com/oracle
oracle.com
Top Related