The Time is Now! Migrating from OWB to ODI 12c

www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics

The Time is Now!

2

Stewart Bryson

medium.com/@stewartbryson @stewartbryson linkedin.com/in/stewartbryson

Migrating from Oracle Warehouse Builder to Oracle Data Integrator 12c

http://www.redpillanalytics.com

mailto:[email protected]?subject=

http://twitter.com/RedPillA

© 2014 RED PILL Analytics

Who is Red Pill Analytics?


Red Pill Analytics: Leadership

4

Stewart Bryson !

Oracle ACE Director !

Previously From: Rittman Mead

Informix Software !

Data Warehousing & Business Intelligence

since 1996

Kevin McGinley !

Oracle ACE Director !

Previously From: Accenture

BI Consulting Group !

Data Warehousing & Business Intelligence

since 1997


mailto:[email protected]



About Red Pill Analytics

Other Services

!

• Expert Retainer • Support • Training

5

!

!

•DevOps for BI and DI •Hosted or On-premise •Used in Capacity Analytics

Capacity Analytics

!

•Development-as-a-service •Agile methodology • Faster/cheaper than traditional consulting • Subscription billing





Oracle Technology Network White Paper

6





Oracle Technology Network White Paper

7

http://bit.ly/owb-odi

Google Search: otn odi bryson





Oh Captain My Captain


Goodbye Old Friend


Migrations are Hard


Add Value


Have a Mission Statement


Mission Statement

13

Any task undertaken to migrate content from OWB to ODI will add immediate value to our BI stakeholders.





ODI 12c: Overview

14

Flow-Based Declarative Design with Component-Based

Development

New Mappings replace interfaces and allow multi-step

design patterns

Reusable Mappings and Datasets for Declarative Design

similar to 11g





ODI 12c: Overview

15

Components Provide Multi-Step Process





ODI 12c: Overview

15

Components Provide Multi-Step Process

Data Lineage built into the Mapping Editor





ODI 12c: Overview

16

Warehouse Builder-like Components including

Pivot/Unpivot, Join, Expression, Set, etc.





ODI 12c: Overview

17

Component-Style KM’s encapsulate modular, reusable

pieces of logic specific to particular Components





ODI 12c: Overview

18





Installing ODI 12c: Enterprise Installation

19

Enterprise Installation includes Fusion Middleware

and supports Standalone Colocated Agents

Handles configuration of the Master and Work repositories and other FMW tables (Work

Tables)

JEE Agents can utilize Weblogic Coherence for high

availability






20

Warning: FMW Needs to be installed

for JEE Agents






20

Warning: FMW Needs to be installed

for JEE Agents

Ignore This!The Enterprise

Installation installs WLS and FMW

behind the scenes






21

Features Specifically Selected

WLS and FMW are installed as an internal

feature






22

We get all this cool stuff!






23





Repository Creation Utility

24

Recent Oracle Database versions, including 11gR2 and 12c, as well as DB2 and SQL

Server

Master Repository contains Topology information, Work

Repository contains development information

Creates Fusion Middleware tables, including Fusion Work

Tables





Agents

25





Agents

26

ODI Agents execute load processes, including compiled

Load Plans and Scenarios

Somewhat comparable to the OWB Control Center (much, much better)

Deployed as JEE Agents or Standalone Agents





Domain Configuration

27

Use built-in FMW and WLS functionality to “harden” our

ODI Agents

Our JEE-deployed ODI Agent is comparable to an ETL Server

Utilizing Coherence gives us performance increases in

certain circumstances, fault tolerance, etc.






28


Templates make it easy to mix and match deployed functionality to

our domain






29

Service Table Schema (STB) stores relevant

information from our install to

feed the templates






30

In the end, we have a WLS

Domain complete with Admin Server and

Managed Server






31





OWB Runtime Integration

32

Execute OWB processes as standard ODI processes, including Mappings and

Process Flows

Full execution from OWB Control Center captured in standard ODI audit tables

Orchestrate our loads wherever we want: ODI Load Plans or OWB Process Flows





OWB Runtime Integration Topology

33

Data Server configured with

OWB Workspace Owner






34

JDBC Configuration to

the OWB Workspace






35

Configure the Workspace Owner

as the Physical Schema






36

Setting a Logical Schema to

abstract the Physical Schema and Data Server






37





OWB Runtime Integration Execution

38

Use the ODI Package Editor and the new

OdiStartOwbJob tool

We can execute Mappings or

Process Flows

OdiStartOwbJob "-WORKSPACE=OWBREP" "-LOCATION=OWF_MGR" "-OBJECT_NAME=SBATCH/MAIN_LOAD" "-OBJECT_TYPE=PROCESSFLOW"






39

ODI Scenario for Process Flow






39


Main Process Flow execution



39



Sub-Process Flow execution




39


Mapping Execution



Sub-Process Flow execution Mapping Execution



40

Full Auditing Results



41


Migration Utility

42

Supports 11.2.0.4 of Warehouse Builder with Patch

17830453

Apply Patch 17053768 to Data Integrator if not already

included in release

Command-line utility to perform the OWB to ODI

Migration


Migration Utility: Run Modes

FAST_CHECK Performs a read-only check of the OWB repository and reports back the items than can and cannot be migrated.

DRY_RUN Performs a migration to ODI using the ODI 12c SDK, but does not perform a commit at the end of the process.

RUN (Default) Executes the migration and commits migrated objects to the target ODI 12c repository.

43


Migration Utility: Configuration

44

ODI Master Repository Connection Information



44


ODI Work Repository Information



44



OWB Workspace Information



44



Migration Options

OWB Workspace Information



45



45

Configuration FileODI and OWB passwords



46

Migration Results


Migrated Mappings

Migrated Models


47



48


Oracle Warehouse Builder (Our Old Friend)

50

Wildly Successful •Flow-based design •High-performing code (EL-T) •Ships with Database 10g and 11g •PL/SQL Code Generation •Freemium model

Deprecated •Certified (not shipped) with Database 12c • In maintenance mode Limited Integration Capabilities


Oracle Data Integrator 12c

52

Best of both worlds! •Now with flow-based design •High-performing code (EL-T) •Knowledge Module architecture •Enterprise Manager integration •Fusion Middleware integration

Is there value in changing? • “Our organization has a major investment in OWB” • “Our mappings and process flows just work”


If Only There Was a Way to…

53


If Only There Was a Way to…

54

Maintain investment in OWB •Continue to execute legacy code •Only port our code when it presents value to do so

Blaze new trails with ODI 12c •Fantastic new features are dying to be used •The Knowledge Module (KM) architecture pays huge dividends

Deploy a coherent strategy •Shouldn’t feel like two tools


It’s not an easy choice


Phase 1: Run OWB Process Flows in ODI

56


Phase 1: Run OWB Process Flows in ODI

57


Phase 2: Orchestrate our OWB Mappings in ODI

58



59



60


Phase 3: Re-implement in ODI When Necessary

61



62



63

The Time is Now! Migrating from OWB to ODI 12c

Technology

Transcript of The Time is Now! Migrating from OWB to ODI 12c