UKOUG Tech 15 - Migration from Oracle Warehouse Builder to Oracle Data Integrator - Eurocontrol Case...

T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)

E : info@rittmanmead.comW : www.rittmanmead.com

Rittman Mead ConsultingEurocontrol - Migration from OWB to ODI

UKOUG Tech 2015 - BirminghamJérôme Françoisse

Jérôme Françoisse

•Consultant for Rittman Mead‣Oracle BI/DW Architect/Analyst/Developer

•ODI Trainer•Providing ODI support on OTN Forums•ODI 12c Beta Program Member•Blogger at http://www.rittmanmead.com/blog/•Email : jerome.francoisse@rittmanmead.com•Twitter : @JeromeFr

Rittman Mead

•Oracle Gold partner with offices in Europe, US (Atlanta)

•UKOUG Partner of the year in 4 categories in 2015

•Provide consulting, training, global managed services for customers around the world

•80+ consultants including 1 Oracle ACE Director and 2 Oracle ACEs

•All expert in Oracle BI, DW, EPM and Analytics tech

•Skills in broad range of supporting Oracle tools: OBIEE, OBIA, ODIEE, Essbase, Oracle OLAP, GoldenGate, Exadata, Endeca

•Blog : http://www.rittmanmead.com/blog/•Twitter : @rittmanmead

Agenda

•OWB & ODI•Migration Utility•Migration steps•Migration execution•Fixing•Opportunities

History of OWB & ODI

•2000 - First release of OWB•2006‣Oracle buys Sunopsis - ODI 10g‣OWB Standard and Enterprise Editions

•2008 - Roadmap to merge the products•2009 - Oracle Data Integrator Enterprise Edition•2010 - ODI 11g•2013‣ODI 12c‣OWB is not shipped with Oracle DB 12c ‣Last OWB release - 11.2.0.4

•2015 - End of standard support for OWB

Similarities

•Leverage database capabilities as data transformation engine•Objects are stored in the database•Flow-based development•Can use the ELT paradigm

Flow-based

ETL vs ELT

•Reduce network transfer•Reduce hardware costs•Set-based operations

•Leverage target platform capabilities•Referential Integrity Check

ODI Differences

•Heterogeneous (Oracle, IBM DB/2, SQL Server, Flat files, Webservices, Hadoop, Cloud Apps, …)

•No deployment, the code is generated at run-time•Jobs are executed by agents•Repository stored on any RDBMS (not only Oracle)•Separation between Physical and Logical implementation•Customisation and Reusability thanks to Knowledge Modules•Parallelism•Real-time loading

create table<%=odiRef.getTable("L", "INT_NAME", "A")%>(<%=odiRef.getColList("", "[COL_NAME]\t[DEST_WRI_DT] " + odiRef.getInfo("DEST_DDL_NULL"), ",\n\t", "", "INS or UPD")%>)

ODI Differences

•Heterogeneous (Oracle, IBM DB/2, SQL Server, Flat files, Webservices, Hadoop, ...)

•No deployment, the code is generated at run-time•Jobs are executed by agents•Repository stored on any RDBMS (not only Oracle)•Separation between Physical and Logical implementation•Customisation and Reusability thanks to Knowledge Modules•Parallelism•Real-time loading

Migration Utility

•Needs OWB 11.2.0.3+•Patch OWB 17830453or 21977765 (12.2.1)

•ODI 12.1.2 : Patch 17053768•ODI 12.1.3+ : No Patch•Based on ODI SDK•Command Line Interface•3 modes :‣Fast Check‣Dry Run‣Run

Migration Utility

•Supported‣Module / Location Model and Topology‣All data objects (except sequences) Datastores‣Mapping (Classic, Code-template and Pluggable) Mapping

•Unsupported‣Process flows‣PL/SQL objects (procs, packages, ...)‣Oracle Discoverer / OBIEE metadata‣Queues, Streams, CDC, User Defined Types‣Dimension and cube, cursors‣Data quality, data profiles, data auditors‣Configuration details (security, OMB scripting, ...)

Migration steps• Installing Oracle DB 12c• Installing ODI 12.1.2 and patch 17053768•Plug the homebrew scheduler in ODI•Upgrading a clone OWB repository to 11.2.0.4 and apply patch 17830453•Execute migration utility in Dry Run mode•Fix issues on OWB side• Installing ODI 12.1.3•Build Knowledge Modules for missing loading strategies.•Execute migration utility in Run mode•Clean up topology•Synchronise Test environment and Dev environment•For each mapping : Change knowledge modules, remove unnecessary objects, unit test

•Creation of Load Plans•Testing

OWB Architecture

DevelopmentRepository

Deployment

DDR_SRC

DDR_STG

SWH_SRC

SWH_STG

OWB_REPO

PE21 - Dev

PE5 - Prod

Sources and Targets

ODI Architecture

Repositories ExecutionContext

DDR_SRC

DDR_STG

PE21 - Dev

PE16 - Test

PE9ODI_DEV

Sources and Targets

MASTER

PE9ODI_TEST

PE9ODI_PROD

SWH_SRC

SWH_STG

PE5 - Prod

HF_TEST

MASTER

DDR_SRC

DDR_STG

DDR_SRC

DDR_STG

HF_PROD

Migration UtilityODI_MASTER_USER=DEV_ODI_REPOODI_MASTER_URL=jdbc:oracle:thin:@porape9.ops.cfmu.eurocontrol.be:1542/odi_devODI_MASTER_DRIVER=oracle.jdbc.OracleDriverODI_USERNAME=SUPERVISORODI_WORK_REPOSITORY_NAME=WORKREP_DEVOWB_WORKSPACE_OWNER=OWB_REPOSOWB_URL=porape9.ops.cfmu.eurocontrol.be:1542:owb2odiOWB_WORKSPACE_NAME=OWB_REPOS

MIGRATION_MODE=DRY_RUN

MIGRATION_OBJECTS=PROJECT.SWH.MODULE.DDR;PROJECT.SWH.MODULE.EUACA.MAPPING.UWB_LDIM_IMATCH_METHOD_010_INS;

MIGRATION_STRATEGY=CREATE

Migration UtilityODI_MASTER_USER=DEV_ODI_REPOODI_MASTER_URL=jdbc:oracle:thin:@porape9.ops.cfmu.eurocontrol.be:1542/odi_devODI_MASTER_DRIVER=oracle.jdbc.OracleDriverODI_USERNAME=SUPERVISORODI_WORK_REPOSITORY_NAME=WORKREP_DEVOWB_WORKSPACE_OWNER=OWB_REPOSOWB_URL=porape9.ops.cfmu.eurocontrol.be:1542:owb2odiOWB_WORKSPACE_NAME=OWB_REPOS

MIGRATION_MODE=DRY_RUN

MIGRATION_OBJECTS=PROJECT.SWH.MODULE.DDR;PROJECT.SWH.MODULE.EUACA.MAPPING.UWB_LDIM_IMATCH_METHOD_010_INS;

MIGRATION_STRATEGY=CREATE

oracle@prisme111: cd /opt/oracle/11.2.0.4/owb/bin/unixoracle@prisme111: ./migration.sh ODI_MASTER_REP_PASSWD ODI_SUPERVISOR_PASSWD OWB_WORKSPACE_PASSWD migration.config

Migration Utility: Report

Migration Utility: Result

Mappings not migrated

•Errors in input connectors - Fixed with ODI 12.1.3•Procedure inside mappings - Wrapping into functions•No “Delete” loading strategy - Development of new IKM Knowledge Module

•No “Update” only mode with the Merge syntax - Edit of existing IKM

PSM - Scheduler

PSM - Monitoring

PSM - Parameter

PSM - Scheduler

Cleanup Topology

Migration is a good opportunity for...

•Re-assess the business rules•Define Data Stewardship•Refactoring•Delete unused batch•Complete missing documentation

Execution OWB jobs from ODI

•Register in Topology•ODI Tool : OdiStartOwbJob•Monitor in Operator

UKOUG Tech 15 - Migration from Oracle Warehouse Builder to Oracle Data Integrator - Eurocontrol Case...

Data & Analytics

Transcript of UKOUG Tech 15 - Migration from Oracle Warehouse Builder to Oracle Data Integrator - Eurocontrol Case...

Subledger Accounting Implementation Options Presented to the UKOUG Oracle Financials SIG by Nic Atkin.

Oracle Support Update UKOUG AIM SIG 27 th March 2012 Antony Higginson.

Oracle Support Update UKOUG DB SIG Phil Davies. 2 Agenda Oracle Support New Releases – Oracle Database – Enterprise Manager Patch Set Releases Life Time.

Audit Considerations for your 11i implementation Richard Byrom Oracle Applications Consultant UKOUG November 2004.

Oracle Support Update UKOUG AIM SIG 27 th March 2012

Embracing Database Diversity: The New Oracle / MySQL DBA - UKOUG

UKOUG APPS 14: Optimizing Performance for Oracle EPM Systems

Strategies to Move Forward with Oracle Forms - UKOUG Tech13

UKOUG Tech 15 - Oracle Platinum Partner · UKOUG Tech 15 Common Mistakes When Implementing Oracle Enterprise Manager 12c Philip Brown @pbedba

Example R usage for oracle DBA UKOUG 2013

Running Oracle EBS in the cloud (UKOUG APPS16 edition)

Stack Traces and Flame Graphs for Oracle Troubleshooting Luca Canali, CERN UKOUG, Birmingham, December 6 th, 2015.

Evolution of Performance Management: Oracle 12c adaptive optimizations - ukoug 2015

UKOUG Applications Conference & Exhibition … · 16 UKOUG Applications Conference & Exhibition 16 UKOUG JD Edwards Conference & Exhibition 16 UKOUG Technology Conference & Exhibition

REPORTING from ORACLE EBS 1 BeLife.co.uk: UKOUG May 2009 REPORTING from ORACLE EBS.

Oracle clusterware 11gR2 - WordPress.com · Oracle clusterware 11gR2 UKOUG TEBS 2010 Frits Hoogland ... security –Technical)security,)performance Blog: h6p ... –Oracle clusterware

Ukoug Slides

Ukoug tech13 bi_publisher_and_bi_em12c_dhananjay_papde_v1

Compressing Very Large Data Sets in Oracle Luca Canali, CERN UKOUG Conference Birmingham, December 1 st, 2009.

UKOUG Tech15 - Deploying Oracle 12c Cloud Control in Maximum Availability Architecture