Data Migration

Project ScenarioChallenges:Tight deadlines5-15 EnvironmentsSqueezed time for migration activitiesNumerous legacy systems to integrate with100% quality requirement

AgendaIntroduction Migration to OracleMigration to Oracle using open interface / APIMigration to Oracle Oracle UtilitiesMigration to Oracle Third party toolsMigration from Oracle

Introduction: Migration scenariosData migration: moving data from one database to another.

Types of migration:Migrating in an upgrade processMigrating legacy data in a new implementationMigrating between CRP, Test and Prod instancesPropagating master reference data

Introduction: Types of DataSetup Data (Application Object Library (AOL) data)ProfilesFlexfield definitionsMaster DataItemsCustomersTransaction DataPurchase OrdersInvoices

Migration to OracleSupported:Entering data via the UIUsing Open Interfaces / APIsUsing Oracle data migration utilities

Not supported:Direct manipulation of the data in the database.

Migration to Oracle using open interface / APISample usage of open interfaces for conversion purpose:

3-step process:Load: Get the data from the flat file into the staging tableValidate: Assess the quality of the data by calculating and evaluating metrics.Transfer: Insert data into interface table / call APIs with values in staging table

Run the conversion programs in the E-Business Suite for security / accessability / traceability.

Migration to Oracle using open interface / APILoad program: generic shell script taking name of SQL*Loader control file as parameter.Validate program: 1 per interface.Transfer program: 1 per interface.

FNDLOADUsage: OA Instance OA InstanceSetup Master TransactionalExtract Import

Moves OA data between database and text file representations. Can be modified with any editorUse to synchronize databases during new installations or maintenance.Good for moving data between test and prod instances, e.g. in upgrade projects.Command-line based tool, requires technically skilled personell to executeNo cost, well documented.

Migration to Oracle Oracle Utilities

Migration to Oracle Oracle UtilitiesFNDLOAD

An example of downloading is:FNDLOAD apps/apps@devdb 0 Y DOWNLOAD testcfg.lct out.ldt FND_APPLICATION_TL APPSNAME=FND

An example of uploading is:FNDLOAD apps/apps@custdb 0 Y UPLOAD fndapp.lct fnd1234.ldt

Oracle Applications System Administrators Guide, Appendix C

iSetupUsage: OA Instance OA InstanceSetup Master TransactionalExtract Import

Load original configuration multiple timesLoad modified configurationLoad extractLoad subset of extractMigration to Oracle Oracle UtilitiesiSetup.oracle.comCentral instanceiSetup ConfiguratoriSetup MigratoriSetup Reporter

iSetup

Typical scenario:Configurator -> CRPMigrator -> SystemtestMigrator -> Production

Complete or partial collection of dataCopy data within the same instance (e.g. from one OU to another)

http://isetup.oracle.com, Oracle iSetup User GuideMigration to Oracle Oracle Utilities

ADIUsage: Anything OA InstanceSetup Master TransactionalExtract Import

ADI is a spreadsheet-based extension of Oracle Applications.Limited to GL and FA dataImport Text File feature to simplify work with Oracle Assets. (Assets and physical inventory).

Applications Desktop Integrator Users Guide

Migration to Oracle Oracle Utilities

DataLoad ClassicUsage: Anything OA InstanceSetup Master TransactionalExtract Import

Automated loading using copy & pasteIf there is a form for it, DataLoad can insert data into itFreeware, no supportQuick and easyNot suitable for large volume loads, out of sync problem.

http://www.dataload.net/

Migration to Oracle Third party tools

DataLoad ProfessionalUsage: Anything OA InstanceSetup Master TransactionalExtract Import

Automated secure loadingFast because load is processed on the middle tierInexpensiveRequires some technical skillsSuitable for medium volumesRerun load multiple times in test environment until 100% successful.

http://www.dataload.net/


Mercury WinRunnerUsage: Anything OA InstanceSetup Master TransactionalExtract Import

A testing tool, but has functionality that makes it suitable for automating data loadWork similarly to DataLoad Classic, but more reliable.Records keystrokes and creates playback script for you.ExpensiveIf you have it for testing, might as well use it for migration too.

http://www.mercury.com/us/products/quality-center/functional-testing/winrunner/Migration to Oracle Third party tools

Mercury Object MigratorUsage: OA Instance OA InstanceSetup Master TransactionalExtract Import

Utilizes FNDLOADRuns as a concurrent program within OAVersion control, reporting capabilities, rollback functionalityUseful if you have to synchronize lots of instances or require elaborate version control.

http://www.mercury.com/us/products/it-governance-center/change-management/extensions/object-migrator.html


Various adapters / connectorsUsage: Anything AnythingSetup Master TransactionalExtract Import

Insevo (formerly Taviz)LibradosiWay Software

Bi-directional connectivityAny application, any platform, any data structure, any message typeReduced risk

http://wwws.sun.com/software/connectors/


Migration from OracleMake your own customized outbound interfaces using Materialized views

Customizations must be non-invasiveComplete controlSeveral pitfalls:A change in any column in any table, even if it not selected in the MV definition, will lead to refresh of the MVCannot distinguish between inserts and updates of the MVTriggers work differently on MVs than regular tables

Migration from OracleBusiness Event System

Most significant events are predefinedDefine your own eventsElegant way to trigger workflow activities / custom logicNon-invasive

Questions?Trond SteensnaesConsultant, [email protected]: 0047 93011983

*******Open interfaces are tables in which you put your data to be imported and then you run an concurrent job to do the actual import into the core tables.APIs (Application Program Interfaces) are code objects provided for you that you can call from a wrapper with the data to be imported as parameters.

Do consider manual entry of the data as an option. Not necessary always elegant, but it might sometimes actually be the cheapest alternative.

All product user guides warn you not to use SQL to manipulate data. This might compromise the integrity of the relational data and your data become corrupt.**Set up a document with the file structure definition describing the data requirements to be met by the legacy data.Data cleansing and enrichment should be done prior to extract of legacy data.

****.lct file: configuration file (predefined by Oracle)..ldt file: data file.

Sample usage:New installation: Concurrent programs from your DEV environment.Maintenance: updated lookup values

Limited to migrating these datatypes: Concurrent program definitions Request groups Lookup types and lookup values Profile options and profile option values Flexfields setup data Attachments definitions Messages Security information

*Consists of:iSetup Configurator: Setup a new Oracle E-business Suite instanceiSetup Migrator: Migrate setup data from one Oracle E-Business Suite instance to another. iSetup Reporter: Generate reports on setup data

Migrator: Create selection set. A Selection Set is a collection of setup objects for which you want to extract data. Add filters as required. Create a snapshot. This extract takes a source instance and a selection set as input.

Key benefits:Reduced implementation riskLower implementation costReduced implementation timeAutomatic validation and verification of setup dependenciesReduced time to migrate setup information to multiple instancesAuditing capabilities to identify setup errors

**The Import Text File feature allows you to import data from legacy systems, thirdparty payables applications, barcode scanners, or any other flat data file sources, toyour worksheet. Using the Import Text File feature, you can map fields in yoursource file to fields in your worksheet. Once you have created the map using theImport Text File feature, you can save it and apply it to future conversions.

*****Formerly Kintana Object Migrator***There are 2 issues to keep in mind when designing/building/testing outbound interfaces:1. The trigger on materialized views issue.Triggers work somewhat differently on materialized views than they do on tables. This is caused by the way the RDBMS maintains these views.Oracle maintains a materialized view in this way: first it DELETES the row (if it exists already) and then INSERTs a new (the updated) row. So for updates, it actually deletes the exisiting row and inserts a new one. This is the reason why an AFTER UPDATE trigger will never fire.The second anomaly is that if you use refresh on commit, it will perform one DELETE/INSERT for each table referenced in the definition of the materialized view that has changed since the last commit in the session.So for a MV that joins three base tables, an INSERT of a new row in the MV results in the following sequence of events: INSERT DELETE INSERT- DELETE INSERT (if no commit has been performed until the three base tables were populated). An UPDATE of two of the three base tables will lead to the following sequence of events: DELETE-INSERT-DELETE-INSERT (if no commit was performed until both updates were executed). It is important that you find out if the interface shall export data upon creation only, or also after the information is updated.

2. When is the data eligible for export?If one joins three tables in the definition of the materialized view, the data will not appear in the view until the user has provided data in all tables, unless outer joins are used. When selecting values from several tables in the materialized view it is important to have an understanding of when data will be inserted into the materialized view.You should select from tables in either the materialized view or in the trigger:Select from the table in the materialized view if the row from that table is required before export of the information. (Or you can outer join to this table if the row is not required before export.)If changes to the row in the table shall trigger export it needs to be included in the materialized view, if not implement it as a lookup in the trigger.*The Business Event System is an application service that uses the Oracle Advanced Queuing (AQ) infrastructure to communicate business events between systems. The Business Event System consists of the Event Manager, which lets you register subscriptions to significant events, and event activities, which let you model business events within workflow processes. Business events represent integrations points. There are in excess of 500 predefined events in 11i.

When a local event occurs, the subscribing code is executed in the sametransaction as the code that raised the event. Subscription processing caninclude:executing custom code on the event information, sending event information to a workflow processsending event information to other queues or systems.

Oracle Workflow is a component of Oracle9i Integration, a comprehensive set of infrastructure and tools for integrating enterprise applications and enabling e-business marketplaces. Oracle Workflow leverages the Oracle technology stack, including the Oracle9i database and Oracle Advanced Queuing, to enable application integration at the business process level. Business event messages from Oracle Workflow can be placed on or received from Oracle9i Advanced Queues, providing support for Oracle Net as well as HTTP and HTTPS communication protocols. The Oracle9i Messaging Gateway feature enables integration between Oracle Workflow and other messaging solutions such as IBM MQSeries and TIBCO Rendezvous. Oracle Workflow is also incorporated as part of Oracle9i Integration in the Oracle E-Business Suite, an integrated suite of applications for the enterprise designed to transform your business into an e-business.

Data Migration

Documents

Transcript of Data Migration