GoldenGate Whitepaper Oracle 8i 9i to 10g 11g Database Migration

Eliminating DatabasE DowntimE whEn UpgraDing or migrating from oraclE 8i or 9i to oraclE 10g or 11g

Publication Date: february 2009

Author: alok pareek, Vice president of technology, goldengate software, inc.

abstract:

Eliminating database downtime poses a significant challenge for IT organizations that need to upgrade or migrate mission-critical database environments running Oracle database software versions 8i or 9i to Oracle 10g or 11g. That is particularly true for critical applications that must provide continuous or near-continuous operations to users who increasingly expect uninterrupted availability of online service. Any outage of an application or website, even if that outage is scheduled or “planned,” has an impact on the revenue and reputation of the business.

The purpose of this paper is to present a solution that tackles the specific challenge of upgrading or migrating an Oracle 8i or 9i database to Oracle software version 10g or 11g – without taking any database downtime. Key technology components used in this solution are GoldenGate Software’s platform for transactional data management and Oracle’s Cross Platform Transportable Tablespace feature.

The details within this paper will showcase key technical strengths of the solution, including how to achieve a rolling upgrade/migration for this scenario; using a clone database to offload instantiation and conversions; keeping transactions in synch across the databases; managing “partial” or phased migrations or upgrades; conducting data verification post-upgrade/migration; and implementing an easy and reliable failover strategy.

whitE papEr

© 2009 GoldenGate Software, Inc.

2 © 2009 GoldenGate Software, Inc.

whitE papEr

tablE of contEnts

Section Page Number1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

3. Concepts and Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

GoldenGate’s Technology for Transactional Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Database Upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Database Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Standby Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Transportable Tablespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Cross Platform Transportable Tablespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Clone Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

RMAN (Recovery Manager) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

4. A Zero-Downtime Approach Using GoldenGate and Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Upgrade vs. Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

5. Upgrade and Migration Challenges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Business Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Technical Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

6. Available Solutions: 8i or 9i ➔ 10g or 11g Upgrade or Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Database Upgrade Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Export/Import. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

SQL Plus As Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Transportable Tablespaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

SQL*Loader. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Oracle Data Guard, Oracle Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

The GoldenGate Zero-Downtime Upgrades Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

7. Overview of GoldenGate’s Technology Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

On-Disk Queues aka “Trail files”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Delivery (Apply) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

GoldenGate Veridata® . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

8. Overview of The Oracle Cross Platform Transportable Tablespace Feature . . . . . . . . . . . . . . . . . . . . . . . . 13

9. Detailed Steps Using a Platform Migration Case Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Migration without Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Migration with Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Failback Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Partial Migration Using Bi-Directional Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

10. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1© 2009 GoldenGate Software, Inc.

1. introDUction

Today, mission-critical applications must provide continuous operations to users who increasingly expect uninterrupted availability of online services. Any outage of an application or website, even if that outage is scheduled, has an impact on the revenue and reputation of the business. In fact, many e-commerce businesses report that a half-day outage would incur millions of dollars in financial losses. For databases that host the data store for these mission-critical applications, availability requirements have become stringent.

From that high availability standpoint, there are essential events that still require application downtime: modifying hardware or database software, upgrading applications, applying software patches, migrating to different computing architectures, etc. Since such events are not conceived via system or data failures, they are aptly classified as planned outages. Empirical data from non-trivial applications deployed in very large database (VLDB) environments demonstrates that minimizing downtime to handle planned outages is a complex, time consuming, error prone, and costly process.

For all of those reasons, many organizations choose to postpone those essential IT events as long as possible. If such an event is a move to Oracle 10g or 11g, there is a better solution available today.

The purpose of this paper is to present a solution that takes the specific problem of either upgrading or migrating an Oracle 8i or 9i database to Oracle software version 10g or 11g without taking any database downtime (excluding application switchover time).

Eliminating database downtime for upgrades or migrations in mission-critical Oracle database environments running Oracle database software versions 8i or 9i poses a significant challenge for IT organizations. Oracle Corp. does not provide a database rolling upgrade solution when upgrading the database from Oracle version 8i or 9i to Oracle version 10g or 11g.1 As such, the solution described and illustrated in this paper is the only proven solution that limits database downtime to mere seconds during the migration or upgrade from 8i or 9i to Oracle 10.x or higher. In the case of applications running on 8i, there’s a strong sense of urgency and concern to upgrade to Oracle version 10, as the 8i product has reached end-of-life support from Oracle.

The key technology components used in this solution to achieve zero-database-downtime migrations are GoldenGate Software, Inc.’s software platform for transactional data management and Oracle’s Cross Platform Transportable Tablespace feature. Oracle’s Cross Platform Transportable Tablespace feature facilitates high-speed data movement of Oracle data from one machine/platform to another; it is a recommended though not mandatory method during instantiation of a target database that eventually participates in a rolling upgrade.


whitE papEr

2. goals

This paper is created primarily for advanced database users.

The main goal of this paper is to increase awareness of a solution for minimizing database downtime during a database upgrade or migration under the circumstances outlined in the introduction. As such, resource issues such as disk space, software-licensing costs, etc., are not discussed – not because such issues are unimportant, but because the need to minimize downtime or completely avoid it outweighs other considerations in demanding high availability environments.

Furthermore, this document is not intended to repeat the upgrade steps, typical warnings, and preparatory details associated with an Oracle database upgrade. There are many articles and notes that exist on the Oracle MetaLink customer support portal, in various books, and on the web, which are helpful towards understanding the operational procedures and complexities associated with upgrades and migrations.

As one of the lead developers of the Cross Platform Transportable Tablespace feature in 10g at Oracle, I struggled with minimizing downtime when migrating very large 8i or 9i databases from one platform to another. My aim therefore is to present a novel solution that allows for a database upgrade/migration from 8i or 9i to 10g/11g without taking database downtime or having to make tablespaces read-only, or take any locks. The solution presented leverages the benefits of real-time synchronization, clone databases, rolling upgrades, transportable and cross-platform transportable tablespaces, and failback. Every effort is made to avoid overhead at the primary database to ensure application availability (activities such as instantiation, file conversion for cross platform transport, etc., are offloaded to a clone).

A secondary goal is also to explain how to minimize the “wall clock time” required to perform the entire upgrade. Experienced users will realize that wall clock time can be on the order of days or weeks, yet the downtime to the application can still be near zero. The key here, as will be explained in the following pages, is to offload work processing to additional database copies.

3. concEpts anD tErminologY

Throughout this document, we refer to the production database as the SOURCE and the secondary copy as the TARGET database.

The following concepts and terminology deserve review to fully comprehend the solution proposed in this white paper:

3.1 GoldenGate’s Technology for Transactional Data ManagementGoldenGate’s software platform provides guaranteed change data capture, routing, transformation, and delivery, across heterogeneous business systems in real time. It captures transactions non-intrusively from a source database by reading transaction logs, transforms when needed, propagates, and applies those transactions with guaranteed integrity to another database (target) in real time. GoldenGate’s processes run continuously (and can even be bi-directional) and support high-volume, rapidly changing environments, moving thousands of transactions per second with very low impact. The target database is a transactional replica at a logical level; it can be functionally leveraged for multiple applications including a rolling database upgrade.


Aside from the differentiating ability of GoldenGate to handle heterogeneous database environments, readers familiar with Oracle data movement technologies may ruminate on the perceived resemblance between Oracle Streams and GoldenGate. This collation, however, is purely conceptual. The underlying architectures are significantly different. For example, Streams in Oracle 9i has problems scaling in high-throughput environments. In addition, it is limited in its support for data types (e.g. LONG) and for certain kernel functionality (e.g. log_parallelism). These issues are not present in GoldenGate’s platform.

GoldenGate’s software is designed for managing high transaction volumes in real time without impacting throughput or commit time latencies on the source database. The transaction capture component does not rely on the shared memory of the database instance to stage transactions or associated metadata, thereby avoiding any resource contention with the ongoing application workload.

Not relevant to this specific topic, but noteworthy for this audience nevertheless, is that GoldenGate can also be used when the target and source database software come from different vendors, and GoldenGate can apply changed data in real-time to JMS message queues or topics as well.

3.2 Database UpgradeA database upgrade advances the Oracle software release number from one version to another. There are two primary ways to perform an upgrade:

3.2.1 In place

An in-place upgrade renders the database inaccessible for business applications while the database software is being upgraded. This procedure entails running an upgrade script (e.g. for u0902000.sql), recompiling invalid PL/SQL etc., and experiencing downtime that is usually not acceptable in most mission-critical environments.

The intent of this paper is to not to address in-place upgrades.

3.2.2 Rolling Upgrade

The term rolling upgrade refers to upgrading different databases or different instances of the same database (in a Real Application Clusters environment) one at a time, without stopping the database. Oracle doesn’t allow a rolling database upgrade in Oracle 9i.

A rolling upgrade (database) consists of the following high-level steps:

1. The application points to the production database running software version VOLD

2. A secondary logical database copy is constructed running software version VOLD

3. The secondary database copy is upgraded to the next database version VNEW

4. The secondary and primary databases are synchronized

5. The primary database is shut down

6. The application is pointed to the secondary database

7. The primary is kept in the same version VOLD for failback reasons


whitE papEr

3.3 Database MigrationA migration allows for the underlying operating system or hardware platform to be changed. In Oracle, on-disk file formats are not homogeneous across platforms. Under 10g compatibility, the on-disk structures for platforms that appear in v$TRANSPORTABLE_PLATFORM are identical, but the endian format could differ.

Moving an Oracle database across different operating systems is a common requirement in many computing environments.

3.4 Standby DatabaseA standby database is a copy of the main production database that is maintained for high availability or disaster tolerance purposes. The standby database can be physical or logical.

In the physical standby copy, the database is kept on recovery mode (i.e. the redo logs of the production database are applied to a mounted copy of the production database – the hardware/OS limitations here are obvious since redo across Oracle platforms is not compatible).

A logical standby is a copy of the production database that contains the same objects, but doesn’t physically match the structure of the production database copy. For example, a standalone table EMP in the physical database could be represented as obj# 3143 whereas its logical equivalent at the standby site could be represented as obj# 5931. It is maintained by instantiating a logical equivalent of the production database and replaying the SQL that modifies the production database at the standby site.

3.5 Transportable TablespaceTransportable Tablespace was a feature introduced in Oracle 8i that allows for non-system tablespaces to be moved from one database to another by physically grafting the tablespaces’ datafiles into the target database’s control file and importing object metadata into the target database’s dictionary.

It has three main phases:

1. Exporting the metadata (dictionary data for the objects)

2. Transferring the datafiles from one database to another

3. Importing the metadata and datafiles

3.6 Cross Platform Transportable TablespaceCross Platform Transportable Tablespace is an extension of the Transportable Tablespace feature and allows tablespaces to be transported across database platforms. This feature can only be used after the database compatibility has been advanced to 10.0.0.0 or higher.

This whitepaper makes use of this feature along with GoldenGate’s software to migrate an 8i or 9i database to a 10g or 11g database.

3.7 Clone DatabaseA clone database is a database constructed using a restored backup of an existing database, recovered to a point in time, and opened.

3.8 RMAN (Recovery Manager)RMAN (Recovery Manager) is an Oracle tool that manages the process of making backups and managing the process of restoring and recovering from them. It is also used for conversion of endian systems during a cross platform CONVERT.


4. a ZEro-DowntimE approach Using golDEngatE anD oraclE

Since an in-place upgrade requires application downtime during the upgrade process, it is not feasible in critical HA environments. Therefore we consider only rolling upgrades henceforth.

Using GoldenGate in conjunction with Oracle technologies, a rolling upgrade or a rolling migration can be performed without taking any application downtime—other than the very minimal time required for application switchover, typically less than one minute and in most cases, only seconds.

4.1 Upgrade vs. MigrationIn general, a migration is more complex than an upgrade given that a migration includes both a software upgrade and a migration to a different platform, and often there are on-disk incompatibilities across the datafiles/logfiles/controlfiles of the source and target database platforms.

As such, this paper will use a real-life case study of migrating an existing Oracle database on the Sun (Solaris™ OE (32-bit)) platform running on 9i to Linux (Linux IA (64-bit)) on 10g.

Readers who are interested only in an upgrade should skip the cross platform transport steps covered in Section 8.

5. UpgraDE anD migration challEngEs

Such upgrade and migration projects pose challenges for mitigating business risk and navigating technology issues. Some of the more common and critical challenges include:

5.1 Business Challenges

5.1.1 Impact on Revenue

Downtime equates to lost revenue. Many businesses (e.g. retail, travel, banking, etc.) no longer have the extended downtime windows for planning upgrades/migrations. Many database environments, for scalability or cost saving reasons, would like to take advantage of recent trends in technology such as deploying low cost storage (new hardware), or other advances in clusterware and software. The following table (Figure 1) illustrates the average downtime costs associated with various businesses.

Industry Segment/Application Average Cost per Hour of Downtime

Financial Markets $6.45 million

Credit Card Sales $2.65 million

Pay-Per-View Television $150,000

Home Shopping $113,000

Airline Reservations $89,000

Teleticket Sales $69,000

Shipping $28,000Source: International Data Corporation and Sun Microsystems (http://www.sun.com/datacenter/continuity/availability/)

Figure 1: Average Cost of Downtime for Business Applications


whitE papEr

5.1.2 Customer Expectations/Loyalty

In the Internet era, businesses must continue to support online processing beyond conventional business hours. For the user, the cost of switching over to the competition is non existent or minimal. For example, let’s look at what happened when eBay took a 22-hour outage several years ago on a Thursday. In a combined study by Nielsen Media Research and NetRatings Inc., which measures website usage, it was found that during eBay’s downtime, the average time spent per person at Yahoo’s auction site the next day increased from 7 minutes to 18 minutes and the number of people using Yahoo’s site skyrocketed from 62,000 on Thursday to 135,000 on Saturday. Financial losses to eBay were estimated at $3 million to $5 million.2

5.1.3 Interdependencies, Business Reputation

E-commerce has resulted in digital business partnerships where non-availability of critical data at one site may also result in loss of business services at multiple sites (via data access from either web services, direct querying, or pre-configured batch loading). Extended downtime is becoming difficult to plan for since significant coordination is required with business partners. This results in postponement of upgrades/migrations, which has associated indirect costs (e.g. running the existing software with complex workarounds for bugs, not being able to take advantage of new software functionality in the later releases, etc.).

5.2 Technical ChallengesThe following presents some of the serious issues that need to be dealt with when considering a zero-database-downtime upgrade/migration. Once these issues are well understood, one gains clarity into why a unified solution using GoldenGate and Oracle’s Cross Platform Transportable Tablespace provides the best way to perform the upgrade or migration for HA environments.

5.2.1 Instantiation

The first problem that needs to be addressed when performing a rolling upgrade is choosing the method by which the target database is instantiated. In other words, how do you create a first version of the target database?

The complexity of the problem is magnified when the target database is on a different platform, since physical backups cannot simply be restored at the target and converted into a clone.

To handle multi-terabyte sized databases, procedures such as export/import are tedious and error prone. Failures during the instantiation or global data consistency are not adequately addressed with methods that are time consuming, unless some form of locking is used. Extracting data from the production database over a period of time results in a performance impact on the source database that is unacceptable. There’s also interference with the normal mechanics of the database cache manager affecting buffer allocation and flushing algorithms, resource management algorithms, etc., since the typical OLTP database is not optimized to handle long running queries, table scans, etc.

A good instantiation solution therefore must properly address:

½ Global data consistency

½ Efficiency (speed)

½ Performance (i.e. low impact on the production database)

½ Manageability


5.2.2 Incremental Data Movement

The next post-instantiation challenges are to determine:

i) The point that demarcates the last committed transaction picked up by the instantiation process with the subsequent transactions submitted against the production database; and

ii) The method by which the subsequent transactions get propagated to the target database.

Depending on the instantiation method, one or both problems may be pertinent. The following example depicts this challenge (See Figure 2).

Figure 2: High-Level Instantiation of a New Target Database for Upgrades/Migrations

Example:

1. Tables MASTER and SLAVE have a parent-child relationship via a foreign key constraint.

2. The instantiation method (e.g. export, SQL copy, etc.) picks up a read consistent version of table MASTER as of SCN 23.12360.

3. The instantiation method picks up a read consistent version of table SLAVE as of SCN 23.12777.

4. Transaction TM inserts new row R1 into table MASTER at SCN 24.12890.

There are several problems with this method. First, how are transactions handled that are active as of SCN 23.12360 but commit after SCN 23.98745? Second, the MASTER and SLAVE tables are from different points in time, and as a result are inconsistent with each other from a referential integrity point of view. And third, the data submitted subsequent to SCN 23.12360 for the MASTER table (in this case row R1) still needs to be moved.

Thus, a good solution that addresses incremental data movement must:

i) Ensure global consistency

ii) Identify a clean instantiation termination point

iii) Allow transactions from that point onward to be propagated to the target database

iv) Handle any failures during incremental data propagation (e.g. loss of network, media failure, etc.)

Source

Bi-Directional TDM Configuration

Target

RouteLAN/WAN/Web/IP

Delivery Target Trail

Capture Source Trail

Capture

CaptureSource Trail

DeliveryTarget Trail

Compare & Verify

Apply

Compare & Verify

exp_norows.dmpexp_xtts.dmp

IGNORE = Y

Capture

Capture

Apply

Apply

Compare & Verify

Failover


IGNORE = Y

T1

Production database:generating redo atSCN 23.12345

Begin Instantiation:database clock atSCN 23.12355

End Instantiation:database clock atSCN 23.98745

Dprod9i / Sun Solaris

Dprod backup Dpitr 9i ➔ 10g

Sun Solaris

Dtarget 10g Linux

Time

T3 T100

2 3

4

5

8

6 7

1211

13

10



Sun Solaris

Dtarget 10g Linux

2 3

4

5

8

6 7

1211

14

16

13

10


whitE papEr

5.2.3 Performance Impact

Another major concern during the upgrade is to understand what kind of impact is experienced by the application running on the production database. The upgrade steps should not degrade performance on the production database such that the application service levels (SLAs) are compromised.

5.2.4 Staging Areas

Adequate disk resources need to be configured and managed when addressing rolling upgrades or migrations. Especially when a no-downtime migration is required – proper consideration must be given to handling the disk requirements for the clone database.

5.2.5 Change Management

Changes other than those made by DML need to be addressed during the upgrade/migration process. Examples of these are datafile additions, PL/SQL package creations, etc. In general if the overall solution is fast (i.e. O[hours] ) such changes can be restricted since they are infrequent operations.

5.2.6 Special Handling of Legacy Data or Certain Datatypes

Any datatypes that are not supported by the solution must be identified ahead of time and dealt with separately. These are usually objects that Export/Import doesn’t work with, such as certain application tables in the SYS schema and typically those that are fairly small in size. These simply can be recreated at the target.

Additional handling is required to support certain datatypes when upgrading/migrating from Oracle 8i environments.

5.2.7 Verification

Testing that the source and target database are synchronized needs to be conducted post upgrade/migration — any out-of-synch conditions at the new source and/or failback database could pose risks to the business and should be identified and acted upon. Conducting this verification should be possible even as ongoing transactions are being processed at the source database.

A good solution must therefore address fast and efficient comparison of data across the two databases, i.e. the database should not acquire any locks during the verification period. In-flight transactions must be accounted for during the comparison.

5.2.8 Failback

Once the upgrade/migration takes place, a failback strategy must be in place to avoid any downtime and data loss in case the new environment is not stable. Therefore, all transactions that have been processed at the target (the new production database) need to be propagated to the original production database in a consistent and manageable manner for a successful failback.


6. aVailablE solUtions: 8i or 9i ➔ 10g or 11g UpgraDE or migration

Scenario:Unload/

LoadExport/ Import

Transportable Tablespaces

Backup/Roll Forward

Standby DatabasesData Guard Streams

GoldenGate

8i/9i ➔ 10g/11g Yes Yes Yes No No No1 Yes

8i/9i ➔ 10g/11gcross platform

Yes Yes No No No No1 Yes

9i ➔ 10g/11g RAC/ASM

Yes Yes Yes No No No1 Yes

8i ➔ 9i Yes Yes Yes No No No Yes

8i ➔ 9icross platform

Yes Yes No No Yes2 No3 Yes

Non-Oracle ➔ 10g/11g

Yes No No No No No3 Yes

Downtime Weeks/Days Hours/Minutes Minutes/Seconds

Extended Downtime Real-Time

1. From Oracle 9.2 to Oracle 10g+ it may work for very limited data types and low-volume databases.2. Requires transient logical standby and does not allow the use of standby systems for failover during the upgrade process.3. May work for limited data types and low-volume databases.

Figure 3: An Overview of Downtime Categories for Different Solutions, Given Various Oracle Upgrade and Migration Scenarios

There are a number of possible solutions that may be evaluated to help with a project to move onto 10g or 11g. The figure above (Figure 3) depicts whether these solutions support different possible migration or upgrade scenarios, along with associated time categories—from extended downtime to real-time switchover.

Those and several other common methods and solutions are summarized below with their respective advantages and disadvantages, illustrating where significant shortfalls exist for environments seeking minimal downtime and high availability:

6.1 Database Upgrade AssistantThis method only works for an in-place upgrade. This doesn’t solve the rolling upgrade scenario, nor does it address platform migration.

6.2 Export/ImportA full export can be done from an 8i or 9i database, followed by an import into an existing vanilla 10g database. For Oracle 10g to 11g upgrades you can use the Data Pump Export utility and import the data into an existing vanilla 11g database using Data Pump Import. Oracle’s Export utility extracts data from the database and writes it to a proprietary file (called an export dump file). Import processes the data and recreates all the objects (tables, indexes, packages, etc.). Thus the time required for an export/import upgrade is high. With Oracle 10g Data Pump you can parallelize the process more easily but the time required will be still significant.


whitE papEr

An export in FULL mode does have advantages in that it moves all database objects; it also supports both upgrades and migrations. However, in a high availability environment, there are three significant disadvantages to using Export/Import:

1. Volume-Time Relationship: The time required to extract the data is dependent on the size of the data. Extracting and reinserting high volumes of data can take days or even weeks for large databases.

2. Ongoing Transactions: Ongoing changes are not captured by Export, nor is there a way to demarcate the boundaries across SCNS that allow the incremental data to be re-exported. This effectively implies application downtime during the export/import process.

3. Recoverability: Failures during export and/or import are not easy to recover from.

6.3 SQLPlus As CopyUsing this method, all schemas/tables can be captured from one database and recreated on another database over a database link. An example of schema copy is as follows:

COPY FROM SCOTT/TIGER@LOCALDB TO SCOTT/TIGER@REMOTEDB

This method doesn’t scale nor can it be viewed as a serious way to move high volumes of data. It has a high database impact, is unmanageable, and doesn’t support incremental data movement.

6.4 Transportable Tablespaces Transportable tablespaces have been used effectively to minimize planned outages, since the time to transport avoids extracting data and instead relies on integrating datafiles directly into the target database. It’s not a complete solution, because it doesn’t address incremental data movement and also imposes restrictions on the production application by rendering the user data to read-only workloads. Also, the method cannot be used for migrations.

The high level flow of the upgrade process using transportable tablespaces is as follows:

1. Create a target database in 10g or 11g using install utilities. Also install any components and applications as necessary. Pre-create all users, roles, Java packages, views, and additional objects not picked up by transport.

2. Extract any unsupported objects using export or recreate object commands.

3. Make all non SYSTEM tablespaces read only.

4. Unplug the non SYSTEM tablespaces using the export utility – this extracts only the dictionary metadata associated with the non SYSTEM tablespaces.

5. Transport the datafiles in the non SYSTEM tablespaces to the target database.

6. Plug the datafiles into the target database using the import utility in transportable mode.

7. Run manual verification tests to ensure that the data is synchronized with the primary.

This method can achieve the upgrade at disk speeds of file transfer; but the drawback is that the application is inaccessible – so the downtime could be extended – and as previously noted, does not support migrations.


6.5 SQL*LoaderThis method involves scanning all the tables and writing the data into flat files, and using the SQL loader utility to reload the data into the target database. It is tedious, slow, and doesn’t address all database object movement. It suffers from the same disadvantages as Export/Import and SQL Copy

6.6 Oracle Data Guard, Oracle StreamsThe above two solutions do not support a rolling upgrade from 8i or 9i to 10g or 11g. For 10g to 11g upgrades you can use a physical standby configuration but you have to use transient logical standby during the upgrade process. In addition, during the upgrade process the standby system will not be available for failover. Please note that there are some limitations with using logical standby particularly with data types. (See References, 3 and 4).

6.7 The GoldenGate Zero-Downtime Upgrades SolutionUsing its real-time, heterogeneous data movement technology GoldenGate offers Zero-Downtime Migration and Upgrade solutions that address a minimal database downtime from 8i or 9i to 10g or 11g.

For example, the upgrade to Oracle 10g using GoldenGate consists of the following high-level steps:

1. Perform Initial Setup of a standby database running 8i or 9i software using an existing database backup.

2. Upgrade the standby database to 10g.

3. Synchronize the standby database with the production database.

4. Test in active/live mode.

5. Switchover the application to the standby database.

6. Upgrade the primary database to 10g after comprehensive application testing at standby.

For a migration scenario to Oracle 10g, the high-level steps of GoldenGate’s Zero-Downtime Upgrades solution are as follows:

1. Perform Initial Setup of a clone database running 8i or 9i software using an existing database backup on the same platform.

2. Upgrade the clone database to 10g, advance compatibility to 10.0.0.0.

3. Create a vanilla 10g database on the new platform with 10.0.0.0 compatibility.

4. Move the data from the clone database to the new target database using Oracle’s Cross Platform Transportable Tablespace feature.

5. Synchronize the new target database with the original 8i or 9i production database.

6. Test in active/live mode.

7. Switchover the application to the standby database.

8. Upgrade the primary database to 10g after comprehensive application testing at standby.

From here, Sections 7-8 describe the technologies that will be used to conduct the zero-downtime upgrade or migration. Section 9 discusses the steps in greater detail. Since a migration is a more complicated procedure than an upgrade, the detailed steps use a case study of migrating a 9i database on one platform to a 10g database on another platform.

Please note that for upgrading or migrating to Oracle 11g, database versions prior to Oracle 9.2.0.4 require the step to upgrade to Oracle 10g first before upgrading to 11g. For upgrading from 10g to 11g you can refer to Oracle’s documented approach.3


whitE papEr

7. oVErViEw of golDEngatE’s tEchnologY componEnts

There are a number of key technology components that comprise GoldenGate’s software platform and are relevant to the subject at hand

Figure 4: High-Level Architecture Diagram of Goldengate’s Software Products, Running Bi-Directionally and Verifying Data Between Two Databases

7.1 CaptureGoldenGate® Capture is a process that runs on the source database system. It is multithreaded in an Oracle RAC environment. The process mines transactions from the Oracle redo log and propagates transactions over the network to an on-disk queue. Only committed transactions are written to the queues.

7.1.1 Initialization

GoldenGate Capture can be used either to initialize the target database directly, or to start propagating transactions against an existing target database from a given point.

7.1.2 Change Capture

Only table DML operations are captured from the source database, i.e. updates to indexes, system tables, etc. are not captured.

7.2 On-Disk Queues or “Trail Files”GoldenGate® Trail Files can be conceptualized as a persistent ordered set of committed transactions generated by the GoldenGate Capture process. GoldenGate Trail Files describe DML operations (inserts, updates, and deletes) along with transactional context as captured from the source database.

Source


Target

RouteLAN/WAN/Web/IP



Capture

CaptureSource Trail


Compare & Verify

Apply

Compare & Verify


IGNORE = Y

Capture

Capture

Apply

Apply

Compare & Verify

Failover


IGNORE = Y

T1






Sun Solaris

Dtarget 10g Linux

Time

T3 T100

2 3

4

5

8

6 7

1211

13

10



Sun Solaris

Dtarget 10g Linux

2 3

4

5

8

6 7

1211

14

16

13

10


7.3 Delivery (Apply)The GoldenGate® Delivery process runs on the target system. It reads the captured data from the Trail Files and applies the captured transactions at the target using dynamic SQL. To maintain synchronization between the source and target databases, GoldenGate applies data changes to the target tables using native database calls, statement caches, and local database access. To ensure data and referential integrity, GoldenGate applies data changes in the transaction commit order that occurred at the source database.

7.4 GoldenGate Veridata®

GoldenGate Veridata is a stand-alone, high-speed comparison solution that identifies and reports data discrepancies between two databases without interrupting ongoing business processes. It allows data discrepancies to be isolated for testing and troubleshooting. GoldenGate Veridata is ideal for conducting data validation after the rolling upgrade, once the source and target are fully operational and running different versions of Oracle. It can also help to determine if a failback is needed, in case of any risky data anomalies.

8. oVErViEw of thE oraclE cross platform transportablE tablEspacE fEatUrE

The Cross Platform Transportable Tablespace procedure, to transport all non SYSTEM tablespaces, is described below since it will be used as part of the rolling upgrade/migration process:

1. Ensure compatibility of source and target database is at 10.0.0.0 or higher.

2. Make all non SYSTEM tablespaces read only.

3. Unplug the non SYSTEM tablespaces using the export utility – this extracts only the dictionary metadata associated with the non SYSTEM tablespaces.

4. Convert the datafiles associated with the non SYSTEM tablespaces using the RMAN CONVERT functionality. If the cross platform transport is across the same endian system, then conversion may not be required.

5. Transport the converted datafiles in the non SYSTEM tablespaces to the target database.

6. Plug the datafiles into the target database using the import utility in transportable mode.


whitE papEr

9. DEtailED stEps Using a platform migration casE EXamplE

We use a case study of moving an Oracle 9i database on Sun Solaris to Oracle 10g on Linux and discuss the detailed implementation steps. These steps can be simplified and adopted to accomplish a rolling upgrade.

Two scenarios are described next: A migration without the addition of a failback solution, and the same migration with the addition of a failback.

9.1 Migration without Failback

Figure 5: Cross-Platform Migration (without Failback)

1. Address change management by restricting creation of newer packages, tablespaces, etc. during the migration process. (Not depicted)

2. Start GoldenGate Capture process (captures consistent scn = Qscn) at the production database Dprod.

3. Do a point-in-time recovery of an existing backup of Dprod until Qscn in a separate staging area. Call this database Dpitr.

4. Upgrade Dpitr to 10g on Solaris. Advance compatibility to 10.0 or higher.

5. Set up a vanilla 10g database on Linux. Call this database Dtarget.

6. Take a full export without rows at Dpitr. Call the generated export exp_norows.dmp. This will pick up any objects that are not picked up as part of the transportable tablespace export.

7. Unplug the user tablespaces from Dpitr using the Oracle Cross Platform Transportable Tablespace feature using source side endian conversion. (Note the conversion would not be required if the endian systems were the same.) This is the step that avoids any performance degradation and does not require any quiescing at Dprod. This step will create a small export dump file. Call this exp_xtts.dmp.

Source


Target

RouteLAN/WAN/Web/IP



Capture

CaptureSource Trail


Compare & Verify

Apply

Compare & Verify


IGNORE = Y

Capture

Capture

Apply

Apply

Compare & Verify

Failover


IGNORE = Y

T1






Sun Solaris

Dtarget 10g Linux

Time

T3 T100

2 3

4

5

8

6 7

1211

13

10



Sun Solaris

Dtarget 10g Linux

2 3

4

5

8

6 7

1211

14

16

13

10


8. Plug the set of tablespaces from Dpitr into Dtarget using the Cross Platform Transportable Tablespace feature. Use the exp_xtts.dmp file created from Step 7. Note that the plugged in tablespaces are in read-only mode.

9. Make the set of user tablespaces in Dtarget Read Write. (Not depicted)

10. Do a NOROWS import with IGNORE=Y option at Dtarget using the exp_norows.dmp dump file.

11. Start GoldenGate Apply process at Dtarget and synchronize up to the changes generated since Qscn.

12. If any data types are not supported by Transportable Tablespace or GoldenGate, then do a special export/import of these objects from Dprod to Dtarget.

13. Use GoldenGate Veridata to verify that the data at Dprod and Dtarget is synchronized.

14. Switchover the application from Dprod to Dtarget. (Not depicted)

The above procedure offloads any quiescing, conversion work to a clone database, and takes advantage of GoldenGate’s incremental real-time data capture and delivery to eliminate the downtime to zero (excluding the application switchover time).

As an alternative to using the Cross Platform Transportable Tablespace feature in step 6, you can use a full export with data and import into a vanilla database, in which case steps 7, 8, and 9 do not apply. And in step 10 you can import the data as well as all database objects.

For upgrading from Oracle 10g to 11g you can use Data Pump Export in parallel to improve performance.

9.2 Migration with Failback

Figure 6: Cross-Platform Migration (with Failback)

Source


Target

RouteLAN/WAN/Web/IP



Capture

CaptureSource Trail


Compare & Verify

Apply

Compare & Verify


IGNORE = Y

Capture

Capture

Apply

Apply

Compare & Verify

Failover


IGNORE = Y

T1






Sun Solaris

Dtarget 10g Linux

Time

T3 T100

2 3

4

5

8

6 7

1211

13

10



Sun Solaris

Dtarget 10g Linux

2 3

4

5

8

6 7

1211

14

16

13

10


whitE papEr

1. Address change management by restricting creation of newer packages, tablespaces, etc. during the migration process. (Not depicted)

2. Start GoldenGate Capture process (captures consistent scn = Qscn) at the production database Dprod.

3. Do a point-in-time recovery of an existing backup of Dprod until Qscn in a separate staging area. Call this database Dpitr.

4. Upgrade Dpitr to 10g on Solaris. Advance compatibility to 10.0 or higher.

5. Set up a vanilla 10g database on Linux. Call this database Dtarget.

6. Take a full export without rows at Dpitr. Call the generated export exp_norows.dmp. This will pick up any objects that are not picked up as part of the transportable tablespace export.

7. Unplug the user tablespaces from Dpitr using the Oracle Cross Platform Transportable Tablespace feature using source side endian conversion. (Note the conversion would not be required if the endian systems were the same.) This is the step that avoids any performance degradation and does not require any quiescing at Dprod. This step will create a small export dump file. Call this exp_xtts.dmp.

8. Plug the set of tablespaces from Dpitr into Dtarget using the Oracle Cross Platform Transportable Tablespace feature. Use the exp_xtts.dmp file created from Step 7. Note that the plugged in tablespaces are in read-only mode.

9. Make the set of user tablespaces in Dtarget Read Write.

10. Do a NOROWS import with IGNORE=Y option at Dtarget using the exp_norows.dmp dump file.

11. Start the GoldenGate Delivery process at Dtarget and synchronize up to the changes generated since Qscn.

12. If any data types are not supported by Transportable Tablespace or GoldenGate then do a special export/import of these objects from Dprod to Dtarget.

13. After GoldenGate eliminates the lag between Dprod and Dtarget use GoldenGate Veridata to verify that the data at Dprod and Dtarget is synchronized.

14. Start GoldenGate Capture process at Dtarget.

15. Switchover the application from Dprod to Dtarget. (Not depicted)

16. Start GoldenGate Apply on Dprod.

9.3 Failback StepsDuring the real-time bi-directional data movement GoldenGate allows two databases to support transaction processing. This makes failback a trivial process, if the new database is not stable:

1. Stop the application at Dtarget (the new primary) running 10g or 11g

2. Once GoldenGate has applied all transactions from Dtarget to Dprod, switch application to Dprod

3. Declare Dprod as the new primary

9.4 Partial Migration Using Bi-Directional SynchronizationIn certain environments, it may be possible to migrate a limited number of applications over to the target database on 10g or 11g while running the rest of the applications against the 8i or 9i source database. This can be done based on logical application partitioning or geographical partitioning.


For example, if the application supports 100 bank branches, only one branch might be switched over to the new database until the new 10g (or 11g) environment is deemed stable after appropriate testing and validation. GoldenGate can be run bi-directionally to synchronize the data across the source and the target systems, thus supporting multi-master database environments.

10. sUmmarY

To summarize, the key technical strengths of the solution are as follows:

½ A rolling upgrade/migration – using two databases

½ No instantiation using primary database – by offloading to a clone database

½ Conversions – offloaded to a clone staging database

½ Synchronize transactions across databases

½ Verify data replication and transaction integrity – using GoldenGate Veridata

½ Easy failover strategy – using a bi-directional GoldenGate configuration

Using GoldenGate’s software platform and Oracle’s Cross Platform Transportable Tablespace feature, a rolling upgrade or even a migration can be done with zero database downtime and only very minimal application switchover downtime. This combined solution enables companies to start utilizing new database features provided in Oracle 10g and 11g releases without impacting business operations.

aboUt thE aUthor

Alok Pareek is the Vice President of Technology for GoldenGate Software, previously working as its Software Architect for High Availability solutions. Prior to joining GoldenGate, he had over 10 years of server development experience at Oracle in the Recovery/High Availability area. His significant (patent-filed) contributions at Oracle include development of the Cross Platform Transportable Tablespace feature (10g), Multi threaded redo generation (9i), Multiple block size cache support (9i), and Whole database transport (10.2). He was responsible for the redo generation component of the database from 8i to 10.2. He led the technical team responsible for high-speed data movement across platforms as part of Oracle’s cost-cutting measures. He has presented at major industry events, numerous regional Oracle user groups, and various user conferences on the topics of recovery and high availability. Alok holds a graduate degree in Computer Science from Stanford University, where his research area was Database Systems.

rEfErEncEs

1. 10.10 Upgrade with Logical Standby Database (http://download.oracle.com/docs/cd/B19306_01/server.102/b14238/toc.htm)

Using a logical standby database enables you to accomplish upgrades for database software and patch sets with almost no downtime. Note: This capability is not available for upgrading from Oracle9i to Oracle Database 10g.

2. Dembeck, Chet. “Yahoo Cashes in on Ebay’s Outage,” E-Commerce Times, June 18, 1999 (http://www.ecommercetimes.com/perl/story/545.html)

3. Oracle 11g upgrade documentation (http://download.oracle.com/docs/cd/B28359_01/server.111/b28300/toc.htm)

4. Oracle MetaLink note about 10g to 11g upgrade with the transient logical standby (https://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=300479.1)

• Lowell, David E.; Saito, Yasushi; Samberg, Eileen J. Devirtualizable Virtual Machines Enabling General, Single-Node, Online Maintenance, Hewlett Packard Laboratories

• GoldenGate 8.0 Operations Guide for Windows and UNIX


whitE papEr

contact information

GoldenGate Software, Inc.

301 Howard Street

San Francisco, CA 94105 USA

Tel: +1 415-777-0200

Email: [email protected]

For a list of worldwide offices and for more company information, please visit:

www.goldengate.com

aboUt golDEngatE softwarE

GoldenGate Software Inc. is a leading provider of high availability and real-time data integration solutions for improving the availability, accessibility, and performance of critical data across heterogeneous enterprise IT environments. More than 500 customers worldwide, including Visa, Bank of America, US Bank, UBS, Sabre Holdings, DIRECTV, Comcast, MGM Mirage, Chase Paymentech, AMD, Mayo Foundation, Retail Decisions, and Overstock.com, rely on GoldenGate solutions. The company broadens its global market reach through strategic relationships with leading technology vendors including ACI Worldwide, Amdocs, Business Objects, Cerner, Fujitsu, GE Healthcare, HP, IBM, Ingres, Microsoft, Oracle, and Teradata.

GoldenGate Whitepaper Oracle 8i 9i to 10g 11g Database Migration

Technology

Transcript of GoldenGate Whitepaper Oracle 8i 9i to 10g 11g Database Migration