Logical Stdby and RAC - Zero Downtime Upgrade NO Good OCM

Using Logical Standby (SQL Apply) for near zero downtime upgrade

of two node Oracle RAC database from 10.2.0.5 to 11.2.0.3

In the article you will have a look at an example of using a logical standby

database, also known as SQL Apply, to perform a near zero downtime upgrade of

a two node Oracle RAC database from version 10.2.0.5 to version 11.2.0.3. The

two node cluster configuration is described here in detail. For the testing

both primary and standby databases share the same Oracle GI cluster. Earlier

I have upgraded the cluster to 11.2.0.3 and installed Oracle RDBMS 11.2.0.3

binaries on each node. In the article a physical standby database will be

build and converted to a logical database that will be used for the upgrade.

Each node has a separate non-shared $OH for the GI and RDBMS versions

installed. The table below summarizes some environment details used in the

article.

Node1 Node2

Linux

hostname

raclinux1 raclinux2

10.2.0.5 $OH /u01/app/oracle/product/10.2.0/db_1

11.2.0.3 $OH /u01/app/oracle/product/11.2.0/db_3

Production

SID

RAC10G1 RAC10G2

Logical

Standby SID

RACSTB1 RACSTB2

RAC10G is a 10.2.0.5 database and a logical standby database RACSTB will be

built and used for upgrade to 11.2.0.3. Cluster provides a shared pool of

storage that consists of DATA and DATADG disk groups. DATADG hosts RAC10G

files and DATA hosts RACSBT files. The upgrade process goes through the

following conceptual stages illustrated in the table below.

Stage Primary site Logical Standby

Site

Production

Database

1 RAC10G is ONLINE

running 10.2

RACSTB is LS 10.2

and SQL Apply is

running

RAC10G

2 RAC10G is ONLINE

running 10.2

RACSTB is LS 10.2

and SQL Apply is

stopped

RAC10G

3 RAC10G is ONLINE

running 10.2

RACSTB is LS is

being upgraded to

11.2.0.3 and SQL

apply is stopped

RAC10G

4 RAC10G is ONLINE

running 10.2

RACSTB is LS

11.2.0.3 and SQL

apply is started

to sync with

primary

RAC10G
http://gjilevski.wordpress.com/2011/09/29/virtualization-using-oracle-vm-virtualbox-for-building-two-node-oracle-rac-11gr2-11-2-0-2-cluster-where-oracle-11-2-0-2-rdbms-coexists-with-10gr2-10-2-0-5-rdbms-based-on-linux-oel-5-4-2/

5 (Switchover) THIS

IS THE ONLY

DOWNTIME (Depend on

the switchover

time)

RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

deferred)

RAC10G is LS

running 10.2 ( SQL

Apply is stopped)

RACSTB

6 RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

deferred)

RAC10G is LS that

is being upgraded

to 11.2.0.3 ( SQL

Apply is stopped)

RACSTB

7. RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

enabled)

RAC10G is LS

running 11.2.0.3

and SQL apply is

started to sync

with primary

RACSTB

8 (Switchover) RAC10G is ONLINE

running 11.2.0.3

RACSTB is LS

11.2.0.3 and SQL

Apply is running

RAC10G

9. Adjust the

COMATIBLE parameter

for both databases

RAC10G is ONLINE

running 11.2.0.3

RACSTB is LS

11.2.0.3 and SQL

Apply is running

RAC10G

LS Logical Standby

Both RAC10G and RACSTB are two node RAC databases. Only is certain cases

CLUSTER_DATABASE parameter is set to FALSE. The article aim at documenting

steps involved in a near zero downtime upgrade using logical standby and can

be used for later releases, i.e. upgrade from 10.2.0.X, 11.1.0.X or 11.2.0.X

to 11.2.0.X+1 as Oracle has not changed the interface for creating and

managing standby databases from 10.2 to 11.2 significantly. There are some

data type restrictions that are not subject to coverage here but can be

examined in detail from Oracle documentation listed in the reference section.

1. Build logical standby database (SQL Apply)

Building a logical standby requires existing physical standby database.

Thus, in the first section you will look at building a physical standby

and in the second section you will convert the physical standby to

logical standby.

1.1 Building a physical standby

In this section you will look at the detailed steps to build a

physical standby database(for sake of testing in the same

cluster). For information related to building 11.2 standby

databases look here.

1.1.1 Enable force logging on the primary database (RAC10G) by

running

SQL> alter database force logging;

1.1.2 Make sure that archive log mode is enabled
file:///D:/Documents/blog/virtualization-using-oracle-vm-virtualbox-for-building-two-node-oracle-rac-11gr2-11-2-0-2-cluster-where-oracle-11-2-0-2-rdbms-coexists-with-10gr2-10-2-0-5-rdbms-based-on-linux-oel-5-4-2

1.1.3 Verify that password file is created.

1.1.4 Make sure that you can login as sysdba using the password

file.

SQL> connect sys/sys1@rac10g as sysdba

Connected.

SQL>

1.1.5 Configure standby redo log on the primary database

(RAC10G).

Oracle documentation states that you need to have n+1

standby redo logs if there are n redo logs.

In my case I have two threads with two groups each. I will

create three groups of standby redo log for each thread.

SQL> select * from v$log;

GROUP# THREAD# SEQUENCE# BYTES MEMBERS ARC STATUS

FIRST_CHANGE# FIRST_TIM

---------- ---------- ---------- ---------- ---------- --- ---------------

- ------------- ---------

1 1 59 52428800 2 YES INACTIVE

3216590 23-JUN-12

2 1 60 52428800 2 NO CURRENT

3270128 23-JUN-12

3 2 49 52428800 2 NO CURRENT

3270168 23-JUN-12

4 2 48 52428800 2 YES INACTIVE

3158521 22-JUN-12

SQL> select * from v$logfile;

GROUP# STATUS TYPE MEMBER IS_

---------- ------- ------- ---------------------------------------- ---

2 ONLINE +DATADG/rac10g/onlinelog/group_2.264.757 NO

480057

2 ONLINE +DATADG/rac10g/onlinelog/group_2.265.757 YES

480061


480051


480053


---------- ------- ------- ---------------------------------------- ---


480381


480385


480391



---------- ------- ------- ---------------------------------------- ---

480395

8 rows selected.

SQL>

The following standby redo logs that are created.

alter database add standby logfile thread 1 group 5 size 52428800;






SQL> SELECT GROUP#,THREAD#,SEQUENCE#,ARCHIVED,STATUS FROM V$STANDBY_LOG;

GROUP# THREAD# SEQUENCE# ARC STATUS

---------- ---------- ---------- --- ----------

5 1 0 YES UNASSIGNED






6 rows selected.

SQL>

1.1.6 Set tnsnames aliases in tnsnames.ora for communication

between the database and the standby. Here RAC10G and

RACSTB will be used for redo shipment. Pay attention to the

PR and ST that will be used to startup and shutdown a

database using a password file authentication.

# tnsnames.ora Network Configuration File:

/u01/app/oracle/product/10.2.0/db_1/network/admin/tnsnames.ora

# Generated by Oracle configuration tools.

LISTENERS_RAC10G =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = raclinux1-vip)(PORT = 1521))


)

LISTENERS_RACSTB =

(ADDRESS_LIST =



)

RAC10G2 =

(DESCRIPTION =


(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = RAC10G)

(INSTANCE_NAME = RAC10G2)

)

)

RAC10G1 =

(DESCRIPTION =


(CONNECT_DATA =



(INSTANCE_NAME = RAC10G1)

)

)

RAC10G =

(DESCRIPTION =



(LOAD_BALANCE = yes)

(CONNECT_DATA =



)

)

LISTENERS_RACSTB =

(ADDRESS_LIST =



)

RACSTB2 =

(DESCRIPTION =


(CONNECT_DATA =


(SERVICE_NAME = RACSTB)

(INSTANCE_NAME = RACSTB2)

)

)

RACSTB1 =

(DESCRIPTION =


(CONNECT_DATA =



(INSTANCE_NAME = RACSTB1)

)

)

RACSTB =

(DESCRIPTION =



(LOAD_BALANCE = yes)

(CONNECT_DATA =



)

)

RACSTBC =

(DESCRIPTION =



(CONNECT_DATA =

(SID = RACSTB1)

)

)

RAC10GC =

(DESCRIPTION =



(CONNECT_DATA =

(SID = RAC10G1)

)

)

PR =

(DESCRIPTION =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = raclinux1-vip)(PORT =

1521))

)

(CONNECT_DATA =

(SID = RAC10G1)

)

)

ST =

(DESCRIPTION =

(ADDRESS_LIST =


1521))

)

(CONNECT_DATA =

(SID = RACSTB1)

)

)

[oracle@raclinux1 admin]$

1.1.7 Set static listener.ora

Make a static listener entries on each node

[oracle@raclinux1 admin]$ cat listener.ora

# listener.ora Network Configuration File:

/u01/app/oracle/product/10.2.0/db_1/network/admin/listener.ora


SID_LIST_LISTENER_RACLINUX1 =

(SID_LIST =

(SID_DESC =

(GLOBAL_DBNAME = RACSTB)

(ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1)

(SID_NAME = RACSTB)

)

(SID_DESC =

(GLOBAL_DBNAME = RAC10G)


(SID_NAME = RAC10G)

)

(SID_DESC =

(GLOBAL_DBNAME = RAC10G_DGMGRL)


(SID_NAME = RAC10G)

)

(SID_DESC =

(GLOBAL_DBNAME = RACSTB_DGMGRL)


(SID_NAME = RACSTB)

)

)

LISTENER_RACLINUX1 =

(DESCRIPTION_LIST =

(DESCRIPTION =


1521))

)

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.21)(PORT =

1521))

)

)


[oracle@raclinux2 admin]$ cat listener.ora

# listener.ora.raclinux2 Network Configuration File:

/u01/app/oracle/product/10.2.0/db_1/network/admin/listener.ora.racl

inux2


SID_LIST_LISTENER_RACLINUX2 =

(SID_LIST =

(SID_DESC =



(SID_NAME = RACSTB)

)

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RACSTB)

)

)

LISTENER_RACLINUX2 =

(DESCRIPTION =

(ADDRESS_LIST =


1521))

(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.22)(PORT =

1521)(IP = FIRST))

)

)


1.1.8 Modify the init parameters on RAC10G

alter system set LOG_ARCHIVE_CONFIG='DG_CONFIG=(RAC10G,RACSTB)'

scope=both sid='*';

alter system set LOG_ARCHIVE_DEST_1='LOCATION=+DATADG/

VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=cRAC10G'

scope=both sid='*';

alter system set LOG_ARCHIVE_DEST_2='SERVICE=RACSTB LGWR ASYNC

VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=RACSTB'

scope=both sid='*';

alter system set LOG_ARCHIVE_DEST_STATE_1=ENABLE scope=both

sid='*';


sid='*';

alter system set LOG_ARCHIVE_MAX_PROCESSES=30 scope=both sid='*';

alter system set FAL_SERVER=RACSTB scope=both sid='*';

alter system set FAL_CLIENT=RAC10G scope=both sid='*';

alter system set DB_FILE_NAME_CONVERT='+DATA/','+DATADG/'

scope=spfile sid='*';

alter system set LOG_FILE_NAME_CONVERT='+DATA/','+DATADG/'

scope=spfile sid='*';

alter system set STANDBY_FILE_MANAGEMENT=AUTO scope=both sid='*';

1.1.9 Prepare init.ora file for RACSTB

Look at the full initRACSTB.ora file in the Appendix.

DB_NAME=RAC10G

DB_UNIQUE_NAME=RACSTB

LOG_ARCHIVE_CONFIG='DG_CONFIG=(RAC10G,RACSTB)'

DB_FILE_NAME_CONVERT='+DATADG/','+DATA/'

LOG_FILE_NAME_CONVERT='+DATADG/','+DATA/'

LOG_ARCHIVE_FORMAT=%t_%s_%r.dbf

LOG_ARCHIVE_DEST_1='LOCATION=+DATA/

VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=RACSTB'

LOG_ARCHIVE_DEST_2='SERVICE=RAC10G LGWR ASYNC

VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=RAC10G'

LOG_ARCHIVE_DEST_STATE_1=ENABLE

LOG_ARCHIVE_DEST_STATE_2=ENABLE

REMOTE_LOGIN_PASSWORDFILE=EXCLUSIVE

STANDBY_FILE_MANAGEMENT=AUTO

FAL_SERVER=RAC10G

FAL_CLIENT=RACSTB

1.1.10 Create directories for bdump,cdump etc

Create the following direcories

mkdir /u01/app/oracle/admin/RACSTB

cd /u01/app/oracle/admin/RACSTB

mkdir adump bdump cdump dpdump hdump pfile udump

1.1.11 Create password file for the standby RACSTB database

orapwd file=orapwRACSTB1 entries=100 password=sys1

orapwd file=orapwRACSTB2 entries=100 password=sys1

1.1.12 Make a RAC10G backup and run a script to create the

standby. Look in the Appendix for output of the script.

backup database plus archivelog;

backup current controlfile;

BACKUP CURRENT CONTROLFILE FOR STANDBY;

run {

allocate channel tst type disk;

allocate channel tst1 type disk;

allocate auxiliary channel tststby type disk;

duplicate target database for standby DORECOVER;

}

1.1.13 Create a spfile for the standby database.

SQL> create spfile='+DATA/RACSTB/spfileRACSTB.ora' from

pfile='/tmp/initRACSTB.ora';

File created.

SQL>

Create pfiles

[oracle@raclinux2 dbs]$ cat initRACSTB2.ora

SPFILE='+DATA/RACSTB/spfileRACSTB.ora'

[oracle@raclinux2 dbs]$







1.1.14 Register the database with Oracle Clusterware for

management as a cluster resource.

srvctl add database -d racstb -o

/u01/app/oracle/product/10.2.0/db_1 -p

+DATA/RACSTB/spfileRACSTB.ora -r physical_standby -y automatic

srvctl add instance -d racstb -i racstb2 -n raclinux2

srvctl add instance -d racstb -i racstb1 -n raclinux1

1.1.15 Start redo apply

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT

FROM SESSION;

To start real-time apply, issue the following statement:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT

LOGFILE;

1.1.16 Verify that RACSTB is created successfully

SQL> select PROTECTION_MODE, PROTECTION_LEVEL, OPEN_MODE,

DATABASE_ROLE from v$database;

PROTECTION_MODE PROTECTION_LEVEL OPEN_MODE DATABASE_ROLE

-------------------- -------------------- ---------- --------------

--

MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE MOUNTED PHYSICAL

STANDBY

SQL>

SQL> select sequence#, applied from v$archived_log order by

sequence#;

SEQUENCE# APP

---------- ---

47 NO

48 NO

49 NO

50 NO

51 NO

52 NO

53 NO

54 NO

55 NO

55 NO

56 NO

SEQUENCE# APP

---------- ---

56 NO

57 NO

57 NO

58 NO

58 YES

58 NO

59 NO

59 NO

59 YES

60 NO

60 NO

SEQUENCE# APP

---------- ---

60 YES

61 YES

61 NO

61 NO

62 NO

62 NO

62 YES

63 NO

63 NO

63 NO

64 NO

SEQUENCE# APP

---------- ---

65 NO

66 NO

67 NO

68 YES

68 NO

69 YES

69 NO

70 YES

70 NO

71 YES

71 NO

SEQUENCE# APP

---------- ---

72 YES

72 NO

73 NO

73 YES

74 YES

74 NO

50 rows selected.

SQL>

SQL> select sequence#, applied from v$archived_log order by

sequence#;

SEQUENCE# APP

---------- ---

58 YES

59 YES

60 YES

61 YES

62 YES

63 YES

64 YES

65 YES

66 YES

68 YES

69 YES

SEQUENCE# APP

---------- ---

70 YES

71 YES

72 YES

73 YES

74 YES

75 YES

76 YES

77 YES

19 rows selected.

SQL>

SQL> COLUMN NAME FORMAT A18

SQL> COLUMN VALUE FORMAT A16

SQL> COLUMN TIME_COMPUTED FORMAT A24

SQL> SELECT * FROM V$DATAGUARD_STATS;

NAME VALUE UNIT

TIME_COMPUTED

------------------ ---------------- ------------------------------

------------------------

apply finish time +00 00:00:00.0 day(2) to second(1) interval

24-JUN-2012 02:01:26

apply lag +00 00:00:12 day(2) to second(0) interval

24-JUN-2012 02:01:26

estimated startup 165 second

24-JUN-2012 02:01:26

time

standby has been o N

24-JUN-2012 02:01:26

pen

transport lag +00 00:00:00 day(2) to second(0) interval

24-JUN-2012 02:01:26

SQL>

1.1.17 Verify that standby database operates successfully.

You can perform two switchovers and archive some redo logs

and make sure that that logs get applied on the standby.

To perform a switchover on the primary issue

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH

SESSION SHUTDOWN;

Shutdown and mount the primary. The primary is already a

new standby. Monitor the original standby until you have

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

--------------------

TO PRIMARY

On the original standby issue the following command to

switch it to primary.

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY WITH SESSION

SHUTDOWN;

Database altered.

SQL>

Restart the original standby which becomes a new primary.

Perform few switches on the primary and monitor that redo

get applied on the standby. Perform another switch and test

to return back to the starting point and confirm that each

database can operate in each role.

You can monitor the role switches as follows

On the standby




-------------------- -------------------- ---------- --------------

--

MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE MOUNTED PHYSICAL

STANDBY

SQL>

On the primary




-------------------- -------------------- ---------- --------------

--

MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE READ WRITE PRIMARY

SQL>

1.1.18 This concludes physical standby database creation.

1.2 Converting the physical standby to logical standby

1.2.1 Make sure that physical standby is operational

1.2.2 Stop the redo apply on the physical standby by issuing the

following command:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

Database altered.

SQL>

1.2.3 Prepare the primary database for role transition by setting

an additional archive log destination that is to be used

only if the primary is in a logical standby role to archive

its own archive logs received from the primary in contrast

to archiving logs in primary role from transactions against

the primary.

alter system set LOG_ARCHIVE_DEST_3='LOCATION=+DATA/

VALID_FOR=(STANDBY_LOGFILES,STANDBY_ROLE) DB_UNIQUE_NAME=RAC10G'

scope=both sid='*';


sid='*';

1.2.4 Build a dictionary by executing on the primary database the

following command.

EXECUTE DBMS_LOGSTDBY.BUILD;

1.2.5 Convert the physical standby database to logical standby by

issuing the following statement on the standby database.

Before that make sure that CLUSTER_DATABASE=false.

SQL> ALTER DATABASE RECOVER TO LOGICAL STANDBY RACSTB;

Database altered.

SQL>

1.2.6 Adjust init parameters on the logical standby and check the

existing archive log destinations.

Add log_archive_dest_3 for the same purpose as explained in

1.2.3

alter system set LOG_ARCHIVE_DEST_3='LOCATION=+DATA/

VALID_FOR=(STANDBY_LOGFILES,STANDBY_ROLE)

DB_UNIQUE_NAME=RACSTB' scope=both sid='*';

alter system set LOG_ARCHIVE_DEST_STATE_3=ENABLE

scope=both sid='*';

1.2.7 Open the logical standby database

ALTER DATABASE OPEN RESETLOGS;

1.2.8 Start SQL apply

SQL> SQL>

SQL> ALTER DATABASE START LOGICAL STANDBY APPLY IMMEDIATE;

Database altered.

SQL>

1.2.9 Verify that Logical Standby database is properly

configured.

On the primary you will see something like this

select name,PROTECTION_MODE, PROTECTION_LEVEL, OPEN_MODE,


select thread#, sequence#, file_name, applied from dba_logstdby_log

where applied!= 'YES';

SQL> select name,PROTECTION_MODE, PROTECTION_LEVEL, OPEN_MODE,


NAME PROTECTION_MODE PROTECTION_LEVEL OPEN_MODE

--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------

RAC10G MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE READ WRITE

PRIMARY

SQL>

On the Logical standby you will see something like this




--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------

RACSTB MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE READ WRITE

LOGICAL STANDBY

SQL>

Lets see how the SQL Apply progresses

07:42:34 SQL> set timing on

07:42:40 SQL> set linesize 200

07:42:50 SQL> column FILE_NAME format a30

07:43:03 SQL> column FILE_NAME format a30

07:43:07 SQL> select thread#, sequence#, file_name, applied from

dba_logstdby_log where applied!= 'YES';

THREAD# SEQUENCE# FILE_NAME APPLIED

---------- ---------- ------------------------------ --------

1 295 +DATA/racstb/archivelog/2012_0 CURRENT

6_25/thread_1_seq_295.944.7868

86237

1 296 +DATA/racstb/archivelog/2012_0 NO

6_25/thread_1_seq_296.945.7868

86239


6_25/thread_1_seq_297.948.7868

87229


---------- ---------- ------------------------------ --------


6_25/thread_1_seq_298.949.7868

87229


6_25/thread_1_seq_299.950.7868

87235


6_25/thread_1_seq_300.961.7868


---------- ---------- ------------------------------ --------

91899


6_25/thread_1_seq_301.960.7868

91899


6_25/thread_1_seq_302.959.7868

91891



---------- ---------- ------------------------------ --------

6_25/thread_1_seq_303.979.7869

06821


6_26/thread_1_seq_304.989.7869

58509


6_26/thread_1_seq_305.988.7869

58505


---------- ---------- ------------------------------ --------


6_26/thread_1_seq_306.993.7869

58985


6_25/thread_2_seq_98.943.78688

6227


6_25/thread_2_seq_99.942.78688

6219


---------- ---------- ------------------------------ --------


6_25/thread_2_seq_100.946.7868

86879


6_25/thread_2_seq_101.962.7868

91899


6_25/thread_2_seq_102.958.7868


---------- ---------- ------------------------------ --------

91861


6_25/thread_2_seq_103.963.7868

91955


6_25/thread_2_seq_104.964.7868

91955



---------- ---------- ------------------------------ --------

6_25/thread_2_seq_105.972.7869

03265


6_25/thread_2_seq_106.973.7869

03277


6_25/thread_2_seq_107.978.7869

06833


---------- ---------- ------------------------------ --------


6_26/thread_2_seq_108.991.7869

58537


6_26/thread_2_seq_109.990.7869

58523


6_26/thread_2_seq_110.992.7869

58965


---------- ---------- ------------------------------ --------

25 rows selected.

Elapsed: 00:00:00.01

07:43:16 SQL>




---------- ---------- ------------------------------ --------


6_26/thread_1_seq_304.989.7869

58509


6_26/thread_1_seq_305.988.7869

58505


6_26/thread_1_seq_306.993.7869

58985


---------- ---------- ------------------------------ --------


6_26/thread_2_seq_108.991.7869

58537


6_26/thread_2_seq_109.990.7869

58523


6_26/thread_2_seq_110.992.7869


---------- ---------- ------------------------------ --------

58965


6_26/thread_2_seq_111.994.7869

58995

7 rows selected.

Elapsed: 00:00:00.01



no rows selected

Elapsed: 00:00:01.81

07:51:21 SQL>

The section from the alert log of the logical standby that

corresponds to the period above is in the Appendix.

1.2.10 For the upgrade strategy to be successful you need to

make sure that the logical standby Data guard performs as

expected ,i.e. each database in the following stages

performs adequately in its role and the primary always

propagates changes to standby which in turn applies

accordingly. I would suggest for you to perform two

switchovers and make sure that Data guard Logical standby

operates as expected.

Primary Role Logical Standby

Role

RAC10G RACSTB

After 1st switchover RACSTB RAC10G

After 2nd switchover RAC10G RACSTB

Lets briefly outline the procedure for switchover to the

logical standby database. Steps are to be executed in an

order.

a. On the primary database check the status.

SQL> select switchover_status from v$database;

SWITCHOVER_STATUS

--------------------

SESSIONS ACTIVE

SQL>

b. Prepare the current primary database for the switchover.

SQL> alter database prepare to switchover to logical standby;

Database altered.

SQL>


SWITCHOVER_STATUS

--------------------

PREPARING SWITCHOVER

SQL>

c. Prepare the target logical standby database for the

switchover.

SQL>ALTER DATABASE PREPARE TO SWITCHOVER TO PRIMARY;


SWITCHOVER_STATUS

--------------------


SQL>

d. Ensure the current primary database is ready for the

future primary databases redo stream. Look for TO

LOGICAL STANDBY on the original primary by issuing the

following statement on the primary database.


SWITCHOVER_STATUS

--------------------

TO LOGICAL STANDBY

SQL>

e. Switch the primary database to the logical standby

database role.

SQL> alter database commit to switchover to logical standby;

Database altered.

SQL>

f. Ensure all available redo has been applied to the target

logical standby database that is about to become the new

primary database. Issue the following SQl statement on

the standby and look for TO PRIMARY status.


SWITCHOVER_STATUS

--------------------

TO PRIMARY

SQL>

g. Switch the target logical standby database to the

primary database role by issuing the following SQL

statement on the logical standby.

SQL> alter database commit to switchover to primary;

Database altered.

SQL>

h. Verify that role transition took place start SQL apply

and make sure that works. For illustration for the 1st

role transition you will have something similar.

Before:




--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------


PRIMARY

SQL>





--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------


LOGICAL STANDBY

SQL>

After:




--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------


PRIMARY

SQL>





--------- -------------------- -------------------- ---------------

-----

DATABASE_ROLE

----------------


LOGICAL STANDBY

SQL>

i. This concludes the Logical Switchover testing

1.2.11 This concludes Logical standby configuration.

2. Prepare and perform a near zero downtime upgrade

In this section you will look at the steps required for:

1. Preparation for upgrade.

2. Actual upgrade.

The upgrade process goes through the following conceptual stages

illustrated in the table below as far as the logical standby database

is concerned. I will upgrade the standby database once it is in stage

3 and will repeat the same upgrade procedure to the original primary

database once it is in stage 6.

Stage Primary site Logical Standby

Site

Production

Database

1 RAC10G is ONLINE

running 10.2

RACSTB is LS 10.2

and SQL Apply is

running

RAC10G

2 RAC10G is ONLINE

running 10.2

RACSTB is LS 10.2

and SQL Apply is

stopped

RAC10G

3 Upgrade to RACSTB RAC10G is ONLINE

running 10.2

RACSTB is LS is

being upgraded to

11.2.0.3 and SQL

apply is stopped

RAC10G

4 Conduct final

application testing

on the LS before

switch production

RAC10G is ONLINE

running 10.2

RACSTB is LS

11.2.0.3 and SQL

apply is started

to sync with

RAC10G

in the next stage. primary

5 (Switchover) THIS

IS THE ONLY

DOWNTIME (Depend on

the switchover

time)

RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

deferred)

RAC10G is LS

running 10.2 ( SQL

Apply is stopped)

RACSTB

6 Upgrade RAC10G RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

deferred)

RAC10G is LS that

is being upgraded

to 11.2.0.3 ( SQL

Apply is stopped)

RACSTB

7. RACSTB is ONLINE

running 11.2.0.3

(archive log

destination is

enabled)

RAC10G is LS

running 11.2.0.3

and SQL apply is

started to sync

with primary

RACSTB

8 (Switchover) RAC10G is ONLINE

running 11.2.0.3

RACSTB is LS

11.2.0.3 and SQL

Apply is running

RAC10G

9. Adjust the

COMATIBLE parameter

for both databases

RAC10G is ONLINE

running 11.2.0.3

RACSTB is LS

11.2.0.3 and SQL

Apply is running

RAC10G

LS Logical Standby

2.1 Preparation for upgrade

i. Make sure that you have 11.2.0.3 cluster already running(

installed and patched)

ii. Make sure that you install 11.2.0.3 RDBMS binaries on each

node of the cluster.

iii. Run pre-upgrade information tool and make sure that any

errors and warnings are addressed prior to actual upgrade

(pay attention to invalid objects and obsolete or

deprecated init parameters). Run statistics collection. Run

utlu112i.sql that comes with 11.2.0.3. Output from the run

is in the Appendix.

iv. Make sure that you have a flashback database feature

enabled on both databases RAC10G and RACSTB. Note that in

10.2 you need to bounce the instance, downtime involved, to

set it up.

shutdown immediate

startup mount

alter database flashback on;

v. Make sure that you create a guaranteed restore point

,requires cluster_database=false, on both RAC10G and RACSTB

vi. On both RAC10G and RACSTB you should have.

SQL> create restore point pre_upgrade guarantee flashback database;

Restore point created.

SQL>

SQL> column TIME format a40

SQL> column NAME format a30

SQL> select * from v$restore_point;

SCN DATABASE_INCARNATION# GUA STORAGE_SIZE TIME

NAME

---------- --------------------- --- ------------ -----------------

----------------------- ------------------------------

3629546 2 YES 8192000 25-JUN-12

12.02.27.000000000 AM PRE_UPGRADE

SQL>

vii. Stop 10g listener on standby database

viii. Stop SQL apply on the standby database

SQL> alter database stop logical standby apply;

SQL>

ix. Set cluster_database=false on the logical standby database

and shutdown the logical standby database.

2.2 Performing an upgrade

Stopping the logical standby database at the end of the

previous sections marks the beginning of 3rd stage.

2.2.1 Copy the init file to the new 11.2.0.3 location

On node1

cp /u01/app/oracle/product/10.2.0/db_1/dbs/initRACSTB1.ora

/u01/app/oracle/product/11.2.0/db_3/dbs/initRACSTB1.ora

On node2

cp /u01/app/oracle/product/10.2.0/db_1/dbs/initRACSTB2.ora

/u01/app/oracle/product/11.2.0/db_3/dbs/initRACSTB2.ora

2.2.2 Copy the password files

cp /u01/app/oracle/product/10.2.0/db_1/dbs/orapwRACSTB1

/u01/app/oracle/product/11.2.0/db_3/dbs/orapwRACSTB1

cp /u01/app/oracle/product/10.2.0/db_1/dbs/orapwRACSTB2

/u01/app/oracle/product/11.2.0/db_3/dbs/orapwRACSTB2

2.2.3 Login as oracle user that has 11.2.0.3 environment set and

export ORACLE_SID=RACSTB1

2.2.4 Invoke sqlplus and from sqlplus create a text parameter file.

SQL> create pfile='/tmp/i1.ora' from

SPFILE='+DATA/RACSTB/spfileRACSTB.ora';

File created.

SQL>

2.2.5 Edit the test parameter file so that you remove the

*_dump_dest parameters and make sure that cluster_database

parameter is set to false.

2.2.6 Make sure that tnsnames.ora entries from 10.2 $OH and included

in 11.2.0.3 tnsnames.ora file in 11.2.0.3 $OH

2.2.7 From the new $OH/rdbms/admin and invoke sqlplus / as sysdba.

Database was shutdown earlier.

startup migrate pfile='/tmp/i1.ora';

spool /tmp/catupgrd.log

@catupgrd.sql

Wait for the upgrade to complete. It finishes with shutting

down the database. Examine for errors and fix the problem if

any. Proceed further upon successful completion of

catupgrd.sql.

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL>

SQL>

SQL>

SQL> DOC

DOC>######################################################################

#

DOC>######################################################################

#

DOC>

DOC> The above sql script is the final step of the upgrade. Please

DOC> review any errors in the spool log file. If there are any errors in

DOC> the spool file, consult the Oracle Database Upgrade Guide for

DOC> troubleshooting recommendations.

DOC>

DOC> Next restart for normal operation, and then run utlrp.sql to

DOC> recompile any invalid application objects.

DOC>

DOC> If the source database had an older time zone version prior to

DOC> upgrade, then please run the DBMS_DST package. DBMS_DST will

upgrade

DOC> TIMESTAMP WITH TIME ZONE data to use the latest time zone file

shipped

DOC> with Oracle.

DOC>

DOC>######################################################################

#

DOC>######################################################################

#

DOC>#

SQL>

SQL> Rem Set errorlogging off

SQL> SET ERRORLOGGING OFF;

SQL>

SQL> REM END OF CATUPGRD.SQL

SQL>

SQL> REM bug 12337546 - Exit current sqlplus session at end of

catupgrd.sql.

SQL> REM This forces user to start a new sqlplus session in

order

SQL> REM to connect to the upgraded db.

SQL> exit

Disconnected from Oracle Database 11g Enterprise Edition Release

11.2.0.3.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage

Management, OLAP,

Data Mining and Real Application Testing options


SQL>

2.2.8 Create spfile create SPFILE='+DATA/RACSTB/spfileRACSTB.ora' from pfile='/tmp/i1.ora';

2.2.9 Startup the database and from sqlplus run the following

commands and scripts.

startup

@utlu112s.sql

@catuppst.sql

@utlrp.sql

You can look for the output of the scripts in the Appendix.

utlu112s.sql displays vital information for the status of the

upgrade. Catuppst.sql performs some additional tasks.

Utlrp.sql recompiles an invalid database objects.

2.2.10 Set the parameter and restart the instance

alter system set cluster_database=true scope=spfile sid='*';

2.2.11 Modify listeners in 11.2.0.3 (SCAN Listener and Node

Listeners) and reload the listeners.

On Node 1.

[grid@raclinux1 admin]$ cat listener.ora


/u01/app/11.2.0.3/grid/network/admin/listener.ora


ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1 = ON

SID_LIST_LISTENER =

(SID_LIST =

(SID_DESC =



(SID_NAME = RACSTB1)

)

(SID_DESC =



(SID_NAME = RAC10G1)

)

(SID_DESC =




)

(SID_DESC =




)

)

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_HA = ON

SID_LIST_LISTENER_SCAN1 =

(SID_LIST =

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RACSTB)

)

(SID_DESC =



(SID_NAME = RACSTB)

)

)

LISTENER =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER))

)

ADR_BASE_LISTENER = /u01/app/grid

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER = ON

LISTENER_SCAN1 =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER_SCAN1))

)

ADR_BASE_LISTENER_SCAN1 = /u01/app/grid

[grid@raclinux1 admin]$

On Node 2.

[grid@raclinux2 admin]$ cat listener.ora


/u01/app/11.2.0.3/grid/network/admin/listener.ora


ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1 = ON

SID_LIST_LISTENER =

(SID_LIST =

(SID_DESC =




)

(SID_DESC =




)

(SID_DESC =




)

(SID_DESC =




)

)

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_HA = ON

SID_LIST_LISTENER_SCAN1 =

(SID_LIST =

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RAC10G)

)

(SID_DESC =



(SID_NAME = RACSTB)

)

(SID_DESC =



(SID_NAME = RACSTB)

)

)

LISTENER =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER))

)

ADR_BASE_LISTENER = /u01/app/grid

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER = ON

LISTENER_SCAN1 =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER_SCAN1))

)

ADR_BASE_LISTENER_SCAN1 = /u01/app/grid

[grid@raclinux2 admin]$

Reload 11.2 Listeners

2.2.12 Remove the database from CRS/GI from 10gR2 home and add the

database to CRS/GI from 11.2.0.3 home

From 10.2 $OH

[oracle@raclinux1 dbs]$ srvctl remove database -d racstb -f

[oracle@raclinux1 dbs]$ srvctl config database -d racstb

PRKR-1001 : cluster database racstb does not exist

PRKO-2005 : Application error: Failure in getting Cluster Database

Configuration for: racstb


From 11.2.0.3 $OH

srvctl add database -d RACSTB -o /u01/app/oracle/product/11.2.0/db_3 -c

RAC -y AUTOMATIC -p +DATA/RACSTB/spfileRACSTB.ora -r logical_standby

srvctl add instance -d RACSTB -i RACSTB2 -n raclinux2

srvctl add instance -d RACSTB -i RACSTB1 -n raclinux1

[oracle@raclinux1 admin]$ srbctl config database -d RACSTB

-bash: srbctl: command not found

[oracle@raclinux1 admin]$ srvctl config database -d RACSTB

Database unique name: RACSTB

Database name:

Oracle home: /u01/app/oracle/product/11.2.0/db_3

Oracle user: oracle

Spfile: +DATA/RACSTB/spfileRACSTB.ora

Domain:

Start options: open

Stop options: immediate

Database role: LOGICAL_STANDBY

Management policy: AUTOMATIC

Server pools: RACSTB

Database instances: RACSTB1,RACSTB2

Disk Groups:

Mount point paths:

Services:

Type: RAC

Database is administrator managed

[oracle@raclinux1 admin]$ srvctl status database -d RACSTB

Instance RACSTB1 is not running on node raclinux1

Instance RACSTB2 is not running on node raclinux2


2.2.13 Restart the logical standby database.

SQL>shutdown immediate;

srvctl start database -d

SQL> select * from v$active_instances;

INST_NUMBER INST_NAME

----------- ------------------------------------------------------------

1 raclinux1.gj.com:RACSTB1


SQL>

On primary

SQL> select PROTECTION_MODE, PROTECTION_LEVEL, OPEN_MODE, DATABASE_ROLE from

v$database;


-------------------- -------------------- ---------- ----------------

MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE READ WRITE PRIMARY

SQL>

On standby

SQL> select PROTECTION_MODE, PROTECTION_LEVEL, OPEN_MODE, DATABASE_ROLE from

v$database;


-------------------- -------------------- -------------------- ---------------

-

MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE READ WRITE LOGICAL STANDBY

SQL>

2.2.14 Start SQL apply on the Logical standby database RACSTB (

stage 4th) by issuing the following statement:

alter database start logical standby apply immediate;

Take a note that RACSTB is 11.2.0.3 and can still apply redo

log changes generated by the primary database 10.2. Issue a

log switch on the primary and wait until all redo accumulated

during the Logical Standby upgrade from the primary database

get applied to the Logical standby database. Monitor SQL apply

status on the standby database using this query on Logical

standby database RACSTB and look at the return no rows as an

indicator that the database are synchronized.

SQL> select thread#, sequence#, file_name, applied from dba_logstdby_log

where applied!= 'YES';

no rows selected

SQL>

2.2.15 Perform a final testing on the logical standby database

RACSTB before the switchover. Make sure that application can

function with the upgraded database. So far the production

database has been RAC10G running 10.2.0.5 instead of 11.2.0.3.

Only if you are in a go situation perform the actual

application switchover to 11.2.0.3 database as shown in the

next step.

2.2.16 Take a downtime for the application to point to the new

production database RACSTB.

2.2.17 Perform a switchover whereby RACSTB becomes a primary

database and RAC10G becomes a logical standby. Take a note

that log_archive_dest_state_2 is set to defer in order to

suspend sending redo logs to the new logical standby as 10.2

cannot apply changes from 11.2. Steps are as described earlier

and outlined below.

On the primary database issue the following statements.

SQL> alter database prepare to switchover to logical standby;

Database altered.


SWITCHOVER_STATUS

--------------------


SQL>

On the standby database issue the following statements.

SQL> ALTER DATABASE PREPARE TO SWITCHOVER TO PRIMARY;

Database altered.


SWITCHOVER_STATUS

--------------------


SQL>

Check what you have on the primary


SWITCHOVER_STATUS

--------------------

TO LOGICAL STANDBY

SQL>

On the primary issue the following sql statement.

SQL> alter database commit to switchover to logical standby;

Database altered.

SQL>

On the logical standby issue the following


SWITCHOVER_STATUS

--------------------

TO PRIMARY

SQL>

SQL> show parameter log_archive_dest_2

NAME TYPE VALUE

------------------------------------ ----------- -------------------------

-----

log_archive_dest_2 string SERVICE=RAC10G LGWR ASYNC

VALI

D_FOR=(ONLINE_LOGFILES,PRIMARY

_ROLE)

DB_UNIQUE_NAME=RAC10G

log_archive_dest_20 string








NAME TYPE VALUE

------------------------------------ ----------- -------------------------

-----



SQL> show parameter log_archive_dest_2_state

SQL> show parameter log_archive_dest_state_2

NAME TYPE VALUE

------------------------------------ ----------- -------------------------

-----

log_archive_dest_state_2 string ENABLE

log_archive_dest_state_20 string enable










SQL>


SWITCHOVER_STATUS

--------------------

TO PRIMARY

SQL>

On the standby

SQL> alter system set log_archive_dest_state_2=defer scope=both sid='*';

System altered.

SQL>

SQL> alter database commit to switchover to primary;

Database altered.

SQL>

Now The Primary is 11.2.0.3




--------- -------------------- -------------------- --------------------

DATABASE_ROLE

----------------


PRIMARY



----------- ------------------------------------------------------------



SQL> select * from v$version;

BANNER

--------------------------------------------------------------------------

------

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit

Production

PL/SQL Release 11.2.0.3.0 - Production

CORE 11.2.0.3.0 Production

TNS for Linux: Version 11.2.0.3.0 - Production

NLSRTL Version 11.2.0.3.0 - Production

SQL>

The new Logical standby is 10.2.0.5

SQL> select name,PROTECTION_MODE, PROTECTION_LEVEL,

OPEN_MODE, DATABASE_ROLE from v$database;

NAME PROTECTION_MODE

PROTECTION_LEVEL OPEN_MODE DATABASE_ROLE

------------------------------ -------------------- -------

------------- ---------- ----------------

RAC10G MAXIMUM PERFORMANCE MAXIMUM

PERFORMANCE READ WRITE LOGICAL STANDBY

SQL>

SQL> select * from v$instance;

INSTANCE_NUMBER INSTANCE_NAME HOST_NAME

VERSION STARTUP_T STATUS PAR THREAD#

ARCHIVE LOG_SWITCH_WAIT LOGINS SHU

--------------- ---------------- --------------------------

-------------------------------------- ----------------- --

------- ------------ --- ---------- ------- ---------------

---------- ---

DATABASE_STATUS INSTANCE_ROLE ACTIVE_ST BLO

----------------- ------------------ --------- ---

1 RAC10G1 raclinux1.gj.com

10.2.0.5.0 25-JUN-12 OPEN NO 1

STARTED ALLOWED NO

ACTIVE PRIMARY_INSTANCE NORMAL NO

SQL> show parameter cluster

NAME TYPE VALUE

------------------------------------ ----------- ----------

--------------------

cluster_database boolean FALSE

cluster_database_instances integer 1

cluster_interconnects string

SQL> select name,PROTECTION_MODE, PROTECTION_LEVEL,

OPEN_MODE, DATABASE_ROLE from v$database;

NAME PROTECTION_MODE

PROTECTION_LEVEL OPEN_MODE DATABASE_ROLE

------------------------------ -------------------- -------

------------- ---------- ----------------

RAC10G MAXIMUM PERFORMANCE MAXIMUM

PERFORMANCE READ WRITE LOGICAL STANDBY

SQL>

2.2.18 Start the application to use RACSTB as a production

database.

2.2.19 Upgrade the RAC10G new logical standby database. This

procedure is essentially the same as upgrading RACSTB. So

below are the steps. Refer to RACSTB upgrade for more detail

explanation.

Make sure that initRAC10G.ora parameter file is in 11.2.0.3

$OH. Make sure that password files are in the new 11.2.0.3 $OH

Copy Init.ora and Password files on each node

On node1

cp /u01/app/oracle/product/10.2.0/db_1/dbs/initRAC10G1.ora

/u01/app/oracle/product/11.2.0/db_3/dbs/initRAC10G1.ora

On node2

cp /u01/app/oracle/product/10.2.0/db_1/dbs/initRAC10G2.ora

/u01/app/oracle/product/11.2.0/db_3/dbs/initRAC10G2.ora

On node 1

cp /u01/app/oracle/product/10.2.0/db_1/dbs/orapwRAC10G1

/u01/app/oracle/product/11.2.0/db_3/dbs/orapwRAC10G1

On node 2

cp /u01/app/oracle/product/10.2.0/db_1/dbs/orapwRAC10G2

/u01/app/oracle/product/11.2.0/db_3/dbs/orapwRAC10G2

Prepare init file

[oracle@raclinux1 dbs]$ cat initRAC10G1.ora

SPFILE='+DATADG/RAC10G/spfileRAC10G.ora'


Run the following script to identify any issues for upgrading

the database. Make sure that address any errors, warnings,

recommendations or suggestion. The detailed output is in the

Appendix. Make sure that you change any obsolete or

deprecated parameters and recompile invalid errors.

SQL> @/u01/app/oracle/product/11.2.0/db_3/rdbms/admin/utlu112i.sql

Create a text parameter file

SQL> create pfile='/tmp/i2.ora' from

SPFILE='+DATADG/RAC10G/spfileRAC10G.ora';

File created.

SQL>

Remove core_dump_dest and background_dump_dest user_dump_dest

parameters from the text parameter file and make sure that

cluster_database is false.

Perform a clean shutdown of RAC10G logical standby database.

Invoke from 11.2.0.3 $OH sqlplus / as sysdba and execute the

following statements.

startup

@catupgrd.sql

startup

create SPFILE='+DATADG/RAC10G/spfileRAC10G.ora' from pfile='/tmp/i2.ora';

@utlu112s.sql

@catuppst.sql

@utlrp.sql

Pay attention to any errors. In case of error(s) troubleshoot

and start from the catupgrd.sql script.

Set cluster_database=true and restart the database

Remove old configuration and register the database with

CRS/GI.

From OLD $OH

srvctl remove database -d rac10g -f

From NEW $OH

srvctl add database -d RAC10G -o /u01/app/oracle/product/11.2.0/db_3 -c

RAC -y AUTOMATIC -p +DATADG/RAC10G/spfileRAC10G.ora

srvctl add instance -d RAC10G -i RAC10G2 -n raclinux2

srvctl add instance -d RAC10G -i RAC10G1 -n raclinux1

2.2.20 Synchronize the archive logs after the RAC10G logical

standby database has been upgraded to 11.2.0.3.

On the new primary enable the archiving destination that was

deliberately set to defer state.

alter system set log_archive_dest_state_2=enable scope=both sid='*';

On the LOGICAL STANDBY start SQL apply by executing the SQL

statement.

alter database start logical standby apply immediate;

Check until all redo logs are applied on the logical standby

database that is, 0 rows returned by the query on the logical

standby.

select thread#, sequence#, file_name, applied from dba_logstdby_log where

applied!= 'YES';

Increase the processes init parameter in order to avoid ORA-

16081: insufficient number of processes for APPLY error on the

RAc10G logical standby.

alter system set processes=400 scope=spfile sid='*';

2.2.21 Perform a switchover to return RAC10G as a primary database

and RACSTB as a logical standby database.

Issue on the primary database.

On the primary :


SWITCHOVER_STATUS

--------------------

TO STANDBY

SQL>

SQL> ALTER DATABASE PREPARE TO SWITCHOVER TO LOGICAL STANDBY;

Database altered.


SWITCHOVER_STATUS

--------------------


SQL>

On standby execute :

SQL> ALTER DATABASE PREPARE TO SWITCHOVER TO PRIMARY;

Database altered.


SWITCHOVER_STATUS

--------------------


SQL>

Wait until on primary you see


SWITCHOVER_STATUS

--------------------

TO LOGICAL STANDBY

SQL>

On the primary issue

ALTER DATABASE COMMIT TO SWITCHOVER TO LOGICAL STANDBY;

Wait until You get on the standby the following status


SWITCHOVER_STATUS

--------------------

TO PRIMARY

SQL>

Issue on the standby

ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY;

Verify switchover

On primary




DATABASE_ROLE

--------- -------------------- -------------------- -------------------- -

---------------


PRIMARY



----------- ------------------------------------------------------------

1 raclinux1.gj.com:RAC10G1

2 raclinux2.gj.com:RAC10G2

SQL>

On Logical standby




--------- -------------------- -------------------- --------------------

DATABASE_ROLE

----------------


LOGICAL STANDBY



----------- ------------------------------------------------------------



SQL>

Start SQL apply on the Logical Standby

SQL> alter database start logical standby apply immediate;

Database altered.

SQL>

2.2.22 This concludes the upgrade from 10.2.0.5 to 11.2.0.3 using

a logical standby database for a near zero downtime.

Summary

In the article you looked at a way to use a logical standby database for a

near zero downtime upgrade from Oracle RAC 10.2.0.5 to Oracle RAC 11.2.0.3.

The article described a way to create a physical standby database and convert

it to a logical standby database. The article was written for migrating from

Oracle 10.2.0.5 to 11.2.0.3 but can be used for any migration from Oracle

10.2.0.X or 11.1.0.X or 11.2.0.X to Oracle 11.2.0.X+1. Due to the fact that

Oracle has not changed the way user manages Data Guard physical standby and

Data Guard logical database from 10.2 to 11.2 in the way of interface and

commands and SQL statements. Pay attention to the data type limitations which

are described in the Oracle official documentation listed in the Reference

section to consider if the approach described in the article is a fit for

you.

References:

1. Upgrade & Migrate on Multi-Terabyte Mission Critical RAC with Near Zero

Downtime : Jane Brown Senior Database Administrator

2. Oracle Data Guard Concepts and Administration 10g Release 2 (10.2) 2

B14239-05

3. Oracle Data Guard Concepts and Administration 11g Release 2 (11.2)

E25608-03

Appendix

[oracle@raclinux1 admin]$ cat /tmp/initRACSTB.ora

RACSTB1.__db_cache_size=394264576

RACSTB2.__db_cache_size=398458880

RACSTB1.__java_pool_size=4194304

RACSTB2.__java_pool_size=4194304

RACSTB1.__large_pool_size=4194304

RACSTB2.__large_pool_size=4194304

RACSTB1.__shared_pool_size=180355072

RACSTB2.__shared_pool_size=176160768

RACSTB1.__streams_pool_size=0

RACSTB2.__streams_pool_size=0

*.audit_file_dest='/u01/app/oracle/admin/RACSTB/adump'

*.background_dump_dest='/u01/app/oracle/admin/RACSTB/bdump'

*.cluster_database_instances=2

*.cluster_database=true

*.compatible='10.2.0.5.0'

*.core_dump_dest='/u01/app/oracle/admin/RACSTB/cdump'

*.db_block_size=8192

*.db_create_file_dest='+DATA'

*.db_create_online_log_dest_1='+DATA'

*.db_domain=''

*.db_file_multiblock_read_count=16

*.db_file_name_convert='+DATADG/','+DATA/'

*.db_name='RAC10G'

*.db_unique_name='RACSTB'

*.db_recovery_file_dest='+DATA'

*.db_recovery_file_dest_size=10737418240

*.dispatchers='(PROTOCOL=TCP) (SERVICE=RACSTBXDB)'

*.fal_client='RACSTB'

*.fal_server='RAC10G'

RACSTB2.instance_number=2

RACSTB1.instance_number=1

*.job_queue_processes=10

*.log_archive_config='DG_CONFIG=(RAC10G,RACSTB)'

*.log_archive_dest_1='LOCATION=+DATA/ VALID_FOR=(ALL_LOGFILES,ALL_ROLES)

DB_UNIQUE_NAME=RACSTB'

*.LOG_ARCHIVE_DEST_2='SERVICE=RACSTB LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE)

DB_UNIQUE_NAME=RAC10G'

*.log_archive_dest_state_1='ENABLE'

*.log_archive_dest_state_2='ENABLE'

*.log_archive_format='%t_%s_%r.dbf'

*.log_archive_max_processes=30

*.log_file_name_convert='+DATADG/','+DATA/'

*.open_cursors=300

*.pga_aggregate_target=196083712

*.processes=150

*.remote_listener='LISTENERS_RACSTB'

*.remote_login_passwordfile='exclusive'

*.sga_target=590348288

*.standby_file_management='AUTO'

RACSTB2.thread=2

RACSTB1.thread=1

*.undo_management='AUTO'

RACSTB2.undo_tablespace='UNDOTBS2'

RACSTB1.undo_tablespace='UNDOTBS1'

*.user_dump_dest='/u01/app/oracle/admin/RACSTB/udump'


RMAN> run {

allocate channel tst type disk;

allocate channel tst1 type disk;

allocate auxiliary channel tststby type disk;

duplicate target database for standby DORECOVER;

}

Starting backup at 23-JUN-12

using channel ORA_DISK_2

using channel ORA_DISK_1

channel ORA_DISK_1: starting full datafile backupset

channel ORA_DISK_1: specifying datafile(s) in backupset

including standby control file in backupset

channel ORA_DISK_1: starting piece 1 at 23-JUN-12

channel ORA_DISK_1: finished piece 1 at 23-JUN-12

piece

handle=+DATADG/rac10g/backupset/2012_06_23/ncnnf0_tag20120623t233025_0.554.786756627

tag=TAG20120623T233025 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:05

Finished backup at 23-JUN-12

RMAN> 2> 3> 4> 5> 6>

released channel: ORA_DISK_1

released channel: ORA_DISK_2

allocated channel: tst

channel tst: sid=148 instance=RAC10G1 devtype=DISK

allocated channel: tst1

channel tst1: sid=147 instance=RAC10G1 devtype=DISK

allocated channel: tststby

channel tststby: sid=150 instance=RACSTB1 devtype=DISK

Starting Duplicate Db at 23-JUN-12

contents of Memory Script:

{

set until scn 3422282;

restore clone standby controlfile;

sql clone 'alter database mount standby database';

}

executing Memory Script

executing command: SET until clause

Starting restore at 23-JUN-12

channel tststby: starting datafile backupset restore

channel tststby: restoring control file

channel tststby: reading from backup piece

+DATADG/rac10g/backupset/2012_06_23/ncnnf0_tag20120623t232526_0.539.786756329

channel tststby: restored backup piece 1

piece

handle=+DATADG/rac10g/backupset/2012_06_23/ncnnf0_tag20120623t232526_0.539.786756329

tag=TAG20120623T232526

channel tststby: restore complete, elapsed time: 00:00:09

output filename=+DATA/racstb/controlfile/current.392.786756729

Finished restore at 23-JUN-12

sql statement: alter database mount standby database

WARNING: DB_FILE_NAME_CONVERT resulted in invalid ASM names; names changed to diskgroup

only.


{


set newname for tempfile 1 to

"+data";

switch clone tempfile all;

set newname for datafile 1 to

"+data";


"+data";


"+data";


"+data";


"+data";


"+data";

restore

check readonly

clone database

;

}



executing command: SET NEWNAME

renamed temporary file 1 to +data in control file







Starting restore at 23-JUN-12


channel tststby: specifying datafile(s) to restore from backup set

restoring datafile 00001 to +DATA




+DATADG/rac10g/backupset/2012_06_23/nnndf0_tag20120623t232654_0.545.786756415


piece

handle=+DATADG/rac10g/backupset/2012_06_23/nnndf0_tag20120623t232654_0.545.786756415

tag=TAG20120623T232654



channel tststby: specifying datafile(s) to restore from backup set





+DATADG/rac10g/backupset/2012_06_23/nnndf0_tag20120623t232654_0.546.786756415


piece

handle=+DATADG/rac10g/backupset/2012_06_23/nnndf0_tag20120623t232654_0.546.786756415

tag=TAG20120623T232654


Finished restore at 23-JUN-12


{

switch clone datafile all;

}


datafile 1 switched to datafile copy

input datafile copy recid=14 stamp=786756865

filename=+DATA/racstb/datafile/system.390.786756745



filename=+DATA/racstb/datafile/undotbs1.389.786756747



filename=+DATA/racstb/datafile/sysaux.387.786756811



filename=+DATA/racstb/datafile/users.388.786756747



filename=+DATA/racstb/datafile/example.386.786756811



filename=+DATA/racstb/datafile/undotbs2.385.786756811


{


recover

standby

clone database

delete archivelog

;

}



Starting recover at 23-JUN-12

starting media recovery

archive log thread 1 sequence 67 is already on disk as file

+DATADG/rac10g/archivelog/2012_06_23/thread_1_seq_67.540.786756371







archive log filename=+DATADG/rac10g/archivelog/2012_06_23/thread_1_seq_67.540.786756371

thread=1 sequence=0


thread=2 sequence=0


thread=1 sequence=68


thread=2 sequence=57

media recovery complete, elapsed time: 00:00:03

Finished recover at 23-JUN-12

Finished Duplicate Db at 23-JUN-12

released channel: tst

released channel: tst1

released channel: tststby

RMAN>

Alert log from the Logical standby

LOGMINER: Parameters summary for session# = 22

LOGMINER: Number of processes = 3, Transaction Chunk Size = 201

LOGMINER: Memory Size = 30M, Checkpoint interval = 150M

LOGMINER: SpillScn 5587203, ResetLogScn 388265

LOGMINER: summary for session# = 22

LOGMINER: StartScn: 0 (0x0000.00000000)

LOGMINER: EndScn: 0 (0x0000.00000000)

LOGMINER: HighConsumedScn: 5586390 (0x0000.00553dd6)

LOGMINER: session_flag: 0x1

LOGMINER: Read buffers: 16

Tue Jun 26 07:45:32 2012

LOGMINER: session#=22 (Logical_Standby$3), reader MS00 pid=98 OS id=9687 sid=127 started

Tue Jun 26 07:45:33 2012

LOGMINER: session#=22 (Logical_Standby$3), builder MS01 pid=99 OS id=9689 sid=129 started

Tue Jun 26 07:45:33 2012

LOGMINER: session#=22 (Logical_Standby$3), preparer MS02 pid=100 OS id=9691 sid=131

started

Tue Jun 26 07:46:00 2012

db_recovery_file_dest_size of 10240 MB is 7.26% used. This is a

user-specified limit on the amount of space that will be used by this

database for recovery-related files, and does not reflect the amount of

space available in the underlying filesystem or ASM diskgroup.

Tue Jun 26 07:46:24 2012

LOGMINER: Turning ON Log Auto Delete

LOGMINER: Begin mining logfile for session 22 thread 1 sequence 295,

+DATA/racstb/archivelog/2012_06_25/thread_1_seq_295.944.786886237



LOGMINER: End mining logfile for session 22 thread 1 sequence 295,


Tue Jun 26 07:46:35 2012









Tue Jun 26 07:46:37 2012

LOGSTDBY Analyzer process AS00 started with server id=0 pid=101 OS id=9826

Tue Jun 26 07:46:38 2012

LOGSTDBY Apply process AS04 started with server id=4 pid=105 OS id=9836

Tue Jun 26 07:46:38 2012


Tue Jun 26 07:46:38 2012


Tue Jun 26 07:46:38 2012


Tue Jun 26 07:46:38 2012


Tue Jun 26 07:47:03 2012

LOGMINER: thread 1 disabled

Tue Jun 26 07:47:05 2012





Tue Jun 26 07:47:55 2012

LOGMINER: thread 1 enabled

Tue Jun 26 07:47:58 2012

LOGMINER: Turning ON Log Auto Delete


















+DATA/racstb/archivelog/2012_

Logical Stdby and RAC - Zero Downtime Upgrade NO Good OCM

Documents

Transcript of Logical Stdby and RAC - Zero Downtime Upgrade NO Good OCM