Presentation 12c grid_upgrade

Basel Bern Lausanne Zurich Düsseldorf Frankfurt/M. Freiburg i. Br. Hamburg Munich Stuttgart Vienna

Upgrade 8 nodes 11GR2 cluster to 12c.

Everything going smoothly ?

Let’s see…

© 2013TechEvent 27 September 2013 2

Agenda

Target 11GR2 infrastructure

Pre-upgrade Checks and restrictions

Run CVU to validate readiness

Upgrade the GRID

Upgrade the RDBMS

Post upgrade task

Q/A

Trivadis

Tech-Event

27 Sept 2013



OS is OEL 6.4 x64

3 server pools

PMU: ACTIVE_SERVERS=node3 node5 node6 node8

LUDO: ACTIVE_SERVERS=node1 node7 (rac one node)

TST: ACTIVE_SERVERS=node2 node4 (rac one node)



NAME TARGET STATE SERVER STATE_DETAILS

-------------------------------------------------------------------

ora.ludo.db

1 ONLINE ONLINE node1 Open

ora.pmu.db



3 ONLINE OFFLINE Instance Shutdown




ora.vico.db


© 2013


TechEvent 27 September 2013 7

The apex application



Unset all environment variables

$ unset ORACLE_BASE

$ unset ORACLE_HOME

$ unset ORACLE_SID

$ unset TNS_ADMIN

$ unset ORA_NLS10

$ unset ORA_CRS_HOME (under oracle support)

Also, ensure that the $ORACLE_HOME/bin path is

removed from your PATH environment variable.



To upgrade existing Oracle Grid Infrastructure

installations from Oracle Grid Infrastructure 11g

Release 2 (11.2.0.2) to a later release, you must

apply patch 11.2.0.2.3 (11.2.0.2 PSU 3) or later.

Back Up the Oracle Software Before Upgrades!



CVU runs the appropriate system checks

automatically, and either prompts you to fix

problems, or provides a fixup script to be run on all

nodes in the cluster before proceeding with the

upgrade.

./runcluvfy.sh stage -pre crsinst -upgrade -n node1, node2,

node3, node4, node5, node6, node7, node8 -rolling -src_crshome

/oracle/grid/11.2.0.3 -dest_crshome /oracle/grid/12.1.0.0 -

dest_version 12.1.0.1.0 -fixup -verbose

dest_version must be: 12.1.0.1.0



For my cluster the test failed because of an NTP

issue. Timing between servers was ok but the time

server was not the same.

Solution for me was to set the same time server for

all nodes.

An other option would have been to disable NTP

during the upgrade phase and let CTSS run in

active mode.

I did not report other issues for the check.



NTP common Time Server Check started...

PRVF-5408 : NTP Time Server "94.23.228.130" is common only to the following nodes "node4"

PRVF-5408 : NTP Time Server "149.20.64.28" is common only to the following nodes "node8,node5"


PRVF-5408 : NTP Time Server "145.238.203.14" is common only to the following nodes

"node3,node8,node7,node6"










PRVF-5416 : Query of NTP daemon failed on all nodes


Upgrade the GRID


Upgrade the GRID

Run ./runInstaller


Upgrade the GRID


Upgrade the GRID

Initially I have decide to just upgrade without new functionalities


Upgrade the GRID

But to use some of them, you have to reinstall the grid


Upgrade the GRID

So I choose YES


Upgrade the GRID


Upgrade the GRID

Cool feature to group nodes to upgrade


Upgrade the GRID


Upgrade the GRID

11gR2 alertNode1.log

[ctssd(3025)]CRS-2405:The Cluster Time Synchronization Service on host node1 is shutdown by user

12c alertNode1.log

[client(28956)]CRS-1808:Node-specific configuration for node node1 in Oracle Local Registry was

upgraded to version 12.1.0.1.0.

[ohasd(29129)]CRS-2112:The OLR service started on node node1.

[ohasd(29129)]CRS-1301:Oracle High Availability Service started on node node1.

...

[crsd(9720)]CRS-1012:The OCR service started on node node1.

2013-09-06 21:22:19.214:

[crsd(9720)]CRS-1201:CRSD started on node node1.

2013-09-06 21:22:44.713:

[client(9981)]CRS-1804:Node node1 upgraded to version 12.1.0.1.0.


Upgrade the GRID

Databases are automaticaly relocated to other nodes during the upgrade

thanks to policy managed


-------------------------------------------------------------------

ora.ludo.db


ora.pmu.db




4 ONLINE OFFLINE Instance Shutdown,STARTING

5 ONLINE OFFLINE Instance Shutdown,STARTING


ora.vico.db



Upgrade the GRID


Upgrade the GRID

Oups, what the hell going on!! Let’s see the logs


Upgrade the GRID

Log says: +GRID does not have enough space. Required space is 3358 MB ,

available space is 161 MB!

I would have appreciated a message during prerequisite checks

Actions:

1. Added more disks to the GRID diskgroup

2. Retried the configuration

3. Failed again as the available space is not updated

1. So I skipped the configuration and close the installer

2. Checked the log for the command that needs to be run

3. Run the command(s) manually


Upgrade the GRID

-bash-4.1$ /oracle/grid/12.1.0.0/bin/dbca -silent -createDatabase -templateName

MGMTSeed_Database.dbc -sid -MGMTDB -gdbName _mgmtdb -storageType ASM -

diskGroupName GRID -datafileJarLocation /oracle/grid/12.1.0.0/assistants/dbca/templates -

characterset AL32UTF8 -autoGeneratePasswords -oui_internal

Copying database files

1% complete

..

35% complete

Creating and starting Oracle instance

37% complete

..

58% complete

Registering database with Oracle Grid Infrastructure

64% complete

Completing Database Creation

68% complete

..

100% complete

Look at the log file "/oracle/gridbase/cfgtoollogs/dbca/_mgmtdb/_mgmtdb.log" for further details.


Upgrade the GRID

-bash-4.1$ /oracle/grid/12.1.0.0/oui/bin/runConfig.sh ORACLE_HOME=/oracle/grid/12.1.0.0

MODE=perform ACTION=configure RERUN=false $*

Setting the invPtrLoc to /oracle/grid/12.1.0.0/oraInst.loc

perform - mode is starting for action: configure

perform - mode finished for action: configure

You can see the log file: /oracle/grid/12.1.0.0/cfgtoollogs/oui/configActions2013-09-07_01-35-58-PM.log

Warning: if you do not set ORACLE_SID=-MGMTDB the process will create a new database!

This manual installation step should normally be avoided if you make sure you have 5Gb free

in your GRID diskgroup.


Upgrade the GRID

After the upgrade process some listeners failed to start and the post

upgrade check script failed of 4 nodes on the VIP section.

If reboot the nodes one by one and everything was ok after that.

Post-check for cluster services setup was successful.

The plug-in Oracle Cluster Verification Utility has successfully been performed

The action configuration has successfully completed


Upgrade the GRID

At the end

You should see something like this in the alertlog!

[crsd(9720)]CRS-1001:The OCR was formatted using version 4.

[cssd(9016)]CRS-1626:A Configuration change request completed successfully

[cssd(9016)]CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2

node3 node4 node5 node6 node7 node8 .

[crsd(9720)]CRS-1005:The Oracle Clusterware upgrade was completed. Version has

changed from [186647296, 11.2.0.3.0] to [202375424, 12.1.0.1.0].

Details in /oracle/grid/12.1.0.0/log/node1/crsd/crsd.log.


Upgrade the GRID


-------------------------------------------------------------------

ora.ludo.db

1 ONLINE ONLINE node5 Open,STABLE

ora.mgmtdb


ora.pmu.db

1 ONLINE OFFLINE Instance Shutdown,STABLE






ora.vico.db


Note: I have started to reboot from node8. The first node that is ok get the

mgmtdb management database.


Upgrade the RDBMS


Upgrade the RDBMS

CVU runs the appropriate system checks automatically in

the RAC before proceeding with the upgrade.

Before running CVU, make sure the dbsnmp account is

unlocked and the password is set. You can specify and other

account.

Run CVU:/oracle/db/12.1.0.0/bin/cluvfy stage -pre dbinst -

upgrade -src_dbhome /oracle/db/11.2.0.3 -dest_dbhome

/oracle/db/12.1.0.0 -dest_version 12.1.0.0.0 –verbose

…

ASM and CRS versions are compatible

Database version "12.1.0.0.0" is compatible with the

Clusterware version "12.1.0.1.0".

Pre-check for database installation was successful.


Upgrade the RDBMS

Run the installer and install “software only” across all nodes


Upgrade the RDBMS

Would have preferred to be prompt for root credentials…


Upgrade the RDBMS

To upgrade a database, you must run the upgrade assistant

from a node where the database is running

Is Rolling Upgrade possible?

YES but using:

Using Data Guard SQL Apply

Using Physical Standby Databases and Transient Logical

Standby


Upgrade the RDBMS

Run dbua from the node where the target db is running


Upgrade the RDBMS


Upgrade the RDBMS

Script can be generated to fix up upgrade prerequisites


Upgrade the RDBMS

Adjust some settings, the parallelism one is cool


Upgrade the RDBMS

Check the summary…


Upgrade the RDBMS

Strange error as the oracle user has the right to do it.


Upgrade the RDBMS

You can then check the Upgrade Result


Upgrade the RDBMS

Important

It’s not possible to convert a 11gR2 database to a container

database.

To use this feature you have:

Create a 12c container database

Upgrade your 11gR2 databases to 12c

Plug your upgraded 12c databases in the container database

© 2013

Upgrade the RDBMS


-------------------------------------------------------------------

ora.ludo.db


ora.mgmtdb


ora.pmu.db







ora.vico.db




Post upgrade tasks


Post upgrade tasks

Remove Grid 11gR2 home

You must first change the permission and ownership of the

previous release Grid home

#chmod -R 755 /oracle/grid/11.2.0.3

#chown -R grid /oracle/grid/11.2.0.3

Use the Oracle Grid Infrastructure 11g standalone deinstallation

tool to remove the previous release Grid home.

And not the 12c one as stated in the documentation

Download it from here: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html


Post upgrade tasks

./deinstall -home /oracle/grid/11.2.0.3

####################### CHECK OPERATION SUMMARY #######################

Oracle Grid Infrastructure Home is: /oracle/grid/12.1.0.0

The cluster node(s) on which the Oracle home deinstallation will be performed

are:node1,node2,node3,node4,node5,node6,node7,node8

Oracle Home selected for deinstall is: /oracle/grid/11.2.0.3

Inventory Location where the Oracle home registered is: /oracle/oraInventory

The home being deconfigured is NOT a configured Grid Infrastructure home (/oracle/grid/12.1.0.0)

ASM was not detected in the Oracle Home

Do you want to continue (y - yes, n - no)? [n]: Y

…

####################### CLEAN OPERATION SUMMARY #######################

The home being deconfigured is NOT a configured Grid Infrastructure home (/oracle/grid/12.1.0.0)

Successfully detached Oracle home '/oracle/grid/11.2.0.3' from the central inventory on the local node.

Successfully deleted directory '/oracle/grid/11.2.0.3' on the local node.

Successfully detached Oracle home '/oracle/grid/11.2.0.3' from the central inventory on the remote nodes

'node5,node2,node3,node7,node6,node4,node8'.

Successfully deleted directory '/oracle/grid/11.2.0.3' on the remote nodes


Oracle Universal Installer cleanup completed Successfully.

Oracle deinstall tool successfully cleaned up temporary directories.

#######################################################################


Post upgrade tasks

Remove RDBMS11gR2 home

After having completed all 11gR2 database upgrades

Use the Oracle Grid Infrastructure 11g standalone deinstallation

tool to remove the previous release Grid home.

And not the 12c one as stated in the documentation

Download it from here: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html


Post upgrade tasks

./deinstall -home /oracle/db/11.2.0.3

####################### CHECK OPERATION SUMMARY #######################

Oracle Grid Infrastructure Home is: /oracle/grid/12.1.0.0

The cluster node(s) on which the Oracle home deinstallation will be performed

are:node1,node2,node3,node4,node5,node6,node7,node8

Oracle Home selected for deinstall is: /oracle/db/11.2.0.3

Inventory Location where the Oracle home registered is: /oracle/oraInventory

No Enterprise Manager configuration to be updated for any database(s)

No Enterprise Manager ASM targets to update

No Enterprise Manager listener targets to migrate

Checking the config status for CCR

node1 : Oracle Home exists with CCR directory, but CCR is not configured








CCR check is finished

Do you want to continue (y - yes, n - no)? [n]: YES


Post upgrade tasks

######################### CLEAN OPERATION END #########################

####################### CLEAN OPERATION SUMMARY #######################

Cleaning the config for CCR

As CCR is not configured, so skipping the cleaning of CCR configuration

CCR clean is finished

Successfully detached Oracle home '/oracle/db/11.2.0.3' from the central inventory on the local

node.

Successfully deleted directory '/oracle/db/11.2.0.3' on the local node.

Successfully detached Oracle home '/oracle/db/11.2.0.3' from the central inventory on the remote

nodes 'node5,node2,node3,node7,node6,node4,node8'.

Successfully deleted directory '/oracle/db/11.2.0.3' on the remote nodes


Oracle Universal Installer cleanup was successful.

Oracle deinstall tool successfully cleaned up temporary directories.

#######################################################################

############# ORACLE DEINSTALL & DECONFIG TOOL END #############


Post upgrade tasks

Upgrade ASM DiskGroup attributes to 12,1,0,0

BUT, think about compatible parameter of the database!

You will not be able to start your database if you do not update

the parameter to “12.1.0.0.0” before changing ASM DiskGroup

attributes.


Post upgrade tasks

After the upgrade, think to update

Documentation,

Procedure,

DRP scenarios,

…

Welcome to 12c!


Thank you very much for your attention

?

Presentation 12c grid_upgrade

Technology

Transcript of Presentation 12c grid_upgrade