Oracle RAC

123
Oracle RAC Oracle Real Application Clusters (RAC) Installation Installation steps for Oracle RAC on Windows 2003 Enterprise x64 Edition. Clusterware Oracle Clusterware Installation Clusterware Pre-Installation Networking Each node in the cluster must have two network interfaces. One interface, called public is used by clients to connect to the cluster. The other interface, called private is used for cluster network traffic only between nodes. Note, the names of the network interfaces must be exactly the same on each node in the cluster and have the same case.

Transcript of Oracle RAC

Page 1: Oracle RAC

Oracle RACOracle Real Application Clusters (RAC)

InstallationInstallation steps for Oracle RAC on Windows 2003 Enterprise x64 Edition.

ClusterwareOracle Clusterware Installation

Clusterware Pre-Installation NetworkingEach node in the cluster must have two network interfaces. One interface, called public is used by clients to connect to the cluster. The other interface, called private is used for cluster network traffic only between nodes.

Note, the names of the network interfaces must be exactly the same on each node in the cluster and have the same case.

For the private interfaces we will use the subnet 10.1.1.0, using 10.1.1.180 for rac1 and 10.1.1.182 for rac2.

For the public interfaces we will use the subnet 192.168.1.0 using 192.168.1.180 for rac1 and 192.168.1.182 for rac2.

Page 2: Oracle RAC

In addition, Clusterware will assign a "virtual" IP address to each node in the cluster. These IP addresses must not be defined in the Windows Network Settings. Clusterware takes case of the VIP assignment.

For the virtual ips we will use 192.168.1.181 for rac1 and 192.168.1.183 for rac2.

To configure the public interface. right click on the interface and select Properties.

.Select "Internet Protocol (TCP/IP)" and click the Properties button.

Page 3: Oracle RAC

Note, change the default gateway and DNS server to appropriate values for your network.

Page 4: Oracle RAC

Repeat this process to configure the private interface.

The public interface must also be listed first in the network binding order. This can be done by clicking Advanced Settings from the Advanced menu of the Network Connections window.

Page 5: Oracle RAC

If the public interface is not listed first, use the "up arrow" button to move it to the top of the connection list.

Finally, Media Sensing needs to be disabled for networking on the nodes. Media Sensing is a feature used to detect if network interfaces are in a link state. When a link goes down, Windows will unbind protocols from the interface, this can cause problems with Clusterware and must be disable.

Using the Registry Editor (Start->Run "regedit").

Add a new DWORD value in HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters called DisableDHCPMediaSense and set it's value to 1.

Page 6: Oracle RAC

Clusterware Pre-Installation StorageIn this example, I have defined 3 persistent disks shared between my two guests: 2 x 10GB and a 40GB disk. The intent is to use the 10GB disks for the OCR and Voting disks and the 40GB disk for data and flash partitions.

Using the Disk Management interface in Windows, right click on the 10GB disk and select New Partition.Click Next.

Page 7: Oracle RAC

Select Extended partition and click Next.

Use the entire disk for the partition and click Next.

Page 8: Oracle RAC

Click Finish to create the partition.

Now, we need to create a new logical disk. Right click on the newly created partition and select New Logical Disk. Click Next.

Page 9: Oracle RAC

Select Logical drive and click Next.

Use the entire disk for the partition and click Next.

Page 10: Oracle RAC

Select "Do not assign a drive letter or drive path" and click Next.

Select "Do not format this partition" and click Next.

Page 11: Oracle RAC

Click Finish to create the partition.

Repeat this process for the second 10GB disk.

Page 12: Oracle RAC

For the 40GB, following the steps above, create an extended partition occupying the entire disk, but create 2 logical partitions of 35GB and 5GB.

Each node may need to be restarted before they can see the partitions on the shared disks. After a restart, Windows may assign a drive letter to the partitions. These drive letters should be removed through the Disk Management interface before proceeding.

Right click on the volume and select "Change Driver Letter and Paths...".

Highlight the drive letter and click Remove.Click the Yes button to remove the letter.

Clusterware InstallDownload Oracle Clusterware Release 2 (10.2.0.1.0) from http://download.oracle.com/otn/nt/oracle10g/10201/102010_win64_x64_clust...

Unzip the software to a temporary staging area that will be called $STAGING_DIR from this point onwards.

Included as part of the installation software is the Cluster Verification Utility (CVU). This utility can be used to verify the cluster before and after installation. Before the cluster software has been installed, this utility is available in the $STAGING_DIR\10210_win64_x64_clusterware\clusterware\cluvfy directory.

Page 13: Oracle RAC

Execute the following from a command prompt:> runcluvfy.bat stage -pre crsinst -n rac1,rac2 -verbose

The tool should generate the following output:

Performing pre-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "rac1"  Destination Node                      Reachable?                ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Node reachability check passed from node "rac1".

Checking user equivalence...

Check: User equivalence for user "Administrator"  Node Name                             Comment                   ------------------------------------  ------------------------  rac2                                  passed                    rac1                                  passed                  Result: User equivalence check passed for user "Administrator".Checking administrative privileges...

Checking node connectivity...

Interface information for node "rac2"  Interface Name                  IP Address                      Subnet            ------------------------------  ------------------------------  ----------------  public                          192.168.1.183                   192.168.1.0       public                          192.168.1.182                   192.168.1.0       private                         10.1.1.182                      10.1.1.0       

Interface information for node "rac1"  Interface Name                  IP Address                      Subnet            ------------------------------  ------------------------------  ----------------  public                          192.168.1.181                   192.168.1.0       public                          192.168.1.180                   192.168.1.0       private                         10.1.1.180                      10.1.1.0       

Page 14: Oracle RAC

Check: Node connectivity of subnet "192.168.1.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  rac2:public                     rac2:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac1:public                     rac1:public                     yes             Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) rac2,rac1.

Check: Node connectivity of subnet "10.1.1.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  rac2:private                    rac1:private                    yes             Result: Node connectivity check passed for subnet "10.1.1.0" with node(s) rac2,rac1.

Suitable interfaces for the private interconnect on subnet "192.168.1.0":rac2 public:192.168.1.183 public:192.168.1.182rac1 public:192.168.1.181 public:192.168.1.180

Suitable interfaces for the private interconnect on subnet "10.1.1.0":rac2 private:10.1.1.182rac1 private:10.1.1.180

ERROR: Could not find a suitable set of interfaces for VIPs.

Result: Node connectivity check failed.

Checking system requirements for 'crs'...

Check: Operating system version   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          Windows Server 2003       Windows Server 2003       passed      rac1          Windows Server 2003       Windows Server 2003       passed    Result: Operating system version check passed.

Check: Total memory   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------

Page 15: Oracle RAC

  rac2          1023.38MB (1047944KB)     512MB (524288KB)          passed      rac1          1023.38MB (1047944KB)     512MB (524288KB)          passed    Result: Total memory check passed.

Check: Swap space   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          2.43GB (2550756KB)        1GB (1048576KB)           passed      rac1          2.43GB (2550756KB)        1GB (1048576KB)           passed    Result: Swap space check passed.

Check: System architecture   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          64-bit                    64-bit                    passed      rac1          64-bit                    64-bit                    passed    Result: System architecture check passed.

Check: Free disk space in "C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp" dir  Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          6.25GB (6551952KB)        400MB (409600KB)          passed      rac1          5.79GB (6066360KB)        400MB (409600KB)          passed    Result: Free disk space check passed.

System requirement passed for 'crs'

Pre-check for cluster services setup was unsuccessful on all the nodes.

Note, the tool will fail with the error "Could not find a suitable set of interfaces for VIPs.". This is normal as we have used a private subnet (10.1.1.0) for the VIP interfaces.

If no other errors have occurred, now run the setup.exe found in $STAGING_DIR\10210_win64_x64_clusterware\clusterware.

Page 16: Oracle RAC

Click Next.

Page 17: Oracle RAC

Enter the name and path for the cluster home and click Next.

Page 18: Oracle RAC

The installer will now run prerequisite checks. Click Next.

Enter a name for the cluster. We also need to add our second node. Click Add and enter in the information for rac2.

Page 19: Oracle RAC

Click Next.

Page 20: Oracle RAC

The interface type needs to be changed to "Public" for our public interface. Click Edit.

Page 21: Oracle RAC

Click Next when done.

Page 22: Oracle RAC

Now we need to select partitions that will be used as the Oracle Cluster Registry (OCR) and the Voting disk. Highlight the partition to be used for the OCR disk, click Edit and mark it.

Now, highlight the second partition and mark it as the Voting disk.

Page 23: Oracle RAC

Click Next when done.

A warning dialog will pop up recommending that an OCR mirror disk be created for redundancy. Click OK.

Page 24: Oracle RAC

A warning dialog will pop up recommending that an Voting mirror disk be created for redundancy. Click OK.

For the purposes of this example, we do not need any redundancy as the install is in a virtual machine. Click Next to continue.

The summary screen should now be displayed.

Page 25: Oracle RAC

Click Install to begin the installation.

Page 26: Oracle RAC

Once the installation has completed, the configuration assistants will run.

The Virtual Private IP Configuration Assistant will fail for the same reason that the CVU utility failed due to the private subnet being used. Click OK to continue

Page 27: Oracle RAC

Another warning dialog will pop up. Click OK to close it.

Click Exit to close the installer.

Page 28: Oracle RAC

We now need to the run VIP Configuration Assistant manually since it failed during the install. From the $CLUSTER_HOME\bin directory, run vipca.bat and click Next.

Page 29: Oracle RAC

Select the public interface and click Next.

Page 30: Oracle RAC

Now enter in the hostnames for the VIP interfaces for each node. If your hosts file or DNS is setup correctly, the IP Address column should fill in automatically. Click Next when done.

Page 31: Oracle RAC

A summary screen should now be shown. Click Finish.

The assistant should now run. Click OK when it is finished.

Page 32: Oracle RAC

A dialog will now be display showing the configuration results. Click Exit to close the assistant.

Clusterware should now be installed and running on both nodes. To verify the installation, run the CVU utility from $CLUSTER_HOME\bin

> cluvfy stage -post crsinst -n rac1,rac2, -verbose

Performing post-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "rac1"  Destination Node                      Reachable?             

Page 33: Oracle RAC

  ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Node reachability check passed from node "rac1".

Checking user equivalence...

Check: User equivalence for user "Administrator"  Node Name                             Comment                   ------------------------------------  ------------------------  rac2                                  passed                    rac1                                  passed                  Result: User equivalence check passed for user "Administrator".

Checking Cluster manager integrity...

Checking CSS daemon...  Node Name                             Status                    ------------------------------------  ------------------------  rac2                                  running                   rac1                                  running                 Result: Daemon status check passed for "CSS daemon".

Cluster manager integrity check passed.

Checking cluster integrity...

  Node Name                             ------------------------------------  rac1rac2                          

Cluster integrity check passed

Checking OCR integrity...

Checking the absence of a non-clustered configuration...All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...OCR of correct Version "2" exists.

Checking data integrity of OCR...Data integrity check for OCR passed.

OCR integrity check passed.

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"

Page 34: Oracle RAC

  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons  Node Name     CRS daemon                CSS daemon                EVM daemon  ------------  ------------------------  ------------------------  ----------  rac2          yes                       yes                       yes         rac1          yes                       yes                       yes      

Checking CRS health...

Check: Health of CRS  Node Name                             CRS OK?                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          yes                       exists                    passed      rac1          yes                       exists                    passed    Result: Check passed.

Checking existence of ONS node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          no                        exists                    passed      rac1          no                        exists                    passed    Result: Check passed.

Page 35: Oracle RAC

Checking existence of GSD node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          no                        exists                    passed      rac1          no                        exists                    passed    Result: Check passed.

Post-check for cluster services setup was successful.

Clusterware Upgrade from 10.2.0.1 to 10.2.0.3Before we install anymore software, the Clusterware install should be upgraded from 10.2.0.1 to 10.2.0.3.

The first thing to do is to download the patch 5337014 from Metalink.

Unzip the patch to a temporary directory on any of the two nodes and run setup.exe and click Next.

Page 36: Oracle RAC

Select the Clusterware home and click Next.

Page 37: Oracle RAC

Cluster installation should already be selected at this point. Click Next to continue.

Page 38: Oracle RAC

The summary screen should be displayed at this point. Click Next to continue.

Page 39: Oracle RAC

At this point the patch software will be deployed to all the nodes in the cluster.

Page 40: Oracle RAC

The patch software should be now be deployed to each node. Click Exit to close the installer.

Page 41: Oracle RAC

At this point, the patch has not been installed. It has only be staged to each node in the cluster. We now need to apply the patch on a rolling basis to each node. NOTE: the following steps need to be repeated for each node in the cluster.

The first thing we need to do is to stop all the Oracle services. From a command prompt, execute the following commands:

> net stop OracleCRService> net stop OracleEVMService> net stop OracleCSService> net stop "Oracle Object Service"> net stop OracleClusterVolumeService

To verify that all Oracle services have stopped, there should be no output from the following command:

> net start | findstr Oracle

It is also a good idea to stop the Microsoft Distributed Transaction Coordinator, as it may have loaded some Oracle libraries that need to be patched.

Page 42: Oracle RAC

> net stop MSDTC

Now, go to the $CLUSTER_HOME\install directory and run the patch installer:

> patch102.bat

The command may fail with the following error message:

Failed to patch all the files in the ORA_CRS_HOME C:\oracle\product\10.2.0\crsPlease ensure that all processes running from the home have beenstopped.

If the MSDTC has already been stopped, this is most likely due to two ons.exe processes that are still running. Using the Windows Task Manager to kill these two processes.

Now execute the command again and if successful it should display the following message:

Successful upgrade of this node to Oracle Cluster Ready Services 10203

Automatic Storage Management (ASM)Oracle Automatic Storage Management (ASM) Installation

Page 43: Oracle RAC

Automatic Storage Management (ASM) InstallationOracle ASM is a new feature of Oracle 10g used for storage management which is consistent across all server and storage platforms. ASM provides many advantages, including:

adding additional disks requires no downtime I/O distribution filesystem is not buffered allowing for direct I/O kernelized asynchronous I/O

Run the cluster verification cluvfy to verify the cluster is ready for a database installation by executing the following command:

d:\oracle\product\10.2.0\crs\bin> cluvfy stage –pre dbinst –n all –r 10gR2

The following output should be generated:Performing pre-checks for database installation

Checking node reachability...

Check: Node reachability from node "rac1"  Destination Node                      Reachable?                ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Node reachability check passed from node "rac1".

Checking user equivalence...

Check: User equivalence for user "Administrator"  Node Name                             Comment                   ------------------------------------  ------------------------  rac2                                  passed                    rac1                                  passed                  Result: User equivalence check passed for user "Administrator".Checking administrative privileges...

Checking node connectivity...

Interface information for node "rac2"  Interface Name                  IP Address                      Subnet            ------------------------------  ------------------------------  ----------------  public                          192.168.1.183                   192.168.1.0       public                          192.168.1.182                   192.168.1.0    

Page 44: Oracle RAC

  private                         10.1.1.182                      10.1.1.0       

Interface information for node "rac1"  Interface Name                  IP Address                      Subnet            ------------------------------  ------------------------------  ----------------  public                          192.168.1.181                   192.168.1.0       public                          192.168.1.180                   192.168.1.0       private                         10.1.1.180                      10.1.1.0       

Check: Node connectivity of subnet "192.168.1.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  rac2:public                     rac2:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac2:public                     rac1:public                     yes               rac1:public                     rac1:public                     yes             Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) rac2,rac1.

Check: Node connectivity of subnet "10.1.1.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  rac2:private                    rac1:private                    yes             Result: Node connectivity check passed for subnet "10.1.1.0" with node(s) rac2,rac1.

Suitable interfaces for the private interconnect on subnet "192.168.1.0":rac2 public:192.168.1.183 public:192.168.1.182rac1 public:192.168.1.181 public:192.168.1.180

Suitable interfaces for the private interconnect on subnet "10.1.1.0":rac2 private:10.1.1.182rac1 private:10.1.1.180

ERROR: Could not find a suitable set of interfaces for VIPs.

Page 45: Oracle RAC

Result: Node connectivity check failed.

Checking system requirements for 'database'...

Check: Operating system version   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          Windows Server 2003       Windows Server 2003       passed      rac1          Windows Server 2003       Windows Server 2003       passed    Result: Operating system version check passed.

Check: Total memory   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          1023.38MB (1047944KB)     512MB (524288KB)          passed      rac1          1023.38MB (1047944KB)     512MB (524288KB)          passed    Result: Total memory check passed.

Check: Swap space   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          2.43GB (2550756KB)        1GB (1048576KB)           passed      rac1          2.43GB (2550756KB)        1GB (1048576KB)           passed    Result: Swap space check passed.

Check: System architecture   Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          64-bit                    64-bit                    passed      rac1          64-bit                    64-bit                    passed    Result: System architecture check passed.

Check: Free disk space in "C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp" dir  Node Name     Available                 Required                  Comment     ------------  ------------------------  ------------------------  ----------  rac2          6.1GB (6399648KB)         400MB (409600KB)          passed      rac1          5.02GB (5263824KB)        400MB (409600KB)          passed    Result: Free disk space check passed.

System requirement passed for 'database'

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"

Page 46: Oracle RAC

  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"  Node Name                             Running                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons  Node Name     CRS daemon                CSS daemon                EVM daemon  ------------  ------------------------  ------------------------  ----------  rac2          yes                       yes                       yes         rac1          yes                       yes                       yes      

Checking CRS health...

Check: Health of CRS  Node Name                             CRS OK?                   ------------------------------------  ------------------------  rac2                                  yes                       rac1                                  yes                     Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          yes                       exists                    passed      rac1          yes                       exists                    passed    Result: Check passed.

Checking existence of ONS node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          no                        exists                    passed      rac1          no                        exists                    passed    Result: Check passed.

Checking existence of GSD node application   Node Name     Required                  Status                    Comment     ------------  ------------------------  ------------------------  ----------  rac2          no                        exists                    passed      rac1          no                        exists                    passed    Result: Check passed.

Page 47: Oracle RAC

Pre-check for database installation was unsuccessful on all the nodes.

Once again the tool will report a failure due to private subnet being using for the virtual IPs. If this is the only error, we can proceed with the installation.

A database home must be installed to host the ASM instance.

Download Oracle 10g Release 2 (10.2.0.1.0) x64 for Windows fromhere and extract it to a temporary directory. Run setup.exe.

Click Next.

Select Enterprise Edition and click Next.

Page 48: Oracle RAC

Specify a name and location for the ASM home and click Next. Note, this home must be not be the same as the cluster home

Page 49: Oracle RAC

Select Cluster Installation and select all the nodes in the cluster. Click Next to continue.

Page 50: Oracle RAC

The installer will now perform prerequisite checks. Click Next.

Page 51: Oracle RAC

Select Install database Software only and click Next. We will patch the ASM installation before the instance is created.

Page 52: Oracle RAC

Review the installation summary. Click Install to continue.

Page 53: Oracle RAC

The installer will now deploy create the ASM on each node in the cluster.

Page 54: Oracle RAC
Page 55: Oracle RAC

Once the install has completed, click Exit to close the installer.

ASM Upgrade from 10.2.0.1 to 10.2.0.3ASM should be upgraded to patch level 2 (10.2.0.3) before the instance is created.

The first thing to do is to download the patch 5337014 from Metalink.

Unzip the patch to a temporary directory on any of the two nodes and run setup.exe and click Next.

Page 56: Oracle RAC

Select the ASM home and click Next.

Page 57: Oracle RAC

Cluster installation should already be selected. Click Next to continue.

Page 58: Oracle RAC

The installation summary should now be displayed. Click Install to continue.

Page 59: Oracle RAC

The patch will now be staged to each node in the cluster.

Page 60: Oracle RAC

Configuration assistants will be run to apply the patch to each node.

Page 61: Oracle RAC
Page 62: Oracle RAC

The patch should now be installed. Click Exit to close the installer.

RAC Listener ConfigurationFrom the $ASM_HOME\bin directory, run the Oracle Net Configuration Assistant:

c:\oracle\product\10.2.0\asm_1\bin> netca.bat

Select Cluster configuration and click Next.

Page 63: Oracle RAC

Select all the nodes in the cluster and click Next.

Choose Listener configuration and click Next.

Page 64: Oracle RAC

Select Add and click Next.

Enter a name for the listener and click Next.

Page 65: Oracle RAC

Select the TCP protocol and click Next.

Select the default port number of 1521 and click Next.

Page 66: Oracle RAC

The listener is now configured. Click Next to close the assistant.

Page 67: Oracle RAC

ASM Instance CreationAfter patching the software inside the $ASM_HOME, we are now ready to create the instance. The first thing we need to do is stamp the partitions that we would like ASM to manage.

From the $ASM_HOME\bin directory, run the asmtool utility:

c:\oracle\product\10.2.0\asm_1\bin> asmtoolg.exe

Select Add or change label and click Next.

Select the 35GB partition and enter "DATA" as the prefix to stamp this partition with. Click Next.

Page 68: Oracle RAC

Click Next.

Page 69: Oracle RAC

Click Finish to close the tool.

We now need to repeat the above steps to specify our partition to be used for flash recovery.From the $ASM_HOME\bin directory, run the asmtool utility:

c:\oracle\product\10.2.0\asm_1\bin> asmtoolg.exe

Select Add or change label and click Next.

Page 70: Oracle RAC

Select the 35GB partition and enter "FLASH" as the prefix to stamp this partition with. Click Next.

Click Next.

Page 71: Oracle RAC

Click Finish to close the tool.

Now from the $ASM_HOME\bin directory, start the Database Configuration Assistant:

Page 72: Oracle RAC

c:\oracle\product\10.2.0\asm_1\bin\dbca.bat

Select Oracle Real Application Clusters database and click Next.

Select Configure Automatic Storage Management and click Next.

Page 73: Oracle RAC

Make sure all the nodes in the cluster are selected and click Next.

Page 74: Oracle RAC

Enter a password for the SYS account and select the Create initialization parameter file (IFILE) option, then click Next.

Page 75: Oracle RAC

Click OK to start the instance creation.

It will take a few minutes to complete the instance creation.

We now need to configure the disk groups. Click Create New.

Page 76: Oracle RAC

To create the DATA disk, select the disk that was stamped as DATA earlier by the asmtool utility, mark it as External and click OK.

Page 77: Oracle RAC

Repeat this for the FLASH disk. Click Create New. select the disk stamped as FLASH, mark it as External and click OK.

Page 78: Oracle RAC
Page 79: Oracle RAC

We should now have two disks configured. Click Finish to close the assistant.

Database InstallationGuide to installing and patching an Oracle database in a Clusterware environment.

Database Home InstallationDownload Oracle 10g Release 2 (10.2.0.1.0) x64 for Windows fromhere and extract it to a temporary directory. Run setup.exe.

Page 80: Oracle RAC

Click Next.

Select Enterprise Edition and click Next.

Page 81: Oracle RAC

Specify a home for the RAC database. This home must be different than both the ASM and Clusterware homes from our previous installs. Click Next to continue.

Page 82: Oracle RAC

Select Cluster Installation and ensure that all the nodes are selected. Click Next.

Page 83: Oracle RAC

Click Next once all the prerequisite checks have been completed.

Page 84: Oracle RAC

Select No and click Next.

Page 85: Oracle RAC

Select Install database Software only and click Next.

Page 86: Oracle RAC

Review summary and click Install.

Page 87: Oracle RAC

The installer will now install the Oracle home.

Page 88: Oracle RAC
Page 89: Oracle RAC

Click Exit to close the installer.

Database Home Upgrade from 10.2.0.1 to 10.2.0.3The database software should be upgraded to patch level 2 (10.2.0.3) before the instance is created.

The first thing to do is to download the patch 5337014 from Metalink.

Unzip the patch to a temporary directory on any of the two nodes and run setup.exe and click Next.

Page 90: Oracle RAC

Select the RAC home from our previous install and click Next.

Page 91: Oracle RAC

Cluster installation mode should already be selected. Click Next to continue.

Page 92: Oracle RAC

Review the summary before installing. Click Install to continue.

Page 93: Oracle RAC

The patch will now be staged to each node in the cluster.

Page 94: Oracle RAC

The patch will now be applied to each node.

Page 95: Oracle RAC
Page 96: Oracle RAC

Click Exit to close the installer.

Database Instance CreationFrom the Oracle database home bin directory, run the Database Configuration Assistant.

c:\oracle\product\10.2.0\db_1\bin> dbca.bat

Page 97: Oracle RAC

Select Oracle Real Application Clusters database and click Next.

Select Create a Database and click Next.

Page 98: Oracle RAC

Make sure every node in the cluster is selected and click Next.

Page 99: Oracle RAC

Select the General Purpose template and click Next. Note, at this point if you have specific requirements for your database such as a different character set, Custom Database may be a better choice.

Page 100: Oracle RAC

Enter a global database name and a SID. Click Next to continue.

Page 101: Oracle RAC

Check the Configure the Database with Enterprise Manager checkbox (it should be checked by default) and click Next.

Page 102: Oracle RAC

Enter a password for the SYS account and click Next.

Page 103: Oracle RAC

Select Automatic Storage Management (ASM) and click Next.

Page 104: Oracle RAC

Select the disk group that we stamped as DATA and click Next.

Page 105: Oracle RAC

Click Next.

Page 106: Oracle RAC

Use the Browse button to find the disk group we stamped as FLASH and select it for the flash recovery area. Click Next to continue.

Page 107: Oracle RAC

Click Next.

Page 108: Oracle RAC

Do not specify any services. Click Next.

Page 109: Oracle RAC

Click Next.

Page 110: Oracle RAC

Tablespaces can be defined at this point or later through the Enterprise Manager. Click Next to continue.

Page 111: Oracle RAC

Click Finish.

Page 112: Oracle RAC

A summary of the selected options is shown at this point. Click OK to create the database.

Page 113: Oracle RAC

This will take a few minutes.

Page 114: Oracle RAC

The instance is now created. Click Exit to close the assistant.

The instance will now be started on each node.

Page 115: Oracle RAC

Using the SYS account, we can now log into the Oracle Enterprise Manager using a browser. A shortcut to the proper URL should be in the Start menu, in this case it is http://rac1:1158.

Re-installing Oracle RAC After Failed Installation

Page 116: Oracle RAC

I encountered a situation where I needed to re-install an Oracle RAC installation.

The two nodes in my cluster were fresh installs of Win2003 R2 Enterprise x64 Edition. In this case I needed to reuse the shared disk between the two nodes.

Running the cluster verification utility (runcluvfy.bat), my two machines passed cluster pre-install check:

> runcluvfy.bat stage -pre crsinst -n rac1,rac2 -verbose.

The problem happened during the actual install itself during Storage Configuration stage. The two partitions that I had previously selected as my OCR and Voting disks were not listed.

Checking the installer logs (these are normally found in C:\Program Files\Oracle\Inventory\logs and will be called installActionsYYYY-MM-DD_hh-mm-ssAMPM.log), I found the following:

nodeList = rac1,rac2INFO: Query Returned: B:,E:,F:,G:,H:,I:,J:,K:,L:,M:,N:,O:,P:,Q:,R:,S:,T:,U:,V:,W:,X:,Y:,Z:INFO: Setting variable 'sl_Drives' to 'B:,E:,F:,G:,H:,I:,J:,K:,L:,M:,N:,O:,P:,Q:,R:,S:,T:,U:,V:,W:,X:,Y:,Z:'. Received the value from a code block.INFO: Calling Query ClusterPreinstQueries1.2.1  getPartitionInfoINFO: Query Returned: ocrcfg,1,1,10229,votedsk1,2,1,10229,,3,1,35001,,3,2,5946INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3,2'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3,2,5946'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3,2,5946,'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3,2,5946,,'. Received the value from a code block.INFO: Setting variable 'sl_OHPartitionsAndSpace' to 'NO_VAL,3,1,35001,,,,3,2,5946,,,'. Received the value from a code block.INFO: Calling Query NtServicesQueries10.2.0.2.0  ntGetAllOracleServicesINFO: Query Returned: OracleRemExecService,OracleClusterPreinstServiceINFO: Calling Query ClusterPreinstQueries1.2.1  RemoveClusterPreinstService

The important line the above mess is:

Page 117: Oracle RAC

INFO: Query Returned: ocrcfg,1,1,10229,votedsk1,2,1,10229,,3,1,35001,,3,2,5946

Apparently, the Oracle Installer is smart enough to recognize that 2 of my 4 partitions have labels "ocrcfg" and "votedsk1" and is not listing them because as far as it knows, theses partitions are in use by another cluster.

I tried deleting my partitions, re-sizing them, re-formatting with full NTFS and then restarting the Oracle Installer. Once again, the same problem occurred, the disks still had these labels.

It turns out Oracle includes two utilities that are part of the Clusterware install for removal of disk labels and low-level formatting of partitions

guioracleobjmanager.exe logpartformat.exe

The catch here is that you need to retrieve the files from CLUSTER_HOME\bin directory. If you do not have Clusterware installed on another machine, you will not be able to get access to these files and may need to call Oracle support to get copies of them.

In my case, I was able to copy these executables and the appropriate DLLs to my install machines.

Run the guioracleobjmanager.exe tool.

As we can see above, this utility lists any labels that Oracle has applied to the shared storage. We can see the same names we saw in the install log: ocrcfg and votedsk1.

From a command prompt, we now need to format this partitions using the logpartformat.exe utility:

> logpartformat /q \\.\ocrcfg> logpartformat /q \\.\votedsk1

Page 118: Oracle RAC

Once both partitions have been formatted, we need to remove the labels from the disks using the Oracle Object Manager.

Check the checkbox on each line with the disk labels that we just formatted and select the Commit menu item from the Options menu.

The partitions should now appear in Oracle Universal Installer during the Clusterware installation.