Oracle RAC 11g Installation on ZFS Oracle Database 11g [11.2.0.4] RAC Installation on OEL 5.8
using ZFS Shared Storage
ORACLE 11g RAC on ZFS INSTALLATION
[Oracle Enterprise Linux 5.8]
Required Software
Table-1
Type Software Image & Install
OS Oracle Linux 5.8 OracleLinux-R5-U8-Server-x86_64-dvd.iso
DB Oracle Database 11.2.0.4 p13390677_112040_Linux-x86-64_1of7 p13390677_112040_Linux-x86-64_2of7 p13390677_112040_Linux-x86-64_3of7
VM VM VirtualBox VirtualBox-4.3.2-90405-Win.exe
ZFS ZFS Storage Appliance 700_1 “Sun ZFS Storage Appliance” image
Network Architecture
Table-2
Interface Network IP Full Name Name Entry
eth0 public 192.168.10.101 rac01.taysi.com rac01 HOSTS File
eth0 public 192.168.10.102 rac02.taysi.com rac02 HOSTS File
eth0 virtual 192.168.10.111 rac01-vip.taysi.com rac01-vip HOSTS File
eth0 virtual 192.168.10.112 rac02-vip.taysi.com rac02-vip HOSTS File
eth0 scan 192.168.10.121 rac-scan.taysi.com rac-scan DNS
eth0 scan 192.168.10.122 rac-scan.taysi.com rac-scan DNS
eth0 scan 192.168.10.123 rac-scan.taysi.com rac-scan DNS
eth1 private 192.168.20.101 rac01-priv.taysi.com rac01-priv HOSTS File
eth1 private 192.168.20.102 rac02-priv.taysi.com rac02-priv HOSTS File
eth0 Dns 192.168.10.100 racdns.taysi.com racdns HOSTS File
RAC Node 1 Creation
INSTALLATION
Start the VirtualBox and create a new virtual machine by clicking the “New” button. Enter the OS details in
the popup window and then click the “Next” button.
Image-1
After adjusting the memory size, click “Next” button.
Image-2
Choose the “Create a virtual hard drive now” option and click “Create” button.
Image-3
Choose VDI option and click the “Next” button.
Image-4
Choose “Dynamically allocated” and click the “Next” button.
Image-5
Select the location of the image file, adjust the disk size (30GB) and click the “Create” button.
Image-6
Start the virtual machine.
Image-7
Select the OS image file and click the “Start” button.
Image-8
Hit the “Enter” button and continue.
Image-9
Skip the testing and continue.
Image-10
Click the “Next” button.
Image-11
Select the language and click the “Next” button.
Image-12
Select the keyboard and then click the “Next” button.
Image-13
Check the “Review and modify partitioning layout” option and click the “Next” button.
Image-14
Click the “Yes” button and continue.
Image-15
Select the /boot partition and click the “Edit” button.
Image-16
Set the /boot partition size as 150MB click the “OK” button.
Image-17
Leave the “swap” and “/root” partitions as they are and click the “Next” button.
SWAP SPACE RECOMMENDATION:
If RAM between 2 GB and 8 GB then 2 times the size fo RAM.
If RAM between 8 GB to 32 GB then 1.5 times the size of RAM.
If RAM greater than 32 GB then 32 GB.
Image-18
Accept the loader settings and click the “Next” button.
Image-19
Leave the network page as it is, we‟re going to configure it later. Click the “Next” button.
Image-20
Select the region/city and click the “Next” button.
Image-21
Enter a root password and click the “Next” button.
Image-22
Select “Customize now” option to decide the packages to be installed and then click the “Next” button.
Image-23
Select the required groups from the below screen.
Required Packages for Oracle RAC installation:
GNOME Desktop Environment
Editors
Graphical Internet
Text-based Internet
Development Libraries
Development Tools
Server Configuration Tools
Administration Tools
Base
System Tools
X Window System
Image-24
After selecting the required packages click the “Next” button and continue.
Image-25
Click the “Next” button to begin installation.
Image-26
Installing…
Image-27
Reboot.
Image-28
Press the “Forward” button to continue installation after the reboot.
Image-29
Agree the licence agreement and click the “Forward” button.
Image-30
Disable the firewall and click the “Forward” button. Click “Yes” for the security warning.
Image-31
Disable Secure Linux and click the “Forward” button. Click “Yes” for the security warning.
Image-32
Do NOT enable the Kdump, click the “Forward” button to continue.
Image-33
Adjust the Date & Time and then click the “Forward” button to continue.
Image-34
Do NOT create any user, click the “Forward” button to continue.Click the “Continue” button for the warning
popup.
Image-35
Click the “Forward” button to continue.
Image-36
Click the “Finish” button to finish installation. And reboot the machine.
Image-37
NETWORK CONFIGURATION - VIRTUAL BOX
Add new network interface on VirtualBox
Image-38
Click the Network link on the left then the “Add host-only network” button on the right. You should have 2
network interfaces for RAC installation. Then click the “OK” button.
Image-39
Change the settings of the newly installed RAC01 Node, right click the RAC01 tab then choose the
“Settings…” from the popup menu.
Image-40
Configure the adapter settings of the newly installed RAC01 Node
Image-41
Add a second adapter for RAC01 as seen below, and then click the “OK” button.
Image-42
Change the IP settings of VirtualBox adapters from Control Center > Network and Sharing Center. Click
VirtualBox Host-Only Network.
Image-43
Change the properties as below:
Image-44
Do the same thing for VirtualBox Host-Only Network#2 with the following values below:
Image-45
NETWORK CONFIGURATION - RAC01 SERVER NODE
Edit /etc/sysconfig/network [ROOT]
NETWORKING=yes NETWORKING_IPV6=yes HOSTNAME=rac01.taysi.com
Edit /etc/resolv.conf [ROOT]
nameserver 192.168.10.100 search taysi.com
Edit /etc/hosts [ROOT]
127.0.0.1 localhost.taysi.com localhost # Public 192.168.10.101 rac01.taysi.com rac01 192.168.10.102 rac02.taysi.com rac02 # Private 192.168.20.101 rac01-priv.taysi.com rac01-priv 192.168.20.102 rac02-priv.taysi.com rac02-priv # Virtual 192.168.10.111 rac01-vip.taysi.com rac01-vip 192.168.10.112 rac02-vip.taysi.com rac02-vip # SCAN -> TO DNS (written here just for info) #192.168.10.121 rac-scan.taysi.com rac-scan #192.168.10.122 rac-scan.taysi.com rac-scan #192.168.10.123 rac-scan.taysi.com rac-scan
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 [ROOT]
# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none HWADDR=08:00:27:1F:03:1F ONBOOT=yes NETMASK=255.255.255.0 IPADDR=192.168.10.101 GATEWAY=192.168.10.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes
Edit /etc/sysconfig/network-scripts/ifcfg-eth1 [ROOT]
# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth1 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:d6:01:6f TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=192.168.20.101 GATEWAY=192.168.20.1 USERCTL=no IPV6INIT=no PEERDNS=yes
INSTALL RPMS
Mount the Linux OS iso file. Machine > Settings > Storage
If some other image is already mounted here, remove the mounted image and reboot. When the Linux is
up again, mount the OracleLinux Image as seen below.
Image-46
Note: Do not forget to remove the image after you‟re done, else it will try to boot from image next time
you reboot the server.
Install RPMs -> cd /media/OL5.8\ x86_64\ dvd\ 20120229/Server/ (should be mounted somewhere under
/media)
Just copy the whole command and paste it in the terminal and enter. [ROOT]
rpm -Uvh binutils-2.* rpm -Uvh compat-libstdc++-33* rpm -Uvh elfutils-libelf-0.* rpm -Uvh elfutils-libelf-devel-* rpm -Uvh gcc-4.* rpm -Uvh gcc-c++-4.* rpm -Uvh glibc-2.* rpm -Uvh glibc-common-2.* rpm -Uvh glibc-devel-2.* rpm -Uvh glibc-headers-2.* rpm -Uvh ksh-2* rpm -Uvh libaio-0.* rpm -Uvh libaio-devel-0.* rpm -Uvh libgcc-4.* rpm -Uvh libstdc++-4.* rpm -Uvh libstdc++-devel-4.* rpm -Uvh make-3.* rpm -Uvh sysstat-7.* rpm -Uvh unixODBC-2.* rpm -Uvh unixODBC-devel-2.*
Note: If dependecies fail, just resubmit the whole command, sometimes some libraries must be installed
before others.
KERNEL SETTINGS
Edit /etc/sysctl.conf [ROOT]
fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmall = 2097152 kernel.shmmax = 1054504960 kernel.shmmni = 4096 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default=262144 net.core.rmem_max=4194304 net.core.wmem_default=262144 net.core.wmem_max=1048586
Note: Issue the command “/sbin/sysctl -p” after changing kernel values.
SECURITY LIMITS
Edit /etc/security/limits.conf for oracle and grid users. [ROOT]
oracle hard memlock 50000000 oracle soft memlock 50000000 oracle soft core unlimited oracle hard core unlimited oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 4096 oracle hard nofile 65536 oracle soft stack 10240 grid hard memlock 50000000 grid soft memlock 50000000 grid soft core unlimited grid hard core unlimited grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 4096 grid hard nofile 65536 grid soft stack 10240
Add the following line to /etc/pam.d/login [ROOT]
session required pam_limits.so
CREATE GROUPS and USERS
USER PRIMARY GROUPS SECONDARY GROUPS grid oinstall asmadmin,asmdba,dba oracle oinstall dba,asmdba
Note: The group and user IDs between the two nodes must be exactly the same.
Create groups and users [ROOT]
groupadd oinstall groupadd asmadmin groupadd asmdba groupadd dba useradd -g oinstall -G asmadmin,asmdba,dba grid useradd -g oinstall -G dba,asmdba oracle
Note: Give passwords to the users after creation.
CREATE DIRECTORIES
Table-3
PATH ENV OWNER MOD /u01/app/grid ORACLE_BASE
[grid’s] grid:oinstall 775
/u01/app/11.2.0/grid ORACLE_HOME [grid’s]
grid:oinstall 775
/u02/app/oracle ORACLE_BASE [oracle’s]
oracle:oinstall 775
/u02/app/oracle/product/11.2.0/db_home ORACLE_HOME [oracle’s]
oracle:oinstall 775
Note: For grid user, ORACLE_HOME should NOT be a sub-dir of ORACLE_BASE. It should be a seperate
maybe same level directory for its ownership will be changed with root.sh script.
Create directories [ROOT]
mkdir -p /u01/app/grid mkdir -p /u01/app/11.2.0/grid mkdir -p /u02/app/oracle mkdir -p /u02/app/oracle/product/11.2.0/db_home chown -R grid:oinstall /u01 chown -R oracle:oinstall /u02 chmod -R 775 /u01 chmod -R 775 /u02
SETUP FILES
Create a directory for setup files and copy the the installation files in here using an FTP tool. [ROOT]
mkdir /setup # Copy the following files into this path: # p13390677_112040_Linux-x86-64_1of7.zip # p13390677_112040_Linux-x86-64_2of7.zip # p13390677_112040_Linux-x86-64_3of7.zip # Unzip the files unzip p13390677_112040_Linux-x86-64_1of7.zip unzip p13390677_112040_Linux-x86-64_2of7.zip unzip p13390677_112040_Linux-x86-64_3of7.zip
CLUSTER VERIFICATION RPM
Go to the path /setup/grid/rpm and install the single rpm package residing in this path. [ROOT]
rpm -Uvh cvuqdisk-1.0.9-1.rpm
DECONFIGURE NTP
Deconfigure [ROOT]
service ntpd stop chkconfig ntpd off mv /etc/ntp.conf /etc/ntp.conf.orig rm /var/run/ntpd
Note: This is a virtual env. test installation. In a real env. you may want to configure NTP instead of
disabling it.
CONFIGURE BASH PROFILES
Switch to oracle user and edit the bash_profile [ORACLE]
# Oracle Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac01.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u02/app/oracle;export ORACLE_BASE ORACLE_HOME=/u02/app/oracle/product/11.2.0/db_home;export ORACLE_HOME ORACLE_SID=RAC1;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi
Switch to grid user and edit the bash_profile [GRID]
# Grid Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac01.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u01/app/grid;export ORACLE_BASE ORACLE_HOME=/u01/app/11.2.0/grid;export ORACLE_HOME ORACLE_SID=RAC1;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi
RAC Node 2 Creation
Change the default machine folder to the path where your RAC01 resides.
Image-47
Shutdown the RAC01 Server and clone the virtual machine.
Image-48
Enter “RAC02” for the clone name and check “Reinitialize the MAC address of all network cards”
checkbox. Click the “Next” button to continue.
Image-49
Confirm that the “Full clone” option is selected. Click the “Clone” button to begin the cloning process.
Image-50
START THE RAC02 SERVER AND CHANGE THE NETWORK CONFIGURATIONS OF RAC02 NODE
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 [ROOT]
# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:d5:92:19 NETMASK=255.255.255.0 IPADDR=192.168.10.102 GATEWAY=192.168.10.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes
Edit /etc/sysconfig/network-scripts/ifcfg-eth1 [ROOT]
# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth1 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:40:4d:a9 NETMASK=255.255.255.0 IPADDR=192.168.20.102 GATEWAY=192.168.20.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes
Delete files with .bak extension and reboot the machine. [ROOT]
rm /etc/sysconfig/network-scripts/ifcfg-eth0.bak rm /etc/sysconfig/network-scripts/ifcfg-eth1.bak
Note: You can also use Linux GUI to change network settings by choosing System > Administration >
Network.
Change the hostname -> edit /etc/sysconfig/network [ROOT]
NETWORKING=yes NETWORKING_IPV6=yes HOSTNAME=rac02.taysi.com
CHANGE THE BASH PROFILES OF RAC02 NODE
Switch to oracle user and edit the bash_profile [ORACLE]
# Oracle Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac02.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u02/app/oracle;export ORACLE_BASE ORACLE_HOME=/u02/app/oracle/product/11.2.0/db_home;export ORACLE_HOME ORACLE_SID=RAC2;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi
Switch to grid user and edit the bash_profile [GRID]
# Grid Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac02.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u01/app/grid;export ORACLE_BASE ORACLE_HOME=/u01/app/11.2.0/grid;export ORACLE_HOME ORACLE_SID=RAC2;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi
# At this point we‟ve completed the configuration of RAC Nodes.
DNS Server Creation
From Oracle 11.2.0.2 onward, we need a DNS server to define SCAN addresses.
Just create a new Oracle Linux VM Server as we did in step “RAC Node 1 Creation”, but this time allocate
only 512MB of memory and a limited disk space. Install all the RPMs that we did earlier. Name the server
“RACDNS”
Edit /etc/hosts [ROOT]
127.0.0.1 racdns.racdomain.com racdns localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 192.168.10.100 racdns.taysi.com racdns
Edit /etc/named.conf [ROOT]
zone "taysi.com" IN { type master; allow-update { none; }; file "taysi.com.zone"; }; zone "10.168.192.in-addr.arpa." IN { type master; file "10.168.192.zone"; allow-update { none; }; };
Edit -> vi /var/named/taysi.com.zone [ROOT]
$TTL 86400 @ IN SOA localhost root.localhost ( 42 ; serial (d. adams) 3H ; refresh 15M ; retry 1W ; expiry 1D ) ; minimum IN NS localhost localhost IN A 127.0.0.1 rac01 IN A 192.168.10.101 rac02 IN A 192.168.10.102 rac01-priv IN A 192.168.20.101 rac02-priv IN A 192.168.20.102 rac01-vip IN A 192.168.10.111 rac02-vip IN A 192.168.10.112 rac-scan IN A 192.168.10.121 rac-scan IN A 192.168.10.122 rac-scan IN A 192.168.10.123
Edit -> vi /var/named/10.168.192.zone [ROOT]
$ORIGIN 10.168.192.in-addr.arpa. $TTL 1H @ IN SOA dnserver.taysi.com. root.dnsserver.taysi.com. ( 2 3H 1H 1W 1H ) 10.168.192.in-addr.arpa. IN NS dnsserver.taysi.com. 101 IN PTR rac01.taysi.com. 102 IN PTR rac02.taysi.com. 111 IN PTR rac01-vip.taysi.com. 112 IN PTR rac02-vip.taysi.com. 121 IN PTR rac-scan.taysi.com. 122 IN PTR rac-scan.taysi.com. 123 IN PTR rac-scan.taysi.com.
Edit -> vi /etc/resolv.conf [ROOT]
search taysi.com nameserver 192.168.10.100
Copy zone files [ROOT]
cd /var/named cp /usr/share/system-config-bind/profiles/default/named/* .
Edit -> vi /etc/sysconfig/network [ROOT]
NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=racdns.racdomain.com
Edit on both RAC01 and RAC02 nodes -> vi /etc/resolv.conf [ROOT]
nameserver 192.168.10.100 search taysi.com
On DNS server test the DNS resolution of both nodes [ROOT]
[root@racdns ~]# nslookup rac01 Server: 192.168.10.100 Address: 192.168.10.100#53 Name: rac01.taysi.com Address: 192.168.10.101 [root@racdns ~]# nslookup rac02 Server: 192.168.10.100 Address: 192.168.10.100#53 Name: rac02.taysi.com Address: 192.168.10.102
Installation and Configuration of ZFS
Download Oracle ZFS simulator from Oracle web site.
Then run the VirtualBox and import by File > Import Appliance.
Image-51
Open appliance
Image-52
Choose the ovf file and click the “Open” button.
Image-53
Click the “Next” button to continue.
Image-54
Click the “Import” button to begin import process.
Image-55
It may take 3-5 minutes to import…
Image-56
Start the virtual machine.
Image-57
Press the „Enter‟ key to select the Storag of interest. (Here we have only one…)
Image-58
Press any key to begin configuration
Image-59
Config as seen below and press “ESC” then “1” to save config.
Image-60
Login to web console using a compatible browser by using the link -> https://192.168.10.200:215/
Enter username and password you prvieded in the configuration.
Press the “Commit” button without changing anything.
Image-61
Press the “Commit” button without changing anything.
Image-62
Press the “Commit” button without changing anything.
Image-63
Press the “Commit” button without changing anything.
Image-64
PRESS the “+” button left to “Available Pool” to add a pool.
Image-65
Allocate all devices (10 disks, since we‟ll not create any other pool) and then “Commit”
Image-66
Choose storage profile (in our case striped is sufficient)
Image-67
Commit
Image-68
After skipping the register part, the Installation is done.
ADD FILESYSTEMS
Table-4
File System Record Size Syn. Write Cache Dev. Usage
SHAREDCONFIG01 128K Throughput All data & metadata
SHAREDCONFIG02 128K Throughput All data & metadata
SHAREDCONFIG03 128K Throughput All data & metadata
ORCL_DATA_FS 8K Throughput All data & metadata
ORCL_RECO_FS 128K Throughput All data & metadata
ORCL_REDO_FS 128K Throughput All data & metadata
File System Architecture
Table-5
NAME PATH MOUNT POINT
Clusterware Base /u01/app/grid Local File System
Clusterware Home /u01/app/11.2.0/grid Local File System
OCR File 1 /sharedconfig01/ocr1 ZFS: SHARED_CONFIG to /sharedconfig01
OCR File 2 /sharedconfig02/ocr2 ZFS: SHARED_CONFIG to /sharedconfig02
OCR File 3 /sharedconfig03/ocr3 ZFS: SHARED_CONFIG to /sharedconfig03
VDSK File 1 /sharedconfig01/vdsk1 ZFS: SHARED_CONFIG to /sharedconfig01
VDSK File 2 /sharedconfig02/vdsk2 ZFS: SHARED_CONFIG to /sharedconfig02
VDSK File 3 /sharedconfig03/vdsk3 ZFS: SHARED_CONFIG to /sharedconfig03
Standart Database Base /uxx/app/<DB_OWNER> Local File System
Standart Database Home /uxx/app/<DB_OWNER>/product/11.2.0/db_home Local File System
Standart Database Datafiles /uxx/app/<DB_OWNER>/<DB_NAME>/data/data
ZFS: <DB_NAME>_DATA_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/data
Standart Database Index /uxx/app/<DB_OWNER>/<DB_NAME>/data/index
Standart Database Undo TS /uxx/app/<DB_OWNER>/<DB_NAME>/data/undo
Standart Database Temp TS /uxx/app/<DB_OWNER>/<DB_NAME>/data/temp
Standart Database Redo Log Group 1 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo01 ZFS: <DB_NAME>_REDO_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/redo
Standart Database Redo Log Group 2 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo02
Standart Database Redo Log Group 3 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo03
Standart Database Backup Dir /uxx/app/<DB_OWNER>/<DB_NAME>/reco/backup ZFS: <DB_NAME>_RECO_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/reco
Standart Database Arc. Log Dir 1 /uxx/app/<DB_OWNER>/<DB_NAME>/reco/reco01
Standart Database Arc. Log Dir 2 /uxx/app/<DB_OWNER>/<DB_NAME>/reco/reco02
EBS Database Base /uxx/app/<DB_OWNER> Local File System
EBS Database Home /uxx/<DB_NAME>/tech_st/11.2.0 Local File System
EBS Database Datafiles /uxx/<DB_NAME>/apps_st/data/data
ZFS: <DB_NAME>_DATA_FS to /uxx/<DB_NAME>/data
EBS Database Index /uxx/<DB_NAME>/apps_st/data/index
EBS Database Undo TS /uxx/<DB_NAME>/apps_st/data/undo
EBS Database Temp TS /uxx/<DB_NAME>/apps_st/data/temp
EBS Database Redo Log Group 1 /uxx/<DB_NAME>/apps_st/redo/redo01 ZFS: <DB_NAME>_REDO_FS to /uxx/<DB_NAME>/redo
EBS Database Redo Log Group 2 /uxx/<DB_NAME>/apps_st/redo/redo02
EBS Database Redo Log Group 3 /uxx/<DB_NAME>/apps_st/redo/redo03
EBS Database Backup Dir /uxx/<DB_NAME>/apps_st/reco/backup ZFS: <DB_NAME>_RECO_FS to /uxx/<DB_NAME>/reco
EBS Database Arc. Log Dir 1 /uxx/<DB_NAME>/apps_st/reco/reco01
EBS Database Arc. Log Dir 2 /uxx/<DB_NAME>/apps_st/reco/reco02
Add filesystems from Shares Tab by clicking the “+” button left to the string Filesystems
Image-69
Create the filesystem as follows
Image-70
Edit the filesystem by clicking the pencil icon of the file system.
Image-71
SHARES > GENERAL
Uncheck the “Inherit from project” checkbox and then change the Database record size according to the
table-4 values.
Image-72
SHARES > PROTOCOLS
Uncheck the “Inherit from project” checkbox and then add NFS exceptions as seen blow. Those IP adresses
are the RAC01 and RAC02 public IPs. /24 means IPv4 protocol. Check the ROOT ACCESS checkboxes.
Image-73
Repeat the steps between Image-69 and Image-73 to define te filesystems in the listed in table-4.
DEFINE MOUNT POINTS
Create paths [ROOT]
mkdir -p /sharedconfig01 mkdir -p /sharedconfig02 mkdir -p /sharedconfig03 mkdir -p /u02/app/oracle/ORCL/redo mkdir -p /u02/app/oracle/ORCL/data mkdir -p /u02/app/oracle/ORCL/reco chown -R grid:oinstall /sharedconfig01 chown -R grid:oinstall /sharedconfig02 chown -R grid:oinstall /sharedconfig03 chown -R oracle:oinstall /u02/app/oracle/ORCL
Edit fstab on both RAC01 and RAC02 nodes. Reboot the servers after editing. [ROOT]
192.168.10.200:/export/SHAREDCONFIG01 /sharedconfig01 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/SHAREDCONFIG02 /sharedconfig02 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/SHAREDCONFIG03 /sharedconfig03 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_DATA_FS /u02/app/oracle/ORCL/data nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_RECO_FS /u02/app/oracle/ORCL/reco nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_REDO_FS /u02/app/oracle/ORCL/redo nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0
Test on both nodes RAC01 and RAC02 that the mounts are OK.
[root@rac01 grid]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 26G 6.9G 17G 29% / /dev/sda1 145M 24M 114M 17% /boot tmpfs 1002M 0 1002M 0% /dev/shm 192.168.10.200:/export/SHAREDCONFIG 49G 32K 49G 1% /u01/sharedconfig 192.168.10.200:/export/ORCL_DATA_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/data 192.168.10.200:/export/ORCL_RECO_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/reco 192.168.10.200:/export/ORCL_REDO_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/redo
CRS Installation
Provide SSH connectivity [ROOT]
Go to the grid installation files path …grid/sshsetup and run the ssh setup script (On RAC01 or RAC02)
cd /setup/grid/sshsetup ./sshUserSetup.sh -user oracle -hosts "rac01 rac02" -advanced -exverify -confirm ./sshUserSetup.sh -user grid -hosts "rac01 rac02" -advanced -exverify -confirm
Verify the cluster structure before the installation
cd /setup/grid ./runcluvfy.sh stage -pre crsinst -n rac01,rac02 -verbose > result.txt
Examine the output file “result.txt”, at the end of the file we should see the phrase:
Pre-check for cluster services setup was successful.
BEGIN CRS INSTALLATION
su - grid cd /setup/grid/ ./runInstaller
Image-74
Image-75
Image-76
Image-77
Image-78
Image-79
Image-80
Image-81
Image-82
Image-83
Image-84
Image-85
Image-86
Image-87
Image-88
Image-89
Image-90
Image-91
Database Software Installation
su - oracle cd /setup/database ./runInstaller
Image-92
Image-93
Image-94
Image-95
Image-96
Image-97
Image-98
Image-99
Image-100
After executing the root.sh scripts, the software installation is done.
Database Installation
On both nodes RAC01 and RAC02 [with oracle user]
su - oracle mkdir -p /u02/app/oracle/ORCL/data/data mkdir -p /u02/app/oracle/ORCL/data/index mkdir -p /u02/app/oracle/ORCL/data/undo mkdir -p /u02/app/oracle/ORCL/data/temp mkdir -p /u02/app/oracle/ORCL/redo/redo01 mkdir -p /u02/app/oracle/ORCL/redo/redo02 mkdir -p /u02/app/oracle/ORCL/redo/redo03 mkdir -p /u02/app/oracle/ORCL/reco/backup mkdir -p /u02/app/oracle/ORCL/reco/reco01 mkdir -p /u02/app/oracle/ORCL/reco/reco02 chown -R oracle:oinstall /u02/app/oracle/ORCL chmod -R 775 /u02/app/oracle/ORCL
Run dbca command with oracle user
dbca
Image-101
Image-102
Image-103
Image-104
Image-105
Image-106
Image-107
Image-108
Image-109
Image-110 (Also select the charset of the database here)
Image-111
Image-112
Image-113
Installation is DONE...
Top Related