STEP BY STEP ORACLE RAC

12C (FLEX ASM)

INSTALLATION ON LINUX

X86_64 (VM)

Erman Ibrisim ermanibrisim@gmail.com

18.08.2016

V1.0

Contents

1. Prepare Operating System

2. GRID Infrastructure Installation

3. RDBMS Installation

4. Container Database Creation

STEP BY STEP ORACLE 12C (FLEX ASM) INSTALLATION ON

LINUX X86_64 (VM)

- Summary

Node 1 Node 2

Operating System OEL 6.5 OEL 6.5

Hostname rac12c1.gantek.com rac12c2.gantek.com

Puclic IP Address 192.168.60.110 192.168.60.111

VIP IP Address 192.168.60.112 192.168.60.113

PRIV IP Address 192.168.100.100 192.168.100.101

Scan IP Addresses

192.168.60.115

192.168.60.116

192.168.60.117

Scan name rac12c-scan

Cluster Name rac12c-cluster

GNS VIP Address 192.168.60.120

ASM Home /u01/app/12.1.0/grid

ASM Version 12.1.0.2

DB Home /u02/app/oracle/product/12.1.0/db

DB Version 12.1.0.2

1. Prepare Operating System

- Edit /etc/hosts file.

$ cat /etc/hosts 127.0.0.1 localhost localhost.localdomain 192.168.60.110 rac12c1.gantek.com rac12c1 192.168.60.111 rac12c2.gantek.com rac12c2 #VIP IP 192.168.60.112 rac12c1-vip.gantek.com rac12c1-vip 192.168.60.113 rac12c2-vip.gantek.com rac12c2-vip #PRIV 192.168.100.100 rac12c1-priv.gantek.com rac12c1-priv 192.168.100.101 rac12c2-priv.gantek.com rac12c2-priv

- Disable selinux.

$ vi /etc/selunix/config . . . SELINUX = disabled

- Install oracle-rdbms-server-12cR1-preinstall rpm

yum install oracle-rdbms-server-12cR1-preinstall

or download the rpm from http://public-yum.oracle.com/ and install.

rpm –ivh oracle-rdbms-server-12cR1-preinstall

- Enter DNS records to DNS server. To configure DNS server refer my post.

rac12c-scan

192.168.60.115

192.168.60.116

192.168.60.117

- Set DNS server to all nodes.

[oracle@rac12c1 ~]$ cat /etc/resolv.conf search gantek.com nameserver 192.168.60.102

[oracle@rac12c2 ~]$ cat /etc/resolv.conf search gantek.com nameserver 192.168.60.102

- Test the DNS records.

[root@rac12c1 ~]$ nslookup rac12c-scan Server: 192.168.60.102 Address: 192.168.60.102#53 Name: rac12c-scan.gantek.com Address: 192.168.60.116 Name: rac12c-scan.gantek.com Address: 192.168.60.117 Name: rac12c-scan.gantek.com Address: 192.168.60.115

- Start ntpd service with “-x” option.

[root@rac12c1 ~]$ cat /etc/sysconfig/ntpd # Drop root to id 'ntp:ntp' by default. OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid" [root@rac12c1 ~]$ service ntpd start [root@rac12c1 ~]$ chkconfig ntpd on [root@rac12c2 ~]$ cat /etc/sysconfig/ntpd # Drop root to id 'ntp:ntp' by default. OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid" [root@rac12c2 ~]$ service ntpd start [root@rac12c2 ~]$ chkconfig ntpd on

- Create 2 shared disk on vm for ASM disks.

- Add shared disks to other nodes.

Note: You should add disk with same order to all nodes.

- Format shared disks on node1.

[root@rac12c1 ~]# fdisk /dev/sdd Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First sector (61-1048575, default 61): 2048 Last sector or +size or +sizeM or +sizeK (2048-1048575, default 1048575): Using default value 1048575 Command (m for help): w The partition table has been altered!

[root@rac12c1 ~]# fdisk /dev/sde Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First sector (61-1048575, default 61): 2048 Last sector or +size or +sizeM or +sizeK (2048-1048575, default 1048575): Using default value 1048575 Command (m for help): w The partition table has been altered!

[root@rac12c1 ~]# fdisk -l /dev/sdd Disk /dev/sdd: 12.9 GB, 12884901888 bytes 255 heads, 63 sectors/track, 1566 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x385d52f9 Device Boot Start End Blocks Id System /dev/sdd1 1 1566 12578863+ 83 Linux [root@rac12c1 ~]# fdisk -l /dev/sde Disk /dev/sde: 12.9 GB, 12884901888 bytes 255 heads, 63 sectors/track, 1566 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x302b6a29 Device Boot Start End Blocks Id System /dev/sde1 1 1566 12578863+ 83 Linux

- İssue partprobe commad on other nodes.

[root@rac12c2 ~]# partprobe

- Change owner of the asm disks.

[root@rac12c1 ~]# chown oracle:oinstall /dev/sdd1 [root@rac12c1 ~]# chown oracle:oinstall /dev/sde1

[root@rac12c2 ~]# chown oracle:oinstall /dev/sdd1 [root@rac12c2 ~]# chown oracle:oinstall /dev/sde1

- Add ‘Internal Network’ adapter to all nodes for interconnect.

- Configure interconnect interfaces.

[root@rac12c1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 TYPE=Ethernet ONBOOT=yes BOOTPROTO=none IPADDR=192.168.100.100 PREFIX=24 GATEWAY=255.255.255.0 DEFROUTE=yes IPV4_FAILURE_FATAL=yes

IPV6INIT=no NAME="System eth1" ARPCHECK=no [root@rac12c1 ~]# service network restart

[root@rac12c2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 TYPE=Ethernet ONBOOT=yes BOOTPROTO=none IPADDR=192.168.100.101 PREFIX=24 GATEWAY=255.255.255.0 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth1" ARPCHECK=no [root@rac12c2 ~]# service network restart

2. GRID Infrastructure Installation

- Install oracleasm RPMs.

[root@rac12c1 ~]# rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64 [root@rac12c1 ~]# rpm -ivh oracleasmlib-2.0.12-1.el6.x86_64 [root@rac12c1 ~]# rpm -qa | grep oracleasm oracleasm-support-2.1.8-1.el6.x86_64 oracleasmlib-2.0.12-1.el6.x86_64

[root@rac12c2 ~]# rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64 [root@rac12c2 ~]# rpm -ivh oracleasmlib-2.0.12-1.el6.x86_64 [root@rac12c2 ~]# rpm -qa | grep oracleasm oracleasm-support-2.1.8-1.el6.x86_64 oracleasmlib-2.0.12-1.el6.x86_64

- Configure oracleasm.

[root@rac12c1 ~]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface [oracle]:

Default group to own the driver interface [oinstall]: Start Oracle ASM library driver on boot (y/n) [y]: Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done [root@rac12c2 ~]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface [oracle]: Default group to own the driver interface [oinstall]: Start Oracle ASM library driver on boot (y/n) [y]: Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done

- Create ASM disks.

[root@rac12c1 ~]# oracleasm createdisk DATA /dev/sdd1 [root@rac12c1 ~]# oracleasm createdisk FRA /dev/sde1

- Issue “oracleasm scandisks” on other nodes.

[root@rac12c2 ~]# oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@rac12c2 ~]# oracleasm listdisks DATA FRA

- Unzip the binaries and change the owner of grid folder.

[root@rac12c1 ~]# unzip p13390677_112040_Linux-x86-64_3of7.zip [root@rac12c1 ~]# unzip p13390677_112040_Linux-x86-64_4of7.zip [root@rac12c1 ~]# chown –R oracle:oinstall grid/

- Start the installer.

[oracle@rac12c1 ~]$ cd grid [oracle@rac12c1 ~]$ ./runInstaller

- If passwordless ssh failed, refer my post to configure manually.

- Here is the root.sh outputs.

[root@rac12c1 ~]# /u01/app/12.1.0/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/12.1.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params 2016/08/15 11:35:36 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2016/08/15 11:36:02 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2016/08/15 11:36:03 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert 2016/08/15 11:36:49 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.evmd' on 'rac12c1' CRS-2672: Attempting to start 'ora.mdnsd' on 'rac12c1' CRS-2676: Start of 'ora.evmd' on 'rac12c1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac12c1' CRS-2676: Start of 'ora.gpnpd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac12c1' CRS-2672: Attempting to start 'ora.gipcd' on 'rac12c1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac12c1' succeeded CRS-2676: Start of 'ora.gipcd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac12c1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac12c1' CRS-2676: Start of 'ora.diskmon' on 'rac12c1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac12c1' succeeded ASM created and started successfully. Disk Group DATA created successfully. CRS-2672: Attempting to start 'ora.crf' on 'rac12c1' CRS-2672: Attempting to start 'ora.storage' on 'rac12c1' CRS-2676: Start of 'ora.storage' on 'rac12c1' succeeded CRS-2676: Start of 'ora.crf' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac12c1' CRS-2676: Start of 'ora.crsd' on 'rac12c1' succeeded CRS-4256: Updating the profile Successful addition of voting disk 6919a759fed34f3ebf0f39778f0b6032. Successfully replaced voting disk group with +DATA.

CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 6919a759fed34f3ebf0f39778f0b6032 (ORCL:DATA) [DATA] Located 1 voting disk(s). CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac12c1' CRS-2673: Attempting to stop 'ora.crsd' on 'rac12c1' CRS-2677: Stop of 'ora.crsd' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.storage' on 'rac12c1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac12c1' CRS-2677: Stop of 'ora.storage' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.drivers.acfs' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac12c1' CRS-2673: Attempting to stop 'ora.ctssd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.evmd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.asm' on 'rac12c1' CRS-2677: Stop of 'ora.mdnsd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.crf' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.asm' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac12c1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac12c1' CRS-2677: Stop of 'ora.cssd' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac12c1' CRS-2677: Stop of 'ora.gipcd' on 'rac12c1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac12c1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'rac12c1' CRS-2672: Attempting to start 'ora.evmd' on 'rac12c1' CRS-2676: Start of 'ora.mdnsd' on 'rac12c1' succeeded CRS-2676: Start of 'ora.evmd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac12c1' CRS-2676: Start of 'ora.gpnpd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac12c1' CRS-2676: Start of 'ora.gipcd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac12c1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac12c1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac12c1' CRS-2676: Start of 'ora.diskmon' on 'rac12c1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac12c1' CRS-2672: Attempting to start 'ora.ctssd' on 'rac12c1' CRS-2676: Start of 'ora.ctssd' on 'rac12c1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac12c1' CRS-2676: Start of 'ora.asm' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'rac12c1' CRS-2676: Start of 'ora.storage' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.crf' on 'rac12c1' CRS-2676: Start of 'ora.crf' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac12c1' CRS-2676: Start of 'ora.crsd' on 'rac12c1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: rac12c1 CRS-6016: Resource auto-start has completed for server rac12c1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2016/08/15 11:42:05 CLSRSC-343: Successfully started Oracle Clusterware stack CRS-2672: Attempting to start 'ora.net1.network' on 'rac12c1' CRS-2676: Start of 'ora.net1.network' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gns.vip' on 'rac12c1' CRS-2676: Start of 'ora.gns.vip' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gns' on 'rac12c1' CRS-2676: Start of 'ora.gns' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c1' CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac12c1' CRS-2676: Start of 'ora.asm' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac12c1' CRS-2676: Start of 'ora.DATA.dg' on 'rac12c1' succeeded CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac12c1' CRS-2673: Attempting to stop 'ora.crsd' on 'rac12c1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac12c1' CRS-2673: Attempting to stop 'ora.cvu' on 'rac12c1' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac12c1' CRS-2673: Attempting to stop 'ora.oc4j' on 'rac12c1'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac12c1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac12c1' CRS-2673: Attempting to stop 'ora.gns' on 'rac12c1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' CRS-2673: Attempting to stop 'ora.rac12c1.vip' on 'rac12c1' CRS-2677: Stop of 'ora.cvu' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac12c1' CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac12c1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac12c1' CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac12c1' CRS-2677: Stop of 'ora.asm' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c1' CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.rac12c1.vip' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.scan3.vip' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.scan1.vip' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.gns' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.gns.vip' on 'rac12c1' CRS-2677: Stop of 'ora.scan2.vip' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.gns.vip' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.oc4j' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac12c1' CRS-2677: Stop of 'ora.ons' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac12c1' CRS-2677: Stop of 'ora.net1.network' on 'rac12c1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac12c1' has completed CRS-2677: Stop of 'ora.crsd' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac12c1' CRS-2677: Stop of 'ora.drivers.acfs' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.evmd' on 'rac12c1' CRS-2673: Attempting to stop 'ora.storage' on 'rac12c1' CRS-2677: Stop of 'ora.gpnpd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.storage' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac12c1' CRS-2677: Stop of 'ora.evmd' on 'rac12c1' succeeded CRS-2677: Stop of 'ora.asm' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac12c1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac12c1' CRS-2677: Stop of 'ora.cssd' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac12c1' CRS-2677: Stop of 'ora.crf' on 'rac12c1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac12c1' CRS-2677: Stop of 'ora.gipcd' on 'rac12c1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac12c1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'rac12c1' CRS-2672: Attempting to start 'ora.evmd' on 'rac12c1' CRS-2676: Start of 'ora.mdnsd' on 'rac12c1' succeeded CRS-2676: Start of 'ora.evmd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac12c1' CRS-2676: Start of 'ora.gpnpd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac12c1' CRS-2676: Start of 'ora.gipcd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac12c1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac12c1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac12c1' CRS-2676: Start of 'ora.diskmon' on 'rac12c1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac12c1' CRS-2672: Attempting to start 'ora.ctssd' on 'rac12c1' CRS-2676: Start of 'ora.ctssd' on 'rac12c1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac12c1' CRS-2676: Start of 'ora.asm' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'rac12c1' CRS-2676: Start of 'ora.storage' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.crf' on 'rac12c1' CRS-2676: Start of 'ora.crf' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac12c1' CRS-2676: Start of 'ora.crsd' on 'rac12c1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-2664: Resource 'ora.DATA.dg' is already running on 'rac12c1' CRS-6017: Processing resource auto-start for servers: rac12c1

CRS-2672: Attempting to start 'ora.oc4j' on 'rac12c1' CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'rac12c1' CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'rac12c1' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac12c1' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'rac12c1' succeeded CRS-2676: Start of 'ora.oc4j' on 'rac12c1' succeeded CRS-6016: Resource auto-start has completed for server rac12c1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2016/08/15 11:45:46 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@rac12c2 ~]# /u01/app/12.1.0/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/12.1.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params 2016/08/15 11:46:13 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2016/08/15 11:46:37 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2016/08/15 11:46:38 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful 2016/08/15 11:47:55 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac12c2' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac12c2' CRS-2677: Stop of 'ora.drivers.acfs' on 'rac12c2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac12c2' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'rac12c2' CRS-2672: Attempting to start 'ora.evmd' on 'rac12c2' CRS-2676: Start of 'ora.mdnsd' on 'rac12c2' succeeded CRS-2676: Start of 'ora.evmd' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac12c2' CRS-2676: Start of 'ora.gpnpd' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac12c2' CRS-2676: Start of 'ora.gipcd' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac12c2' CRS-2676: Start of 'ora.cssdmonitor' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac12c2' CRS-2672: Attempting to start 'ora.diskmon' on 'rac12c2' CRS-2676: Start of 'ora.diskmon' on 'rac12c2' succeeded CRS-2676: Start of 'ora.cssd' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac12c2' CRS-2672: Attempting to start 'ora.ctssd' on 'rac12c2' CRS-2676: Start of 'ora.ctssd' on 'rac12c2' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac12c2' CRS-2676: Start of 'ora.asm' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.storage' on 'rac12c2' CRS-2676: Start of 'ora.storage' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.crf' on 'rac12c2' CRS-2676: Start of 'ora.crf' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac12c2' CRS-2676: Start of 'ora.crsd' on 'rac12c2' succeeded CRS-6017: Processing resource auto-start for servers: rac12c2 CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' CRS-2672: Attempting to start 'ora.net1.network' on 'rac12c2' CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c2' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac12c1' succeeded

CRS-2676: Start of 'ora.net1.network' on 'rac12c2' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac12c1' CRS-2672: Attempting to start 'ora.ons' on 'rac12c2' CRS-2677: Stop of 'ora.scan1.vip' on 'rac12c1' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac12c2' CRS-2676: Start of 'ora.scan1.vip' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac12c2' CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac12c2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac12c2' CRS-2676: Start of 'ora.ons' on 'rac12c2' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac12c2' succeeded CRS-2676: Start of 'ora.asm' on 'rac12c2' succeeded CRS-6016: Resource auto-start has completed for server rac12c2 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2016/08/15 11:51:23 CLSRSC-343: Successfully started Oracle Clusterware stack 2016/08/15 11:51:39 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

- ‘ifconfig –a’ outputs after installation complete.

[root@rac12c1 ~]# ifconfig -a eth0 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.110 Bcast:192.168.60.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe33:36d3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:314120 errors:0 dropped:0 overruns:0 frame:0 TX packets:5257811 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23305902 (22.2 MiB) TX bytes:13376038881 (12.4 GiB) eth0:1 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.120 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:2 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.112 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:4 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.117 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:5 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.115 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 08:00:27:BC:0B:C5 inet addr:192.168.100.100 Bcast:192.168.100.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:febc:bc5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7842 errors:0 dropped:0 overruns:0 frame:0 TX packets:9611 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:6236056 (5.9 MiB) TX bytes:8636280 (8.2 MiB) eth1:1 Link encap:Ethernet HWaddr 08:00:27:BC:0B:C5 inet addr:169.254.173.187 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:5818 errors:0 dropped:0 overruns:0 frame:0 TX packets:5818 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:17878495 (17.0 MiB) TX bytes:17878495 (17.0 MiB)

[root@rac12c2 ~]# ifconfig -a eth0 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.111 Bcast:192.168.60.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe33:36d3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5250539 errors:0 dropped:0 overruns:0 frame:0 TX packets:312209 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7830981089 (7.2 GiB) TX bytes:23225168 (22.1 MiB)

eth0:1 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.116 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:2 Link encap:Ethernet HWaddr 08:00:27:33:36:D3 inet addr:192.168.60.113 Bcast:192.168.60.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 08:00:27:BC:0B:C5 inet addr:192.168.100.101 Bcast:192.168.100.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:febc:bc5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:9864 errors:0 dropped:0 overruns:0 frame:0 TX packets:8918 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:8909508 (8.4 MiB) TX bytes:7791457 (7.4 MiB) eth1:1 Link encap:Ethernet HWaddr 08:00:27:BC:0B:C5 inet addr:169.254.180.241 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:1523 errors:0 dropped:0 overruns:0 frame:0 TX packets:1523 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1627409 (1.5 MiB) TX bytes:1627409 (1.5 MiB)

- crsctl stat res –t output.

[root@rac12c1 u01]# /u01/app/12.1.0/grid/bin/crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr ONLINE ONLINE rac12c1 STABLE ONLINE ONLINE rac12c2 STABLE ora.DATA.dg ONLINE ONLINE rac12c1 STABLE ONLINE ONLINE rac12c2 STABLE ora.LISTENER.lsnr ONLINE ONLINE rac12c1 STABLE ONLINE ONLINE rac12c2 STABLE ora.net1.network ONLINE ONLINE rac12c1 STABLE ONLINE ONLINE rac12c2 STABLE ora.ons ONLINE ONLINE rac12c1 STABLE ONLINE ONLINE rac12c2 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE rac12c2 STABLE ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE rac12c1 STABLE ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE rac12c1 STABLE ora.MGMTLSNR 1 ONLINE ONLINE rac12c1 169.254.173.187 192. 168.100.100,STABLE ora.asm 1 ONLINE ONLINE rac12c1 Started,STABLE 2 ONLINE ONLINE rac12c2 Started,STABLE 3 OFFLINE OFFLINE STABLE ora.cvu 1 ONLINE ONLINE rac12c1 STABLE ora.gns 1 ONLINE ONLINE rac12c1 STABLE ora.gns.vip 1 ONLINE ONLINE rac12c1 STABLE ora.mgmtdb 1 ONLINE ONLINE rac12c1 Open,STABLE ora.oc4j 1 ONLINE ONLINE rac12c1 STABLE ora.rac12c1.vip 1 ONLINE ONLINE rac12c1 STABLE

ora.rac12c2.vip 1 ONLINE ONLINE rac12c2 STABLE ora.scan1.vip 1 ONLINE ONLINE rac12c2 STABLE ora.scan2.vip 1 ONLINE ONLINE rac12c1 STABLE ora.scan3.vip 1 ONLINE ONLINE rac12c1 STABLE --------------------------------------------------------------------------------

3. RDBMS Installation

- Unzip the binaries and change owner of the database folder.

[root@rac12c1 ~]# unzip p13390677_112040_Linux-x86-64_1of7.zip [root@rac12c1 ~]# unzip p13390677_112040_Linux-x86-64_2of7.zip [root@rac12c1 ~]# chown -R oracle:oinstall database/

- Start the installer.

[oracle@rac12c1 ~]$ cd database [oracle@rac12c1 ~]$ ./runInstaller

- Root.sh outputs.

[root@rac12c1 ~]# /u02/app/oracle/product/12.1.0/db/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u02/app/oracle/product/12.1.0/db Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.

[root@rac12c2 ~]# /u02/app/oracle/product/12.1.0/db/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u02/app/oracle/product/12.1.0/db Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.

4. Container Database Creation

- Start database configuration assistant (DBCA).

[oracle@rac12c1 ~]# /u02/app/oracle/product/12.1.0/db/bin/dbca

- Connect Enterprise Manager Express

https://rac12c1.gantek.com:5500/em

- Connect container database

[root@rac12c1 ~]# su - oracle [oracle@rac12c1 ~]$ . oraenv ORACLE_SID = [beetle] ? contdb The Oracle base has been set to /u01/app/oracle [oracle@rac12c1 ~]$ sqlplus sys/oracle123@contdb as sysdba SQL*Plus: Release 12.1.0.2.0 Production on Thu Aug 18 15:08:50 2016 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Advanced Analytics and Real Application Testing options SQL> sho parameter name NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ cell_offloadgroup_name string db_file_name_convert string db_name string contdb db_unique_name string contdb global_names boolean FALSE instance_name string contdb2 lock_name_space string log_file_name_convert string pdb_file_name_convert string processor_group_name string service_names string contdb

References;

Grid Infrastructure Installation Guide for Linux

Real Application Clusters Installation Guide for Linux and UNIX