tow nodes Oracle 12c RAC on virtualbox

Oracle 12c Clustering

A two-node RAC installationOn

VirtualBox

Efficient Learning

• Research• Repeat• Share• Gain experience and have fun

Agenda• Target and Resources• Session 1

– Install Oracle Linux on Virtual Box– Prepare it for Oracle Installation– Clone a second node and make it work

• Session 2 or maybe a Session 3– IP planning and Configure DNS– Install Grid Infrastructure– Install Oracle RDBMS and Create Database– Verifying & Exploring RAC

• New topic volunteering opportunities

Target• To build up a two-node RAC based on Oracle 12c hosted by

Oracle Linux 6.5 and virtualized by Oracle VirtualBox

racnd1 racnd2

Private network

public network

ASM1

orcl1orcl2

ASM2

Cluster SCAN:orcl-scanSID:orcl

Virtual network

192.168.1.20192.168.1.10

192.168.1.30/31/32

192.168.1.21192.168.1.11

-- VirtualBox-- No NTP-- Simple Storage Solution

Resources• J:\ICS\ICSDev\Applications\Software\RACLab

– GridAndORA: contains Oracle 12c database and Grid Infrastructure installation disks

– OraLinux: Oracle Unbreakable Enterprise Linux 6.5– VirtualBox: Oracle VirtualBox installer for windows– RAC2NodesExample: a completed installation

• J:\ICS\ICSDev\Applications\Software\VMWare Images\RAC12C– Two nodes RAC built in VMWare workstation.– More complex with iSCSI as shared storage, NTP setup and

OpenFiler as NTP Server and host of shared storage

Another Example

oelnd1 oelnd2

Private network

public network

ASM1

orcl1 orcl2

ASM2

openFiler

ASM4

ntp

iscsi

iscsi iscsi

Cluster SCAN:oelndSID:orcl

Virtual network

ntp ntp

iSCSI targets

VMWareOpenFileriSCSINTP

Session 1

– Install Oracle Linux on Virtual Box– Install client additions– Prepare it for Oracle Installation– Prepare shared storage– Clone a second node and make it work• Remove clone of shared disks• Define shared disks• Reconfigure network interfaces

Prepare directories and resources

– Make directory d:\VboxVms and d:\R12cResource– Copy oracleLinux6.5_V41362-01 to d:\

R12cResource; copy all files under J:\ICS\ICSDev\Applications\Software\RACLab\GridAndORA to d:\R12cResource

– Extract all the zip files in d:\R12cResource. You will have database for RDBMS and grid for grid infrastructure

Install Oracle Linux on Virtual Box• Start VirtualBox. • Change default machine folder to D:\VBVMs• New to create new virtual machine

Install Oracle Linux

• Memory size: 2560mb• Hard drive: create a virtual hard drive now:

VDI, Dynamically allocated,30GB• Network: first two as host-only adapter, third

one as NAT. Make all enabled

Install Oracle Linux

• Start the VM, choose the Linux ISO as startup disk

Install Oracle Linux• Skip media checking• Name it as racnd1• Network:

– Eth0: auto connect, manual ip 192.168.1.10/255.255.255.0– Eth 1:auto connect, manual ip 192.168.2.10/255.255.255.0– Eth2: not auto connect, DHCP

• Install as desktop so that x-windows server be installed. Check HA is you like.

Install VirtualBox client additions• After rebooting, logon with root.• Enabled eth2 and connect to internet

– yum install gcc kernel-uek-devel-3.8.13-16.2.1.el16uek.x86_64– Reboot

• DevicesInsert Gest Additions… run and let it finish. Eject the CD.

• Mouse cursor is no longer captured• Shared folder should be supported now

Prepare Oracle Installation

• Enable eth2• yum install oracle-rdbms-server-12cR1-preinstall– this will download all the dependencies that are

required by oracle installation. Create oracle user, oinstall group, dba group. Log file is at /var/log/oracle-rdbms-server-12cR1-preinstall/results/orakernel.log

• Verify users and groups are created– ls /etc/passwd ls /etc/group

• passwd oracle

Prepare Oracle Installation

• Make oracle base– mkdir -p /u01/app/oracle– chown -R oracle:oinstall /u01

• Define variable– vi /home/oracle/.bash_profile• export ORACLE_BASE= /u01/app/oracle

Prepare Shared Storage

• shutdown now –h• Configure racnd1’s storage to have two shared

disks:ASM1 and ASM2– mkdir D:\VBVMs\SharedDisks– Create new disk under SATA controller– VDI, Fixed size, 6GB


• File->Virtual Media Manager to make them shareable

• Start VM


• fdisk –l to verify sdb and sdc are available• Create partition on new disks– fdisk /dev/sdb• n,p,1,use default, use default,w

– fdisk /dev/sdc• n,p,1,use default, use default,w

– fdisk –l to verify

Prepare Shared Storage• Make persistent storage reference

– echo options=-g >/etc/scsi_id.config– less /etc/scsi_id.config to confirm– prepare /etc/udev/rules.d/99-ora-asm-devices.rules by running the

following scripts:i=1cmd="/sbin/scsi_id -g -u -d"for disk in sdb sdc ; do cat <<EOF >> /etc/udev/rules.d/99-ora-asm-devices.rulesKERNEL=="sd?1", BUS=="scsi", PROGRAM=="$cmd /dev/\$parent", RESULT=="`scsi_id -g -u -d /dev/$disk`", NAME="asm-disk$i", OWNER="oracle", GROUP="dba", MODE="0660"EOF i=$(($i+1))done

Prepare shared storage

• Refresh udev– /sbin/partprobe /dev/sdb1 /dev/sdc1– /sbin/udevadm test block/sdb/sdb1– /sbin/udevadm test /block/sdb/sdc1– /sbin/udevadm control --reload-rules– /sbin/start_udev– ls /dev/asm* should be able to see the two disks

Disable Features/Services• Disable selinux– Edit /etc/sysconfig/selinux

• selinux=disabled

• Disable NTP so that OUI will decide to use Cluster Time Synchronization Service (CTSS) which is part of Oracle Clusterware.– mv /etc/ntp.conf /etc/ntp.conf.orig

• Disable firewall– /etc/rc.d/init.d/iptables status– /etc/rc.d/init.d/iptables stop– chkconfig iptables off

Clone an VM

• Now we’ve got an VM that is almost all prepared for Oracle installation

• Clone it so that we will save time on preparing the second node.– Machine/clone… full clone. This will take a while– Name it racnd2 with MAC address reinitialized– Reconfigure storage

• Remove newly cloned and add shared disks• Remove the newly cloned disks permanently from Virtual

Media Manager

Clone an VM• Power on and configure the network

– Remove the ones cloned from racnd1• System eth0,1,2

– Auto eth3: manual 192.168.1.20/255.255.255.0– Auto eth4:manual 192.168.2.20/255.255.255.0– Auto eth5: not connecting automatically, DHCP– Change hostname to be racnd2

• /etc/sysconfig/network– HOSTNAME=racnd2

– rm /etc/udev/rules.d/70-persistent-net.rules• Reboot to automatically regenerate the rule file

– Verify the shared storage is still available• Ls /dev/asm*

Session2

– IP planning – Configure DNS– Install Grid Infrastructure– Install Oracle RDBMS and Create Database– Verification: get the feet wet on RAC

IP Planning• racnd1

– 192.168.1.10 racnd1– 192.168.2.10 racnd1-priv– 192.168.1.11 racnd1-vip

• racnd2– 192.168.1.20 racnd2– 192.168.2.20 racnd2-priv– 192.168.1.21 racnd2-vip

Single client access name:orcl-scan

• 192.168.1.30 orcl-scan• 192.168.1.31 orcl-scan• 192.168.1.32 orcl-scan

• Virtual IP addresses must be in same network as public ip addresses. They are used by cluster ware to implement its failover mechanism. E.g. in tnsnames.ora

• Private IP is for cluster’s internal communications(cache fusion, interconnect).• Public IP is normal IP address. DBAs use them to access the server• SCAN is a single service name known by clients. A name that represents the

cluster.

Configure DNS

• Make racnd1 as DNS server– Enable internet by enabling eth2– yum install bind-libs bind bind-utils– /etc/named.conf• gedit /etc/named.conf• Replace the content with the content in the embedded

object• Save and close //

// named.conf

//

// Provided by Red Hat bin d package to configure the ISC BIND named(8) DNS

// server as a caching only nameserver (as a lo calhost DNS resolver only).

//

// See /usr/share/doc/bind*/sample/ for example named configuration files.

//

options {

li sten-on port 53 { 127.0.0.1;192.168.1.10; };

li sten-on -v6 port 53 { ::1; } ;

directory "/var/named";

dump-file "/var/named/data/c ache_dump.db";

statistics-file "/var/named/data/named_stats.txt";

memstatis tics-file "/var/named/data/named_mem_stats.txt";

recursio n yes;

dnssec-enable yes ;

dnssec-val idation yes;

dnssec-loo kaside auto;

/* Path to ISC DL V key */

bindkeys-file "/etc/named.iscdlv.key";

managed-keys-di rectory "/var/named/dynamic";

};

logging {

channel default_debug {

file "data/named.run";

severity dynamic;

};

};

zone " localdomain." IN {

type master;

file "lo caldomain.zone";

allow-update { none; };

};

zone " 1.168.192.in-addr.arpa." IN {

type master;

file "1.168.192.in-ad dr.arpa";

allow-update { none; };

};

include "/etc/named.rfc1912.zones";

include "/etc/named.ro ot.key";

Configure DNS

• Make racnd1 DNS server – continued– gedit /var/named/localdomain.zone• Fill it with the content in the embedded object.

– Gedit /var/named/1.168.192.in-addr.arpa• Fill it with the content in the embedded object

$TTL 86400

@ IN SOA localhost root.localhost (

42 ; serial (d. adams)

3H ; refresh

15M ; retry

1W ; expiry

1D ) ; minimum

IN NS localhost

localhost IN A 127.0.0.1

racnd1 IN A 192.168.1.10

racnd2 IN A 192.168.1.20

racnd1-priv IN A 192.168.2.10

racnd2-priv IN A 192.168.2.20

racnd1-vip IN A 192.168.1.11

racnd2-vip IN A 192.168.1.21

orcl-scan IN A 192.168.1.30



$TTL 1H

@ IN SOA racnd1.localdomain. root.racnd1.localdomain. ( 2

3H

1H

1W

1H )

1.168.192.in-addr.arpa. IN NS localdomain.

10 IN PTR racnd1.localdomain.

20 IN PTR racnd2.localdomain.

11 IN PTR racnd1-vip.localdomain.

21 IN PTR racnd2-vip.localdomain.

30 IN PTR orcl-scan.localdomain.



Configure DNS• Make racnd1 as DNS server – continued

– Start DNS• Service named start

– Start it automatically• chkconfig named on

• Make both racnd1 and racnd2 to use DNS– Edit the adapter of 192.168.1 network

to have 192.168.1.10 as their name server.

Configure hosts

• On racnd1, Replace /etc/hosts with content in embedded document– gedit /etc/hosts

– On racnd2, replace /etc/hosts with content in embedded document• gedit /etc/hosts

– Restart both VMs– Verify nslookup

• nslookup racnd2 or racnd1• nslookup racnd1 on racnd2

127.0.0.1 racnd1 localhost racnd1.localdomain localhost.localdomain localhost4 localhost4.localdomain4

::1 racnd1 localhost racnd1.localdomain localhost.localdomain localhost6 localhost6.localdomain6

192.168.1.10 racnd1

192.168.1.20 racnd2

192.168.2.10 racnd1-priv

192.168.2.20 racnd2-priv

192.168.1.11 racnd1-vip

192.168.1.21 racnd2-vip

192.168.1.30 orcl-scan

192.168.1.31 orcl-scan

192.168.1.32 orcl-scan

127.0.0.1 racnd2 localhost racnd2.localdomain localhost.localdomain localhost4 localhost4.localdomain4

::1 racnd2 localhost racnd2.localdomain localhost.localdomain localhost6 localhost6.localdomain6

192.168.1.10 racnd1

192.168.1.20 racnd2

192.168.2.10 racnd1-priv

192.168.2.20 racnd2-priv

192.168.1.11 racnd1-vip

192.168.1.21 racnd2-vip

192.168.1.30 orcl-scan

192.168.1.31 orcl-scan

192.168.1.32 orcl-scan

Test SSH

• Query package– rpm -qa --queryformat "%{NAME}-%{VERSION}-%

{RELEASE} (%{ARCH})\n"| grep ssh• On racnd1, as oracle– ssh racnd2

• On racnd2, as oracle– ssh racnd1

• Make sure ssh is working. If not, correct it.

Install Grid Infrastructure

• Configure the shared folders– In VirtualBox racnd1’s setting dialog, add D:\

R12cResource– Reboot the racnd1, logon as oracle– su – and mount the shared folder

• Mkdir –p /media/ R12cResource• mount -t vboxsf R12cResource /media/R12cResource• Exit

– cd /media/R12cResource/grid– ./runInstaller


• Skip software update• Install and configure oracle Grid Infrastructure for a cluster• Configure a standard cluster• Typical installation—otherwise has to be familiar with GNS the

first• SCAN name: orcl-scan• Add racnd2• Setup SSH connectivity

– Provide password,setup then test• Identify network interfaces to make sure private and public

networks are correctly associated with corresponding network


• Yes


• Accept the default software location• Storage type is ASM• sysasm password

is password


• Discovery disk by using /dev/asm*• Choose all the disks• Redunency as external


• Accept default inventory directory definition• Automatically run configuration scripts• Fix the cvuqdisk-1.0.9-1 and check again. • agree to Run fix-up scripts • Ignore other (two) warning on prerequisite checks• Review summary and install. – Slow process– Tail –f /u01/app/oraInventory/logs/installActionsyyyy-

mm-dd_xx-xx-xxPM.log


• Simple verification– Ps –ef|grep asm– export ORACLE_SID=ASM1; export

ORACLE_HOME=/u01/app/12.1.0/grid– cd /u01/app/12.1.0/grid/bin– ./asmca to view ASM instances and the disk group– crsctl stat res –t to view cluster component status

Install Oracle 12c RDBMS and create database

On racnd1– cd /media/R12cResource/database– ./runInstaller– Do not provide email and skip software update– Create and configure database– Server class– Oracle RAC database installation– Admin managed– Choose all the nodes– Typical install

Install Oracle 12c RDBMS and create database

• Storage type: ASM• Uncheck CDB

Some screens

Reminder: save response file for further study on silent installation

Run script as root

Create database

Enabled accounts

• Enabled scott and hr accounts for future use

Verification

• ps –ef|grep orcl• . oraenv– orcl1

• sqlplus / as sysdba• select instance_name,host_name from

gv$instance;

Exploring the RAC

• Connect to cluster• Transparent Application Failover

Connect to a cluster• This can be done on any machine that is not part of the cluster,

but should be in same network as cluster’s public network– From a third VW if your resource is enough– From VM’s host if an Oracle client is installed– From one of node with cluster instance name: orcl. This is simplest

method, but you have to believe it is working as if you connect from third computer

• I adopt method 2– http://www.oracle.com/technetwork/topics/winx64soft-089540.html

• instantclient-basiclite-windows.x64-12.1.0.1.0• instantclient-sqlplus-windows.x64-12.1.0.1.0• Unzip them to same folder

http://www.oracle.com/technetwork/topics/winx64soft-089540.html

http://www.oracle.com/technetwork/topics/winx64soft-089540.html

Configure host network• Go to property of VirtualBox Host-Only Network in network

connections• Bring up the IPv4’s properties

– Change IP to be 192.168.1.1, DNS to be 192.168.1.10– In DNS, add localdomain as DNS suffix– Test the network by

• ping 192.168.1.10 • Ping racnd1.localdomain• Ping racnd1• Ping orcl-scan

– Verify SCAN is running • ./svrctl status scan• ./svrctl status scan_listener

Connect from host

• Using basic connection descriptor– sqlplus scott/[email protected]/orcl– sqlplus scott/tiger@orcl-scan/orcl– But accessing to individual instance will fail because

local listener is not accessible directly remotely. It has to go through SCAN listener the first. The SCAN listener pick one with less work load to serve the client• sqlplus scott/[email protected]/orcl1• sqlplus scott/[email protected]/orcl

mailto:scott/[email protected]/orcl



mailto:scott/[email protected]/orcl1

mailto:scott/[email protected]/orcl1

Failover is not working?• Start one node, connect to it from client. Then start second

node.• Now shutdown first node, querying database will return end

of file error• Restart first node, and re-issue query will be reported not

connected to oracle.

Transparent Application Failover

• Applications and users are automatically and transparently reconnected to another system, applications and queries continue uninterrupted, and the login context is maintained. But transaction will be aborted and transaction management has to be done inside application.

Configure TAF• It’s through client side configuration. I guess the SCAN is only

meaningful here in term of new connections can still connect to same database as long as one instance is still running.

• tnsnames.oraraclab = (DESCRIPTION = (FAILOVER = ON) (LOAD_BALANCE = OFF) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.11)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.21)(PORT = 1521)) ) (CONNECT_DATA = (SERVICE_NAME = orcl) (FAILOVER_MODE = (TYPE = SELECT) (METHOD = PRECONNECT) (BACKUP=192.168.1.21) ) ) )

Test Failover 1• sqlplus sys@rablab as sysdba

– Check instance that is serving

• On racnd1, abort servicesrvctl status database -d orclsrvctl stop instance -d orcl -i orcl1 -o abort

• Rerun the query. Check which node is serving. It should failover to racnd2

• Start the node racnd1, then shutdown node racnd2srvctl start instance -d orcl -i orcl1srvctl stop instance -d orcl -i orcl2 -o abort

• Rerun the query, check which node is serving. It should go back to racnd1

COLUMN instance_name FORMAT a13

COLUMN host_name FORMAT a9

COLUMN failover_method FORMAT a15

COLUMN failed_over FORMAT a11

SELECT DISTINCT

v.instance_name AS instance_name,

v.host_name AS host_name,

s.failover_type AS failover_type,

s.failover_method AS failover_method,

s.failed_over AS failed_over

FROM v$instance v, v$session s

WHERE s.username = 'SYS';

Test Failover 2

• Configure another TNS name orcllab without failover enabled

• Create test package under scott• Run this script to retrieve result from a slow query

select * from table(AMATest.f_SlowResponse (restSecond => 1));

• Non-failover connection, returns no row• Failover connection returns rows normally but

using longer time

CREATE OR REPLACE PACKAG E AMATes t

IS

Type recRow is Rec ord

(

f1 varc har2(30) ,

f2 number

) ;

TYPE Rec Tabl e IS TABLE O F recRow;

FUN CTIO N f_Sl owResponse(restSecond int)

return AM ATest.RecTable PIPELI NED D ETERMIN IS TIC;

EN D AMATest;

/

CREATE OR REPLACE PACKAG E BODY AMATest

is

/*

exa mpl e:

se l ect * f rom tabl e(AMATest.f_S lowRes ponse (restS ec ond => 2)) ;

*/

FUN CTIO N f_Sl owResponse(restSecond int)

return AM ATest.RecTable PIPELI NED D ETERMIN IS TIC

is

begi n

FO R aRec I N (sel ect ena me,deptno from sc ott.em p)

LOO P

pipe ROW(a Rec);

dbms_l ock.sleep(restS econd);

EN D LO OP;

return;

end;

end AMATest;

/

orcllab =

(DESCRIPTION =

(FAILOVER = ON)

(LOAD_BALANCE = OFF)

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.30)(PORT = 1521))

)

(CONNECT_DATA =

(SERVICE_NAME = orcl)

)

)

Q&A

• Sparkle new topics

tow nodes Oracle 12c RAC on virtualbox

Self Improvement

Transcript of tow nodes Oracle 12c RAC on virtualbox