Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud...

28
COLLABORATE 14 – IOUG Forum Cloud Computing 1 | Page Extreme Oracle DB Infrastructure as a Service White Paper Extreme Oracle DB Infrastructure As A Service Charles Kim, Viscosity North America ABSTRACT Learn the secrets of the trade to rapidly provisioning Oracle DB Infrastructure-As-A-Service. This extreme session will cover topics of delivering Linux-As-S-Service, RAC-As-A-Service, ASM-As-A-Service, Database-As-A-Service, Backup-As-A- Service, and even Data-Guard-As-A-Service . Advanced techniques to deploy enterprise RAC and non-RAC database deployments in an automated fashion will be shared . Save days and even weeks of deploy time by attending this session. There is no reason why you as a DBA or Architect, should not be able to deploy a fully patched RAC environment from bare metal Linux and create a RAC database in less than one hour. Anyone deploying RAC or even non-RAC will learn the secret sauce and knowledge of how to properly deploy mission critical systems that is repeatable and consistent. Learn to deploy a fully patched (11.2.0.3 or 11.2.0.4 with PSU x or 12.1 with PSU x) two node RAC in less than one hour. Learn how to automate database builds and to leverage golden image database templates. We can't forget about multi-tenant deployment of Oracle 12c Pluggable Databases. Learn how to deploy pluggable databases (PDB) and to migrate PDBs and significantly increase your database consolidation density. The details of adding nodes to an existing clusters and removing nodes from the cluster will also be disseminated. TARGET AUDIENCE Any DBA, System Administrators, or Architect who want to save time from doing mundane database administrative tasks and do things more important should attend this session. EXECUTIVE SUMMARY DBAs start to work on RAC or stand-alone database builds, they are given a server as a blank canvas. We have to mold what we have and create an end product that has to become a masterpiece that performs well. We need to tweak from the Operating System level and work down the stack. In this two hour session, our goal is to demonstrate how you can rapidly provide: Linux as a Service RAC as a service ASM as a service Database as a service (DBaaS). We will proceed to the other MAA components and look at how to rapid provide Backup and Recovery as a service and Data Guard as a service. And if you are licensed for the multitenant option or are just curious to learn more about consolidation and achieving a deeper level of consolidation, we will extend DBaaS to encompass the pluggable database. LINUX AS A SERVICE The greatest advantage of choosing Oracle Enterprise Linux (Oracle Linux) over Red Hat Enterprise Linux was that Oracle provides a single RPM to literally setup and configure the Linux environment which includes setup of pertinent kernel parameters, unix accounts, unix groups, home directories, and most important, all the RPMs with all the dependencies. Earlier in 2009, Oracle launched their public yum repository which offers a free and convenient way to install packages from the Enterprise Linux. Obviously, the public yum server is offered with limited support.

Transcript of Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud...

Page 1: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

1 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Extreme Oracle DB Infrastructure As A Service

Charles Kim, Viscosity North America

ABSTRACT Learn the secrets of the trade to rapidly provisioning Oracle DB Infrastructure-As-A-Service. This extreme session will cover topics of delivering Linux-As-S-Service, RAC-As-A-Service, ASM-As-A-Service, Database-As-A-Service, Backup-As-A-Service, and even Data-Guard-As-A-Service . Advanced techniques to deploy enterprise RAC and non-RAC database deployments in an automated fashion will be shared . Save days and even weeks of deploy time by attending this session. There is no reason why you as a DBA or Architect, should not be able to deploy a fully patched RAC environment from bare metal Linux and create a RAC database in less than one hour. Anyone deploying RAC or even non-RAC will learn the secret sauce and knowledge of how to properly deploy mission critical systems that is repeatable and consistent. Learn to deploy a fully patched (11.2.0.3 or 11.2.0.4 with PSU x or 12.1 with PSU x) two node RAC in less than one hour. Learn how to automate database builds and to leverage golden image database templates. We can't forget about multi-tenant deployment of Oracle 12c Pluggable Databases. Learn how to deploy pluggable databases (PDB) and to migrate PDBs and significantly increase your database consolidation density. The details of adding nodes to an existing clusters and removing nodes from the cluster will also be disseminated.

TARGET AUDIENCE Any DBA, System Administrators, or Architect who want to save time from doing mundane database administrative tasks and do things more important should attend this session.

EXECUTIVE SUMMARY DBAs start to work on RAC or stand-alone database builds, they are given a server as a blank canvas. We have to mold what we have and create an end product that has to become a masterpiece that performs well. We need to tweak from the Operating System level and work down the stack. In this two hour session, our goal is to demonstrate how you can rapidly provide: • Linux as a Service

• RAC as a service • ASM as a service

• Database as a service (DBaaS). We will proceed to the other MAA components and look at how to rapid provide Backup and Recovery as a service and Data Guard as a service. And if you are licensed for the multitenant option or are just curious to learn more about consolidation and achieving a deeper level of consolidation, we will extend DBaaS to encompass the pluggable database.

LINUX AS A SERVICE The greatest advantage of choosing Oracle Enterprise Linux (Oracle Linux) over Red Hat Enterprise Linux was that Oracle provides a single RPM to literally setup and configure the Linux environment which includes setup of pertinent kernel parameters, unix accounts, unix groups, home directories, and most important, all the RPMs with all the dependencies. Earlier in 2009, Oracle launched their public yum repository which offers a free and convenient way to install packages from the Enterprise Linux. Obviously, the public yum server is offered with limited support.

Page 2: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

2 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Starting with Oracle Database 11g Release 2 and Oracle Linux 6 including Unbreakable Enterprise Kernel (UEK), Oracle replaced the oracle-validated RPM with the oracle-rdbms-server-11gR2-preinstall RPM (the Oracle RDBMS Server 11gR2 Pre-install RPM) to pre-configure the Linux operating system for installation of Oracle Grid Infrastructure and Database. To successfully install the Oracle Database 11g Release 2 on Oracle Linux 6, there are quite a bit of pre-installation and configuration that have to be met. Using the Oracle RDBMS Server 11gR2 Pre-install RPM, you can complete majority of the pre-installation configuration tasks and automate the Linux server build. If you are leveraging Red Hat Linux, the public yum repository was considered to be of no or little use. In this section, we will show you how to leverage Oracle’s oracle-rdbms-server-11gR2-preinstall or oracle-rdbms-server-12c-preinstall RPM for your Red Hat environment as still maintain your support and maintenance from Red Hat.

CONFIGURE ORACLE-RDBMS-SERVER-12C-PREINSTALL FOR RED HAT In this section, you will learn how to configure the oracle-rdbms-server-12c-preinstall RPM so that we can leverage this RPM on Red Hat Linux. First install two RPMs to build both binary/source software packages and to aid in development of RPM packages. yum install rpm-build rpmdevtools –y

Create linux user called mockbuild. We need to keep the same username that oracle used to compile it’s package to avoid a rpmbuild error: useradd mockbuild -p mockbuild -m -s /bin/bash

su to the mockbuild user and execute the rpmdev-setuptree command: [root@duppy yum.repos.d]# su - mockbuild [mockbuild@duppy ~]$ rpmdev-setuptree The rpmdev-setuptree program will create the ~/rpmbuild directory and a set of subdirectories (e.g. SPECS and BUILD), which you will use for creating your packages. [mockbuild@duppy ~]$ ls -l rpmbuild total 20 drwxrwxr-x. 2 mockbuild mockbuild 4096 Jan 26 19:12 BUILD drwxrwxr-x. 2 mockbuild mockbuild 4096 Jan 26 19:12 RPMS drwxrwxr-x. 2 mockbuild mockbuild 4096 Jan 26 19:12 SOURCES drwxrwxr-x. 2 mockbuild mockbuild 4096 Jan 26 19:12 SPECS drwxrwxr-x. 2 mockbuild mockbuild 4096 Jan 26 19:12 SRPMS

We will try a mock installation of the source RPM [mockbuild@rh6u3 ~]$ rpm -ivh oracle-rdbms-server-11gR2-preinstall-1.0-6.el6.src.rpm warning: oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.src.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY 1:oracle-rdbms-server-12c########################################### [100%]

Attempting to install the source rpm with the rpm –ihv command will returns the prompt back instantly. We must check the .spec file located in the rpmbuild/SPEC subdirectory We need to remove the line in the SPEC file that has the hard dependency on the Unbreakable Enterprise Kernel (uek) [mockbuild@duppy ~]$ vi rpmbuild/SPECS/oracle-rdbms-server-12cR1-preinstall.spec #System requirement Requires:procps module-init-tools ethtool initscripts Requires:bc bind-utils nfs-utils util-linux-ng pam Requires:xorg-x11-utils xorg-x11-xauth # -- get rid of the line below # Requires:kernel-uek Requires:smartmontools

Page 3: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

3 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Next, execute the rpmbuild to create the .rpm file which will create the binary package used to encapsulate software to be installed. [mockbuild@duppy ~]$ rpmbuild -ba rpmbuild/SPECS/oracle-rdbms-server-12cR1-preinstall.spec

The -ba options will build source and binary packages from. We will build binary and source packages (after doing the %prep, %build, and %install stages). Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.9guAHq + umask 022 + cd /home/mockbuild/rpmbuild/BUILD + echo RPM_BUILD_ROOT=/home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 RPM_BUILD_ROOT=/home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 + cd /home/mockbuild/rpmbuild/BUILD + rm -rf oracle-rdbms-server-12cR1-preinstall-1.0 + /bin/tar -xf - + /usr/bin/gzip -dc /home/mockbuild/rpmbuild/SOURCES/oracle-rdbms-server-12cR1-preinstall-1.0.tar.gz + STATUS=0 + '[' 0 -ne 0 ']' + cd oracle-rdbms-server-12cR1-preinstall-1.0 + /bin/chmod -Rf a+rX,u+w,g-w,o-w . + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.evgOV9 + umask 022 + cd /home/mockbuild/rpmbuild/BUILD + cd oracle-rdbms-server-12cR1-preinstall-1.0 + exit 0 Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.yoXMaT + umask 022 + cd /home/mockbuild/rpmbuild/BUILD + cd oracle-rdbms-server-12cR1-preinstall-1.0 + rm -rf /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 + mkdir -p -m 755 /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/sysconfig/oracle-rdbms-server-12cR1-preinstall + mkdir -p -m 755 /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/usr/bin + mkdir -p -m 755 /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/rc.d/init.d + mkdir -p -m 755 /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/security/limits.d + mkdir -p -m 700 /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/var/log/oracle-rdbms-server-12cR1-preinstall/results + install -m 700 oracle-rdbms-server-12cR1-preinstall-verify /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/sysconfig/oracle-rdbms-server-12cR1-preinstall + install -m 700 oracle-rdbms-server-12cR1-preinstall-verify /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/usr/bin + install -m 600 oracle-rdbms-server-12cR1-preinstall.param /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/sysconfig/oracle-rdbms-server-12cR1-preinstall + install -m 700 oracle-rdbms-server-12cR1-preinstall-firstboot /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/rc.d/init.d + touch /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/etc/security/limits.d/oracle-rdbms-server-12cR1-preinstall.conf + ln -f -s /etc/sysconfig/oracle-rdbms-server-12cR1-preinstall/oracle-rdbms-server-12cR1-preinstall-verify /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64/usr/bin/oracle-rdbms-server-12cR1-preinstall-verify + /usr/lib/rpm/check-rpaths /usr/lib/rpm/check-buildroot + /usr/lib/rpm/brp-compress + /usr/lib/rpm/brp-strip + /usr/lib/rpm/brp-strip-static-archive + /usr/lib/rpm/brp-strip-comment-note Processing files: oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 warning: File listed twice: /etc/sysconfig/oracle-rdbms-server-12cR1-preinstall/oracle-rdbms-server-12cR1-preinstall-verify warning: File listed twice: /etc/sysconfig/oracle-rdbms-server-12cR1-preinstall/oracle-rdbms-server-12cR1-preinstall.param Provides: config(oracle-rdbms-server-12cR1-preinstall) = 1.0-11.el6 oracle-rdbms-server-12cR1-preinstall = 1.0 Requires(interp): /bin/sh /bin/sh /bin/sh /bin/sh

Page 4: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

4 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1 Requires(pre): /bin/sh /etc/redhat-release Requires(post): /bin/sh Requires(preun): /bin/sh Requires(postun): /bin/sh Requires: /bin/bash Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 Wrote: /home/mockbuild/rpmbuild/SRPMS/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.src.rpm Wrote: /home/mockbuild/rpmbuild/RPMS/x86_64/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.0RnMI5 + umask 022 + cd /home/mockbuild/rpmbuild/BUILD + cd oracle-rdbms-server-12cR1-preinstall-1.0 + rm -rf /home/mockbuild/rpmbuild/BUILDROOT/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 + exit 0

Confirm that the RPM was successfully created in the rpmbuild/RPMS/x86_64 subdirectory ls -lR rpmbuild/RPMS/x86_64 rpmbuild/RPMS/x86_64: total 16 -rw-rw-r--. 1 mockbuild mockbuild 14833 Jan 26 19:21 oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64.rpm

You need to save this RPM as it will be the toolkit for you to provision Linux As A Service on bare-metal servers. We can simply perform the bare minimum install and use the RPM that we created to rapid provision a Linux server ready for Oracle installation. As root, you can install the 12cR1 preinstall RPM.

PROVISION RED HAT SERVER WITH ORACLE-RDBMS-SERVER-12C-PREINSTALL When you install Red Hat, you can do a minimal installation of Red Hat 6, or any other installation of your choice (Desktop, Workstation, etc). For security purpose, most production environments run in runlevel 3 and so the minimal installation should be the preferred installation of choice. In this section, we will demonstrate how you can re-purpose the oracle-rdbms-server-12c-preinstall RPM on Red Hat 6 the same way you would do for Oracle Linux 6.

Simply install the oracle-rdbms-server-preinstall RPM and your OS will be completely configured to install/configure Grid Infrastructure and Oracle database software. On Red Hat, we will want to perform a localinstall to install a set of local rpm files.

yum localinstall ~mockbuild/rpmbuild/RPMS/x86_64/oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64.rpm … … Installed: oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-11.el6 Dependency Installed: cloog-ppl.x86_64 0:0.15.7-1.2.el6 compat-libcap1.x86_64 0:1.10-1 compat-libstdc++-33.x86_64 0:3.2.3-69.el6 cpp.x86_64 0:4.4.7-4.el6 gcc.x86_64 0:4.4.7-4.el6 gcc-c++.x86_64 0:4.4.7-4.el6 ksh.x86_64 0:20120801-10.el6 libXmu.x86_64 0:1.1.1-2.el6 libXxf86dga.x86_64 0:1.1.3-2.el6 libXxf86misc.x86_64 0:1.0.3-4.el6 libaio-devel.x86_64 0:0.3.107-10.el6 libdmx.x86_64 0:1.1.2-2.el6 libstdc++-devel.x86_64 0:4.4.7-4.el6 mpfr.x86_64 0:2.4.1-6.el6 ppl.x86_64 0:0.10.2-11.el6 xorg-x11-utils.x86_64 0:7.5-6.el6 xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6

Page 5: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

5 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Complete!

We can now proceed with the Oracle Grid Infrastructure and Database 12c.

BACKGROUND RAC Provisioning is extremely simple if you know the secret sauce. If you know the secret sauce, you will quickly learn that we do not install and configure RAC software and any of the patches anymore. We can simply clone the RAC environment from one cluster to the other. The beauty of cloning the RAC environment (both Grid Infrastructure and Database) is that we are taking a pristine golden image platform and cloning that to other environments. A pristine environment will consist of a fully tested and integrated stack. For example, as of January of 2014, Oracle support released 12.1.0.1.2, 11.2.0.4.1 and 11.2.0.3.9 PSU. Please review MOS ID: 1454618.1: Quick Reference to Patch Numbers for Database PSU, SPU(CPU), Bundle Patches and Patchsets for future updates. Non-RAC environments are even easier to deploy in an automated fashion, as we do not have to worry about components like shared disks, voting disks, cluster registry, networks for private interconnect and public network traffic. If you are leveraging ASM in a non-RAC environment, maintenance of the Grid Infrastructure poses a little overhead to the effort. In a non-RAC environment, we can tar and un-tar Oracle Homes and simply plug them in to the new server. We can do this with RAC also but more infrastructure steps are required and pre-configuration steps are necessary. In this paper, we will discuss how we can easily instantiate the non-RAC environment first; then we will take the same concept and instantiate a RAC environment.

STANDARDIZATION Standardization is crucial to effective and rapid Oracle GI and database software provisioning. As part of the corporate standards, DBAs need to standardize on common directory structures for ORACLE_HOME for Grid Infrastructure, ORACLE_HOME for the Database Home, ORACLE_BASE and INVENTORY_LOCATION. This same logic applies to naming ASM disks (whether you use udev rules or ASMLIB), network interface names, bonded interface names, VIP names, private hostnames, SCAN names, cluster names, and so on. The more you standardize, easier the automation will be and you will be able to achieve the most important aspect of automation, consistent reliable builds. On the database front, standardization for database names, ASM disk groups, initialization parameters, database accounts, stored procedures, triggers, and so on will enable you to automate a consistent and reliable database build.

PREPARE RAC AS A SERVICE The first thing that we have to do is prepare the environment so that we can capture an image that we want to replicate. We want to capture an image of the operating system binary and a golden copy of the database that we want to provision to new environments that we will create. Non-RAC and RAC database provisioning starts with cloning the Grid Infrastructure and Oracle Home binaries. To clone our database environment, we must copy the software stack from the source to the target system that we are trying to provision. The complete binary code that make up the Grid Infrastructure Home and Oracle Home can be archived in a single TAR streaming archive file. We will want to clean up various log and output directories. This will not only clean up our archive file but also reduce the archive in size. Instead of cleaning up various log and output directories, we can simply exclude them while we are creating our archive file. This can be accomplished with the –X option. export GRID_HOME=/u01/app/grid/11.2.0 cd ${GRID_HOME} tar cpfX - ${EXCLUDE_LIST} * > ${NFS_DIR}/Grid.tar

Page 6: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

6 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

The environment variables for EXCLUDE_LIST and NFS_DIR are defined in a master configuration file. You will want to set the values for these environment variables to fit your environment with the export syntax (i.e. EXCLUDE_LIST=/u01/app/oracle/rac/EXCLUDE_GRID.txt and export NFS_DIR=/nfsdir/oracle). Here’s the content of the ${EXCLUDE_LIST} where ORACLE_DB_HOME should be replaced by $ORACLE_HOME and Grid_home should be replaced by $GRID_HOME or $ORACLE_CRS_HOME: ${GRID_HOME}/host_name ${GRID_HOME}/log/host_name ${GRID_HOME}/gpnp/host_name ${GRID_HOME}/crf/* ${GRID_HOME}/log/diag/asmcmd/* ${GRID_HOME}/log/diag/tnslsnr/* ${GRID_HOME}/auth/{crs,css,evm,ohasd}/* ${GRID_HOME}/ccr/hosts/* ${GRID_HOME}/cdata/* ${GRID_HOME}/cfgtoollogs/opatch/lsinv/* ${GRID_HOME}/crs/install/*.txt ${GRID_HOME}/crs/init/* ${GRID_HOME}/evm/init/* ${GRID_HOME}/evm/log/* ${GRID_HOME}/install/*.log ${GRID_HOME}/srvm/admin/oratab.bak.* ${GRID_HOME}/network/admin/*.bak.* ${GRID_HOME}/network/admin/*.ora ${GRID_HOME}/ohasd/init/* ${GRID_HOME}/root.sh ${GRID_HOME}/*ouibak* *.ouibak *.ouibak1 *.log

For RAC environments, there is a small but extremely important pre-requisite to creating an archive file for the Grid Infrastructure software directory. You must first unlock the Grid Infrastructure. Unlocking the Grid Infrastructure involves shutting down the cluster and changing file/directory permissions so that Oracle will have read/write permissions for all the necessary files. To unlock the GI Home, we issue the command: “rootcrs.pl –unlock” from the $GRID_HOME/crs/install directory. Executing the rootcrs.pl command with the –unlock option will shutdown the cluster and yield the following output: [root@rhel59dra install]# ./rootcrs.pl -unlock Using configuration parameter file: ./crsconfig_params CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel59dra' CRS-2673: Attempting to stop 'ora.crsd' on 'rhel59dra' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rhel59dra' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.rhel59drb.vip' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.FRA.dg' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.OCR_VOTE.dg' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.dtccrac.db' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.cvu' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.LSNR_GTRRAC.lsnr' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.oc4j' on 'rhel59dra' CRS-2677: Stop of 'ora.rhel59drb.vip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.LSNR_GTRRAC.lsnr' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rhel59dra' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rhel59dra' CRS-2677: Stop of 'ora.scan3.vip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.scan1.vip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rhel59dra'

Page 7: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

7 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.rhel59dra.vip' on 'rhel59dra' CRS-2677: Stop of 'ora.scan2.vip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.rhel59dra.vip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.cvu' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.registry.acfs' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.dtccrac.db' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.REDO01.dg' on 'rhel59dra' CRS-2677: Stop of 'ora.REDO01.dg' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.FRA.dg' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.oc4j' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.OCR_VOTE.dg' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rhel59dra' CRS-2677: Stop of 'ora.asm' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rhel59dra' CRS-2677: Stop of 'ora.ons' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rhel59dra' CRS-2677: Stop of 'ora.net1.network' on 'rhel59dra' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rhel59dra' has completed CRS-2677: Stop of 'ora.crsd' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.crf' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.ctssd' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.evmd' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.asm' on 'rhel59dra' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rhel59dra' CRS-2677: Stop of 'ora.crf' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.evmd' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.asm' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rhel59dra' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel59dra' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rhel59dra' CRS-2677: Stop of 'ora.cssd' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rhel59dra' CRS-2677: Stop of 'ora.gipcd' on 'rhel59dra' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rhel59dra' CRS-2677: Stop of 'ora.gpnpd' on 'rhel59dra' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel59dra' has completed CRS-4133: Oracle High Availability Services has been stopped. Successfully unlock /u01/app/11.2.0/grid

For non-RAC environments, you do not go through the preparatory steps to unlock the GI Home. You simply create an archive file for the contents of the GI Home. You should, however, exclude all the unnecessary log and output files. We can leverage the same tar command that we issued for the Grid Infrastructure home but create a separate exclude list for the database home.

This%is%where%you%may%want%to%create%a%single%TAR5ball%of%the%/u01%file%system%(or%equivalent)%which%encompasses%all%of%the%Oracle%software%stack%(GI%Home,%DB%Home,%and%other%miscellaneous%binaries%such%as%OSWatcher,%RAC%Check,%RDA,%monitoring%scripts,%and%much%more).%%The%choice%is%yours,%and%you%can%modify%the%scripts%that’s%provided%in%this%paper%as%it%suits%your%business%

For the database Oracle homes, the steps are identical whether you are on RAC or non-RAC. Here’s the tar script with the exclude list: ORACLE_DB_HOME/network/admin/* ORACLE_DB_HOME/network/log/* ORACLE_DB_HOME/dbs/*

Page 8: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

8 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

As the final step, you can create the golden image extract of the Oracle Home: export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db cd ${ORACLE_HOME} tar cpfX - ${EXCLUDE_LIST} * > ${NFS_DIR}/DB.tar

Saavy Unix folks can take advantage of the tar command and redirect the files to the ssh command and extract the files in a single command on the target server. Here’s an example of the single command that can be leveraged as root from the /u01/app/oracle/product/11.2.0 directory: tar cvf – grid |ssh target_host “cd /u01/app/11.2.0; tar xvf –“

PROVISIONING RACAAS Before you start cloning the GI Home, you need to extract the TAR archive with the tar –xvf command. You can perform the extract as either the oracle or root privileged user. Once the software binaries are extracted, you are ready to start provisioning the GI Home.

The steps to performing the GI environment is displayed above. In a nutshell, the cloning process is really nothing more than executing a perl script and following all the instructions afterwards. Then we clone the Grid Home that is fully patched and add the Grid Home to the local oracle inventory. As part of the local inventory registration, creating /etc/oraInst.loc and changing the permissions of the oraInventory location, we must execute root.sh on every node. This root.sh execution is just touches files and permissions and completes instantly. Then we configure the Grid Infrastructure by running config.sh and then complete the process by executing root.sh on each of the RAC nodes. The second round of root.sh execution takes a while longer and all the magic of the cluster configuration happens. To clone the GI stack for RAC, change your directory to the $ORACLE_HOME/clone/bin directory and execute the perl cloning script called clone.pl with the following options: export ORACLE_BASE=/u01/app/oracle export GI_HOME=/u01/app/12.1.0/grid export INVENTORY_LOCATION=/u01/app/oraInventory JAVA_HOME=${GI_HOME}/jdk/bin/java PATH=$PATH:${JAVA_HOME} cd ${GI_HOME}/clone/bin perl ${GI_HOME}/clone/bin/clone.pl -silent ORACLE_HOME=${GI_HOME} ORACLE_HOME_NAME=OraGridHome1 ORACLE_BASE=$ORACLE_BASE -O'"CLUSTER_NODES={rac01,rac02,rac03,rac04}"' -O'"LOCAL_NODE=$(hostname -s)"' CRS=TRUE INVENTORY_LOCATION=${INVENTORY_LOCATION}

To clone the GI stack for non-RAC, we execute same clone.pl script but our options are reduced, and because we do not have the CLUSTER_NODES and LOCAL_NODES parameters, our script is significantly simplified: cat clone_grid_11203.sh cd /u01/app/grid/11.2.0.3/clone/bin

Page 9: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

9 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

perl clone.pl -silent ORACLE_BASE=/u01/app/oracle ORACLE_HOME=/u01/app/grid/11.2.0.3 ORACLE_HOME_NAME=Ora11g_gridhome2 INVENTORY_LOCATION=/u01/app/oraInventory

After we clone the GI Home, we will be prompted to execute root.sh. The root.sh execution registers the GI Home to the oraInventory and set permissions and what not. After executing root.sh, you will prompt you to run config.sh shell script in silent mode or roothas.pl in a non-RAC environment. For RAC builds, we can execute the config.sh in GUI mode (default) but if you want to automate the cloning process, you must figure out how to execute config.sh in silent mode. export DIR=$(pwd) FN=`echo $0 | sed s/\.*[/]//` ${GRID_HOME}/crs/config/config.sh -silent -responseFile \ ${DIR}/${CLUSTER_NAME}.rsp \ -ignorePrereq -ignoreSysPrereqs

The secret sauce of cloning the GI Home for RAC environments is executing the config.sh in GUI mode and saving the response file at the end so that you can manipulate that response file to do your bidding in the future. Executing config.sh in GUI mode, you are prompted for the same set of screens as if you are installing the software. Here’s a screen shot what the saving the response file will look like.

At the summary screen (at the end of the config.sh GUI wizard), you will be provided an option to save the response file (near the right bottom corner of the screen) for the GI configuration. You should save this response file at the end of the GUI config.sh or at the time of GI installation and configuration. The response file that is created are the same identical files. If you perform a diff between the two response files (install versus config.sh), you will see no differences between them. Here’s what the response file looks for Oracle 11g Release 2 Grid Infrastructure after we remove all the comments and blank lines: [oracle@rac1 ~]$ cat grid.11.2.rsp |grep -v ^# |sed -e "/^$/d" oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v11_2_0 ORACLE_HOSTNAME=rac1.viscosityna.com INVENTORY_LOCATION=/u01/app/oraInventory SELECTED_LANGUAGES=en oracle.install.option=CRS_CONFIG ORACLE_BASE=/u01/app/oracle

Page 10: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

10 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

ORACLE_HOME=/u01/app/11.2.0/grid oracle.install.asm.OSDBA=oinstall oracle.install.asm.OSOPER=oinstall oracle.install.asm.OSASM=oinstall oracle.install.crs.config.gpnp.scanName=rac-scan oracle.install.crs.config.gpnp.scanPort=1521 oracle.install.crs.config.clusterName=rac-clust oracle.install.crs.config.gpnp.configureGNS=false oracle.install.crs.config.gpnp.gnsSubDomain= oracle.install.crs.config.gpnp.gnsVIPAddress= oracle.install.crs.config.autoConfigureClusterNodeVIP=false oracle.install.crs.config.clusterNodes=rac1.viscosityna.com:rac1-vip.viscosityna.com,rac2.viscosityna.com:rac2-vip.viscosityna.com,rac3.viscosityna.com:rac3-vip.viscosityna.com oracle.install.crs.config.networkInterfaceList=eth0:192.168.225.0:3,eth1:192.168.10.0:2,eth2:172.16.41.0:1 oracle.install.crs.config.storageOption=ASM_STORAGE oracle.install.crs.config.sharedFileSystemStorage.diskDriveMapping= oracle.install.crs.config.sharedFileSystemStorage.votingDiskLocations= oracle.install.crs.config.sharedFileSystemStorage.votingDiskRedundancy=NORMAL oracle.install.crs.config.sharedFileSystemStorage.ocrLocations= oracle.install.crs.config.sharedFileSystemStorage.ocrRedundancy=NORMAL oracle.install.crs.config.useIPMI=false oracle.install.crs.config.ipmi.bmcUsername= oracle.install.crs.config.ipmi.bmcPassword= oracle.install.asm.SYSASMPassword= oracle.install.asm.diskGroup.name=DATA oracle.install.asm.diskGroup.redundancy=EXTERNAL oracle.install.asm.diskGroup.AUSize=1 oracle.install.asm.diskGroup.disks=/dev/ASMCRS001,/dev/ASMCRS002,/dev/ASMCRS003 oracle.install.asm.diskGroup.diskDiscoveryString=/dev/ASM* oracle.install.asm.monitorPassword= oracle.install.crs.upgrade.clusterNodes= oracle.install.asm.upgradeASM=false oracle.installer.autoupdates.option= oracle.installer.autoupdates.downloadUpdatesLoc= AUTOUPDATES_MYORACLESUPPORT_USERNAME= AUTOUPDATES_MYORACLESUPPORT_PASSWORD= PROXY_HOST= PROXY_PORT=0 PROXY_USER= PROXY_PWD= PROXY_REALM= [ConfigWizard]

Lot of parameters in the response file will not change with new RAC installations. As you can see, if you have a corporate standard for naming conventions, lot of the parameters will be repeated in the new RAC builds. After we execute config.sh, we will be prompted to execute root.sh again. Up to this point, we are merely building our configuration for our RAC Grid Infrastructure deployment. All the magic will happen at the 2nd invocation of root.sh for the Grid Infrastructure home; although, Oracle has built-in lot of pre-configuration and pre-installation checks along the way, you will also detect your failures in this stage also. As the privileged user or as sudo, you will need to execute root.sh to complete your clone of your Grid Infrastructure. For non-RAC environments, we will be prompted to execute roothas.pl instead of config.sh for the stand-alone server: To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user: /u01/app/11.2.0.3/grid/perl/bin/perl -I/u01/app/11.2.0.3/grid/perl/lib -I/u01/app/11.2.0.3/grid/crs/install /u01/app/11.2.0.3/grid/crs/install/roothas.pl

For the ron-RAC environment, our Grid Infrastructure work is done here. We have some post clone work that is needed for ASM but that will be addressed in another section.

PROVISIONING DB HOME We can realistically provision a fully functional database(s) to our customers in matter of minutes instead of hours, days or even weeks. The biggest benefits of cloning the database software stack are:

Page 11: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

11 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

• We do not need to configure X Windows or vncserver on the database server or deal with X-Windows client or vncviewer on our desktops.

• The database home cloning is extremely quick compared to installing the latest Database software, applying PSUs and applying any one-off patches.

• When we clone an Oracle Home from one server to another or from a Golden Image, we take with it all the patches that were performed against the source Oracle Home. The database cloning process is extremely simple as shown in the diagram to the right.

If you look at the perl script (clone.pl), you will see that the cloning of the DB home is similar to the GI home. The logic is very similar to cloning the database stack. We will proceed from the Clone DB Home on each node with the clone.pl script. cd ${GI_HOME}/clone/bin perl ${GI_HOME}/clone/bin/clone.pl -silent \ ORACLE_HOME=${GI_HOME} \ ORACLE_HOME_NAME=OraGridHome1 \ ORACLE_BASE=$ORACLE_BASE \ -O'"CLUSTER_NODES={rac1,rac2,rac3}"' \ -O'"LOCAL_NODE=$(hostname -s)"' \ CRS=TRUE INVENTORY_LOCATION=${INVENTORY_LOCATION}

Executing the clone.pl script for the DB Home produces the folloing output and will ask you to run root.sh on each node. ./runInstaller -clone -waitForCompletion "ORACLE_HOME=/u01/app/oracle/product/11.2.0/db" "ORACLE_HOME_NAME=OraDBHome1" "ORACLE_BASE=/u01/app/oracle" "CLUSTER_NODES={rac1,rac2,rac3}" "LOCAL_NODE=$(hostname -s)" "CRS=TRUE" "INVENTORY_LOCATION=/u01/app/oraInventory" -silent -noConfig -nowait Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 3695 MB Passed Preparing to launch Oracle Universal Installer from /tmp/OraInstall2013-08-25_12-00-06AM. Please wait ...Oracle Universal Installer, Version 11.2.0.3.0 Production Copyright (C) 1999, 2011, Oracle. All rights reserved. You can find the log of this install session at: /u01/app/oraInventory/logs/cloneActions2013-08-25_12-00-06AM.log . Performing tests to see whether nodes rac2,rac3 are available ............................................................... 100% Done. Installation in progress (Sunday, August 25, 2013 12:01:06 AM CDT) ............................................................................... 79% Done. Install successful Linking in progress (Sunday, August 25, 2013 12:01:14 AM CDT) Link successful Setup in progress (Sunday, August 25, 2013 12:03:05 AM CDT) Setup successful End of install phases.(Sunday, August 25, 2013 12:03:29 AM CDT) WARNING: The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. /u01/app/oracle/product/11.2.0/db/root.sh #On nodes rac1 To execute the configuration scripts: 1. Open a terminal window

Page 12: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

12 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

2. Log in as "root" 3. Run the scripts in each cluster node The cloning of OraDBHome1 was successful. Please check '/u01/app/oraInventory/logs/cloneActions2013-08-25_12-00-06AM.log' for more details.

The cloning scripts for the DB home do not take long time at all. Unlike the scripts for the GI home, it is a quick process as it simply registers the Oracle Homes, changes file/directory permissions and performs so cleanup tasks. For non-RAC databases, the steps are identical except we will not provide a CLUSTER_NODES and LOCAL_NODE parameter in our clone.pl script.

PREPARE DBAAS We strongly believe that DBAs should only perform installations once. We believe that DBAs should only patch once for each release of oracle. They should maintain one environment for each version of Oracle they support. DBAs should create tar archives and deploy Oracle environments with tar-balls. There is no reason why DBAs should not be able to provision a database on a new server in matter of minutes. We will focus on creating what we coin creating golden image database templates from an existing database.

What%is%a%golden%image%database%template?%%%Preserve%corporate%standards%for%monitoring%Preserve%corporate%standards%for%redo%log%sizes%and%archive%log%mode%Preserve%corporate%standards%for%security%profiles%and%security%related%roles%Preserve%corporate%standards%for%auditing%Preserve%corporate%standards%for%password%verification%and%authentication%Preserve%corporate%standards%for%common%users%List%goes%on%and%on%and%on%…%

From the $ORACLE_HOME/bin directory, we can execute dbca with the -createCloneTemplate option. The -createCloneTemplate option will shutdown the database, and create a cold backup as backupset using RMAN to the location specified in the -datafileJarLocation parameter. dbca will also create a template configuration (.dbc) file that is XML based that you can manipulate as needed for your automation needs. In our example below, we will call our template name db_gold_01 and place the backup of the database in /bkups/templates as specified by the $BKUP_DIR environment variable. Other relevant parameters are sysDBAUserName and the sysDBAPassword.

Here’s the dbca script in silent mode that will create the golden image database template. $ > cat create_clone_template.ksh export BKUP_DIR=/bkups/templates dbca -silent -createCloneTemplate \

Page 13: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

13 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

-sourceSID RACDB \ -templateName db_gold_01 \ -sysDBAUserName sys \ -sysDBAPassword oracle123 \ -maintainFileLocations false \ -datafileJarLocation $BKUP_DIR

Here’s the output that this script produces: $ ./create_clone_template.ksh Gathering information from the source database 4% complete 8% complete 13% complete 17% complete 22% complete Backup datafiles 28% complete 88% complete Creating template file 100% complete Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/silent.log_2013-05-08_08-36-32-AM" for further details.

Locate the the tempate xml file in the format of .dbc with the find command: $ find . -name '*.dbc' ./assistants/dbca/templates/General_Purpose.dbc ./assistants/dbca/templates/Data_Warehouse.dbc ./assistants/dbca/templates/db_gold_01.dbc

Copy to the .dbc file to your backup directory where your $BKUP_DIR was specified as part of the -datafileJarLocation so that we have everything in a centralized location. cp $ORACLE_HOME/assistants/dbca/templates/db_gold_01.dbc $BKUP_DIR $ > ls -l /bkups/templates total 282784 -rw-r----- 1 oracle oinstall 18497536 May 8 08:37 gb_gold_01.ctl -rw-r----- 1 oracle oinstall 270778368 May 8 08:37 gb_gold_01.dfb

DEPLOY DBAAS Now you created your Golden Template as a database template, what do you do next? How do you leverage it? With a single dbca command, you can have your cake and eat it too. Here’s the dbca script that will save you ton of time. dbca -silent \ -createDatabase \ -templateName db_gold_01.dbc \ -gdbName racdb \ -sid racdb \ -SysPassword oracle123 \ -SystemPassword oracle123 \ -emConfiguration NONE \ -redoLogFileSize 500 \ -recoveryAreaDestination FRA \ -storageType ASM \ -asmSysPassword oracle123 \ -diskGroupName DATA \ -listeners LISTENER_RAC \ -characterSet AL32UTF8 \ -nationalCharacterSet AL16UTF16 \ -databaseType MULTIPURPOSE \ -nodelist rac1,rac2,rac3 \ -initparams audit_file_dest=’/oraacfs/trace/racdb/adump' \ -initparams compatible='11.2.0.3' \ -initparams db_create_file_dest='+DATA' \ -initparams db_create_online_log_dest_1='+DATA' \ -initparams db_create_online_log_dest_2='+FRA' \

Page 14: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

14 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

-initparams diagnostic_dest=’/oraacfs' \ -initparams parallel_max_servers=32 \ -initparams pga_aggregate_target=524288000 \ -initparams processes=400 \ -initparams sga_target=4294967296 \ -initparams db_recovery_file_dest='+FRA' \ -initparams db_recovery_file_dest_size=2097152000

ADDING A NODE TO THE CLUSTER Adding a RAC node to an existing cluster is not something that we do every day. Adding a RAC node to an existing cluster is relatively straight forward., you should always exercise the addnode.sh functionality in the lower environments before you attempt to add a node to the production RAC cluster. You need to go through the infrastructure limitations and holes before proceeding to touch the production environment. Here’s are some initial high level instructions:

• On target, create the oraInventory directory as an empty directory and change the ownership to be oracle:dba or whatever meets your corporate security requirements

• On target, make sure /u01/app/12.1.0 directory is created and also empty. Make sure that it is owned by oracle:dba • Run the customized addnode.ksh on an existing RAC node with parameters replaced as oracle. Please notice that this is a

custom script that invokes Oracle-supplied addnode.sh in silent mode • You will be instructed to run root.sh on target node as root

Here’s some caveats to consider. We are disabling pre-addnode checks with the IGNORE_PREADDNODE_CHECKS value to Y. /u01/app/oraInventory has to be pre-created and owned by oracle:dba and be completely empty. /u01/app/12.1.0 has to be owned by oracle:dba. The /u01/app/12.1.0/grid directory cannot exist … even as an empty directory. The addnode.sh script attempts to do a "mkdir grid" as oracle, and if it fails, it dies. We did not use the -noCopy method. The copy the Grid Home from source RAC node to the new RAC node added about 5 minutes to the overall time. Here’s the contents of our custom addnode.ksh script (not the same as the addnode.sh). This script in fact invokes the Oracle-supplied addnode.sh shell script from the $ORACLE_HOME/addnode directory (new in Oracle 12c).

+ASM1 > cat addnode.ksh export ORACLE_HOME=/u01/app/12.1.0/grid export PATH=$ORACLE_HOME/bin:$PATH cd $ORACLE_HOME/addnode rementnnnn export IGNORE_PREADDNODE_CHECKS=Y #./addnode.sh -silent "CLUSTER_NEW_NODES={rhel59drb}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={rhel59b-vip}" -noCopy ./addNode.sh -silent "CLUSTER_NEW_NODES={rhel59drb}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={rhel59drb-vip}" "CLUSTER_NEW_PRIVATE_NODE_NAMES={rhel59drb-priv}"

The ability to rapidly grow (addnode.sh) makes DBaaS feasible. We can easily scale a RAC database as demand grows. We will be lot more confident as we get more comfortable with adding nodes to the existing cluster. Something to think about is that we can add more than one node to an existing cluster at a time.

DROPPING A NODE TO THE CLUSTER As our workload demand shrinks, we have the need to remove an existing RAC node from the cluster. Deleting an existing RAC node from the cluster is extremely simple. The first step to drop a node is first to delete the node with the “crsctl delete node” command. crsctl delete node -n rac03 crsctl delete node -n rac04

Page 15: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

15 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Second, we need to cleanup the orphaned VIPs: [root@rac01 ~]# crsctl delete resource ora.rac04.vip -f [root@rac01 ~]# crsctl delete resource ora.rac03.vip -f

Once we are done with our deletion of node and removal of VIPs, we can check the status of the cluster resources: crsctl stat res -t

PLUGGABLE DATABASES Oracle Database 12c features a new option called Pluggable Database (PDB). With PDB, you now have more incentive to lean towards the schema consolidation. PDB “begins” to eliminate the typical issues that come with schema consolidation; such as namespace collisions, security, and granularity of recovery. The schema consolidation model has consistently provided the most opportunities for reducing operating expenses, since you only have a single big database to maintain, monitor, and manage. Though schema consolidation allows the best ROI (w.r.t CapEX/OPex), you are sacrificing flexibility for compaction. Consolidation and isolation tend to move in opposite directions. The more you consolidate the less capabilities you’ll have for isolation; in contrast, the more you try to isolate, the more you sacrifice benefits of consolidation. Custom (home-grown) apps have been best fit use cases for schema consolidation, since application owners and developers have more control on how the application and schema is built. Let’s review some terminologies before we begin:

• Root Container Database – Or the root CDB (cdb$root) is the name you give it will be name of the instance. The CDB database owns the SGA and running processes. You can have many CDBs on the same database server (each with its own PDBs). You can have up to 252 PDBs per CDB. This number is just an arbitrary number that Oracle specified. As customers driver more and more towards the PDB consolidation model and define a need to increase the maximum number of PDBs, we are sure that Oracle will increase this number to meet their customer requirements.

• You can have a more than one CDB on the same database server, each with it’s own PDBs, allowing DBAs to have a Database Instance consolidation model coupled a schema consolidation. For best scalability, mix in RAC and leverage all the benefits of RAC Services, QoS, and Workload Distribution. The seed PDB (PDB$SEED) is an Oracle supplied system template that the CDB can use to create new PDBs. The seed PDB is named PDB$SEED. One cannot add or modify objects in PDB$SEED.

• Pluggable Database – The PDB databases are sub-containers that serviced by CDB resources. The true beauty of the PDB is its mobility. You can unplug and plug 12c databases into and out of CDBs. You can “create like” new PDBs from existing PDB, like full snapshots.

Everyone needs to know that the Pluggable Database is not a free option. You must pay for use of this feature and is listed as a Multitenant option under the Enterprise Edition Options. The following screen shot is as of January 2014:

Page 16: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

16 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

There are several ways to leverage the pluggable database option. The first option is to leverage dbca to create a CDB/PDB database, maintain, plug/unplug, archive PDBs:

Here’s an example of dbca to create a CDB/PDB. You can create a database with CDB/PDB enabled at the time of creation. In this example, we are creating a database called CDB1 with a pluggable database called VNA.

Page 17: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

17 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Here’s screen shots to demonstrate how you can also leverage dbca to manage pluggable databases.

Here’s an example screen shot of creating PDBs from PDB archives or PDB file sets.

You can also download the Apex application for provisioning PDBs (still in beta as of January 2014): http://www.oracle.com/technetwork/database/multitenant/downloads/multitenant-pdbss-2016324.html

Oracle provides a Multitenant Self-Service Provisioning via Application Express to provide a simplified GUI interface to Oracle Database 12c Multitenant and allows for the self-service provisioning of Pluggable Databases (PDBs). This Apex application is provided to encourage learning about PDBs in development / test environments with powerful PDB features including create, clone, plug, and unplug. Another way to manage PDBs is with Oracle Enterprise Manager Database Express, also referred to as EM Express, is a web-based tool for managing Oracle Database 12c. You can login to the EM Express using a browser of your choice and manage PDBs.

The last and preferred method to provision PDBs is the old fashion command line interface (CLI). The CLI will be our method for initiating the automated scripting that we are discussing in this session. Let’s start with creating a CDB/PDB database with dbca in silent mode:

cd /u01/app/oracle/product/12.1.0/dbhome_1/bin ./dbca -silent \ -createDatabase \ -templateName General_Purpose.dbc \ -gdbName TEST \ -sid TEST \ -SysPassword oracle123 \ -createAsContainerDatabase true \ -numberofPDBs 2 \ -pdbName VNA \ -SystemPassword oracle123 \ -emConfiguration DBEXPRESS \ -redoLogFileSize 100 \ -recoveryAreaDestination FRA \ -storageType ASM \ -asmsnmpPassword oracle123 \ -asmSysPassword oracle123 \ -diskGroupName DATA \ -listeners LISTENER \ -registerWithDirService false \ -characterSet AL32UTF8 \ -nationalCharacterSet AL16UTF16 \ -databaseType MULTIPURPOSE \ -nodelist ol59a,ol59b \ -initparams audit_file_dest='/u01/app/oracle/admin/TEST/adump' \

Page 18: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

18 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

-initparams compatible='12.1.0.0' \ -initparams db_create_file_dest='+DATA' \ -initparams db_create_online_log_dest_1='+DATA' \ -initparams db_create_online_log_dest_2='+FRA' \ -initparams db_recovery_file_dest='+FRA' \ -initparams pga_aggregate_target=100M \ -initparams diagnostic_dest='/u01/app/oracle' \ -initparams parallel_max_servers=8 \ -initparams processes=400 \ -initparams sga_target=524288000 \ -initparams db_recovery_file_dest_size=4322230272

Notice the three lines that is highlighted in red that start with the –createAsContainerDatabase option. All other options to creating dbca in silent mode remain the same.

CREATE PDBS FROM NON-CDB As you saw from the examples and screenshots, creating a PDB is extremely simple with dbca. Now, let’s do this with the command line interface. Creating PDBs, migrating a non-CDB database, or moving PDBs between CDBs requires some PL/SQL interface. In our first example, let’s take a look at creating a PDB from a non-CDB database. Before we can create the XML manifest, the database must be placed in read-only mode. For RAC databases, we can stop the RAC database and then start just the first instance in read-only mode: $ srvctl stop database -d PSOFT $ srvctl start instance -d PSOFT -startoption "READ ONLY" -n rac01

Next, we must create an XML manifest which contains metadata to describe the non-CDB using the DBMS_PDB package. begin dbms_pdb.describe( pdb_descr_file => '/u01/app/oracle/oradata/psoft.xml'); end; / PL/SQL procedure successfully completed.

You can view a sample PDB XML manifest from: http://www.dbaexpert.com/blog/pdb-manifest-xml/ We need to determine whether a pluggable database described by file pdb_descr_file is compatibile with the current CDB. Login to the CDB and execute the DBMS_PDB.CHECK_PLUG_COMPATIBILITY procedure: BEGIN IF dbms_pdb.check_plug_compatibility('/u01/app/oracle/oradata/psoft.xml', 'PSOFTDEV') THEN dbms_output.put_line('TRUE'); ELSE dbms_output.put_line('FALSE'); END IF; END; / FALSE PL/SQL procedure successfully completed.

We can check the PDB_PLUG_IN_VIOLATIONS table to see what the potential issues are: SQL> select * from pdb_plug_in_violations where name='PSOFTDEV'; 18-FEB-14 04.38.34.312314 PM PSOFTDEV Non-CDB to PDB WARNING 0

Page 19: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

19 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

1 PDB plugged in is a non-CDB, requires noncdb_to_pdb.sql be run. PENDING Run noncdb_to_pdb.sql. 18-FEB-14 04.38.34.312821 PM PSOFTDEV Parameter WARNING 0 1 CDB parameter memory_target mismatch: Previous 843055104 Current 0 PENDING Please check the parameter in the current CDB

As you can see from the warning messages, we are OK to proceed to create the PDB database from the non-CDB. The next step is to shutdown the database that will be plugged into our CDB: $ srvctl stop database -d PSOFT

Now, we will connect to our CDB database that we created called VPROD and issue the commands to create the PSOFT pluggable database: SQL> select name, open_mode from V$PDBS; NAME OPEN_MODE ------------------------------ ---------- PDB$SEED READ ONLY VNA MOUNTED SQL> set echo on SQL> @pdb_copy_local.sql SQL> create pluggable database PSOFT using '/u01/app/oracle/oradata/psoft.xml' copy file_name_convert=('+DATA','+DATA'); Pluggable database created.

Note, that when you create the PDB, it is initially in mounted mode with a status of NEW. View the OPEN MODE status of a PDB by querying the OPEN_MODE column in the V$PDBS view or view the status of a PDB by querying the STATUS column of the CDB_PDBS or DBA_PDBS view. Next, we will need to set our container to our new PDB and execute the noncdb_to_pdb.sql script from the $ORACLE_HOME/rdbms/admin directory. This script must be executed before the PDB can be opened for the first time. SQL> alter session set container=PSOFT; Session altered. SQL> @?/rdbms/admin/noncdb_to_pdb.sql … … SQL> SQL> -- leave the PDB in the same state it was when we started SQL> BEGIN 2 execute immediate '&open_sql &restricted_state'; 3 EXCEPTION 4 WHEN OTHERS THEN 5 BEGIN 6 IF (sqlcode <> -900) THEN 7 RAISE; 8 END IF; 9 END; 10 END; 11 / old 2: execute immediate '&open_sql &restricted_state'; new 2: execute immediate ' '; PL/SQL procedure successfully completed.

Page 20: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

20 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

SQL> SQL> WHENEVER SQLERROR CONTINUE;

Here’s a tail-end output of the script. As you can see, the script puts the database back to the state that it was at prior to invocation. The output to this script is rather extensive; for the curious, you can view the complete output from: http://www.dbaexpert.com/blog/noncdb_to_pdb-sample-output/

After the script executed, we need to open the PSOFT PDB in READ WRITE mode:

SQL> alter pluggable database psoft open; Pluggable database altered.

Let’s look at our PDBs and see some details of each PDB:

col pdb_name for a30 set lines 122 trims on select con_id, dbid, name, open_mode, total_size from v$pdbs; select dbid, con_id, guid, con_uid, pdb_name, status from CDB_PDBS; CON_ID DBID NAME OPEN_MODE TOTAL_SIZE ---------- ---------- ------------------------------ ---------- ---------- 2 4082385525 PDB$SEED READ ONLY 283115520 3 3780808255 VNA MOUNTED 0 4 1834925412 PSOFT READ WRITE 1158676480 DBID CON_ID GUID CON_UID PDB_NAME STATUS ---------- ---------- -------------------------------- ---------- ------------------------------ ------------- 3780808255 1 F2B00B9803F75A09E0430A2910AC0F18 3780808255 VNA NORMAL 4082385525 1 F2AFD228598D4F2CE0430A2910AC50CF 4082385525 PDB$SEED NORMAL 1834925412 1 F2B1597C0F080518E0430A2910AC7076 1834925412 PSOFT NORMAL

DIFFERENT WAYS TO CREATE A PDB In addition to the above methods, this section will show additional methods to create a PDB. The beauty of PDB is not only mobility (plug and unplug), which will be demonstrated later, but that we can create/clone a new PDB from a "gold-image PDB". That's real agility and a Database as a Service (DBaaS) play. METHOD 1: CREATE A PDB FROM SEED The easiest way to create a PDB is from the seeded files that Oracle provides. SQL> alter session set container=cdb$root; Session altered. SQL> CREATE PLUGGABLE DATABASE pdbhansolo admin user hansolo identified by hansolo roles=(dba); Pluggable database created. SQL> alter pluggable database pdbhansolo open; Pluggable database altered. SQL> select file_name from cdb_data_files; FILE_NAME -------------------------------------------------------------------------------- +PDBDATA/YODA/E51109E2AF22127AE043EDFE10AC1DD9/DATAFILE/system.280.824693889

Page 21: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

21 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

+PDBDATA/YODA/E51109E2AF22127AE043EDFE10AC1DD9/DATAFILE/sysaux.279.824693893

Notice that it just contains the basic files to enable a PDB. The CDB will copy from the PDB$SEED the System and Sysaux tablesapces and instantiate them in the new PDB. When you create a new PDB from the seed, an administrator for the PDB must be specified in the CREATE PLUGGABLE DATABASE syntax. The administrator will be created as a local user in the PDB and granted the PDB_DBA role locally. METHOD #2: CLONE FROM AN EXISTING PDB CDB$ROOT@YODA> alter session set container=cdb$root; Session altered. CDB$ROOT@YODA> alter pluggable database pdbobi close; Pluggable database altered. CDB$ROOT@YODA> alter pluggable database pdbobi open read only; Pluggable database altered. CDB$ROOT@YODA> CREATE PLUGGABLE DATABASE pdbleia from pdbobi; Pluggable database created. CDB$ROOT@YODA> alter pluggable database pdbleia open; Pluggable database altered. CDB$ROOT@YODA> select file_name from cdb_data_files; FILE_NAME -------------------------------------------------------------------------------- +PDBDATA/YODA/E51109E2AF23127AE043EDFE10AC1DD9/DATAFILE/system.281.824694649 +PDBDATA/YODA/E51109E2AF23127AE043EDFE10AC1DD9/DATAFILE/sysaux.282.824694651 +PDBDATA/YODA/E51109E2AF23127AE043EDFE10AC1DD9/DATAFILE/users.285.824694661 +PDBDATA/YODA/E51109E2AF23127AE043EDFE10AC1DD9/DATAFILE/example.286.824694661 +PDBDATA/YODA/E51109E2AF23127AE043EDFE10AC1DD9/DATAFILE/obiwan.287.824694669

Notice, that the OBI tablespace that we created in PDBOBI came over as part of this Clone process!! Let's create a new clone from an existing PDB, but with a new path. SQL> create pluggable database PDBvader from PDBOBI FILE_NAME_CONVERT=('+PDBDATA/YODA/DD7D8C1D4C234B38E04325AAE80AF577/DATAFILE','+PDBDATA'); Pluggable database created.

METHOD #3: CREATE A PDB AS A SNAPSHOT FROM ANOTHER PDB Cloning technologies have existed for years but it’s never been so easy for DBAs. With Oracle Database 12c Multitenant option and integration with, DBAs simply execute commands from SQL*Plus to: • Create full copy database clones

• Execute and manage snapshot copies of the production or gold image on NetApp, ASM Cluster File System (ACFS) and ZFS Storage Appliance (ZFSSA). Essentially you will need a NAS Appliannce, or any technology that will provide you with COW snapshot. To create a full copy database clone, the DBA would simply issue “create pluggable database ODSPROD from PROD;”. Behind the scene, Oracle Parallel Execution slaves do all the heavy lifting. Internally, each of the full copy clones be assigned a unique global id (GUID) and named and tagged with the new PDB GUID.

Page 22: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

22 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

The snapshot copy is based on underlying storage copy-on-write (COW) technology for the file system and provides faster snapshot copy clones. To create a snapshot copy, the DBA would issue a simple “create pluggable database DEV from PROD snapshot copy; “ command from SQL*Plus. This technology is evolving and will continue to be improved. As of Oracle version 12.1, we are still limited to: 1. Local Clones Only 2. Source PDB cannot be dropped 3. Source PDB cannot be unplugged 4. Both source and cloned PDBs may be opened read-write For additional information on establishing PDB snapshots from another PDB, you are encouraged to view DOC ID 1597027.1: Oracle Multitenant Pluggable Database Snapshot Cloning: Use Cases and Supported Platforms.

PDB DATAFILES FOR ASM Leveraging PDB for RAC and ASM can give DBAs the perception of added complexity. The goal in this section is to show you that it’s not much different than being on the file system. Let’s create couple of new tablespaces in the PSOFT PDB. In this section, you will see how Global Identifier for that PDB impacts the directory structure for the database files inside of ASM. SQL> alter session set container=PSOFT; Session altered. SQL> create tablespace psoft_data; Tablespace created. SQL> create tablespace psoft_index; Tablespace created.

Let’s look at the datafile names for the new tablespaces that we created. SQL> @files FILE_NAME ---------------------------------------------------------------------------------------------------- +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/users.275.839850657 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/sysaux.277.839850657 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/psoft_data.274.839852893 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/psoft_index.273.839852903 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/system.272.839850657 CON_ID FILE_NAME ONLINE_ ---------- ---------------------------------------------------------------------------------------------------- ------- 4 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/users.275.839850657 ONLINE 4 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/sysaux.277.839850657 ONLINE 4 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/psoft_data.274.839852893 ONLINE 4 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/psoft_index.273.839852903 ONLINE 4 +DATA/VPROD/F2B1597C0F080518E0430A2910AC7076/DATAFILE/system.272.839850657 SYSTEM

The datafiles listed in v$datafile differs from cbd_data_files. cdb_data_files only shows datafiles from "open" PDB, so just be careful if you are looking at the correct datafile. The long identifier, " F2B1597C0F080518E0430A2910AC7076", in the OMF name is the GUID or Global Identifier for that PDB. This is not the same as container unique identifier (CON_UID). The con_uid is a local identifier; whereas the GUID is universal. Keep in mind that we can unplug a PDB from one CDB into another CDB, so the GUID provides this uniqueness and streamlines portability.

Page 23: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

23 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

PUTTING IT ALL TOGETHER Rapid RAC provisioning is extremely simple if you know the tricks of the trade to clone the Grid Infrastructure and Database Home. Prior to cloning the Grid Infrastructure home, you will be required create (really manipulate) a response file that encompasses all the information about the cluster such as network interfaces, hostnames, VIPs, SCAN information, private hostnames, ASM or shared disk information, etc. Earlier we displayed what the Grid Infrastructure response file looked like for Oracle Database 11g Release 2. The Grid Infrastructure response file for Oracle Grid Infrastructure 12c has less than 50 parameters to modify (less in Oracle 11g Release 2). [oracle@rac01 ~]$ cat grid.12c.rsp |egrep -v "^#|^$|^\[" |more oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v12.1.0 ORACLE_HOSTNAME=rac01.viscosityna.com INVENTORY_LOCATION=/u01/app/oraInventory SELECTED_LANGUAGES=en oracle.install.option=CRS_CONFIG ORACLE_BASE=/u01/app/oracle ORACLE_HOME=/u01/app/12.1.0/grid oracle.install.asm.OSDBA=oinstall oracle.install.asm.OSOPER=oinstall oracle.install.asm.OSASM=oinstall oracle.install.crs.config.gpnp.scanName=rac-scan oracle.install.crs.config.gpnp.scanPort=1521 oracle.install.crs.config.ClusterType=STANDARD oracle.install.crs.config.clusterName=rac-cluster oracle.install.crs.config.gpnp.configureGNS=false oracle.install.crs.config.autoConfigureClusterNodeVIP=false oracle.install.crs.config.gpnp.gnsOption=CREATE_NEW_GNS oracle.install.crs.config.gpnp.gnsClientDataFile= oracle.install.crs.config.gpnp.gnsSubDomain= oracle.install.crs.config.gpnp.gnsVIPAddress= oracle.install.crs.config.clusterNodes=rac01.viscosityna.com:rac01-vip.viscosityna.com,rac02.viscosityna.com:rac02-vip.viscosityna.com,rac03.viscosit yna.com:rac03-vip.viscosityna.com,rac04.viscosityna.com:rac04-vip.viscosityna.com oracle.install.crs.config.networkInterfaceList=eth0:192.168.225.0:3,eth1:172.16.11.0:5,eth2:172.16.41.0:1 oracle.install.crs.managementdb.configure=true oracle.install.crs.config.storageOption=FLEX_ASM_STORAGE oracle.install.crs.config.sharedFileSystemStorage.votingDiskLocations= oracle.install.crs.config.sharedFileSystemStorage.votingDiskRedundancy=NORMAL oracle.install.crs.config.sharedFileSystemStorage.ocrLocations= oracle.install.crs.config.sharedFileSystemStorage.ocrRedundancy=NORMAL oracle.install.crs.config.useIPMI=false oracle.install.crs.config.ipmi.bmcUsername= oracle.install.crs.config.ipmi.bmcPassword= oracle.install.asm.SYSASMPassword= oracle.install.asm.diskGroup.name=OCRVOTE oracle.install.asm.diskGroup.redundancy=EXTERNAL oracle.install.asm.diskGroup.AUSize=1 oracle.install.asm.diskGroup.disks=ORCL:ASMDISK1,ORCL:ASMDISK2,ORCL:ASMDISK3 oracle.install.asm.diskGroup.diskDiscoveryString= oracle.install.asm.monitorPassword= oracle.install.crs.config.ignoreDownNodes=false oracle.installer.autoupdates.option= oracle.installer.autoupdates.downloadUpdatesLoc= AUTOUPDATES_MYORACLESUPPORT_USERNAME= AUTOUPDATES_MYORACLESUPPORT_PASSWORD= PROXY_HOST= PROXY_PORT=0 PROXY_USER= PROXY_PWD= PROXY_REALM= oracle.install.asm.useExistingDiskGroup=false

In this output, we stripped out all the blank lines, all the lines that start with a # sign (for comments) and headers that start with [. As you can see, of the 50 parameters, lot of the parameters have empty values. Understanding about ½ dozen key parameters are key to being able to manipulate the response to do your bidding. For example, understanding the oracle.install.crs.config.networkInterfaceList parameter (below) is extremely important.

Page 24: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

24 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

oracle.install.crs.config.networkInterfaceList=eth0:192.168.225.0:3,eth1:172.16.11.0:5,eth2:172.16.41.0:1

Each value triplets are separated by a comma. The first component of the value triplet is the interface name. The second components is the Subnet mask, and the last value is the interface type. In Oracle 12c, the values range from 1 to 5 of which 4 and 5 are new to Oracle Database 12c. The value of 1 denotes that the interface is to be used for the public network. A value of 2 indicates that the interface is to be used for the private network. A value of 3 is used to denote that the interface is not to be used. A value of 4 is for ASM traffic for Flex ASM. In this scenario, we would are dedicating a separate network interface for Flex ASM traffic. A value of 5 indicates that the interface is to be leveraged for ASM and the private interconnect. If we have 2 network interfaces (bonded), we would leverage one bonded interface for the public network (1) and the second bonded interface for ASM and private network (5) if we are leveraging Flex ASM. If you standardize, less you have to manipulate the response file. For example, you need to standardize the name of the disk group created at the initial grid cloning process. If you name this disk group OCRVOTE and make it NORMAL with three ASM disks: ORCL:OCRVOTE1, ORCL:OCRVOTE2, and ORCL:OCRVOTE3, this part of the response file would never change. In our example, we needed all of the ASMLIB disks for the OCRVOTE disk group with external redundancy because we only had limited storage. Another big area that you should standard on are things like OraInventory Location, Oracle Base, Grid Infrastructure Homes, Oracle Database Homes, SCAN and VIP names. Other miscellaneous areas that can be standardized are things like cluster name and ports. You should put generic placeholders in the response file. You should create a RAC configuration file that has the changes that you want to make to the response file. Response file generation is a simple global search and replacement from the small configuration file to replace the required parameters in the response file. We put the following menu system ready to rapidly deploy RAC without much thinking process around it. We are constantly making the product better. With each conference, this RAC menu system get more and more robust as additional functionality and hate to say it, a few bug fixes. # ------------------------------------------------------------------------- # # RAC Menu System - racclust # ------------------------------------------------------------------------- # # First Node: rac01 VIP: rac01-vip.viscosityna.com # Second Node: rac02 VIP: rac02-vip.viscosityna.com # Third Node: rac03 VIP: rac03-vip.viscosityna.com # ------------------------------------------------------------------------- # # 10. Prepare Source RAC Cluster for Cloning (as root) # # ------------------------------------------------------------------------- # # 20. Deploy Scripts to all the RAC nodes # ------------------------------------------------------------------------- # # 30. root cleanup and environment check # # ------------------------------------------------------------------------- # # 40. Clone Grid Infrastructure - /u01/app/12.1.0/grid # ------------------------------------------------------------------------- # # 50. Execute config.sh in silent mode # # ------------------------------------------------------------------------- # # 60. Post Clone Checks # # ------------------------------------------------------------------------- # # 70. Clone Database Home - /u01/app/oracle/product/12.1.0/dbhome_1 # ------------------------------------------------------------------------- # # 80. ASM Disk Group Configuration # # ------------------------------------------------------------------------- # # 90. Create RAC Database - PROD # ------------------------------------------------------------------------- # # 100. Post Database Tasks # # ------------------------------------------------------------------------- # # x. Exit # # ------------------------------------------------------------------------- #

Page 25: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

25 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

We will reveal all the functionality to the RAC menu system during our demo at our presentation.

ASM AS A SERVICE Similar to the RAC menu system, we have created a set of ASM related tasks to manage the ASM environment. The ASM menu system includes additional screens to manage ASMLIB, kfed/kfod, ACFS and ASM Disk Group Management. Just about everything you need to administer the ASM environment will be included. You will be able to rapidly provision all components of ASM. # ------------------------------------------------------------------------- # # Automatic Storage Management Menu System # # Primary Host: rac01 Standby Host: racdr01 # ------------------------------------------------------------------------- # # 10. crsctl check has/css # # 11. crsctl stat # # 12. Check for ASMLIB rpms # # ------------------------------------------------------------------------- # # 20. sfdisk ASM Disk (only echos command to execute) # # 21. Configm sfdisk ASM Disk (only echos command to execute) # # 22. dd Write Input File To Block Device # # 23. dd Read Input File From Block Device # # ------------------------------------------------------------------------- # # 25. Prepare ASM Disk (only echos command to execute) # # 26. Prepare ASM Disk Interactive (only echos command to execute) # # ------------------------------------------------------------------------- # # 30. Launch ASMLIB Management Submenu # # ------------------------------------------------------------------------- # # 40. Launch ASM Disk Group Management Submenu # # ------------------------------------------------------------------------- # # 50. View ASM Disk Group and Disks Free Space # # 51. View ASM IO Statistics Inforation # # 52. ASMLIB to Device File Mappings # # 53. Read ASM Disk Header with od # # 70. kfed / kfod Troubleshooting Utilities Submenu # # ------------------------------------------------------------------------- # # 100. Launch ACFS Management Submenu # # x. Exit # # ------------------------------------------------------------------------- # # Enter Task Number:

Please come visit the session as we will reveal all the functionality to the ASM menu system during our demo at our presentation.

DATA GUARD AS A SERVICE Myriads of best practice techniques are provided from Oracle’s Maximum Availability Architecture (MAA) team and industry experts that address Data Guard when it comes to performance, scalability and reliability. Compliance with best practices can help mitigate potential performance and infrastructure issues that ails lot of companies. One of the best ways to comply with industry best practices is with automated processes and procedures. This paper will disseminate fundamental Data Guard best practices and demonstrate how DBAs can automate setup, configuration, monitoring and maintenance of physical standby Data Guard environments with assistance from the Data Guard (DG) Toolkit with concentration on the following topics:

• Building the Physical Standby

• Monitoring and Maintaining the Physical Standby • Configuring Data Guard Broker

• Performing Backup and Recovery with RMAN • Setting Archive Retention

• Performing Switchovers and Failovers

Page 26: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

26 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

# ------------------------------------------------------------------------- # # Data Guard Toolkit # # Primary Host: rac01a Standby Host: rac01dra # Primary DB: VNA Standby DB: VNADR # P SCAN: rac-scan:1521 S SCAN: racdr-scan:1521 # ------------------------------------------------------------------------- # # 10. Launch Preliminary Check Submenu # # ------------------------------------------------------------------------- # # 20. Launch Build Standby Database Submenu # # ------------------------------------------------------------------------- # # 30. Launch the Data Guard Broker Submenu # # ------------------------------------------------------------------------- # # 40. Launch Monitor Physical Standby Data Guard Submenu # # ------------------------------------------------------------------------- # # 50. Launch Monitor Logical Standby Data Guard Submenu # # ------------------------------------------------------------------------- # # 60. Launch Automatic Diagnostic Repository (ADR) CLI Submenu # # ------------------------------------------------------------------------- # # 100. Launch RMAN Backup to Disk Submenu # # ------------------------------------------------------------------------- # # 200. RMAN Backup Primary based on Standby SCN / Recover Standby Database # # ------------------------------------------------------------------------- # # x. Exit # # ------------------------------------------------------------------------- # # Enter Task Number:

Please come visit the session as we will reveal all the functionality to the DG menu system during our demo at our presentation.

BACKUP & RECOVERY AS A SERVICE The Backup and Recovery section leverages the rman2disk shell script from http://www.dataguardbook.com/Downloads/rman2disk/ and encompasses both RMAN backup sets and RMAN backup as copy options. You are encouraged to leverage the backup as copy option and update the baseline copy with incremental backups on a nightly basis especially if you have a VLDB situation. Conceptually, you will rarely need to perform a full level 0 image copy of the database again. Even with the backup as copy option, you should perform a full backup at least once per month or quarter basis. Alternatively, the RMAN2DISK also provides the option to perform level 0 backup sets and even compressed level 0 backup sets. The only caveat is that the DG Toolkit does not provide interfaces to the Media Management Layer (MML) so only D2D backups are supported. # ------------------------------------------------------------------------- # # Data Guard RMAN Backup to Disk Submenu: dg_rman2disk_menu.ksh # Primary Host: rac01a Standby Host: rac01dra # ------------------------------------------------------------------------- # # 10. Enable Block Change Tracking on the VNA1 on rac01a # 20. Perform full level 0 backupset of VNA1 on rac01a # 30. Perform compressed full level 0 backupset of VNA1 on rac01a # 40. Perform backup as copy image backup of VNA1 on rac01a # ------------------------------------------------------------------------- # 50. Perform level 1 incremental backup of VNA1 on rac01a # 60. Perform level 1 compressed incremental backup of VNA1 on rac01a # 70. Perform level 1 incremental backup for recover (UPDATE IMAGE COPY) of VNA1 on rac01a # ------------------------------------------------------------------------- # # 100. Configure archivelog retention policy for VNA1 on rac01a # 110. Configure archivelog retention policy for VNADR1 on rac01dra # ------------------------------------------------------------------------- # # x. Exit # # ------------------------------------------------------------------------- # # Enter Task Number:

Page 27: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

27 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

Page 28: Cloud Computing Extreme Oracle DB Infrastructure As A … · COLLABORATE 14 – IOUG Forum Cloud Computing 3 | Page Extreme Oracle DB Infrastructure as a Service White Paper Next,

COLLABORATE 14 – IOUG Forum Cloud Computing

28 | P a g e Ex t r eme Orac l e DB In f r a s t ruc tu re a s a Se rv i ce Wh i t e Pape r

APPENDICES We made RAC and non-RAC deployment easy with simple to use menu like system for easy and automated RAC deployment. We put all of the automation together and provide it as a simple interface for you to download.

REFERENCES This white paper is a snippet of the Database Infrastructure As A Service book authored by Charles Kim/George Trujillo/Steve Jones. The first release of the book is available for download and is available for iBook. For complete reference and technical details, please download our book from the Apps store. We will offer free updates to the Database Infrastructure As a Service. In this book we will focus on best practices of Oracle Database as a Service with discussions around Linux, consolidation, multi-tenancy, virtualization and cloud deployments. The authors have spent a number of years focused on the virtualization of tier-one high performance platforms. In this book we will share knowledge learned in the field and best practices around deploying Oracle Database-as-a-Service. Whether you are virtualized or running in bare-metal, this book will be your reference of choice for best practices and industry reference architectures.

AUTHOR BIOGRAPHY Charles Kim is an Oracle ACE Director, VMware vExpert, an Oracle Certified DBA, and a Certified RAC Expert. Charles specializes in Exadata, RAC, and Virtualization (VMware and Oracle VM) and authored three books: 1. Oracle Database 11g New Features for DBA and Developers 2. Linux Recipes for Oracle DBAs 3. Oracle Data Guard 11g Handbook 4. Database Infrastructure As a Service Handbook Charles holds certifications in Oracle, VMware, Red Hat Linux, and Microsoft and has over 22 years of Oracle experience on mission and business critical databases. Charles presents regularly at local, regional, national and international Oracle conferences including IOUG Collaborate, VMware World, and Oracle OpenWorld on topics of RAC, ASM, Linux Best Practices, Data Guard Best Practices, VMware virtualization, Oracle VMware virtualization, and 7x24 High Availability Considerations. Charles is the technical editor of the Automatic Storage Management book by Oracle Press and blogs regularly at http://blog.dbaexpert.com and http://oravm.com. Charles is the current Vice President and Vendor Liaison of the Cloud Computing SIG for the Independent Oracle User Group. Charles is a co-author of the upcoming Hadoop-as-a-Service and Expert Exadata book with Pearson.