Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c...

31
A Dell EMC Best Practices Guide Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash Dell EMC Engineering January 2017

Transcript of Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c...

Page 1: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

A Dell EMC Best Practices Guide

Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash Dell EMC Engineering January 2017

Page 2: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

2 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

Revisions

Date Description

January 2017 Initial release

Acknowledgements

Authors: Chidambara Shashikiran, Henry Wong

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this

publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other

trademarks may be the property of their respective owners. Published in the USA [1/19/2017] [Best Practices Guide] [3110-BP-SDS]

Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Page 3: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

3 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

Table of contents Revisions............................................................................................................................................................................. 2

Executive summary ............................................................................................................................................................. 4

1 Product overview .......................................................................................................................................................... 5

1.1 XC Series appliances ......................................................................................................................................... 5

1.2 XC Series all-flash .............................................................................................................................................. 5

1.3 XC Series Acropolis architecture ........................................................................................................................ 6

1.4 XC Series Acropolis Block Services ................................................................................................................... 7

2 Solution infrastructure .................................................................................................................................................. 8

2.1 Physical system configuration ............................................................................................................................ 8

2.1.1 Oracle single instance database configuration ................................................................................................... 8

2.1.2 Oracle RAC database configuration ................................................................................................................. 10

2.1.3 Oracle RAC database configuration using ABS ............................................................................................... 11

2.2 XC Series storage and cluster configuration .................................................................................................... 12

2.3 Network configuration ....................................................................................................................................... 12

3 Sizing hypervisor configuration guidelines ................................................................................................................. 13

3.1 Oracle database VM configuration ................................................................................................................... 13

3.1.1 Processor and memory .................................................................................................................................... 13

3.1.2 XC Series storage container and VMware storage virtualization ..................................................................... 13

3.1.3 VM storage controller and virtual disks ............................................................................................................ 13

3.1.4 VM networking .................................................................................................................................................. 15

3.2 VM guest OS configuration for Oracle guidelines ............................................................................................ 18

3.3 Nutanix CVM ..................................................................................................................................................... 20

3.4 Storage layout for databases ............................................................................................................................ 21

3.4.1 Oracle ASM for Oracle single instance or RAC ................................................................................................ 21

3.4.2 File system for Oracle single instance database .............................................................................................. 23

3.5 Performance monitoring ................................................................................................................................... 23

3.5.1 Nutanix Prism ................................................................................................................................................... 24

3.5.2 vSphere client ................................................................................................................................................... 25

3.5.3 CVM CLI ........................................................................................................................................................... 25

3.5.4 ESX/ESXi CLI ................................................................................................................................................... 26

3.5.5 Oracle EM Express ........................................................................................................................................... 26

A How to identify and query the disk ids/WWNs ........................................................................................................... 27

A.1 How to identify VMware virtual disks in Linux guest......................................................................................... 27

A.2 How to identify XC Series volume group volumes in Linux guest .................................................................... 28

B Configuration details ................................................................................................................................................... 30

C Technical support and resources ............................................................................................................................... 31

C.1 Related resources............................................................................................................................................. 31

Page 4: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

4 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

Executive summary

The Dell EMC™ XC Series Web-Scale Hyperconverged appliance powered by Nutanix™ delivers a highly

resilient, converged compute and storage platform that brings benefits of web-scale architecture to business-

critical enterprise applications such as Oracle®.

The XC Series platform is hypervisor agnostic and software installs quickly for deployment of multiple

virtualized workloads. The XC Series Nutanix platform can deliver storage through multiple protocols such as

NFS, SMB, and iSCSI.

This document provides guidelines for design, configuration, and optimization of Oracle single instance and

Real Application Cluster (RAC) databases applications running on XC Series Nutanix infrastructure. The

document also outlines the different storage presentation methods offered by Nutanix to deploy the Oracle

database application.

Page 5: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

1 Product overview

1.1 XC Series appliances The XC Series is a hyperconverged solution that combines storage, compute, networking, and virtualization

into an industry-proven x86 Dell EMC PowerEdge™ server running Nutanix web-scale software. By

combining the hardware resources from each server node into a shared-everything model for simplified

operations, improved agility, and greater flexibility, Dell EMC and Nutanix together deliver simple, cost-

effective solutions for enterprise workloads. Acropolis Distributed Storage Fabric (DSF) delivers a unified pool

of storage from all nodes across the cluster, using techniques including striping, replication, autotiering, error

detection, failover, and automatic recovery.

The XC Series infrastructure is a scale-out cluster of high-performance nodes, or servers, each running a

standard hypervisor and containing processors, memory, and local storage (consisting of SSD flash for high

performance and high-capacity SATA disk drives). Each node runs virtual machines just like a standard

hypervisor host as displayed in Figure 1.

Nutanix node architecture

In addition, the Acropolis DSF virtualizes local storage from all nodes into a unified pool. Acropolis DSF uses

local SSDs and disks from all nodes to store virtual machine data. Virtual machines running on the cluster

write data to ADSF as if they were writing to shared storage.

1.2 XC Series all-flash Flash technologies are evolving rapidly and flash-based storage is capable of delivering millions of IOPS with

sub-millisecond latency. Also, flash prices have been decreasing, making flash a more cost-effective option

for storage. XC Series appliances are now available in both hybrid and all-flash variants. The blazing

performance, coupled with integrated data efficiency, resiliency and protection capabilities on all-flash variants

makes them the preferred choice for enterprise applications such as Oracle which require higher IOPS and

low latency.

The solution described in this document used a 3-node XC630-10AF all-flash cluster comprising of ten 800GB

SSDs in each node.

Page 6: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

6 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

1.3 XC Series Acropolis architecture Nutanix software provides a hyperconverged platform that uses Acropolis Distributed Storage Fabric (DSF) to

share and present local storage to all the virtual machines in the cluster. The general XC Series Nutanix

architecture is shown in Figure 2.

Nutanix architecture

Acropolis DSF virtualizes the storage across all nodes and presents the same to the hypervisor as one large

pool of shared storage. The DSF replicates writes synchronously to at least one remote XC Nutanix node to

ensure cluster resiliency and availability. Local storage for each XC Nutanix node in the architecture is

presented as one large pool of shared storage to hypervisor.

Each node runs an industry-standard hypervisor — VMware® ESXi®, Microsoft® Hyper-V®, or Acropolis

Hypervisor (AHV) — and the Nutanix Controller VM (CVM). The Nutanix CVM runs the Nutanix software and

serves I/O operations for the hypervisor and all VMs running on that host. Each CVM connects directly to the

local storage controller and its associated disks thereby reducing the storage I/O latency. The data locality

feature ensures virtual machine I/Os are always served by the local CVM on the same hypervisor node,

improving the VM I/O performance regardless of where it runs.

Page 7: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

7 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

1.4 XC Series Acropolis Block Services A feature called Acropolis Block Services (ABS) was released with Acropolis OS 4.7 (AOS). It allows DFS

resources to be exposed directly to a virtualized guest OS or physical hosts using the iSCSI protocol. This

capability enables support for several use cases such as shared storage for Oracle RAC and other

applications that require shared storage.

The XC Series Nutanix storage configuration for ABS is handled through a construct called a volume group

(VG). A VG is a collection of volumes commonly known as virtual disks (vdisks). ABS presents these vdisks to

virtual machines and physical servers using iSCSI protocol. Multiple hosts can share the vdisks associated

with a VG as shown in Figure 3. This is very helpful for shared storage use cases such as Oracle RAC or

Windows server clustering.

XC Nutanix Acropolis Block Services architecture

Page 8: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

8 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

2 Solution infrastructure The configuration and solution components are described in this section.

2.1 Physical system configuration

2.1.1 Oracle single instance database configuration Oracle single-instance database applications can be deployed on virtual machines and can be scaled easily

by adding additional virtualized database instances.

The physical configuration for this environment starts with the basic, three-node XC Series cluster shown in

Figure 4. Single-instance databases can be deployed on VMs on each host of the cluster as shown in the

figure. This architecture enables linear scaling of capacity and performance as you increase number of nodes.

Oracle single-instance configuration

Page 9: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

9 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

As shown in Figure 4, the local storage controller on each host ensures that storage performance as well as

storage capacity increases when additional nodes are added to the XC Series. Each CVM is directly

connected to the local storage controller and its associated disks. By using local storage controllers on each

ESXi host, access to data through Acropolis DSF is localized. It does not require data to be transferred over

the network, thereby improving latency.

Oracle Automatic Storage Management (ASM) is highly recommended for database-related files. The vdisk

presented to each VM is mapped as an Oracle ASM disk and ASM disk groups are carved out using the ASM

disks. The ASM disk groups are also shown in Figure 4.

Page 10: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

10 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

2.1.2 Oracle RAC database configuration Oracle RAC allows running multiple database instances on multiple servers in the cluster against a single

database. The database spans multiple servers but appears as a single unified database to end-user

applications. This architecture helps provide the highest availability and reliability to the Oracle database

applications.

The architecture in Figure 5 is similar to the previous configuration, but the three VMs on each host are

grouped together to form an Oracle RAC.

Oracle RAC configuration

Page 11: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

11 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

2.1.3 Oracle RAC database configuration using ABS The XC Series Nutanix ABS feature enables DSF storage resources to be presented directly to VMs and

physical servers using iSCSI. This configuration is similar to the previous configuration, but the database files

are presented to the VMs using iSCSI. A volume group consisting of multiple vdisks is created to store

database-related files and the vdisks are presented to VMs using iSCSI as shown in Figure 6.

Oracle RAC configuration using ABS

Page 12: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

12 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

2.2 XC Series storage and cluster configuration Each XC430 Series node used in this configuration is comprised of the following hardware components:

Ten 800GB SATA SSDs

Two 16-core Intel® Xeon® E5-2630 v3 2.40GHz processors

Twelve 16GB DDR-4 QR 2133MHz RAM modules (192GB total)

The minimum number of XC Series nodes in a cluster is three. When clustered together, the storage across

three nodes is virtualized together to create a single storage pool, and one or more containers can be created

on top of the storage pool. The storage container is presented to all nodes as shared storage within the

cluster.

The cluster attempts to keep virtual machines and their associated storage on the same cluster node for

performance consistency. However, each cluster node is connected to, and communicates with, the other

nodes on a 10Gb network. This communication allows virtual machines and their associated storage to reside

on different cluster nodes.

2.3 Network configuration Figure 7 shows the ideal network topology used for this solution. Two Dell EMC Networking S4810 switches

are stacked together to provide high availability. It is recommended to use at least two top-of-rack switches for

redundancy. Redundancy across the two switches is provided using the LAG, or stack, connection.

Network configuration

Page 13: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

13 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3 Sizing hypervisor configuration guidelines

3.1 Oracle database VM configuration

3.1.1 Processor and memory Sizing the vCPUs and memory of the virtual machines appropriately requires understanding the Oracle

workload. Avoid overcommitting processor and memory resources on the physical node. Use as few vCPUs

as possible because performance might be adversely impacted when using excess vCPUs due to the

scheduling constraints.

Hyperthreading is a hardware technology on Intel processors that enables a physical processor core to act

like two processors. In general, there is a performance advantage to enabling hyperthreading on the newer

Intel processors.

Each VMware vSphere® physical node also runs a Nutanix CVM. Therefore, consider the resources required

for the CVMs. Only one CVM would be running on a physical node, and it does not move to another physical

node when a failure event occurs.

While it is possible to support multiple Oracle database VMs on a same physical node, for performance

reasons, it is better to spread them out on multiple nodes and minimize the number of database instances

running on the same node. In the case of Oracle RAC, the RAC-instance VMs should run on different physical

nodes. VM-host affinity or anti-affinity rules can be set up for database VMs to define where they can run

within the cluster.

3.1.2 XC Series storage container and VMware storage virtualization On XC Series storage, it is typical to have a single storage container comprised of all SSDs and HDDs in the

cluster so that it can maximize the storage capacity and manage the auto-tiering more efficiently. The single

container is mounted on the ESX hosts through NFS as a datastore. Multiple virtual disks are created from

the same datastore and presented to the guest OS as SCSI disks that can be used by Oracle ASM.

Nutanix ABS allows storage to be presented directly into a non-VM physical host or virtualized guest OS

through iSCSI, bypassing the VMware storage virtualization layer. Additional consideration and extra

configuration steps are required for both Nutanix and the guest OS.

For virtual machines on VMware, it is recommended to present storage as virtual disks with VMware storage

virtualization because it offers a good balance between flexibility, performance, and ease of use.

Find more information on ABS at the Nutanix portal.

3.1.3 VM storage controller and virtual disks Typically, an Oracle database spans across multiple LUNs to increase performance by allowing parallel I/O

streams. In a virtualized environment, multiple virtual disks are used instead. Nutanix recommends to have at

least four to six database virtual disks and add more disks, depending on the capacity requirements, to

achieve better performance.

Page 14: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

14 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

It is best practice to create multiple controllers and separate guest OS virtual disk from database virtual disks.

The guest OS virtual disk should be on the primary controller. Additional controllers are created to separate

the virtual disks for data and log files.

Table 1 shows an example configuration of controllers and virtual disks for an Oracle database VM.

Virtual adapter and vdisk configuration for Oracle database VM

Controller Adapter type Usage Virtual disk

Controller 0 LSI Logic Parallel Guest OS 1 x 100GB

Controller 1 Paravirtual Data files, redo logs 4 x 200GB

Controller 2 Paravirtual Archived logs 2 x 200GB

Use the default adapter type LSI Logic Parallel for SCSI controller 0.

Choose Paravirtual SCSI (PVSCSI) adapter type for controllers where virtual disks are used for data files,

redo logs, and archived logs. The PVSCSI adapter allows greater I/O throughput and lower CPU utilization.

VMware recommends using this for virtual machines with demanding I/O workloads.

3.1.3.1 Shared access for virtual disks By default, VMware does not allow multiple virtual machines to access the same virtual disks. In an Oracle

RAC implementation where multiple database VMs need to access the same set of virtual disks, the default

protection must be disabled by setting the multi-writer option. The option can be found in the virtual machine

setting in the vSphere web client. It can be set when new virtual disks are created or when the virtual machine

is powered down.

Page 15: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

15 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.1.3.2 Queue depth and outstanding disk requests Splitting virtual disks across multiple controllers increases the limits of outstanding I/Os which a virtual

machine supports. For a demanding I/O workload environment, the default queue depth values might not be

sufficient. The default PVSCSI queue depth is 64 per virtual disk and 254 per virtual controller. To increase

these settings, refer to the VMware KB article 2053145.

For vSphere versions prior to 5.5, the maximum number of outstanding disk requests for virtual machines

sharing a datastore/LUN is limited by the Disk.SchedNumReqOutstanding parameter. Beginning with

vSphere version 5.5, this parameter is deprecated and is set per LUN. Review and increase the setting if

necessary. Refer to the VMWare KB article 1268 for details.

3.1.3.3 Enabling virtual disk UUID It is important to correctly identify the virtual disks inside the guest OS before performing any disk-related

operations such as formatting or partitioning the disks. WWN is commonly used as the unique identifier to

identify disks that support it. For the guest OS to properly see this information of the virtual disks, the

EnableUUID parameter must be set to TRUE in the virtual machine configuration. The steps to set this

parameter can be found in appendix A.

3.1.4 VM networking A minimum of two 10GbE interfaces are recommended for each ESXi host. The actual number required

depends on how many vSwitches and the total network bandwidth requirement. Each host should connect to

dual redundant switches for network path redundancy as described in section 2.3. Table 2 shows number of

vSwitches and their target usage.

vSwitches and target use

vSwitch Physical adapters

Speed Network Usage

vSwitchNutanix NA Intra CVM and ESXi host

svm-iscsi-pg vmk-svm-iscsi-pg

Primary storage communication path

vSwitch0 vmnic0 vmnic1

10GB Management network, VM network

Management traffic, VM public traffic, inter-node communication

vSwitch1 vmnic2 vmnic3

10GB VM network: Oracle Private Oracle RAC interconnect

vSwitch2 vmnic4 vmnic5

10GB VM network: iSCSI Dedicated iSCSI traffic to VMs

Page 16: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

16 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.1.4.1 Nutanix local virtual switch Each ESXi host which is part of the cluster has a local vSwitch automatically created as shown in the

following image.

This switch is used for local communication between the CVM and ESXi host. The host has a vmkernel

interface on this vSwitch and the CVM has an interface bound to a port group called svm-iscsi-pg which

serves as the primary communication path to the storage. This switch is created automatically when the

Nutanix operating system is installed. It is recommended not to modify this virtual switch configuration.

3.1.4.2 Management/inter-node network The virtual switch was created using two 10GbE physical adapters, vmnic0 and vmnic1, as shown in the

following image. The management traffic is very minimal but Acropolis DSF needs connectivity across CVMs.

This connectivity is used for synchronous write replication and also Nutanix cluster management. This

network is used for VMware vSphere vMotion® and other cluster management activities as well.

3.1.4.3 Public VM network All public client access to the virtual machines flow through this network. The client traffic can come in bursts

and sometimes the largest amount of data might be transferred between client and VMs. It is recommended

to set this up on the 10Gb network.

Page 17: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

17 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.1.4.4 Oracle RAC interconnect When deploying Oracle RAC, Oracle recommends a setting up a dedicated network for inter-RAC node

traffic. A separate vSwitch can be set up with redundant physical adapters to provide a dedicated RAC

interconnect network. Only RAC traffic should go on this network.

3.1.4.5 iSCSI network Typically, iSCSI is not a normal protocol used in an XC Series Nutanix environment. The ABS feature, part of

Nutanix release 4.7, enables presenting DSF storage resources directly to virtualized guest operating

systems or physical hosts using the iSCSI protocol. If this feature is used, it is recommended to use at least

two 10GbE interfaces dedicated for iSCSI traffic. The vSwitch configuration used for iSCSI connectivity in the

solution is shown in the following image.

Page 18: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

18 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.1.4.6 Other networking best practices Additional networking best practices include the following:

It is recommended to use dedicated NICs on the hosts for management, iSCSI, and RAC-

interconnect traffic. Also, dedicated VLANs are recommended to segregate each type of traffic.

For a standard virtual switch configuration, the default load-balancing policy is recommended: Route

based on originating virtual port.

These attributes help simplify implementation for configurations such as LACP.

The standard network packet size is 1500 MTU. Jumbo frames send network packets in a much

larger size of 9000 MTU. Increasing the transfer unit size allows more data to be transferred in a

single packet which results in higher throughput, and lower CPU utilization and overhead. Use Jumbo

frames only when all the network devices — including the network switches, CVMs, VMs, and ESXi

hosts — on the network path can support the same MTU size.

3.2 VM guest OS configuration for Oracle guidelines Oracle and VMware support most of the mainstream Linux® distributions. Dell recommends Oracle Linux or

Red Hat® Enterprise Linux for running Oracle databases due to the wide customer install base and the strong

supports of the database products by these companies. However, customers might choose any supported

Linux distribution based on their preferences. For this document, the information is based on Oracle Linux.

VMware recommends using the PVSCSI and VMXNET3 drivers for greater performance capability. Many

mainstream Linux distributions might have these drivers included and installed by default. However, in order

to ensure the drivers are at the latest version, obtain the latest version of VMware tools and install them in the

guest OS. Refer to the VMware KB article 1014294 for more information about general VMware tools

installation instructions.

Page 19: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

19 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

Installing and configuring the OS for an Oracle database in a virtual machine is similar to doing so on a

physical host.

1. Install the base OS from a virtual CD or ISO.

2. Update the guest OS, software packages, and bug fixes to the latest version from the vendor’s yum

repository.

3. Install the latest version of VMware tools.

4. Configure the guest OS to synchronize time from a trusted server. This is particularly important if

Oracle RAC will be used. All RAC virtual machines must maintain a synchronized time across the

RAC cluster. It is recommended to use NTP on Oracle Linux 6.x/Red Hat Enterprise Linux 6.x or

chronyd on Oracle Linux 7.x/Red Hat Enterprise Linux 7.x.

5. Transfer the Oracle installation media to the virtual machines. If deploying multiple instances, it might

be beneficial to set up an NFS mount and share the common media through NFS.

6. Disable the NetworkManager service.

7. Disable the avahi_daemon service if Oracle RAC will be deployed on the virtual machine.

8. Oracle recommends tuning the swapping priority on database servers. Set vm.swappiness=5 in

/etc/sysctl.conf.

9. Review the 12c Oracle Database Preinstallation Tasks and 12c Oracle Grid Infrastructure Installation

Checklist. Other Oracle versions can be found on https://docs.oracle.com.

10. Create Oracle OS users and groups (user oracle with primary group of oinstall and user grid with

primary group of oinstall).

11. Present virtual disks to the guest OS in vSphere.

12. Identify virtual disks using the procedures outlined in appendix A.

13. VMware recommends setting the disk timeout value to 180 seconds. This is typically handled by the

VMware tools where a udev rule is created to set this timeout. It has been discovered that this udev

rule is missing in the VMware tools for Oracle Linux 7.x/RHEL 7.x systems. To work around this

issue, the file can be created manually in /etc/udev/rules.d/99-vmware-scsi-timeout.rules with the

following contents:

ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="VMware ",

ATTRS{model}=="Virtual disk", RUN+="/bin/sh -c 'echo 180

>/sys$DEVPATH/timeout'"

14. Reboot the system or run the following command to apply the rule without rebooting:

udevadm trigger --action=add

15. Verify the new setting:

lsscsi –l or

cat /sys/block/sdX/device/timeout

16. Assign ownership and permission to the virtual disks for Oracle and make the setting persistent

across restart. This can be configured using the Linux udev facility. See section 3.4.1 for more

information on configuring Oracle ASM and an udev example.

Page 20: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

20 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

17. For Oracle Linux, install the Oracle preconfiguration script from Oracle’s yum repository. The package

automates some of the Oracle preinstall configuration steps such as adjusting kernel parameters,

configuring the oracle users OS limits, verifying and installing pre-requite packages.

a. Enable ol7-addon or ol6-addon in /etc/yum.repos.d/public-yum-olX.repo:

yum list oracle-rdbms-servers*

oracle-rdbms-servers-12cR1-preinstall

oracle-rdbms-servers-11gR2-preinstall

b. Install the corresponding package version using yum install command.

c. For other Linux distributions, follow the 12c Oracle Database Preinstallation Tasks and 12c

Oracle Grid Infrastructure Installation Checklist. Other Oracle versions can be found on

https://docs.oracle.com.

18. When using virtual disks with VMware storage virtualization, Multipath IO (MPIO) is not required

because each disk will only have a single path in the guest OS.

19. If Nutanix ABS is configured and virtual disks are presented directly to the guest OS through iSCSI,

the Linux iSCSI software initiator must be configured. MPIO with iSCSI disks is no longer required

starting with Acropolis OS 4.7 because it accomplishes session resiliency and HA with the new

external application service IP and iSCSI redirection. For versions prior to 4.7, MPIO with iSCSI disks

are necessary.

20. Nutanix recommends increasing the iSCSI MaxTransferLength to 1MB for large block sequence

workloads with I/O sizes of 1MB or larger. This setting can be changed in /etc/iscsi/iscsid.conf.

node.conn[0].iscsi.MaxRecvDataSegmentLength = 1048576

Find more information on Acropolis Block Services and client iSCSI configuration for use with ABS at the

Nutanix portal.

3.3 Nutanix CVM The Nutanix CVMs are virtual machines running on vSphere. They have direct control of the SSDs and HDDs

on the ESXi hosts through VMDirectPath I/O. They come preconfigured with modest settings so that

customers can start using the storage platform without much startup time. However, for heavy workload

applications such as databases, special attention should be paid to the CVM resources since they serve as

the primary access points to the SSDs and HDDs. During internal database workload testing, the CVMs

processor and memory resources were strained and limited the I/O performance. Dell EMC recommends

increasing each CVM vCPU to a minimum of 12 and memory to 32GB when heavy I/O workloads are

planned.

Continue to monitor the performance of CVMs periodically to fine-tune the configuration to achieve the

optimal performance level. Section 3.5 describes several ways to monitor the performance of CVM and other

infrastructure components.

Page 21: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

21 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.4 Storage layout for databases This section discusses common storage practices for Oracle databases based on Oracle database 12c.

3.4.1 Oracle ASM for Oracle single instance or RAC Oracle ASM is the preferred storage management solution for either single instance database or RAC. Oracle

ASM takes place of the traditional Linux volume manager and file system. When proper ownership and

permission are set on the disks, ASM takes over the management of the disks and create disk groups where

data files reside and are managed automatically. Using Oracle ASM also improves overall performance as it

stripes all files across all disks evenly. Another advantage of ASM is that disks can be added or removed

dynamically without any downtime, and files are restriped across the remaining disks automatically to ensure

optimal performance. ASM also has many advantages over the Logical Volume Manager (LVM) file-system

solution (discussed in section 3.4.2) and it should be considered as the preferred choice whenever possible.

Some of the benefits of ASM include:

Automatic file management

Online data files rebalance across disk volumes

Online addition and removal of disk volumes

Single solution for both volume and file management

The most common way to consume XC Series storage is to mount it as a datastore in the VMware hypervisor

through NFS. Virtual disks are then created through the regular vSphere client and added to the Linux guest

OS virtual machines. Oracle ASM can take control of these virtual disks just like any physical disks. Thus, no

extra configuration steps are required to make XC Series storage work with ASM.

The following Oracle ASM best practices are recommended:

Use Linux device management facility, udev, to assign ownership and access permissions

persistently on disk volumes. Without proper permission, ASM cannot manage and control the disk

volumes. For example, the udev rule specifies a disk volume with the device id/WWN,

36000c294a883a308becef0cd8448725a, creates a symlink in /dev/asmdatadg1, and assigns

grid:oinstall and 0660 permission to the device file.

/etc/udev/rules.d/99-oracle-asm.rules:

KERNEL=="sd*", PROGRAM="/lib/udev/scsi_id --page=0x83 --whitelisted --

device=/dev/%k",RESULT=="36000c294a883a308becef0cd8448725a",

SYMLINK+="asmdatadg1", OWNER:="grid", GROUP:="oinstall",MODE="0660"

To find out how to query the disk id/WWN, refer to appendix A.

Page 22: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

22 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

Oracle supports asynchronous I/O by default. The disk_asynch_io database parameter is set to

true. Oracle recommends using the deadline I/O scheduler for Oracle ASM. Oracle UEK kernels

have the deadline I/O scheduler as default. The Nutanix recommendation is to use the deadline I/O

scheduler for hybrid configurations consisting of SSDs and HDDs. If the default disk I/O scheduler is

not deadline, a udev rule file can be created to set it. The following shows an example of the rule.

60-disk-scheduler.rules:

ACTION=="add|change", KERNEL=="sd*", RUN+="/bin/sh -c '/bin/echo deadline

> /sys$env{DEVPATH}/queue/scheduler'"

For all-flash configurations, Nutanix recommends the NOOP I/O scheduler because it can reduce I/O

latency, improve throughput, and help reduce CPU cycles. The following shows an example of the

udev rule to set IO scheduler as NOOP.

60-disk-scheduler.rules:

ACTION=="add|change", KERNEL=="sd*", RUN+="/bin/sh -c '/bin/echo noop >

/sys$env{DEVPATH}/queue/scheduler'

Although ASM supports volumes of different sizes in the same disk group, to ensure best storage

performance, all volumes should have the same space capacity and performance characteristics.

A minimal of two disk groups should be created: one for data files and one for a fast recovery area.

If Oracle Grid Infrastructure will be installed for Oracle RAC, create a separate disk group to hold the

crs/voting disks.

Oracle ASM offers several levels of data redundancy to increase data availability. In High

Redundancy, ASM creates two mirror copies, and in Normal Redundancy mode, ASM creates one

mirror copy. In External Redundancy mode, there is only one copy of the data. Since XC Series

storage already offers data protection and HA by creating two replicas of the data on the storage level

by default, it is sufficient to choose External Redundancy for the ASM disk groups. However,

customers can choose other redundancy options to increase the level of protection, but must make

sure to factor in the additional storage required by using one of these redundancy modes.

It is recommended to keep the ASM allocation unit size as 1MB which is the default value for ASM

disk groups.

Table 3 shows an example of ASM disk group configuration:

Example ASM disk group configuration

ASM disk group

Number of virtual disks

AU size Description

DATA 4 1 MB Database files; temporary table space; online redo logs; system related table spaces such as SYSTEM and UNDO

FRA 2 1 MB Archive logs and backup data

CRS 1 1 MB Clusterware-related information such as the OCR and voting disks

Page 23: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

23 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.4.2 File system for Oracle single instance database Oracle also supports storing data files on modern file systems such as ext3/ext4. xfs is a high-performance

file system many Linux distributions include as default option. However, as of this time, Oracle has not

certified or tested Oracle databases running on xfs. Therefore, it should not be used for storing data and log

files. For performance reasons, Linux LVM should be considered for any file systems that hold the database

files. LVM provides the capability to distribute files across multiple disks similar to Oracle ASM. The use of file

system with LVM is recommended for a single-instance configuration only. If shared disks are required such

as Oracle RAC, use Oracle ASM instead. The following guidelines should be followed:

Nutanix recommends that a volume group consist of at least four to six volumes; more volumes can

be added later based on the capacity requirements.

All volumes should have the same space capacity and performance characteristics. The maximum

size available to the database is restrained by the smallest volume multiplied by the number of

volumes in the volume group.

Create two volume groups and one logical volume in each volume group: one for database files and

one for archived logs.

Striping the logical volumes can significantly improve the I/O performance.

Create an ext4 file system on the logical volumes. Use of ext4 file system is recommended at this

time. For a complete list of supported file systems, refer to Oracle KB article 1601759.1.

Oracle supports asynchronous I/O by default. The disk_asynch_io database parameter is set to

true.

Set the database parameter filesystemio_options = setall to instruct Oracle to use asynchronous

I/O and direct I/O when the underlying file system supports this.

Using Oracle Managed Files is recommended to simply the administration and management of

database files on file systems.

The Oracle software binaries should be installed into a separate file system, separated from the root

file system and the database file systems.

Table 4 shows an example of LVM configuration.

Example LVM configuration

Volume group

Number of virtual disks

Logical volume/FS

Striped Description

DATA 4 /data Y Database files; temporary table space; online redo logs; system related table spaces such as SYSTEM and UNDO

FRA 2 /fra Y Archive logs and backup data

3.5 Performance monitoring It is important to monitor the performance of various infrastructure components such as CVM, Oracle VM,

CPU, memory, storage, and the network. The following subsections describe the different ways to find this

information. One or multiple methods can be combined to provide a complete picture of the infrastructure

performance.

Page 24: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

24 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.5.1 Nutanix Prism Prism is a rich graphical HTML-based management console for configuring, managing, and monitoring all

components within the infrastructure. It provides excellent performance monitoring and analytic capabilities. It

also gathers certain data from the VMware vSphere and display all relevant information through a single

Prism interface. For example, to see the virtual machines, including CVMs and performance data, click VM >

Table > Check Include Controller VMs.

The Analysis section in Prism allows creating custom charts for almost every aspect of Nutanix platform and

some VMware vSphere metrics, providing invaluable insight into the environment. Customers can easily see

the current and historical trends by sliding the time scale at the top of the screen. Charts can be added from

the existing charts in other pages or can be easily created using a simple popup where components are

selected. Here are some of the useful charts:

Cluster hypervisor IOPS

Cluster hypervisor IO latency

Cluster IO bandwidth

Cluster read/write IOPS

Controller IOPS

Controller latency

Controller bandwidth

Container IOPS

Container latency

Container bandwidth

Disk IOPS

Cluster CPU/memory utilization

Content cache utilization

Page 25: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

25 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.5.2 vSphere client While Prism can see deep into the distributed storage platform, VMware vSphere provides extensive

information on the hypervisor and the VMs. Since CVMs are just other virtual machines managed by vSphere,

many resource statistics can be obtained through the regular vSphere client. To view this information, click

vSphere Web Client > Physical node > VM > Monitor tab > Performance.

3.5.3 CVM CLI The other option to obtain the CVM performance data is to connect directly to each CVM through ssh and run

the non-graphical command-line utilities:

1. ssh to the individual CVM IP addresses and log in with the Nutanix administration account (typically

nutanix).

2. Run the top utility to show real-time system resources such as processor, memory, and swap.

3. Run the free utility to show real-time memory and swap consumption.

4. Run additional Linux performance utilities as desired, such as vmstat, mpstat, and iostat.

The downside of the CLI is that the utilities offer only real-time information. To see historical data and trends,

use vSphere client or Prism.

Page 26: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

26 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

3.5.4 ESX/ESXi CLI To see performance of a specific ESX host, connect directly to the ESX host and run the esxtop command

utility which provides real-time resource utilization:

1. ssh to the ESX host.

2. Log in with the administrator account.

3. Run the esxtop command.

The information pertains only to that specific host and does not include any cluster-wide information. esxtop

breaks down the physical CPU utilization which can be very useful for troubleshooting performance issues.

3.5.5 Oracle EM Express Oracle provides a free web-based tool, Enterprise Manager Database Express (EM Express), for managing

Oracle Database 12c. EM Express is included with the database and does not require any separate middle-

tier application layer so it can be run directly on the database VM.

EM Express offers a friendly web UI. Under the performance hub, customers can find performance data

collected by the database in real-time or from a given period of time in the past. Some useful performance

data includes:

Database resource utilization

Top SQL sessions

SQL monitor

Workload characteristics

Top wait class

Redo logs throughput

IOPS

IO throughput

IO latency

ADDM

Page 27: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

27 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

A How to identify and query the disk ids/WWNs

Correctly identifying the database disks and setting the appropriate ownership and permission on them are

critical steps to take to ensure Oracle can properly take control of the disks. One of the most reliable methods

to identify the correct disks to use is to query the disk ids or WWNs. In the Linux guest, the scsi_id command

is commonly used to query the information. On the storage platform, depending on where the disks are

created (either in VMware or in ABS), the information is extracted from either VMware or the CVM. The

following subsections describe the procedures to query the disk information.

A.1 How to identify VMware virtual disks in Linux guest

Perform the following steps:

1. Set the EnableUUID parameter to TRUE in the virtual machine configuration. Without it set, the Linux

command scsi_id would not return the proper information. Click VM > Edit Settings > VM Options >

Advanced > Configuration Parameters > Add row.

2. Reboot the VM.

3. On the ESXi console, navigate to the VM directory and extract UUIDs from the vmdk files:

[root@XCORANODE1:/]# cd /vmfs/volumes/50eb5287-4ebf3591/oracle-node10

[root@XCORANODE1:/]# ls *vmdk|egrep -v flat|while read file;do echo -n

"$file" : ;awk -F= '/uuid/ {print $NF}' "$file"|sed -e 's/[ |-]//g' ; done

oracle-node10.vmdk :"6000C29336f6899c7fa6abfecaeea954"

crsvote.vmdk :"6000C298843d8fd2b37a500e0ad1ec66"

datadg1.vmdk :"6000C294a883a308becef0cd8448725a"

datadg2.vmdk :"6000C29e2e7074291cbca476ad8cf554"

datadg3.vmdk :"6000C29768dd2bc0ca3af87b113a7b08"

fradg1.vmdk :"6000C290368b65bc7fa1bcbb8d7a4bd4"

fradg2.vmdk :"6000C29f55ee3b293f26e7105f51372d"

oracle-node10.vmdk :"6000C29cacd0ff627d86bd3489c6c5e3"

Page 28: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

28 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

4. On Linux guest (Red Hat/Oracle Linux 6 or 7), extract the UUID with scsi_id commands:

[root@oracle-node10 ~]# ls -1 /dev/sd?|while read device; do echo -n

$device :; /lib/udev/scsi_id --page=0x83 --whitelisted --device=$device;

done

/dev/sda :36000c290368b65bc7fa1bcbb8d7a4bd4

/dev/sdb :36000c29f55ee3b293f26e7105f51372d

/dev/sdc :36000c29336f6899c7fa6abfecaeea954

/dev/sdd :36000c29cacd0ff627d86bd3489c6c5e3

/dev/sde :36000c298843d8fd2b37a500e0ad1ec66

/dev/sdf :36000c294a883a308becef0cd8448725a

/dev/sdg :36000c29e2e7074291cbca476ad8cf554

/dev/sdh :36000c29768dd2bc0ca3af87b113a7b08

5. Match up results from steps 3 and 4 to show the vmdk and Linux device file relationship.

A.2 How to identify XC Series volume group volumes in Linux guest

ABS exposes the Nutanix DSF through the iSCSI protocol directly into the virtualized guests or physical

systems. On the virtualized guests or physical systems, iSCSI services would be required to connect to ABS.

Perform the following steps:

1. Connect to one of the CVMs.

2. Invoke the ncli interface.

3. Issue the volume-group list command. A list of volume groups and the volumes within each volume

group would be shown.

Look for the Disks attribute in the output. All the virtual disks and their attributes including the VM

Disk UUID would be listed.

ncli> volume-group list

Attachment List : [{VM UUID=null, iSCSI Initiator=iqn.1988-

12.com.oracle:a5101ccc36b2}, {VM UUID=null, iSCSI Initiator=iqn.1988-

12.com.oracle:9e4ff9ee93e5}, {VM UUID=null, i

SCSI Initiator=iqn.1988-12.com.oracle:32bfb29df95}]

Description :

iSCSI Target : oraclevg2-160fbe38-8f97-4e5f-b7a8-

dcad2cdb89e4

Disks : [{VM Disk Size=100 GiB (107,374,182,400

bytes), VM Disk UUID=cfced470-276b-4b1e-948a-133c8078409c, Index=0,

Container ID=3222}, {VM Disk Size=100 GiB (107,

Page 29: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

29 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

374,182,400 bytes), VM Disk UUID=45a756e8-e3ff-411c-a354-e29beed44235,

Index=1, Container ID=3222}, {VM Disk Size=100 GiB (107,374,182,400

bytes), VM Disk UUID=27613917-b45f-43d2-96c5-61d

e3bafcbba, Index=2, Container ID=3222}, {VM Disk Size=100 GiB

(107,374,182,400 bytes), VM Disk UUID=6bd0060d-d37b-4cb2-8ce4-

fc669c4f3fc3, Index=3, Container ID=3222}, {VM Disk Size=100 Gi

B (107,374,182,400 bytes), VM Disk UUID=71a592b5-5555-41ea-b27e-

d9ec37d3c84e, Index=4, Container ID=3222}, {VM Disk Size=100 GiB

(107,374,182,400 bytes), VM Disk UUID=7109ed82-4ea7-437c-b

2f3-ca096671437f, Index=5, Container ID=3222}]

UUID : 160fbe38-8f97-4e5f-b7a8-dcad2cdb89e4

Shared : true

Name : ORACLEVG2

4. On Linux guest (Red Hat/Oracle Linux 6 or 7), extract the UUID with scsi_id commands:

[root@oracle-node10 ~]# ls -1 /dev/sd?|while read device; do echo -n

$device :; /lib/udev/scsi_id --page=0x83 --whitelisted --device=$device;

done

5. Match up results from steps 3 and 4 to show the vmdk and Linux device file relationship.

Page 30: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

30 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

B Configuration details

Solution configuration: hardware components Description

XC Series Nutanix cluster

3 x XC630-10AF:

ESXi 6.0

AOS: 4.7.2

2 x Intel Xeon Processor E5-2698 v3 @ 2.30GHz

512 GB RAM (16 x 32GB DIMMs)

10 x 800GB SATA SSD drives

1 x Intel(R) X520/2P I350 rNDC

1 x X520-2 Ethernet Server Adapter

A single VM will be hosted on each server and Oracle RAC database will be configured using the three VMs.

VM configuration 3 x Oracle Linux VMs:

VM version: 11

OS: Oracle Linux 6.7 UEK3

16 vCPUs, 64GB RAM

CVM: 12 vCPUs, 32GB RAM

The vCPU and memory configuration was increased to 16 vCPUs and 64GB RAM from default values on CVM.

Oracle database Oracle 12c R1 RAC database:

Oracle ASM (External Dependency)

Oracle 12c EM Express

Silly Little Oracle Benchmark (SLOB)

The SLOB tool was used to simulate real-world Oracle database transactions.

Network 2 x Dell EMC Networking S4810 switches

Firmware: 9.10

Dual redundant switches were used for high availability.

Performance monitoring

Nutanix Prism

VMware vCenter® performance monitoring

Oracle 12c EM Express

Performance monitoring for the XC Series cluster was done using Nutanix Prism and vCenter.

Oracle database I/O was monitored using Oracle 12c EM Express which is shipped as part of Oracle 12c.

Page 31: Deploying Oracle 12c RAC Database on Dell EMC XC Series ... · PDF file5 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS 1 Product overview 1.1 XC Series

31 Deploying Oracle 12c RAC Database on Dell EMC XC Series All-Flash | 3110-BP-SDS

C Technical support and resources

Dell.com/support is focused on meeting customer needs with proven services and support.

Dell TechCenter is an online technical community where IT professionals have access to numerous resources

for Dell EMC software, hardware and services.

Storage Solutions Technical Documents on Dell TechCenter provide expertise that helps to ensure customer

success on Dell EMC Storage platforms.

C.1 Related resources

See the following referenced or recommended resources related to this document:

Oracle Best Practices Guide: http://go.nutanix.com/rs/nutanix/images/BP-

2000_Oracle_on_Nutanix_Best_Practices.pdf

Nutanix Bible: http://nutanixbible.com/

Nutanix blog: http://longwhiteclouds.com/

Acropolis Block Services: http://go.nutanix.com/rs/031-GVQ-112/images/acropolis-block-services.pdf