WHITE PAPER

EMC SCALEIO FOR ORACLE DATABASE 12C SOLUTIONS: Oracle RAC and Data Guard replication

ABSTRACT

This document showcases EMC ScaleIO clusters as the ideal choice for Oracle RAC®

clusters. It also demonstrates that Oracle Data Guard® replication can be used to

replicate databases between ScaleIO clusters that have Oracle RAC clusters

configured. As a CIO or Enterprise Applications Architect, why buy expensive

dedicated storage systems when your database servers provide you all the

horsepower you need?

June 2016

2

To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local

representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store

Copyright © 2016 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without

notice.

The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with

respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a

particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

Oracle, Oracle Data Guard, Oracle RAC are registered trademarks or trademarks of Oracle Corporation in the United States and/or

other jurisdictions. All other trademarks such as Cisco used herein are the property of their respective owners.

Part Number H15217

3

TABLE OF CONTENTS

Executive Summary ........................................................................................... 4

Test Configuration .............................................................................................. 4

Storage Configuration for the Oracle Database ............................................................... 6

PERFORMANCE TESTS ........................................................................................ 8

OLTP Tests ........................................................................................................ 8

100% Read Workload ................................................................................................. 8

Switchover to Remote Site ........................................................................................ 10

Switchover to Remote Site ........................................................................................ 14

OLTP Workload after Switchover ................................................................................ 15

Data Warehouse Workload ................................................................................ 16

Different Workloads Running on Two Clusters ...................................................... 17

100% Read Workload on SLOB Database, Data Warehouse Query on TPC-H Database...... 18

OLTP Workload on SLOB Database, Data Warehouse Query on TPC-H Database ............... 18

CONCLUSIONS ........................................................................................ 19

4

Executive Summary

Oracle databases need powerful compute nodes as well as storage that offer low latencies. EMC ScaleIO® portfolio consists of software-only option, VxRack® FLEX Node or a fully engineered VxRack System 1000® with FLEX Nodes. All three deployment choices offer both these characteristics in a very small footprint. Each solution can also scale as your business needs grow. This scaling occurs both in the compute and low latency storage areas. In other words, ScaleIO clusters are truly flexible making them optimal for massive, mission critical Oracle® database environment. All of Oracle database solutions work well on ScaleIO configurations.

This document showcases ScaleIO clusters as the ideal choice for Oracle RAC® clusters. It also demonstrates that Oracle Data Guard® replication can be used to replicate databases between ScaleIO clusters that have Oracle RAC clusters configured. If the primary and remote ScaleIO clusters have identical configurations (for example; the number and type of ScaleIO nodes), then the performance measured on the primary and remote clusters will be identical.

EMC ScaleIO is a software-defined platform for block storage – software that creates a server-based SAN from local application servers and storage to deliver flexible and scalable performance and capacity on demand. It is ideal for traditional and modern applications and for those customers evaluating OpenStack deployments. Production and non-production database environments benefit since hyper-convergence allows for extremely fast writes. Its massive parallelism delivers quick recovery and stable, predictable performance.

Oracle Data Guard is a replication solution for Oracle databases. Oracle Data Guard can be used to create, maintain, manage, and monitor one or more standby databases to enable production Oracle databases to survive disasters and data corruptions. Oracle Data Guard maintains these standby databases as copies of the production database. Oracle Data Guard can also be used to switchover to a standby database for planned events like migrations and upgrades.

Test Configuration

For this lab validation, EMC ScaleIO software-only deployment option was used herein referred to as

‘ScaleIO nodes’. The configured system for this lab study is equivalent to any of the VxRack FLEX Node

configurations available. Eight ScaleIO nodes were configured as two clusters. Each cluster had four

ScaleIO nodes. Cisco C240 servers were used as ScaleIO nodes.

5

Figure 1 - Solution architecture. The primary and standby site each has four Cisco UCS 240 M4 servers.

Each Cisco C240 server was equipped as follows: Equivalent to any VxRack FLEX Node configurations

2 x E5-2650 v3 Intel Xeon processors, 10 Cores per Socket

256 GB RAM

2 x 10Gb Ethernet NICs, 1 x 1Gb Ethernet NIC

2 x 400GB 2.5” SATA eMLC SSD drives

21 x 2.5” SAS 10K RPM drives ( one drive dedicated to the Operating System and Database software) (None of the 10K RPM drives were used for the database)

Table 1 - Server Configuration

Operating System Red Hat Enterprise Linux Server release 6.5 (Santiago)

Hyper-converged platform

EMC ScaleIO Software Release 1.32.4504.5

Database Oracle 12c

Table 2 - Server Software Components

Oracle Database 12c Grid Infrastructure was used as the clusterware for Real Application Clusters. Oracle

database 12c was used as the database release software.

The Operating System software, the Grid Infrastructure software and the Database software were placed

on disk partitions of one 10K rpm SAS drive.

One ScaleIO protection domain and one storage pool were configured on each cluster. Each cluster’s

storage pool had an SSD pool that was comprised of the two SSD drives from each of the four nodes in

the cluster.

Figure 2 below showcases details about the SSD pool. This figure shows that there is one storage domain,

domain1 and there is one storage pool, pool1. Two disks from each node compose the storage pool in a

four node cluster.

6

Figure 2 - ScaleIO GUI displaying backend configuration

Storage Configuration for the Oracle Database

Oracle ASM volume manager was used as Oracle database volume manager.

To prepare for importing the ScaleIO volumes into ASM, the ScaleIO SDC agent presented two volumes from the SSD pool to each of the nodes in the cluster.

Two ASM diskgroups were configured: DATA and FRA Figure 3 below shows the following:

1. The two block devices that are mapped to the volumes presented by the ScaleIO SDC.

2. The udev rules file associated a consistently named block device with the underlying ScaleIO SDC device.

3. A list of the two block device entries in the Linux /dev directory mapped to the ASM disks.

7

Figure 3 - Udev rules were used to give persistent names to ScaleIO volumes

Two Oracle ASM disk groups, DATA and FRA, were created from the two ScaleIO volumes with ASM

External redundancy, as ScaleIO uses mesh-mirroring for data protection by default. The below shows the

two disk groups and their properties.

Figure 4 - ASM disk groups

8

After ASM was configured, two sets of workloads (OLTP and Data Warehouse) were loaded in succession and performance tests were run.

PERFORMANCE TESTS

As shown in

Figure 1, there are two Oracle RAC clusters configured on the two ScaleIO clusters with four nodes on

each.

Databases were built on the primary cluster. An Oracle physical standby database was then created on the

other cluster. Oracle Data Guard was configured in maximum performance mode to maintain the standby

database. Oracle Data Guard maintains the standby database by transmitting redo data from the primary

database and then applying the redo to the standby database.

The Data Guard broker command-line interface (the dgmgrl command) was used to switchover the

databases between the primary and replication clusters. Performance of the OLTP and Data Warehouse

workloads was measured at each site.

The following provides details of the two database models tested:

1. OLTP Model: A 600GB tablespace was created, then test data was loaded using the Silly Little

Oracle Benchmark kit (SLOB). SLOB guarantees completely random access to the data. It’s a very

simple, scalable and freely available tool kit anyone can use to test storage suitability for Oracle

Database. SLOB allows users to test varying read/write ratios as well as selectable transaction

logging weight. Details about the SLOB benchmark can be found at https://kevinclosson.net/slob/

2. Data Warehousing Model: A 600GB TPC-H Line item table was populated with data generated using

the DBGEN utility. SQL statements that resulted in Full Table Scans were run. The storage

throughput was then measured.

OLTP Tests

For these tests, two workloads were run:

1. 100% Read workload: This workload resulted in mostly 8KB Reads.

2. 75%/25% Read/Write (R/W) workload: OLTP style.

The goal of these exercises was to increase the physical IO rates until the average service times of the

reads remained at or below one millisecond.

100% Read Workload

Using SLOB, a 100% 8KB read workload with 24 load generators per node was used and the resulting IO

rate on the ScaleIO cluster was measured.

This test shows that the ScaleIO cluster sustained an IO rate of ~ 540,000 reads IOPS while maintaining a

read latency of approximately 1 ms. The screen shot below shows the read workload test.

9

Figure 5 - ScaleIO GUI showing workload during 100% Read workload test

For more information on EMC ScaleIO architecture, click here.

The I/O rate reported by ScaleIO GUI is corroborated by Oracle’s AWR reports.

The AWR report in Figure 6 shows that the IO was evenly balanced among the four RAC instances.

Figure 6 - AWR RAC statistics shows that the workload is evenly spread across Oracle Instances

The data below shows the histograms of I/O latencies. Approximately 60% of the single Oracle block

(8KB) reads completed in less than 1 ms and ~ 92% completed in less than 2 ms. Only 1% of the reads

took longer than 4 ms.

Note: The AWR report shows that there are many ‘db file parallel read’ wait events. The ‘db file parallel read’ occurs

when there is a non-contiguous multi-block read. This means that Oracle issues multiple single block read requests in

one batch. However, storage receives only single block read requests.

10

Figure 7 - Histogram of wait events during 100% Read test

Note: Higher IO rates of 575,000 I/O per second were observed when the number of I/O generators was increased. However, at this I/O rate, the read latency increased to about 1.5msec.

Switchover to Remote Site

The Oracle Data Guard command line interface (dgmgrl) was used to switchover the database from the

primary to the replication site.

The database IO was stopped before the switchover was performed. A screenshot of the switchover is

shown below.

This screenshot in Figure 8 shows that initially, database SIO is the primary and the database SIO1 is the

physical standby database. After the ‘switchover to SIO1’ is executed, database SIO1 becomes the

primary and database SIO is the physical standby.

11

Figure 8 - Screenshot showing successful switchover of database

The IO rate and latencies observed on the remote site were similar to that seen on the primary site.

OLTP Mixed IO Workload Results

When the 75/25 R/W Oracle workload was presented to the database, the 4-node ScaleIO cluster was able

to maintain an I/O rate of ~ 320,000 IOPS. This test ran for 20 minutes.

12

The two screenshots below, one represents the ScaleIO GUI and the other, Oracle Enterprise Manager

(OEM) show that the I/O rate was maintained for the duration of the test.

Figure 9 - ScaleIO GUI showing workload during Mixed IO OLTP test

Figure 10 - Oracle Enterprise Manager showing IO load during Mixed OLTP testFigure 11

shows sections of the AWR report where the average read latency was ~ 1.04 ms and that User I/O class

waits accounted for ~ 60% of all wait times.

Latencies of ~ 1 ms or less are the normal thresholds for read latencies for single Oracle block reads for

Flash drives. The AWR statistics show that this threshold is being honored by the ScaleIO cluster.

The wait event ‘gc current grant 2-way’ is to be expected in an Oracle RAC configuration. The ‘gc current

grant 2-way’ is a message-oriented wait event statistics indicating that no block was received because it

was not cached in any instance. Instead a global grant was given, allowing the requesting instance to read

the block from disk or modify it.

13

Figure 11 - Top 10 Foreground events sorted by total wait time during mixed I/O OLTP tests

The AWR histogram data in Figure 12 shows the distribution of the latencies.

The wait event ‘db file sequential read’ is a single-block random read. The histogram shows that ~ 72% of

the reads completed in less than 1 ms and ~ 87% were less than 2 ms.

Transaction log writes are also a key performance indicator. The AWR histogram data shows that ~30%

of the log file parallel writes completed in less than 1 ms, 70% completed in less than 2 ms and only ~8%

took more than 4 ms.

Figure 12 - Histogram of wait events during mixed IO OLTP test

Figure 13 displays the AWR RAC statistics. This shows that the workload was evenly distributed among the

four Oracle instances.

14

Figure 13 - AWR Oracle RAC statistics shows workload evenly distributed

Switchover to Remote Site

The Oracle Data Guard command line interface (dgmgrl) was used to switchover the database from the

primary to the replication site.

The database IO was stopped before the switchover was performed. A screenshot of the switchover is

shown below.

Figure 14 shows that initially database SIO is the primary and the database SIO1 is the physical standby

database. After the ‘switchover to SIO1’ is executed, database SIO1 becomes the primary and database

SIO is the physical standby.

15

Figure 14 - Screenshot of database switchover to SI01

OLTP Workload after Switchover

An OLTP workload (75/25 R/W) was run on the remote cluster. As expected, the I/O throughput was

similar to the one that was sustained on the original production cluster.

16

Figure 15 -ScaleIO GUI showing workload during Mixed I/O OLTP test on the remote site

Data Warehouse Workload

A 600GB scale TPC-H Lineitem table was populated using the DBGEN utility. Multiple queries that result in

tablescans, were run simultaneously. The resulting I/O throughput was measured using both the ScaleIO

GUI and the Oracle Enterprise Manager.

Both of these showed that the I/O throughput was 7.3GB/sec as shown in Figure 16 and Figure 17.

(The tables of the TPCH database were populated with data from flat files. The flat files were generated

using the dbgen utility and these files were stored (in file systems) on a pool of HDD spindles. The 6.2 TB

that is shown is the sum of the size of the SSD pool (which housed the database files) and the HDD pool

(for the flat files)).

Figure 16 - ScaleIO dashboard- Parallel Query Scan Performance

17

Figure 17 - Oracle Enterprise Manager -Parallel Scan Query performance

The database was switched over to the remote site using the Data Guard cli (dgmgrl) and performance

was measured. The throughput and latencies observed on the remote site was similar to that seen on the

primary site.

Different Workloads Running on Two Clusters

Frequently, customers will want to have different workloads running on both clusters. We demonstrate

this by showing that an OLTP database can be active on one site, and a Data Warehouse database can be

active on the second site.

To do this, we built two databases:

1. A SLOB database that was 300GB in size.

2. A TPCH database that was 300GB in size.

The SLOB database had its primary database on site A and the standby database on Site B. The TPCH

database had its primary on site B and its standby on site A.

Figure 18 – SLOB and TPC-H sharing RAC and ScaleIO clusters

The workloads were run concurrently on both the SLOB and TPCH databases.

18

100% Read Workload on SLOB Database, Data Warehouse Query on TPC-H

Database

First, a 100% read workload was run. The SLOB database was able to sustain an I/O rate of 532,000 I/O

per second. The TPC-H database was able to sustain a bandwidth of 7.3 GB/sec. These I/O rates are

similar to what was observed when only one database was active. Figure 19 shows the IO rate to the

SLOB database during this test.

Figure 19 –I/O requests by I/O function

OLTP Workload on SLOB Database, Data Warehouse Query on TPC-H Database

Next, a 75/25 R/W workload was run on the SLOB database and TPC-H queries were run on the TPC-H

database.

The SLOB database was able to sustain an I/O rate of 224,000 I/O per second and the TPC-H database

was able to sustain a bandwidth of 4.4 GB/sec.

The TPC-H database’s parallel degree had to be reduced so that the redo data of the SLOB database could

be transmitted and applied to its Standby database in a timely manner. In other words, the TPC-H

database bandwidth had to be reduced so that there were sufficient I/O resources available for the SLOB

database’s redo apply.

Figure 20 shows the IO rates on the SLOB database and the TPCH database when an OLTP workload was

run on the SLOB database and TPC-H queries were run on the TPC-H database.

19

Figure 20 - Screenshots of TPC-H and SLOB IOPs

CONCLUSIONS

EMC ScaleIO is software-defined block storage that offers multiple deployment options from a software-

only, VxRack FLEX Node as a hyper-converged appliance to a fully engineered hyper-converged VxRack

System 1000 with FLEX Nodes. Each deployment drives agility, high performance, dense compute

delivering low latencies, while remaining extremely flexible, and maintaining a small footprint.

Performance demonstrated throughout this paper proves that EMC ScaleIO portfolio offers an ideal choice

for Oracle RAC clusters. It also demonstrates that Oracle Data Guard replication can be used to replicate

databases between ScaleIO clusters that have Oracle RAC clusters configured.

For more information on EMC ScaleIO, visit http://www.emc.com/storage/scaleio/index.htm.