Dell EMC Ready Solutions for Oracle: Design for Dell EMC ... · 8 Dell EMC Ready Solutions for...

Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage

With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1

February 2019

H17577

Reference Architecture Guide

Abstract

This reference architecture guide shows that customers can migrate from Oracle

Database 12c to Oracle Database 18c running on the Dell EMC Unity 650F All Flash

storage array with no performance impact to production workloads.

Dell EMC Solutions

Copyright

2 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Databases 12cR2 and 18cR1 Reference Architecture Guide

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA February 2019 Reference Architecture Guide H17577.

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Contents

3 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1


Contents

Chapter 1 Executive Summary 5

Ready Solutions for Oracle ................................................................................... 6

Scope.................................................................................................................... 7

Audience ............................................................................................................... 7

We value your feedback ........................................................................................ 7

Chapter 2 Architecture and Design Considerations 8

Unity 650F storage design .................................................................................. 12

Compute and network design .............................................................................. 14

FC fabric connectivity and zoning ........................................................................ 14

Virtual network design ......................................................................................... 16

VM configuration ................................................................................................. 17

Guest operating system configuration ................................................................. 18

Chapter 3 Oracle Database 12cR2 Performance on Unity 650F 20

Test objectives .................................................................................................... 21

Use cases, test methods, and test results ........................................................... 22

Chapter 4 Oracle Database 18cR1 Performance on Unity 650F 32

Test objectives .................................................................................................... 33

Upgrading Oracle Database 12cR2 to 18cR1...................................................... 34

Use cases, test methods, and test results ........................................................... 36

Chapter 5 Summary 49

Summary ............................................................................................................ 50

Chapter 6 References 52

Dell EMC documentation..................................................................................... 53

VMware documentation ...................................................................................... 53

Oracle documentation ......................................................................................... 53

SLOB documentation .......................................................................................... 53

Appendix A Test Tools, and Database and SLOB Configuration 54

Testing and performance collection tools ............................................................ 55

SLOB dataset customization ............................................................................... 55

Database parameter configuration ...................................................................... 56

SLOB parameter settings (slob.conf)................................................................... 58

Contents

4 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide

Appendix B Equipment List 59

Hardware components ........................................................................................ 60

Software components ......................................................................................... 62

Chapter 1: Executive Summary



Chapter 1 Executive Summary

This chapter presents the following topics:

Ready Solutions for Oracle ................................................................................ 6

Scope ................................................................................................................... 7

Audience .............................................................................................................. 7

We value your feedback ..................................................................................... 7



Ready Solutions for Oracle

Today’s database world faces many challenges including performance stabilization, data

consolidation, rapid creation of production copies, cost of organizational infrastructure

(space and storage), and other factors contributing to TCO. The challenges are becoming

more complex, critical, and intense as organizations are pushed to upgrade their

databases to the latest versions (for example, upgrading Oracle Database 12cR2 to

18cR1) because of business, technical, or infrastructure issues. Because of the

challenging circumstances, it is extremely important to offer and support a reference

architecture that is not only integrated, tested, and validated, but also addresses all the

customers’ pain points.

To address these challenges, Dell EMC Ready Solutions for Oracle provides a reference

architecture that features operational agility, efficiency, stability, resiliency, and storage

savings. Testing and validation of the architecture prove that customers can migrate from

Oracle Database 12cR2 to Oracle Database 18cR1 running on a Dell EMC Unity 650 All

Flash (Unity 650F) storage array with no adverse performance impact to production

workloads. This Dell EMC reference architecture for Oracle yields a faster time-to-value

along with superior performance, significant cost savings, and future-ready scalability.

Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Storage is a reference

architecture consisting of:

Unity 650F storage

Dell EMC PowerEdge R840 and R640 servers

Dell EMC Networking and Dell EMC Connectrix switches

VMware vSphere virtualization (version 6.5)

Oracle Databases 12cR2 and 18cR1

By eliminating the time-consuming and complex process of designing a system, this

tested and validated reference architecture streamlines the purchase and update cycles

for the IT organization and accelerates delivery times of complex mission-critical Oracle

Databases 12cR2 and 18cR1 and related applications. Features of this reference

architecture for Oracle include:

Significant data compression and storage savings, and reduced TCO

Support for simplified upgrading of Oracle Database 12cR2 to Oracle Database

18cR1 with no degradation of performance, as demonstrated by the performance

metrics in this guide

Consistent database performance while running an 80/20 read/write Silly Little

Oracle Benchmark (SLOB) stress test and creating two snapshots (both in 12cR2

and 18cR1 databases) hosted on the Unity 650F storage array

Overview

Design for Unity

reference

architecture




Scope

This reference architecture guide describes the architecture design and the process and

outcome of upgrading an Oracle Database 12cR2 to an Oracle Database 18cR1. The

guide describes how we tested and validated the architecture with multiple datasets and

workloads, with and without database snapshots, to ensure maximum flexibility and to

prove that upgrading the database has no negative impact on data integrity or

performance. This guide describes how to upgrade the database and discusses the

methodology and results of the testing that we conducted on the architecture.

Audience

This guide is for IT administrators, storage administrators, virtualization administrators,

system administrators, IT managers, and personnel who evaluate, acquire, manage,

maintain, or operate Oracle database environments.

We value your feedback

Dell EMC and the authors of this document welcome your feedback. Contact the Dell

EMC Solutions team by email or provide your comments by completing our

documentation survey.

Authors: Oracle Ready Solutions Engineering team, Indranil Chakrabarti, Reed Tucker

Note: The following page of the Oracle space on the Dell EMC Communities website provides

links to additional documentation for Dell EMC solutions for Oracle: Oracle Info Hub for Ready

Solutions.

mailto:[email protected]?subject=Feedback:%20Dell%20EMC%20Ready%20Solutions%20for%20Oracle:%20Design%20for%20Dell%20EMC%20Unity%20All%20Flash%20Unified%20Storage%20Reference%20Architecture%20Guide%20(H17577)

https://www.surveymonkey.com/r/SolutionsSurveyExt

https://community.emc.com/docs/DOC-69044


Chapter 2: Architecture and Design Considerations


Chapter 2 Architecture and Design Considerations


Unity 650F storage design ................................................................................ 12

Compute and network design .......................................................................... 14

FC fabric connectivity and zoning ................................................................... 14

Virtual network design ...................................................................................... 16

VM configuration ............................................................................................... 17

Guest operating system configuration ............................................................ 18




Architecture

This section provides an overview of the physical and logical architecture of the database

environment.

The following figure shows the major hardware components of the reference architecture.

Figure 1. Physical architecture

As shown in the physical architecture diagram, the architecture consists of a server layer,

network layer, and storage layer.

Server layer

The server layer consists of:

PowerEdge R840 virtual database server—One virtual OLTP database–

Oracle12cR2, which is later upgraded to Oracle 18cR1—is deployed in a single VM

running Red Hat Enterprise Linux (RHEL) 7.4 as the guest operating system. The

VM runs on a single PowerEdge R840 server with the VMware ESXi 6.5 hypervisor

installed. The server includes the following network components:

Two dual-port 10 GbE network interface controllers (NICs)—For Oracle

public traffic

Two dual-port 16 Gbps host bus adapters (HBAs)—For SAN traffic

Physical

architecture



A minimum of one 1 GbE management remote Network Daughter Card

(rNDC) or LAN on motherboard (LOM) port—For in-band management of the

server from within the operating system

Dedicated 1 GbE iDRAC Ethernet port—For out-of-band management of the

server

PowerEdge R640 management server—The management server runs the

VMware ESXi 6.5 hypervisor. One 1 Gb rNDC port is used for management traffic

and two 10 GbE ports are used for the workload generation (SLOB I/O toolkit)

traffic.

Network layer

The network layer consists of:

Two 10 GbE network switches—Connect to two 10 Gb ports on the database

server to route the Oracle public traffic

Two 16 Gbps Fibre Channel (FC) fabric switches—Route SAN traffic between

the R840 database/ESXi host and the Unity 650F storage array

One 1 GbE network switch—Routes all management traffic between the

components—ESXi host, management server, switches, and Unity 650F storage

Storage layer

The storage layer consists of one Unity 650F storage array as the FC SAN storage to host

Oracle Database 12c and 18c. The Unity array tested in this reference architecture

consists of:

One disk processor enclosure (DPE) with two storage processor (SP) controllers

Four 16 Gbps front-end FC ports per SP

Usable storage capacity of 48.6 TB

The LAN and SAN design features redundant components and connectivity at every level

to ensure that there is no single point of failure. The design enables the application server

to reach the database server and the database server to reach the storage array even if

any of the following components fail:

One or more NICs or HBA ports

One LAN or FC switch

One or more Unity front-end ports

One Unity SP

For details about SAN zoning best practices and configuration, see FC fabric connectivity

and zoning in Chapter 2.




The following figure shows the logical architecture overview of the database environment.

It shows the multiple layers of infrastructure components in the reference architecture,

along with a high-level overview of the software components that are deployed on the

individual hardware components.

Figure 2. Logical architecture

We tested and validated the following types of Oracle databases in the reference

architecture:

A single-node Oracle Database 12cR2 in a virtual environment—Oracle 12cR2

Grid infrastructure (GI) and a standalone Oracle Database 12cR2 run on one virtual

machine (VM).

A single-node Oracle Database 18cR1 in a virtual environment—Oracle 18cR1

GI and a standalone Oracle Database 18cR1 on one VM. This Oracle Database

18cR1 stack is upgraded from the Oracle Database 12cR2 stack.

As shown in Figure 2, one PowerEdge R840 server is the ESXi host, with ESXi 6.5 U2

hosting a single VM. The VM hosts Oracle Database 12cR2, which we later upgraded to

Oracle Database 18cR1. The R840 server has four Intel 18C CPUs and 1,536 GB RAM.

The VM uses RHEL 7.4 as the guest operating system that runs the Oracle 12cR2

Logical

architecture



software stack, which includes Oracle 12cR2 GI and a standalone Oracle Database

12cR2.

After completing the Oracle Database 12cR2 validation and performance studies, we

upgraded the Oracle 12cR2 software stack to Oracle 18cR1; that is, we upgraded Oracle

12c GI to Oracle 18cR1 GI and Oracle Database 12cR2 to Oracle Database 18cR1.

A management server runs the VMware vCenter Server Appliance (VCSA) and the SLOB

benchmarking tool, which are deployed on two separate VMs.

The Unity 650F storage array hosts the storage volumes of Oracle Database 12c and 18c

as well the VM operating system disks. The Unity 650F storage array has two pools:

VM pool (VM_pool) —Stores the VM operating system LUN, three Oracle GI

clusterware storage LUNs for Oracle Cluster Registry (OCR), the voting disk, and

the GI Management Repository (GIMR).

Oracle Database pool (DB_Pool) —Stores all the database LUNs and the

snapshots of these database LUNs.

This architecture includes the following networks:

LAN/Public network—Provides the public network connection between the

database server and applications. For our test environment, this network connects

the SLOB benchmark server and the database server. This network is based on

10 GbE physical network components, as described in Physical architecture.

SAN network—Provides storage I/O communication between the database server

and the Unity storage array. The 16 Gbps Fibre Channel (FC) network is used for

this SAN network.

Management network—Manages the ESXi host, Unity storage array, and network

switches. This network is based on a 1 GbE physical network, as described in

Physical architecture.

Unity 650F storage design

Dell EMC Unity storage systems support two types of storage pools on all-flash storage

arrays: traditional pools and dynamic pools1. Dynamic pools provide many benefits over

traditional pools. The new pool structure eliminates the need to add drives in multiples of

RAID stripe-widths. Data space and replacement space are spread across the drives

within the pool. For greater flexibility in managing and expanding the pool, better drive

utilization, and improved application I/O, we recommend that you create a small number

of dynamic pools that include large numbers of drives of the same type. When

determining the number of pools, consider that there might be different types of

workloads, and dedicate resources to meet specific performance goals. For additional

Unity storage best practices for Oracle Database, see Dell EMC Unity Storage with Oracle

Databases.

1 Dynamic pools are supported in Unity OS version 4.4.x and higher. Refer to Appendix B for

storage array details

https://www.emc.com/collateral/white-papers/h16765-unity-storage-with-oracle-databases-wp.pdf





The following table details the storage pools that we created for this reference

architecture.

Table 1. Unity storage array pool design

The following table details the database LUNs that we created in the storage pools. In

additional to these database LUNs, we created two snapshots of the database volumes to

study the impact of snapshot creation on database performance and storage capacity.

Table 2. Oracle Database LUN design on Unity storage

LUN Purpose VM_POOL DB_POOL Thin LUN Data reduction enabled?

VM OS VM operating system volume

1 x 500 GB Yes Yes

OCR/VD Clusterware and virtual disk (VD) storage

3 x 50 GB Yes Yes

DATA Database files

4 x 500 GB Yes Yes

REDO REDO logs 4 x 25 GB Yes Yes

TEMP TEMP tablespace

1 x 500 GB Yes Yes

FRA Flash Recovery Area for archive logs

2 x 100 GB Yes Yes

Pool name RAID configuration

Capacity Purpose Number of LUNs

VM_Pool RAID 5 2.8TB VM operating system LUN and Oracle Clusterware LUNs

4

DB_Pool RAID 5 45.8 TB Database LUNs and their snapshots

11 original database LUNs plus additional snapshots upon creation



Compute and network design

We configured the PowerEdge R840 database server as follows:

Installed ESXi 6.5 by using the Dell EMC customized ISO image (Dell Version: A04,

Build# 5310538), which is available on Dell EMC Online Support at VMware ESXi

6.5.

Zoned two dual-port 16 Gb/s HBAs, four initiators in total, and configured them with

the Unity 650F front-end FC ports for high-bandwidth, load-balanced, and highly

available SAN traffic. For the recommended FC connectivity and zoning best

practices, see FC fabric connectivity and zoning. For optimal performance, the two

HBA cards are populated in slots 2 and 5 of the R840 server.

Configured one 1 GbE rNDC or LOM port for the management traffic and two 10

GbE ports for the Oracle public traffic. For more details on the virtual networking

design, see Virtual network design. For optimal performance, we installed the two

10 GbE network adapters in slots 3 and 6 of the R840 server.

Created a single VM with RHEL 7.4 as the guest operating system for the virtual

Oracle standalone databases. For more details about the VM properties and best

practices, see VM configuration.

We configured, monitored, and maintained the ESXi host, virtual networking, and the VM

by using VMware vSphere Web Client and VCSA, which is deployed as a VM on the

management server.

Multipath configuration

We configured multipathing on the ESXi 6.5 host according to the following best practices:

Use vSphere Native Multipathing (NMP) as the multipathing software.

Retained the default selection of round-robin for the native path selection policy

(PSP) on the Unity volumes that are presented to the ESXi host.

Change the NMP round-robin path switching frequency from the default value

(1,000 I/O packets) to 1. For information about how to set this parameter, see Dell

EMC Unity Storage with VMware vSphere.

FC fabric connectivity and zoning

The following figure shows the recommended FC connectivity between the HBAs and the

FC switches, and the connectivity between the FC switches and the Unity storage array.

As shown in the figure, each port in each HBA connects to two separate FC switches, and

two front-end ports from each of the Unity SPs connect to two separate FC switches,

forming two FC fabrics.

ESXi host

configuration

https://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverid=f5gh0

https://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverid=f5gh0

https://www.emc.com/collateral/whitepaper/h16391-dellemc-unity-storage-vmware-vsphere.pdf

https://www.emc.com/collateral/whitepaper/h16391-dellemc-unity-storage-vmware-vsphere.pdf




Figure 3. FC fabric connectivity design

Dell EMC recommends single-initiator zoning when creating zone sets on the FC

switches. For high availability, bandwidth, and load balancing, each initiator or HBA port

on the ESXi host is zoned with four front-end Unity storage ports that are spread across

the two storage controllers or SPs, as shown in the following logical representation of

zone sets.

Figure 4. FC zoning logical representation



Virtual network design

The following diagram shows a high-level overview of the virtual network design that we

implemented in the ESXi host. It also shows the mapping between the virtual switch

network and the physical switch network.

Figure 5. Virtual network design in the ESXi host

The virtual network design supports:

VM and management traffic—The VM network and management traffic uses the

default standard virtual switch (vSwitch), which contains two default standard ports

groups. The Management Network port group provides the VMkernel port vmk0 to

manage the ESXi host from VCSA. The VM Network port group provides the 1 GbE

virtual interfaces for in-band management of the database VM. All management

traffic is routed through the 1 GbE physical rNDC or LOM port on the ESXi server

that is connected to the external 1 GbE management switch virtual or network.

Public traffic—The Oracle public traffic uses an additional dedicated standard

vSwitch to which we assigned two physical 10 GbE uplink ports. For high

availability and load balancing, the two 10 GbE uplink ports reside on two separate

physical network adapters and are connected to two separate 10 GbE physical

network switches. Within this vSwitch, we created a standard port group that

provides the virtual network interface for the Oracle public traffic within the

database VM.




VM configuration

We used the following design principles and best practices to create the database VM:

SCSI controllers—We created multiple SCSI controllers to optimize and balance

the I/O for the different database disks, as shown in the following table. We chose

the controller type VMware Paravirtual for optimal performance.

Table 3. SCSI controller properties in the database VM

Controller Purpose SCSI bus sharing Type

SCSI 0 Guest operating system disk None VMware Paravirtual

SCSI 1 Oracle DATA disks Physical VMware Paravirtual

SCSI 2 Oracle REDO disks Physical VMware Paravirtual

SCSI 3 Oracle OCR, GIMR, FRA, TEMP

Physical VMware Paravirtual

Hard disk drives—We assigned the following properties to all database-related

virtual disks (for example, DATA, REDO, FRA, OCR/VD, and TEMP):

Raw Device Mapping (RDM)—For optimal performance and management

simplicity, all Oracle related disks presented to the ESXi host from the Unity

storage array are mapped directly as raw devices to the database VM.

Virtual Device Node—For load balancing and optimal performance, the SCSI

controllers are assigned as noted in Table 3.

VM vCPU and vMem—The following table lists the amount of virtual CPU (vCPU)

and virtual memory (vMem) that we assigned to the database VM during the testing

of the reference architecture.

Table 4. VM configuration: vCPU and vMem details

vCPUs vMem

Number of vCPUs

Limit (MB) Reservation (GB)

Total (GB) Limit (MB)

18 Unlimited 192 256 Unlimited

Network adapters—We added two network adapters, one for in-band VM or

guest operating system management and one for Oracle public traffic, to the

database VM. We configured the two adapters with the recommended type setting

of VMXNet 3.

Enable disk UUID—In each of the VM options, we added the configuration

disk.enableUUID parameter and set it to TRUE. This setting ensures that the

VMDK always presents a consistent disk UUID to the VM.



Guest operating system configuration

In this reference architecture, we use the following best practices to deploy and configure

RHEL 7.4 as the guest operating system in the VM running the Oracle standalone

database:

Install and configure the operating system, network, storage disks, Oracle 12cR2

Grid, and standalone Oracle Database 12cR2 within the VM, as instructed in the

following Dell EMC knowledge base article: How to deploy Oracle 12c Release 2

Standalone Database on RHEL 7.x

Set up the Oracle Grid and database software prerequisites (required operating

system RPMs, users, groups, kernel parameters, and so on) by using the

information and deployment package in the following Dell EMC knowledge base

article: Dell Oracle Deployment RPMs for Oracle 12cR2 on RHEL7.x

Important best practices include:

For each Oracle virtual disk, create a single partition that spans the entire disk

and has a starting offset of 2,048 sectors.

Ownerships and permissions on the Oracle disks within the VM are established

using UDEV rules. The following example shows a UDEV rule set for one of the

Oracle disks (REDO disk) within the custom /etc/udev/rules/60-oracle-

asmdevices.rules UDEV rules file:

KERNEL=="sd[a-z]*[1-9]", SUBSYSTEM=="block",

PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="3600601600f004300accaed5bd9741db5",

SYMLINK+="oracleasm/disks/ora-redo1", OWNER="grid",

GROUP="asmadmin", MODE="0660"

Note: For this reference architecture, we tested the standalone Oracle Database 18c by

performing an in-place upgrade of the deployed 12cR2 database. For an overview of the upgrade

process, see Upgrading Oracle Database 12cR2 to 18cR1.

As described in Unity 650F storage design, we mapped all Oracle related LUNs that are

presented to the ESXi host from the Unity 650F storage array directly as raw devices to

the database VM through RDM. In compliance with the UDEV rules, we assigned the

ownership of the raw devices to the grid user who is the owner of the Oracle GI and

Oracle Automatic Storage Management (ASM). The device link for these Oracle related

raw devices is /dev/oracleasm/disks/ora-XXX. For example,

/dev/oracleasm/disks/ora-redo1 is the device link for REDO1 LUN/raw device.

The following table shows the Oracle disk groups that are created based on these

LUNs/raw devices. Except for the OCR disk group that uses the normal redundancy (with

triple mirroring), all other disk groups use the external redundancy setting. The coarse

striping setting is also used for DATA, FRA, and OCR disk groups, and the fine-grain

striping setting is used for REDO1, REDO2, and TEMP disk groups.

https://www.dell.com/support/article/us/en/04/how16670/how-to-deploy-oracle-12c-release-2-standalone-database-on-rhel-7x?lang=en


https://www.dell.com/support/article/us/en/04/sln314618/dell-oracle-deployment-rpms-for-oracle-12cr2-on-rhel7x?lang=en




Table 5. ASM disk group design

ASM disk group

Purpose Redundancy ASM striping ASM disk group size (GB)

LUN LUN size (GB)

DATA Data files, control files, undo tablespace

External redundancy

Coarse 2,000 DATA00 500

DATA01 500

DATA02 500

DATA03 500

FRA Archive log files

External redundancy

Coarse 200 FRA0 100

FRA1 100

REDO1 Online redo logs

External redundancy

Fine-grain 50 REDO0 25

REDO1 25

REDO2

Online redo logs

External redundancy

Fine-grain

50 REDO2 25

REDO3 25

TEMP

Temp files External redundancy

Fine-grain

500 TEMP

500

OCR OCR, voting disk, GIMR

Normal redundancy

Coarse

50 OCR0 50

OCR1 50

OCR3 50

Oracle ASM has a feature to move the data to higher performance tracks of the spinning

disks in the compact phase at the end of ASM disk rebalancing. This feature has no

benefit for Dell EMC Unity storage when the physical storage is virtualized and the flash

devices are used. You can disable the rebalancing feature by running the alter

diskgroup command for all the disk groups. The following example shows the command

for the DATA disk group:

SQL> alter diskgroup DATA set attribute '_rebalance_compact' =

'FALSE';

For more information about the ASM disk group guidelines, see Dell EMC Unity Storage

with Oracle Databases. For more information about ASM compact rebalancing, see

Oracle Support note 1902001.1.



Chapter 3: Oracle Database 12cR2 Performance on Unity 650F


Chapter 3 Oracle Database 12cR2 Performance on Unity 650F


Test objectives .................................................................................................. 21

Use cases, test methods, and test results ...................................................... 22




Test objectives

In a typical IT environment, databases might be required for testing, development,

reporting, or online analytics. Usually, these additional databases must be based on

copies of the production databases because the new features or hardware cannot be

tested directly on the production systems themselves.

The Dell EMC Unity 650F storage array’s snapshot feature enables you to create multiple

copies of any database. The objective of the tests described in this chapter is to simulate

a typical customer environment in which we create a baseline Oracle production

database, measure its performance, and then create multiple snapshot copies of it to

measure the impact of creating snapshots. Another objective is to study the data savings

feature of the Unity storage as applied to the Oracle databases and analyze how it can

help customers with data compression and space savings. These savings will ultimately

translate into storage-cost and TCO savings.

In our use cases, we analyze the impact of upgrading Oracle Database 12c to the 18c

version on the related performance and data savings numbers. This data benefits the

DBAs and test/dev engineers who frequently must spend hours managing database

creation and refreshing the environments, often while limited by capacity, performance,

and number of database copies.

The following use cases demonstrate the performance and capacity savings of Oracle

Database 12cR2 running on the Unity 650F storage array, as well as the performance

impact of creating Unity snapshots of Oracle Database 12c:

Use case 1: Deduplication and compression of Oracle Database 12cR2

Use case 2: Oracle Database 12cR2 baseline performance

Use case 3: Unity storage snapshot-creation impact on Oracle Database 12cR2

performance

Use case 4: Deduplication and compression of Oracle Database 12cR2 with data

changes

Our use-case testing included stress and compression testing to produce the performance

numbers that are shown in this reference architecture guide. The data was extracted from

Oracle Automatic Workload Repository (AWR) reports. Compression and performance

results documented in this guide are provided as a reference. The actual numbers you

achieve might vary with your environment.

Note: During all testing, the Unity inline data reduction feature was enabled, as noted in Unity

650F storage design. This feature uses some storage CPU and memory cycles.

http://expertoracle.com/2018/02/06/performance-tuning-basics-15-awr-report-analysis/



Use cases, test methods, and test results

In this use case, we loaded data into Oracle Database 12cR2 on the Unity 650F storage

array to understand the impact of data compression and deduplication, also referred to as

data reduction. We tested this reference architecture by loading approximately 1.2 TB of

data into the Oracle 12cR2 test database using the SLOB multiple schema model. The

native SLOB schema contains highly redundant data and, therefore, is highly compressible.

To remove redundancy and to showcase the Unity array’s data compression capabilities

in the worst-case scenario, we used a custom PL/SQL script to insert randomized and

unique data into the SLOB schema (for details, see Appendix A).

After loading approximately 1.2 TB of data into the Oracle 12cR2 test database, a space

savings of 29.12 percent was realized from the data reduction features in the storage

layer, as shown in the following figure.

Figure 6. Space savings for Oracle Database 12cR2 running on Unity 650F storage

As shown in the following figure, the CLI (sqlplus) interface reflects the schema capacity

as seen by the Oracle database. The database sees the entire 1,224.91 GB of the data

loaded in the IOPS tablespace, which validates that all the reduced physical capacity as

seen on the storage is due to the Unity array's data reduction feature and is transparent to

the database.

Use case 1:

Deduplication

and compression

of Oracle

Database 12cR2

https://kevinclosson.net/slob/




Figure 7. CLI interface (sqlplus) showing schema capacity as seen by the database

In this test, we loaded 100 percent unique data that was generated by the PL/SQL

program. Through the Unity 650F storage compression capabilities, we reduced the

amount of storage space required by 29 percent on a loaded data volume of 1,225 GB.

The actual data reduction numbers might vary, depending on the workload, quality of

data, or other factors that are unique to any typical organization.

Note: The compression numbers that are shown in this guide were generated by Dell EMC

engineers on in-house equipment and are for reference purposes only.

The compression feature of the Dell EMC Unity 650F storage array generates storage

cost savings (CAPEX) and TCO savings for a typical organization. For a similar

demonstration of storage data reduction in Oracle Database 18c, see Use Case 1:

Deduplication and compression of Oracle Database 18c in Chapter 4.

In this use case, we created a standalone Oracle Database 12cR2 on a Dell EMC

PowerEdge R840 server, as shown in the following figure. We ran the performance test

for 30 minutes using SLOB on an OLTP workload featuring an 80/20 read/write mixture.

This database features an 8 KB block size with Automatic Storage Management (ASM) in

a coarse-striped and externally redundant configuration.

Figure 8. Performance testing on 12cR2 DB running on Unity 650F (Test 1)

Use case 2:

Oracle Database

12cR2 baseline

performance



During the stress test, we collected performance data from the AWR report that was

generated by the Oracle database. For database and SLOB parameter settings used

during all test cases, see Appendix A. The following table shows the performance metrics

that we captured from AWR for Test 1. We used these values as the baseline numbers for

comparison in Use Cases 3 and 4.

Table 6. Test 1 performance results

Performance metric Value

IOPS 101,727

Database server CPU utilization (%) 25

Database bandwidth (MB/s) 805

Database response time (milliseconds) 0.32

Transactions per second (TPS) 6,869

The performance metrics in Table 6 show that the IOPS value was over 100,000 with an

average database server CPU utilization of 25 percent and a database response time of

0.32 milliseconds (ms). The database server had plenty of capacity for performing other

tasks while running this Oracle Database 12c.

The database bandwidth was healthy (805 MB/s) and the average response time for

queries was quite fast at 0.32 ms. Also, the database performed 6,869 TPS, which means

the commits and rollbacks were happening very quickly. We use the Test 1 results as a

baseline to later compare these results to those that we obtained while creating snapshots

in Use Case 3.

This use case shows the performance efficiency of Oracle Database 12cR2 on the Unity

650F storage array. In Chapter 4, we compare the results in Table 6 to the results we

obtained when running the upgraded Oracle Database 18cR1 on the Unity 650F array.

The goal of this use case is to study the performance impact of creating Unity snapshots

on Oracle Database 12cR2. This use case involves two tests—Test 2 and Test 3:

In Test 2, we performed a 30-minute stress test using SLOB and, at the same time,

created snapshots to measure performance impact, similar to the testing in Use

Case 2.

In Test 3, we created two snapshots and then performed SLOB stress testing to

observe and understand the resulting change in performance numbers.

Note: Before taking the snapshots, we created a consistency group on the Unity storage array and

added all Oracle database volumes to it. Dell EMC recommends taking snapshots of database

volumes at the consistency group level rather than at the individual database-volume level to

guarantee that the Oracle database snapshots can mount and restart successfully on the

database host.

Use case 3:

Unity storage

snapshot-creation

impact on Oracle

Database 12cR2

performance




The following figures show the methodology for Tests 2 and 3.

Figure 9. Test 2 methodology


The goals of Tests 1, 2, and 3 were to:

Capture the baseline performance of Oracle Database 12cR2 (Test 1)

Capture the performance impact on the baseline database during its snapshot

creation (Test 2)

Capture the performance impact on the baseline database after its snapshot is

created (Test 3)



Compare the following performance metrics from the three tests:

IOPS

TPS

Database server CPU utilization (%)

DB bandwidth

DB response time

IOPS results

The following figure shows the comparison of IOPS among Test 1 (Use Case 2), and

Tests 2 and 3 (Use Case 3) on Oracle Database 12c.

Figure 11. Total IOPS for Test 1, Test 2, and Test 3

As shown in Figure 11, with the snapshots created in Test 2 and Test 3, including the

SLOB stress testing in Test 3, the IOPS dropped less than 0.7 percent in comparison to

Test 1 (the baseline test). The IOPS numbers from these three tests prove that, despite

requiring more system resources like drive I/O to handle metadata writes, this reference

architecture generates impressive IOPS numbers.




Database server CPU utilization results

The following figure shows the database server CPU utilization as captured by the AWR

report for Tests 1, 2, and 3.

Figure 12. Database server CPU utilization (%) comparison for Tests 1, 2, and 3

Creating two snapshots caused virtually no change in CPU utilization in Test 2 and Test 3.

In Test 3, which included stress testing with the SLOB tool after creating snapshots, CPU

utilization decreased very slightly as compared to Test 2. Tests 2 and 3 prove that

snapshot creation and stress testing do not have a major impact on the Oracle Database

12cR2 server running on the Unity 650F storage array.

Any data that is written to either the baseline database or to the snapshot database is

redirected to a new write location in the same storage pool. The Unity storage array uses

metadata to track data blocks belonging to the base objects, and snapshots of metadata

consume more storage system resources such as CPU and memory to handle metadata

updates. Even considering the metadata updates, there was no increase in database CPU

utilization (Test 3) as compared to Test 1 (the baseline test).



Database bandwidth results

The following figure compares database bandwidth in terms of MB/s for Oracle Database

12cR2 for Tests 1, 2, and 3.

Figure 13. Comparative analysis of database bandwidth for Test 1, Test 2, and Test 3

Figure 13 demonstrates that creating two snapshots and performing OLTP operations

such as SLOB data loading do not adversely impact the bandwidth. Therefore, there is no

impact on the performance of Oracle Database 12cR2 running on the Unity 650F storage

array. On the contrary, the database bandwidth increased by 3 to 4 percent during

performance stress testing and snapshot creation.




Database response time results

The following figure shows that the database response remained the same while

snapshots were created during stress testing (Test 2) and when snapshots were created

before the stress testing (Test 3) when compared with the baseline number (Test 1).

Therefore, creating snapshots had no impact on the latency performance of the baseline

Oracle Database 12cR2 running on the Unity 650F storage array, which is notable

performance considering that there was also virtually no increase in the CPU utilization.

Figure 14. Database response time during Test 1, Test 2, and Test 3



Transactions per second results

Transactions per second (TPS) is also known as transaction throughput. The following figure

shows the TPS results in Tests 1, 2, and 3. The figure shows a minimal drop in TPS of less

than 1.3 percent in Test 2 and Test 3 compared with the baseline number from Test 1.

Figure 15. TPS during Test 1, Test 2, and Test 3

The performance metrics from Tests 1, 2, and 3 show that there was minimal impact from

creating snapshots and applying stress testing, and there was no performance impact

when running an Oracle Database 12cR2 on the Unity 650F storage array. This ability to

maintain performance is helpful when you have to create multiple copies of the production

database while the production OLTP workloads are running in parallel.




Test methodology

To further determine the storage capacity savings that is realized through the use of the

deduplication and compression feature of the Unity 650F storage array during a data

change, we increased the data that was loaded during Use Case 1 by 5 percent and

captured the savings in storage capacity. The following diagram shows the comparison of

storage capacity before and after the 5 percent data increase.

Figure 16. Data reduction achieved in 12c database after 5% data insertion

Test results

We performed Use Case 4 to find the data reduction percentage after inserting data in six

new SLOB schemas with 100 percent randomized and unique data generated by a

PL/SQL program. After loading the data in six new schemas, we achieved a space

savings of 26.15 percent, as shown in Figure 16. As described in Chapter 4, we achieved

similar savings by running the same tests on an Oracle Database 18cR1.

Use Case 4:

Deduplication

and compression

of Oracle

Database 12cR2

with data changes



Chapter 4 Oracle Database 18cR1 Performance on Unity 650F


Test objectives .................................................................................................. 33

Upgrading Oracle Database 12cR2 to 18cR1 .................................................. 34

Use cases, test methods, and test results ...................................................... 36




Test objectives

The preceding chapter describes the performance and capacity saving studies of Oracle

Database 12cR2 on the Unity 650F storage array. This chapter focuses on similar studies

of Oracle Database 18cR1. We upgraded Oracle Database 12cR2 to 18cR1, conducted

the same performance and capacity-saving studies, and compared the results. Because

we upgraded the previously used Oracle Database 12cR2, both the 12c and 18c test

databases shared the same data and the same configuration of the entire stack, from the

storage to the ESXi host/guest VM operating system and the Oracle database. Through

the same set of use cases as described in Chapter 3, we can compare the performance

and capacity savings on the Unity 650F storage array before and after upgrading the 12c

database to 18c.

To achieve these objectives, we first upgraded the 12cR2 database to 18cR1 as shown in

the following figure.

Figure 17. Upgrading Oracle Database 12c to 18c



Upgrading Oracle Database 12cR2 to 18cR1

We upgraded the database by using Oracle Data Pump technology and the upgradable

tablespace method. Upgrading from Oracle Database 12cR2 to 18cR1 involves the

following steps:

1. Upgrade the GI from 12cR2 to 18cR1.

2. Upgrade the Oracle Database from 12cR2 to 18cR1.

This section provides a high-level overview of the process. For detailed information about

the Oracle 18cR1 upgrade, see the following Oracle documentation:

Oracle Database Upgrade Guide 18c

Oracle Support MOS note (Doc ID 2418576.1): Oracle 18c - Complete Checklist for

Upgrading to Oracle Database 18c (18.x) using DBUA

To upgrade Oracle 12cR2 GI to 18cR1:

1. Stage the software.

Download the 18c GI binary LINUX.X64_180000_grid_home.zip and unzip

the files to the new 18c GI home: /u01/app/18.3.0/grid

2. Ensure that the prerequisites for upgrading to Oracle 18c GI are met.

Check the version and status of the current clusterware with crsctl commands

and run the clusterware verification utility runcluvfy.sh as shown in this

command:

$ /u01/app/18.3.0/grid/runcluvfy.sh stage -pre hacfg

3. Apply the pre-upgrade GI 27006180 patch on the Oracle 12cR2 (12.2.0.1.0) GI

home (see Doc ID 2414935.1).

Download and unzip this patch (p27006180_122010_Linux-x86-64.zip) to

the /home/grid/patches directory, and then use the opatchauto utility to

apply the patch to 12.2.1.0 GI home as the grid user:

[grid@]$/u01/app/12.2.0/grid/OPatch/opatchauto apply

/home/grid/patches/27006180

–h /u01/app/12.2.0/grid

4. Run the following command to validate that the GI patch 27006180 is applied

successfully on the 12cR2 GI home:

$/u01/app/12.2.0/grid/OPatch/opatch lsinventory

5. Before upgrading the 12cR2 GI, back up the clusterware configuration including

OCR.

6. Shut down the 12cR2 database and run the Oracle GI 18c installer setup.sh,

selecting upgrade the Oracle Grid Infrastructure to upgrade the 12cR2 GI to

18c.

Upgrade 12cR2

Grid Infrastructure

to 18cR1

https://docs.oracle.com/cd/B19306_01/server.102/b14215/dp_overview.htm

https://docs.oracle.com/en/database/oracle/oracle-database/18/upgrd/index.html

https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=397911965322703&id=2418576.1&_adf.ctrl-state=16uj51652w_122






To upgrade Oracle Database 12cR2 to 18cR1:

1. Install Oracle Database 18c software.

Download and unzip the Oracle Database 18c software

LINUX.X64_180000_db_home.zip, and then run the installer to install the

software to the new 18c Oracle home:

/u01/app/oracle/product/18.3.0/dbhome_1

During the installation, select the Set up Software Only configuration option.

Also, during database installation, select the Single Instance database

installation option and select Enterprise Edition for the database edition.

2. Restart the 12cR2 database, and then run the pre-upgrade information tool

(preupgrade.jar) command:

java -jar

/u01/app/oracle/product/18.3.0/dbhome_1/rdbms/admin/preupgra

de.jar TERMINAL

This command checks the current Oracle Database 12cR2 and identifies any

required pre-upgrade actions. The output of this command includes the pre-

upgrade actions and post-upgrade actions.

3. Perform the 18cR1 upgrade with the dbua upgrade utility from the 18c Oracle

Database home page.

The dbua upgrade utility prompts the database to upgrade. To speed up the

upgrade process, select Enable parallel upgrade and Recompile invalid

objects during post upgrade. Once the upgrade is complete, upgrade results

are displayed, as shown in the following figure.

Figure 18. Upgrade results

Upgrade Oracle

Database 12cR2

to 18cR1



Use cases, test methods, and test results

The following use cases for Oracle Database 18cR1, which mirror our 12cR2 use cases,

demonstrate the performance and capacity savings of Oracle Database 18cR1 running on

the Unity 650F storage array, as well as the performance impact of creating Unity

snapshots of Oracle Database 18cR1:

Use case 1: Deduplication and compression of Oracle Database 18cR1

Use case 2: Oracle Database 18cR1 baseline performance

Use case 3: Unity storage snapshot-creation impact on Oracle Database 18cR1

performance

Use case 4: Deduplication and compression of Oracle Database 18cR1 with data

changes

To establish the comparison with the 12cR2 database, we used the same test methods

and test configuration and ensured that the 18cR1 database contained the same data as

the 12cR2 database.

To establish the comparison of the deduplication and compression (data reduction)

savings between the 12c database and the 18c database, we reloaded the 1.2 TB of test

data that was used for Test 1 of the 12c database, as described in Chapter 3. We

observed that the size of the database was 1,223 GB, as shown in the following figure.

We inserted 100 percent randomized and unique data (generated by the PL/SQL

program) stored in 128 SLOB schemas. These schemas are initially created by a SLOB

data load and then the data from those schemas is truncated and repopulated with

randomized data. As shown in the following figure, we achieved a data reduction rate of

29.75 percent.

Figure 19. Data reduction in 18c database

Use Case 1:

Deduplication

and compression

of Oracle

Database 18cR1




When we compare the space savings of Oracle Database 12cR2 with Oracle Database

18cR1, we observe a slight improvement, as shown in the following figure.

Figure 20. Data reduction observed between 12c and 18c databases for Use Case 1

In Use Case 2, we ran the performance test for 30 minutes using SLOB to generate an

OLTP workload with an 80/20 read/write mixture on the PowerEdge R840 server, as

shown in the following figure. We used the same database configuration for the 18cR1

database as we used for the 12cR2 database.

Figure 21. Performance testing on 18cR1 database running on Unity 650F (Test 1)

Use case 2:

Oracle Database

18cR1 baseline

performance



For database and SLOB parameter settings that were used during all test cases, see

Appendix A. During the stress test, we collected performance data from the AWR report

generated by the Oracle database. The following table shows the performance metrics

that we captured from AWR for Test 1. We used these values as the baseline numbers for

comparison with Test 2 and Test 3 in Use Case 3.

Table 7. Test 1 performance results

Performance metric Value

IOPS 104,067

Database server CPU utilization (%) 26

Database bandwidth (MB/s) 823

Database response time (milliseconds) 0.31

Transactions per second (TPS) 7,023

The performance metrics in Table 7 show that the IOPS value was over 100,000 with an

average database server CPU utilization of 26 percent and a database response time of

0.31 milliseconds (ms). The database server had plenty of capacity for performing other

tasks while running this Oracle 18c database.

The database bandwidth was healthy (823 MB/s) and the response time for queries was

quite fast at 0.31 ms. Also, the database performed 7,023 TPS, which means that the

commits and rollbacks were happening very quickly. We use the Test 1 results as a

baseline to later compare these results to those that we obtained while creating snapshots

in Use Case 3.

The goal of Use Case 3 is to study the performance impact of creating Unity snapshots on

the Oracle 18cR1 OLTP database. This use case involves two tests—Test 2 and Test 3:

In Test 2, we performed a 30-minute stress test using SLOB and, at the same time,

created snapshots to measure performance impact, similar to the testing in Use

Case 2.

In Test 3, we created two snapshots and then performed SLOB stress testing to

observe and understand the resulting change in performance numbers.

Note: Before taking the snapshots, we created a consistency group on the Unity storage array and

added all Oracle database volumes to it. Dell EMC recommends taking snapshots of database

volumes at the consistency group level rather than at the individual database-volume level to

guarantee that the Oracle database snapshots can mount and restart successfully on the

database host.

The following figures show the methodology for Test 2 and Test 3.

Use case 3:

Unity storage

snapshot-creation

impact on Oracle

Database 18cR1

performance






As shown in Figure 24, with the snapshots created in Test 2 and Test 3 including the

SLOB stress testing in Test 3, the IOPS dropped less than 0.7 percent in comparison to




Tests 2 and 3 let us study the performance impact after running stress testing on an

Oracle Database 18cR1 with or without snapshots by comparing the following benchmark

parameters:

IOPS

TPS



Database server CPU utilization (%)

DB bandwidth

DB response time

IOPS results

The following figure shows the comparison of IOPS among Test 1 (Use Case 2) and Tests

2 and 3 (Use Case 3) on Oracle Database 18cR1.

Figure 24. Total IOPS numbers for Test 1, Test 2, and Test 3

As shown in Figure 24, with the snapshots created in Test 2 and Test 3, including the

SLOB stress testing in Test 3, the IOPS dropped less than 1 percent in comparison to




The following figure compares the number of IOPS generated by the 12c and 18c

databases during the three tests.




Figure 25. Comparative analysis of total IOPS numbers for Test 1, Test 2, and Test 3

As shown in Figure 25, all three tests generated slightly more IOPS with the 18c database

than with the 12c database. In Tests 2 and 3, despite snapshots being created during

stress testing (Test 2) and before stress testing (Test 3), the reduction in the number of

IOPS is minimal. Also, the IOPS numbers achieved by the 18c database are consistently

higher than those of the 12c database.

Database server CPU utilization results

The following figure shows the database server CPU utilization as captured by the AWR

report for Tests 1, 2, and 3.

Figure 26. Database server CPU utilization (%) comparison for Tests 1, 2, and 3

Creating two snapshots in Test 2 caused database server CPU utilization to increase very

little in Test 2 and Test 3. In Test 3, which included stress testing with the SLOB tool after



creation of snapshots, CPU utilization actually decreased by 1 percent in comparison to

Test 2. Tests 2 and 3 prove that snapshot creation and stress testing have very little

impact on the Oracle Database 18cR1 server running on the Unity 650F storage array.

Any data written to either the baseline database or to the snapshot database is redirected

to a new write location in the same storage pool. The Unity storage array uses metadata

to track data blocks that belong to the base objects, and all snapshots consume more

storage system resources such as CPU and memory to handle metadata updates. Even

considering the metadata updates, there was no significant increase in the database

server CPU utilization percentage. Therefore, there is no significant performance impact

on Test 2 and Test 3 in comparison to Test 1 (the baseline test).

Comparing database server CPU utilization during the tests running on the Oracle 12c

database with those on the 18cR1 database, we find that database server CPU utilization

decreases with the 18cR1 database, as shown in the following figure.

Figure 27. Comparative analysis of database server CPU utilization (%) during Test 1, Test 2 and Test 3

This leaves plenty of unused CPU resources available for other activities.

Database bandwidth results

The following figure compares database bandwidth in terms of MB/s for Oracle Database

18cR1 for Tests 1, 2, and 3.




Figure 28. Comparative analysis of DB bandwidth during Test 1, Test 2, and Test 3

Figure 28 demonstrates that creating two snapshots and performing OLTP operations

such as SLOB data loading do not adversely affect the bandwidth, so there is no impact

on the performance of the Oracle Database 18cR1 running on the Unity 650F storage

array. On the contrary, the database performance (bandwidth) increases by about 2

percent as more workloads are applied along with the creation of snapshots during Test 2.

The following figure compares the bandwidth results for the Oracle 12cR2 and 18cR1

databases during the three tests.

Figure 29. Comparative analysis of database bandwidth during Test 1, Test 2, and Test 3

As shown, the bandwidth numbers that were recorded during 12c and 18c database

testing remained the same or improved slightly despite snapshots being created during



stress testing (Test 2) and before stress testing (Test 3) compared with the benchmarking

test (Test 1).

Database response time results

The following figure shows database performance (response time) as measured in Tests

1, 2, and 3. As shown, the database maintained the same response level during the

creation of snapshots during stress testing (Test 2) and before the stress testing (Test 3)

when compared with the baseline number (Test 1). Creating snapshots and doing stress

testing did not significantly impact the database response time and had no impact on the

latency performance of Oracle Database 18cR1 running on the Unity 650F storage array.

Figure 30. Database response time during Test 1, Test 2, and Test 3

The following figure compares the 12cR2 and 18cR1 database response times during our

testing.




Figure 31. Comparative analysis of database response time during Test 1, Test 2, and Test 3

As shown in Figure 31, the database response time during testing of the 12c and 18c

databases remained nearly the same despite snapshots being created during stress

testing (Test 2) and before stress testing (Test 3) compared with the benchmarking test

(Test 1).

Transactions per second results

The following figure shows the transaction throughput that was achieved in Tests 1, 2, and 3

during the testing of the Oracle Database 18c. The figure shows a minimal drop in TPS of less

than 1 percent in Test 2 and Test 3 compared with the baseline number from Test 1.

Figure 32. TPS during Test 1, Test 2, and Test 3



The following figure compares the 12c and 18c database TPS results.

Figure 33. Comparative analysis of TPS during Test 1, Test 2, and Test 3

A comparison of the 12c and 18c database test results shows a minimal reduction of 2

percent TPS from the creation of two snapshots. Thus, we can conclude that Oracle 12c

and 18c databases running on this reference architecture maintain stable transaction

throughput.

The performance metrics from Tests 1, 2, and 3 show that there was minimal impact from

creating snapshots and applying stress testing, and there was no performance impact

when running an Oracle Database 18cR1 on the Unity 650F storage array. This capability

of minimizing impact on the server performance is helpful when you have to create

multiple copies of the production database for nonproduction purposes.

Test methodology

To further determine the storage capacity savings that is realized through the use of the

deduplication and compression feature of the Unity 650F storage array during a data

change, we increased the data that was loaded during Use Case 1 by 5 percent and

captured the savings in storage capacity. The following diagram shows the comparison of

storage capacity before and after the 5 percent data increase.

Use case 4:

Deduplication

and compression

of Oracle

Database 18cR1

with data changes




Figure 34. Data reduction achieved in 18c database after 5% data insertion

The figure above shows that Unity’s data reduction capabilities yield a space savings of

25.97 percent inside the Unity storage array after adding five percent new data. This

space savings helps the customer to consolidate their data which reduces the cost of

storage and the total cost of ownership. (TCO).



Test Results

We performed the testing in Use Case 4 to find the data reduction percentage after

inserting data in six new SLOB schemas with 100% randomized and unique data

generated by a PL/SQL program. After loading the data in six new schemas, we achieved

a space savings of 25.97 percent, as shown in Figure 34. We achieved similar savings by

running the same tests on an Oracle Database 12c.

The following figure compares the data compression rates that were achieved during our

testing with the 12cR2 and 18cR1 databases.

Figure 35. Comparative analysis of data reduction (%) for Use Case 4 between 12c and 18c databases

Figure 35 shows that the data reduction percentage between Oracle Database 12cR2 and

18cR1 on the Unity 650F storage array is quite similar.

Conclusion

The tests described in Chapters 3 and 4 prove that Dell EMC Ready Solutions for Oracle

designed using the Unity 650F storage array and other Dell EMC hardware for networking

and servers creates a reliable reference architecture that will support the upgrade from

Oracle Database 12cR2 to Oracle Database 18cR1. Snapshot creation results in very little

impact to the performance of these databases. This solution exhibits a simplified upgrade

from 12c R2 to 18cR1. Customers can enjoy the many advantages of Dell EMC product

portfolios along with the advanced features of the Oracle Database 18cR1.

Chapter 5: Summary



Chapter 5 Summary


Summary ........................................................................................................... 50

Chapter 5: Summary


Summary

This Ready Solutions for Oracle Design for Unity All-Flash Unified Storage provides

simple, easy-to-use management by allowing storage administrators to provision storage

with very little setup or planning. This architecture provides predictable and consistent

performance and data savings for every environment in a typical enterprise application.

This validated and tested reference architecture enables the upgrading of Oracle

Database 12cR2 to 18cR1 without performance degradation, providing faster time-to-

value, operational agility, and efficiency.

An examination of AWR reports of the 18cR1 database and the 12cR2 database from

which it was upgraded reveals that there was minimal performance impact after adding

snapshots and running stress testing. Further, the deduplication and compression

features of the Unity 650 All Flash storage array provided significant storage capacity

savings and reduced TCO.

In this reference architecture guide, we have demonstrated the ease of upgrading the

Oracle Database 12cR2 to the Oracle Database 18cR1. The IOPS numbers exceeded

100,000 in all the use cases, with better results in the testing of the 18c database. We

noted a consistent pattern in database server CPU utilization, which ranged between 26

and 28 percent for all the use cases. Database response time was consistently at

approximately 0.31 ms. TPS and bandwidth were improved in the 18cR1 database test

results as compared to the 12cR2 database test results. Finally, running either Oracle

Database 12cR2 or 18cR1 on the Unity 650F array provided notable data compression

rates.

If customers elect to upgrade their Oracle Database 12cR2 to Oracle Database 18cR1 on

Dell EMC products like the PowerEdge Server R840, Unity 650F All-Flash Storage, and

Dell EMC networking products, there will be no adverse impact on database performance,

and performance may actually improve. In summary, here are the key results from our

testing of this Ready Solutions for Oracle:

Support for streamlined upgrades from Oracle Database 12cR2 to 18cR1 with no

performance degradation, based on performance metrics that were extracted from

the Oracle AWR report including:

Total IOPS

TPS

Database server CPU utilization

Database bandwidth

Database response time

Minimal or no performance degradation in 12cR2 and 18cR1 databases when

database snapshots were added during SLOB stress testing or before SLOB stress

testing. Thus, multiple copies of the production database can be created without

significant impact, even while the production database is performing its OLTP tasks.

Data compression rates of more than 25 percent after the insertion of unique data

into both 12cR2 and 18cR1 databases. Favorable data compression results were

Chapter 5: Summary



also observed when 5 percent new data was added to the existing data of both

12cR2and 18cR1 databases.

Chapter 6: References


Chapter 6 References


Dell EMC documentation .................................................................................. 53

VMware documentation .................................................................................... 53

Oracle documentation ...................................................................................... 53

SLOB documentation ....................................................................................... 53

Chapter 6: References



Dell EMC documentation

The following Dell EMC documentation provides additional and relevant information.

Access to these documents depends on your login credentials. If you do not have access

to a document, contact your Dell EMC representative.

Dell EMC Unity 650F All-Flash Storage

Dell EMC Unity Storage with Oracle Databases

How to deploy Oracle 12c Release 2 Standalone Database on RHEL 7.x

Dell EMC Ready Solutions for Oracle

For additional information about Dell EMC Ready Solutions for Oracle, see the Oracle Info

Hub for Ready Solutions on the Dell EMC Community Network.

VMware documentation

The following VMware documentation provides additional and relevant information:

VMware vSphere 6.5, ESXi 6.5, vCenter Server 6.5 Installation and setup

Oracle Databases on VMware Best Practices Guide

Dell EMC Host Connectivity Guide for VMware ESX Server

Oracle documentation

The following Oracle documentation provides additional and relevant information:

Installing Oracle 18c Standalone Database

Upgrading Oracle Grid Infrastructure from Oracle 12c to Oracle 18c

SLOB documentation

The following SLOB documentation provides additional and relevant information:

SLOB Resources

(Blog) EMC Unity Storage Performance testing with Oracle ASM and SLOB

https://shop.dellemc.com/en-us/Product-Family/Unity-Products/Dell-EMC-Unity-650F-All-Flash-Storage/p/EMC-Unity-600F-All-Flash-Storage




https://www.dellemc.com/en-us/solutions/business-applications/oracle/index.htm



https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/oracle/oracle-databases-vmware-best-practices-guide.pdf

https://www.emc.com/collateral/TechnicalDocument/docu5265.pdf

ttps://docs.oracle.com/en/database/oracle/oracle-database/18/cwlin/changes-in-this-release-for-oracle-grid-infrastructure-installation-guide.html#GUID-7E17356F-9B6E-4357-8755-B3A99A984ED4

https://docs.oracle.com/en/database/oracle/oracle-database/18/cwlin/upgrading-oracle-grid-infrastructure.html#GUID-DF76F201-3374-486F-9D19-06276764569F

https://kevinclosson.net/slob/

https://gruffdba.wordpress.com/2017/03/04/testing-emc-unity-storage-performance-with-slob/

Appendix A: Test Tools, and Database and SLOB Configuration


Appendix A Test Tools, and Database and SLOB Configuration

This appendix presents the following topics:

Testing and performance collection tools ....................................................... 55

SLOB dataset customization ............................................................................ 55

Database configuration .................................................................................... 56

SLOB parameter settings (slob.conf) .............................................................. 58




Testing and performance collection tools

To simulate Oracle database workloads, we used SLOB version 2.4, which is a simple Oracle

database schema generation and I/O testing toolkit. We used the SLOB multiple schema model

to generate a 1.2 TB dataset (128 schemas x 9,600 MB each). We simulated an OLTP workload

by configuring SLOB to generate a distribution of 80 percent reads and 20 percent write I/Os.

For details on the SLOB parameter settings, see SLOB parameter settings (slob.conf).

The default schema that is generated by the SLOB tool contains a lot of redundant and

repetitive data and, therefore, is highly compressible. To get more realistic results,

especially for the deduplication and compression study, we modified the original SLOB

dataset and replaced it with a unique dataset using a customized PL/SQL script that uses

Oracle’s dbms_random.string function. For details on the custom script, see SLOB

dataset customization.

We collected the database performance data through the Oracle AWR report. During each

test, we configured AWR to capture data every 3 minutes. From the report, we extracted read,

write, and total IOPS; average I/O latency (in ms); and CPU utilization percentage metrics.

SLOB dataset customization

We replaced the default dataset that was loaded by SLOB with a custom and unique

dataset using the following PL/SQL script:

/* Truncate/remove original rows in tables for all users/schemas

(128 schemas in our case) */

for i in {1..128}

do

sqlplus / as sysdba << EOF

truncate table user${i}.cf1;

exit;

EOF

done

/* Start code for inserting random/unique data */

for i in {1..128}

do

sqlplus / as sysdba <<EOF

declare

type cf_table is table of user1.cf1%rowtype

index by binary_integer;

cfa cf_table;

begin

for i in 1..2450000 loop -- Loop through each row to generate

random and unique data for each column

cfa(i).custid:=i;

cfa(i).c2:=dbms_random.string('p',128);


SLOB workload

testing toolkit

Oracle Automatic

Workload

Repository report




















end loop;

for all i in 1..cfa.count

insert into user${i}.cf1 values cfa(i); -- Insert the above newly

generated random+unique data into each column of each row

end;

/

exit;

EOF

done

Database parameter configuration

The following table provides the database configuration and parameter settings used to

conduct all Oracle Database 12cR2 tests:

Table 8. Oracle 12cR2 standalone database configuration

Category Specification/setting Configuration

Operating system VM guest OS RHEL 7.4

Grid configuration Grid version 12c R2 Grid Infrastructure

Database configuration Database version 12c R2 Standalone Database

Database size 1.2 TB

Database parameter configuration

db_block_size 8 KB

db_file_multiblock_read_count 4

sga_max_size 96 GB

pga_aggregate_target 25 GB

12cR2

configuration




The following table provides the database configuration and parameter settings used to

conduct all Oracle Database 18cR1 tests:

Table 9. Oracle 18cR1 standalone database configuration

Category Specification/setting Configuration

Operating system VM guest OS RHEL 7.4

Grid configuration Grid version 18c R1 Grid Infrastructure

Database configuration Database version 18c R1 Standalone

Database

Database size 1.2 TB

Database parameter settings

db_block_size 8 KB

db_file_multiblock_read_count 4

sga_max_size 96 GB

pga_aggregate_target 25 GB

Note: 18cR1 Grid and Standalone Database were upgraded from existing 12cR2 Grid and

Database, respectively.

18cR1

configuration



SLOB parameter settings (slob.conf)

The following table provides the slob parameter settings we used to conduct all Oracle

Database 12cR2 and 18cR1 tests:

Table 10. Slob parameter settings

Parameter Setting

UPDATE_PCT 20

SCAN_PCT 0

RUN_TIME 1800

WORK_LOOP 0

SCALE 9600M

SCAN_TABLE_SZ 1M

WORK_UNIT 6

REDO_STRESS LITE

LOAD_PARALLEL_DEGREE 1

THREADS_PER_SCHEMA 1

Appendix B: Equipment List



Appendix B Equipment List

This appendix presents the following topics:

Hardware components...................................................................................... 60

Software components ....................................................................................... 62



Hardware components

The reference architecture includes the following primary hardware components:

1 x Dell EMC 4S PowerEdge R840 server

2 x Dell EMC Networking 10 GbE switches

2 x Dell EMC Connectrix 16 Gb/s FC switches

1 x Dell EMC Networking 1 GbE switch

1 x Dell EMC 2S PowerEdge R640 server

Unity 650F storage array

The following tables list the details of the hardware, firmware, and driver components of

the ESXi host server that we used in the tested configuration of this reference

architecture.

Note: Newer and updated BIOS and firmware versions are supported, if available. For the latest

version, go to Dell EMC Online Support.

Table 11. ESXi host server components

Component Description

Server 1 x Dell EMC PowerEdge R840

Chassis2 24 x 2.5 in. chassis with up to 12 SAS/SATA bays + 12 NVME/SAS/SATA bays

Processor 4 x Intel Xeon Gold 6154 18c 3.0 GHz

Memory 1,536 GB (24 x 64 GB QR DDR4 2,666 MT/s LRDIMMs)

Local disks in server 3 x 1.2 TB 10 K SAS 12 Gb/s 2.5 in. HDDs (includes one hot spare)

RAID controller PERC H740P adapter

iDRAC iDRAC9 Enterprise

rNDC or LOM Broadcom 5720 2P 1 Gb Base-T + 2P 10G SFP rNDC

Add-on NICs 2 x Intel 10 GbE 2P X710 adapter

HBAs 2 x QLogic QLE2692 DP 16 Gb/s FC HBAs

Power supplies 2 x 2,000 W

2 Other chassis configurations are supported.

ESXi host server

https://support.emc.com/




Table 12. PowerEdge R840 component firmware and drivers

Component Firmware Driver3

BIOS 1.1.1 N/A

Lifecycle Controller and iDRAC9 Enterprise

3.21.21.21 N/A

QLogic QLE2692 16 Gb/s FC HBAs

09.00.06 2.1.50.0 (qlnativefc)

Intel X710 DP 10 GbE SFP+ adapters

20.06.04.03; 20.6.34.0-1OEM.650.0.0.4598673

2.0.6 (i40en)

Broadcom 5720 2P 1 Gb Base-T + BCM57412 2P 10G SFP rNDC

20.6.16 (BCM5720); 20.06.04.03 (BCM57412)

20.6.34.0 (bnxtnet – 10 GbE)

Dell EMC PERC H740P adapter

50.0.3-0962 7.700.50.00 (lsi-mr3)

Delta 2,000 W power supplies

00.1C.53 N/A

The following table lists the hardware details of the storage array that we used in the

tested configuration of this reference architecture.

Table 13. Unity 650F storage array details

Storage array component Details

Operating system version 4.4.0.1534750794

Number of disk processor enclosures (DPEs)

1

Number of storage processor (SP) controllers

2

Number and type of front-end FC ports 4 x 16 Gb/s SFP+ FC ports per SP

Size per SSD 400 GB

Usable capacity 48.6 TB

RAID type RAID 5

3 Dell EMC customized ISO image (Dell Version: A04, Build# 5310538) was used to deploy the

ESXi hypervisor. Inbox drivers in RHEL 7.4 were used within the guest operating system.

Storage array



In this reference architecture for Oracle, we recommend that you use a dedicated

management server to install VCSA, Oracle Enterprise Manager, Dell EMC OpenManage

Essentials, or other centralized management software applications.

The following table lists the management server and its components that were used in the

tested configuration of this reference architecture.

Table 14. Management server and components

Category Components

Server 1 x Dell EMC PowerEdge R640 server

Chassis 2.5 in. with up to 8 HDDs and 3 PCIe slots

Processor 2 x Intel Xeon Gold 6136 12c 3 GHz

Memory 192 GB (12 x 16 GB DR DDR4 2,667 MT/s RDIMMs)

Local disks 3 x 1.2 TB 10k rpm SAS 12 Gb/s 2.5 in. HDDs (includes 1 hot spare)

RAID controller Dell EMC PERC H740P/H730P

iDRAC Dell EMC iDRAC9 Enterprise

rNDC Broadcom 5720 2P 1 Gb Base-T + BCM57412 2P 10G SFP rNDC

Software components

The following table specifies the versions of the software components of this reference

architecture as deployed in the tested configuration.

Table 15. Software versions

Software Version

VMware ESXi 6.5 [Dell EMC customized ISO image (Dell Version: A04, Build# 5310538)]

VMware vCenter Server Appliance 6.5.0.5500 Build Number 5318154

RHEL (guest operating system) 7.4, kernel version 3.10.0-693.el7.x86_64

Management

server

Dell EMC Ready Solutions for Oracle: Design for Dell EMC ... · 8 Dell EMC Ready Solutions for...

Documents

Transcript of Dell EMC Ready Solutions for Oracle: Design for Dell EMC ... · 8 Dell EMC Ready Solutions for...