Copyright 2015, Oracle and/or its affiliates. All rights reserved. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. 0615

Oracle Real Application Clusters (RAC) Support on Third-Party Clouds An Overview and Clarification

One of the key characteristics of cloud deployments is simplification. Based on this

premise, many of Oracles customers have moved or are considering moving into

the cloud. The Oracle Cloud provides a rich choice of application suites as well as

best-of-breed solutions as part of its strong Software-as-a-Service (SaaS) and

Platform-as-a-Service (PaaS) offerings. In addition, Oracle actively works with

certain Third-Party Cloud vendors to provide an integrated service for the Oracle

Database on their platforms. Oracle also maintains an open policy for Third-Party

Cloud vendors to support the Oracle Database in their infrastructure.

One of the first vendors who utilized this policy was Amazon with the Amazon

Web Services (AWS). Deployments made under this policy enable vendors to run

the Oracle Database as part of their Infrastructure-as-a-Service (IaaS) offering

as long as the infrastructure meets the installation prerequisites for the Oracle

product being offered. A vendor can choose to provide additional deployment

support for the Oracle Database or any other Oracle product in those

environments, which in general are neither tested nor certified by Oracle. Most

Third-Party Cloud vendors therefore choose to collaborate with Oracle on such

support in order to improve the quality of the service offered.

Microsoft with their Oracle in Microsoft Windows Azure1 offering was one of the

first vendors partnering with Oracle. Amazon on the other hand has recently

decided to provide Support for self-managed Oracle RAC on Amazon EC22

without Oracles collaboration.

1 http://www.oracle.com/technetwork/topics/cloud/faq-1963009.html 2 https://aws.amazon.com/about-aws/whats-new/2015/11/support-for-self-managed-oracle-rac-on-ec2/

Oracle RAC Support in the Cloud General Policy and Guidance Oracle RAC is supported on all cloud environments supported by the Oracle Database, as

long as the environment is able to provide the hardware, storage, and networking requirements as specified in the Oracle RAC and Grid Infrastructure documentation.

With the exception of the Oracle Cloud, Oracle has not tested nor certified Oracle RAC in

these environments.

This general policy regulates Oracle RAC support in any cloud environment. In addition and depending on the operating system used, either MOS Note 417770.1 - Oracle Linux Support Policies for Virtualization and Emulation or MOS Note 1563794.1 - Certified Software on Microsoft Windows Server 2012 Hyper-V needs to be considered, as these notes regulate the support of the respective Oracle Software Products in Virtual or Hyper-V based deployments. Currently, only the Oracle Cloud provides integrated Oracle RAC support via the Exadata Cloud Service and the Oracle Database Cloud Service. In light of the Oracle in Microsoft Azure1 agreement, Oracle is working with Microsoft to establish an IaaS-based configuration in order to support Oracle RAC in the Microsoft Azure Cloud.

Meeting Hardware, Storage and Networking Requirements Oracles policy for supporting the Oracle Database and Oracle RAC in the cloud is based on the assumption that the hardware, storage and networking requirements as specified in the Oracle RAC and Grid Infrastructure documentation3 are met. For Oracle RAC, this means that the cloud infrastructure needs to provide shared storage and multiple networks, including the ability to create a private, dedicated network that supports broad- and multi-casting. The documentation, however, does not specify in detail how those requirements need to be met.

With todays technology it is possible to use local or server-based storage and make it appear as shared storage (e.g. via an iSCSI emulator) to the Oracle Database. Todays technology also allows for creating the impression that multiple networks are available to the database, while de facto there is only one physical network been provisioned. Such technologies are generally discouraged, although they might be supported under Oracles policy, if transparent to (I.E. undetectable by) the stack and supported by the system as well as the hardware vendor. The main reason is that these technologies often increase complexity or negatively affect performance or availability. Oracle therefore recommends to meet hardware, storage and networking requirements natively and to avoid any additional and unnecessary layer.

In cloud environments meeting those requirements natively can be a challenge. This is why, Oracle offers to work with Third-Party Cloud vendors to select and configure the required technology that should be used to meet hardware, storage and networking requirements in the best possible way. This collaboration also allows Oracle to adapt its software as required and to accommodate for cloud-specific environments that would otherwise be disqualified for running Oracle RAC.

Oracles policy for supporting Oracle RAC in (Third-Party) Clouds allows for a variety of cloud offerings to qualify as a suitable IaaS solution to host an Oracle RAC configuration.

For even more flexibility, Oracles documentation, does not further specify how exactly those requirements need to be met.

Some technologies that are supported under Oracles policy are discouraged as they negatively affect performance and availability.

3 https://docs.oracle.com/database/121/nav/portal_16.htm

Phases of Support One of the key characteristics of cloud deployments is simplification. Based on this premise, the ideal and fully supported cloud environment includes support through all phases of the database lifecycle; from the initial deployment through operation of the database in the cloud. Most of todays cloud environments use a template-based deployment. This means that a standard deployment model (typically a virtual server or instance) is provided to the user, which is pre-configured and has all the software installed so that the Oracle Database can simply be started after the virtual server was instantiated with the required size. Using such a template to run the Oracle Database or Oracle RAC Database provides an ideal foundation for the Database service, as configuration issues are minimized as much as possible. Template-based deployments for the Oracle Database are currently utilized in the Oracle Cloud as well as the Microsoft Azure Cloud4. Once the Oracle Software is deployed and configured, Oracle will support the operation of the Oracle Database in those environments and will address issues that may be encountered.

Comparing Available Cloud Solutions

Oracle Database Cloud Service

The Oracle Database Cloud Service is not surprisingly the most integrated Oracle Database Cloud Service on the market today. Owning the infrastructure as well as the software offered as part of its PaaS offering, Oracle has designed the Oracle Database Cloud Service and the Oracle Exadata Service to provide customers with the most advanced Oracle Database service available today. Using Oracles Multitenant Database technology furthermore enables smooth transition from on-premises, private database cloud environments into the Oracle Cloud, either for migration purposes or in order to build a Hybrid Cloud environment. Using Oracle RAC on either or both ends of the hybrid cloud deployment provides scalability and high availability and thereby complements the solution.

Microsoft Azure

According to the Oracle and Microsoft Windows Azure1 agreement, Oracle and Microsoft are working together to help customers embrace cloud computing by providing greater choice and flexibility in how to deploy Oracle software. The partnership combines the world-class mission-critical business software of Oracle with the leading private and public cloud platforms of Microsoft. This means, that the Oracle and Microsoft Windows Azure is the premier example of a Third-Party Cloud integration with the Oracle Database software. This is further confirmed by the level of support that is provided for the Oracle Database on Microsoft Azure; For customers who have Oracle software licenses and support, Oracle delivers support directly to the customers running the Oracle software on Windows Server Hyper-V and Windows Azure. As additional services [e.g. an Oracle RAC service] are launched as a result of this partnership, more details will be shared on the support model.1

Customers considering running Oracle RAC in a cloud are best advised to choose an offering, in which the cloud vendor as well as Oracle have a clear and consistent understanding on how to support the Oracle Database deployment through its different phases.

The Oracle Cloud is the gold standard for deploying an Oracle Database Service in the cloud.

Oracle and Microsoft are working together to help customers embrace cloud computing by providing greater choice and flexibility in how to deploy Oracle software.

4 https://azure.microsoft.com/en-us/marketplace/partners/oracle/oracledatabase12centerprise/

Amazon Web Services (AWS)

While AWS was the first Third-Party Cloud vendor supporting the Oracle Databases in their IaaS environment, there has never been a technical integration beyond being able to run a Single Instance Oracle Database on templated AWS EC2 Cloud Instances. Amazon and Oracle established a Licensing of Oracle Software in the Cloud Computing Environment"5 but further arrangements similar to the agreement with the Microsoft Azure Cloud are outstanding. Thus, while Amazon has recently promoted the ability to run RAC on AWS (see Support for self-managed Oracle RAC on Amazon EC22), this service does not meet the level of support that Oracle strives to provide to its customers.

Reviewing Amazons main claims on deploying Oracle RAC on EC2 It is useful to take a look at Amazons main claims regarding Oracle RAC on EC2 support, as most of them either contradict or violate recommendations on how to operate and benefit from an Oracle RAC Database.

Claim 1: Oracle RAC on EC2 allows customers to leverage the elasticity and scalability of Amazon Web Services, as RAC on EC2 provides the same excellent availability

characteristics as RAC on conventionally hosted infrastructure

This is a big claim, which falls short in reality. While it is true that an Oracle RAC deployment on EC2 is possible, it is too cumbersome and limited to be useful for any real use case. In order to understand this point, it is important to recognize the two impediments on EC2 that prevent proper deployment of Oracle RAC: 1. The lack of shared storage, and 2. Missing required network capabilities.

One of the core requirements of Oracle RAC, whether in an on-premise or cloud-based deployment, is shared, concurrently accessible storage. For best reliability and scalability, this capability should be provided natively. There are a variety of alternative ways to create the illusion of shared storage being available to a cluster and some are even supported (see Oracle RAC Technologies Matrix for Linux Platforms6). However, in many real-world use cases, Oracle would discourage their use. In a cloud environment, which is meant to be easy, integrated and fully supported, shared storage is best provided as part of the platform.

Amazons EC2 block storage offering is Elastic Block Store (EBS). EBS has the limitation that a volume cannot be accessed from more than one EC2 instance at the same time (concurrently). While it is possible to attach EBS storage to more than one EC2 instance at the same time, concurrent access, which is needed for Oracle RAC, is impossible.

In order to provide shared storage to Oracle RAC in AWS, Amazon recommends using Amazon Machine Images (AMIs), which allow EC2 instances to be launched as a NAS device. Using iSCSI, the EC2 instance-based NAS devices work around the EBS limitation and act as shared storage for multiple Oracle RAC instances.

This approach not only adds an extra layer of complexity, it also becomes a liability in two ways. First of all, I/O, and hence performance, will be affected. Secondly, the NAS instance

While AWS was the first Third Party Cloud vendor supporting the Oracle Databases in their IaaS environment, there has been no technical integration beyond being able to run a Single Instance Oracle Database on templated AWS EC2 Cloud Instances.

The lack of shared storage and the missing network capabilities required for Oracle RAC will render the RAC on EC2 setup more or less useless for any real use case.

5 http://www.oracle.com/us/corporate/pricing/cloud-licensing-070579.pdf 6 http://www.oracle.com/technetwork/database/clustering/tech-generic-linux-new-086754.html

becomes a single point of failure. Amazon is aware of the I/O performance impact of this approach and therefore suggests using i2.8xlarge instances for the NAS deployment. The i2.8xlarge instances may provide the required I/O capacity, but are based on local, ephemeral solid-state disks. The keyword here is ephemeral, which means that the storage is non-persistent. Needless to say that without persistence this storage will lead to an unacceptable data loss. Amazons proposed solution is deploying two or three of these NAS instances and mirroring the storage between them using ASM Normal or High redundancy. However, should both storage instances fail or be taken down (e.g. for maintenance) at the same time, there will be a complete data loss.

Making matters worse, Amazon recommends deploying the Oracle RAC instances and storage in the same placement group in order to avoid expected network saturation and latency issues, which are again a result of Amazons infrastructure lacking support for required network capabilities.

Oracle RAC requires a private network for the inter-instance database communication and the Oracle cluster heartbeat. In order for a network to qualify as a private network for Oracle RAC, the communication on this network should be exclusively for (the) cluster(s), the network is a non-routed network and it must support multicast as well as broadcast. There are other network requirements that need to be met as documented in the Oracle Grid Infrastructure Installation Guide7, but those listed here are the most important. AWS mainly fails to provide multicasting natively.

Amazon claims to have worked around the lack of native multicasting support by using a point-to-point VPN network for the cluster utilizing ntops n2n which, according to Amazons tutorial, allows a group of hosts to join a VPN community, and also supports multicast IP among the members. While this is technically a possible solution, it comes at a high price.

As an extra layer, the n2n-based solution proliferates the likeliness of a failure, increases complexity and may negatively impact performance. Most interestingly, however, there is actually no full support for the n2n project, as stated on ntops webpage, updated Feb 10th 2014: Although we currently have no time to further develop n2n (we have put the project on hold until we have time to work at it again), this tool is still widely used.8

Consequently, it is surprising, how Amazon can provide Support for self-managed Oracle RAC on Amazon EC2 considering that a major component of the solution is de facto unsupported. Given all the workarounds being used, it is also unclear how the solution as a whole can ever be used so that a developer can have his or her own RAC database, which will allow detection of performance and availability regressions related to the RAC architecture in their software. Any availability test performed in this architecture will always be subject to uncertainty as to whether the failure encountered is a result of the RAC architecture or the architecture used to host the Oracle RAC Database in the first place.

Eventually, the entire Oracle RAC setup as proposed by Amazon is subject to the availability of the placement group. Most conventional infrastructures in use today seek to avoid such an exposure to a single point of failure. Thus, Amazons claim to provide the same excellent availability characteristics as RAC on conventionally hosted infrastructure is most likely to fail.

It surprises, how Amazon can provide Support for self-managed Oracle RAC on Amazon EC2 considering that a major component of the solution is de facto unsupported.

7 https://docs.oracle.com/database/121/CWLIN/networks.htm#CWLIN473 8 http://www.ntop.org/n2n/using-n2n-with-amazon-aws-ec2/

Claim 2: Via a simple step-by-step tutorial, anyone can deploy a scalable Oracle RAC on Amazon EC2. This is done in minutes, rather than weeks.

At first glance, providing a 14-steps tutorial9 to set up an Oracle RAC on Amazon EC2 seems promising. However, looking into each step individually reveals that these are only the high-level steps and there are a number of sub-steps all to be executed manually by the user. Claiming that such a deployment can be performed in minutes, rather than weeks, as is the case with physical on-premises infrastructure, is incorrect in many ways.

The days when on-premises deployments took weeks, even days, are over. Oracle has provided multiple means of deploying an Oracle RAC cluster in minutes in addition to simplifying the installation in general. One way of deploying Oracle RAC in virtually any environment and in basically no time are the Oracle VM Templates for Oracle Database10 which are available for Oracle Single Instance databases as well as Oracle RAC 12c Rel. 1 and 11g Rel. 2 databases. Outside of virtual environments, the Oracle Database Appliance (ODA) has enabled hundreds of customers to deploy a two-node cluster in less than a day.

The main issue with the tutorial and its series of steps is that they need to be manually executed by the user, which is error prone and lacks any of the benefits of using a cloud to provision an Oracle RAC cluster. It is merely a proof of concept that Oracle RAC can run in an AWS environment. In fact, the instructions were originally published by the author as a tutorial11. It is unclear how and why Amazon subsequently decided to announce the very same tutorial to be the foundation of Support for self-managed Oracle RAC on Amazon EC2. This was done without Oracles input and knowledge.

The expectation on a cloud environment must be that an Oracle RAC Database can be deployed either using a one-command deployment option (for example using Oracle VM Templates for Oracle Database and the deploycluster12 tool) or via a Wizard as part of a PaaS deployment, as is the case for the Oracle Cloud. At a minimum, templates should be provided that already contain the Oracle Software and a respective configuration so that all the user needs to do is to create a database, as is the case for the Oracle Database deployed in Microsoft Azure (this feature is currently not supported for Oracle RAC databases). The times of providing a cookbook, which is what Amazons tutorial represents, to install a production or even test Oracle RAC environment, especially in the cloud, are as much over as are the times when on-premises deployments took weeks.

Claim 3: RAC on AWS allows developers to do their development and testing against their own Oracle RAC database, which allows a quick detection of performance and availability

regressions in their software related to Oracle RAC.

The reality is that the deployment of Oracle RAC on EC2 uses so many workarounds, which will impact the performance of the Oracle Database, that any performance or destructive testing performed on this platform is de facto useless.

The days when on-premises deployments took weeks, even days, are over.

The expectation for a cloud deployment must be that an Oracle (RAC) Database can be deployed either using a one-command deployment option or using a Wizard as part of a PaaS deployment.

9 https://aws.amazon.com/articles/7455908317389540 10 http://www.oracle.com/technetwork/server-storage/vm/database-templates-12c-11gr2-1972804.html 11 http://www.slideshare.net/AmazonWebServices/dat303-oracle-on-aws-and-amazon-rds-secure-fast-and-scalable 12 https://blogs.oracle.com/wim/entry/oracle_vm_rac_template_what

The following statement made in Amazons Deploying Scalable Oracle RAC on Amazon EC29 tutorial should be considered carefully: For customers using RAC on conventionally-hosted infrastructure, EC2 can still provide a compelling use case. These customers can test the extent to which their workload will scale on RAC, by determining the number of nodes at which their workload reaches a point of diminishing returns, and begins to scale inversely.

This statement either attests a conceptual misunderstanding of the Oracle RAC technology or if this statement was made based on a performance test conducted in this environment confirms the suspicion that the proposed solution for Oracle RAC in EC2 does not scale because of infrastructural shortcomings.

Oracle RAC is used by thousands of customers and is designed to scale real applications, as the name suggests (Oracle Real Application Clusters). There are, however, some applications that may not immediately benefit from the horizontal scaling that Oracle RAC provides. Most of these applications typically do not scale vertically, either, and Oracle has provided guidance on how to optimize an application should it not meet the expected scalability on an Oracle RAC database13.

In order to test an application for scalability on Oracle RAC most customers, however, do not need more than three to four nodes, as it is a design principle of Oracle RAC that the scalability factor does not decrease as more nodes are added to the cluster assuming the infrastructure is able to sustain the increased load14. Testing an application on three to four nodes therefore typically provides a good indication about the scalability of the application on Oracle RAC in general. That means that Amazons use case proposal for RAC on EC2 either ignores this design principle or assuming this statement was made based on a performance test conducted in this environment confirms that the infrastructure provided for Oracle RAC in the EC2 does not scale.

Summary Oracles policy for supporting Oracle RAC in (Third-Party) Clouds allows for a variety of cloud offering to qualify as a suitable IaaS solution to host an Oracle RAC configuration. Customers considering running Oracle RAC in a cloud are best advised to choose an offering, in which the cloud vendor as well as Oracle have a clear and consistent understanding on how to support the Oracle Database deployment through its different phases. Comparing various solutions available today, the Oracle Cloud provides the best integration and support, as it makes the Oracle Database and Oracle RAC services available as part of its PaaS offering. Microsoft Azure, while providing an Oracle Database service as part of their IaaS offering, has engaged with Oracle and has come to a common understanding on how to best support the Oracle Database in this environment. Amazon falls short in this regard and should be considered carefully as a production RAC service provider despite their latest announcement.

This statement either attests a conceptual misunderstanding of the Oracle RAC technology or confirms the suspicion that the proposed solution for Oracle RAC in EC2 does not scale because of infrastructural shortcomings.

13 http://www.slideshare.net/MarkusMichalewicz/oracle-rac-12c-practical-performance-management-and-tuningoow13con8825 14 More information: http://www.slideshare.net/MarkusMichalewicz/oracle-rac-internals-the-cache-fusion-edition

C O N N E C T W I T H U S

blogs.oracle.com/oracle

facebook.com/oracle

twitter.com/oracle

oracle.com

F O R M O R E I N O R M A T I O N

Contact: 1.800.ORACLE1

Copyright 2015, Oracle and/or its affiliates. All rights reserved. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. 1014