Sun Oracle Maa 060407

8/14/2019 Sun Oracle Maa 060407

1/65

Transitioning Oracle E-Business Suite to the Maximum

Availability Architecture on Sun Platforms

Oracle MAA Team and Sun Market Development

8/14/2019 Sun Oracle Maa 060407

2/65

2

The following is intended to outline our generalproduct direction. It is intended for informationpurposes only, and may not be incorporated into anycontract. It is not a commitment to deliver anymaterial, code, or functionality, and should not be

relied upon in making purchasing decisions.The development, release, and timing of anyfeatures or functionality described for Oraclesproducts remain at the sole discretion of Oracle.

8/14/2019 Sun Oracle Maa 060407

3/65

3

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow

Phase 1 Local Cluster Creation

Phase 2 Two Node RAC Phase 3 Full MAA Platform

Ongoing Switchover and Failover Testing

Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

4/65

4

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow




Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

5/65

5

Maximum Availability Architecture:

MAA, and the MAA Team

Oracle-recommended best practices for HighAvailability Based on proven technologies

Enhanced and validated with new Oracle versions

Goal: reduce the complexity of implementing MAA whileminimizing downtime

Best practices available through white papers and Oracledocumentation

Implemented by the MAA Team

HA engineering experts in Oracles core development group Deep-domain expertise designing, developing, and deploying

HA architectures using Oracle and system technologies, andsupporting the same at customer sites worldwide

8/14/2019 Sun Oracle Maa 060407

6/65

6

MAA for EBS: Target Architecture

Redundancy for local hardware failures Solaris Cluster, Oracle Clusterware, Oracle RAC, ASM

Protection against operator error Flashback database

Redundancy for site-level failures Data Guard Redo Apply

But must we suffer an outage to implement MAA?

8/14/2019 Sun Oracle Maa 060407

7/65 7

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow




Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

8/65 8

Minimizing Outage to Implement MAA

Stage all the changes, then switch Clone the file systems: Applications software, tech stack

Clone the database to create a physical standby ofproduction

Stage as many configuration changes as possible Switch over, complete configuration

8/14/2019 Sun Oracle Maa 060407

9/65 9

Clients

NAS Storage

OracleE-Business

Suite

OracleDatabase

Initial Configuration

SAN

Disk and Tape Storage

8/14/2019 Sun Oracle Maa 060407

10/65 10

MAA Configuration

ClientsPrimarySite DisasterRecovery Site

SANSAN

NAS StorageOracle E-Business Suite

Disk and Tape Storage Disk and Tape Storage

Oracle RAC Database

NAS Storage

8/14/2019 Sun Oracle Maa 060407

11/65

11

Clients

NASStorage

Oracle

E-BusinessSuite

OracleDatabase

SAN



Clients

Oracle

E-BusinessSuite

Two NodeRAC on ASM

SAN


NASStorage

Two Node

RAC Configuration

MAA

ConfigurationClients

Primary

Site

Disaster

Recovery Site

SANSAN

NASStorage

Oracle E-Business Suite


Oracle RAC Database

NASStorage

Clients

Oracle

E-BusinessSuite

NewDatabase Node

SAN


Single Node

RAC Configuration

NASStorage

Original NodeOut-of-Service

Single NodeRAC on ASM

8/14/2019 Sun Oracle Maa 060407

12/65

12

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow




Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

13/65

13

Phase 1: Local Cluster Creation

Clients

NAS Storage

OracleE-Business

Suite

Oracle

Database


SAN


Clients

NAS Storage

OracleE-Business

Suite

Single Node RAC Configuration

SAN


NewDatabase

Node

Original NodeOut-of-service

Single NodeRAC on ASM

8/14/2019 Sun Oracle Maa 060407

14/65

14

Phase 1 Establish Single Node RAC with ASM

Production

DatabaseNod

e

New

DatabaseN

ode

AppsNode

Backup

Database

Switchover

Apps

Clone

Database

Software

PrepareDatabasefor RACand ASM

Prepare

for New

Database

Clone

Apps

Software

Switchoverto New RACDatabaseon ASM

EstablishStandby

Database

Prepare

New

Database

Instance

EstablishOracleClusterware

and ASM

Establish

SolarisCluster

and Shared

Storage

8/14/2019 Sun Oracle Maa 060407

15/65

15

Local Cluster Creation:

Prep Target Server

Patch OS to current recommended levels

Install Solaris Cluster

Install and configure shared disk

Create shared logical volumes Create OCR, Voting, and ASM spfile disk groups - these caneach be 1GB

Create Data and Flash Recovery disk groups.

Install Oracle Clusterware and ASM

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

16/65

16


Prep Current Database for RAC

Add redo threads for the new instance(s)

Add undo tablespace(s) for the new instance(s)

Add the clustering tables to the data dictionary byrunning $ORACLE_HOME/rdbms/admin/catclust.sql

Single Node

RAC Configuration

Do these steps ahead of time in production,

not using DBCA, to reduce and simplifythe steps required during the downtime

8/14/2019 Sun Oracle Maa 060407

17/65

17

Local Cluster Creation: Prep

Current DB for Data Guard

Enable force logging to ensure all activityis written to the redo logs

Add standby redo logs

Create database password files Create them for your final configuration all instance names

Grant SQL*Net access to other database nodes forredo traffic 11i10 enables SQL*Net access control by default

Use OAM to add all appropriate interfaces for your newdatabase nodes, local and remote

Run AutoConfig to generate the new sqlnet.ora file

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

18/65

18


Prep DB Configuration Files

Make configuration changes using the includefile, to avoid conflicts with AutoConfig

For the temporary local standby database, we usedEZConnect to simplify network configurations, for

example: sqlplus sys/manager@ha1db:1521/VIS

We set fewer parameters than for a normal standbyscenario, as this is a temporary setup

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

19/65

19


Clone the DB Oracle Home

Run the Apps pre-clone utility against theproduction database

Copy the software to a new directory on the targetserver (named differently than the original) E.g., /u01/appltop in production; /u01/visdbRAC on target

Use adcfgclone.pl dbTechStackon the target server,to define the new topology You will point to the standby, so will not successfully connect

to a database Configure and restart the listener

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

20/65

20


Establish the Local Standby

Using RMAN, back up the production database,then restore it to the new environment

Start managed recovery: On the primary: set log_archive_dest_state_2 = enable

On the standby: start managed recovery

Validate that redo is being shipped and applied

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

21/65

21


Clone App Tier Software

Clone the Application tier software to a newdirectory structure on the current middle tier(s), soconfiguration can be ready ahead of downtime Run the pre-clone utility

Copy the software to a new directory Run adclonectx.plto define the new topology

Run adcfgclone.pl appsTier, pointing to the new context filecreated above

Single Node

RAC Configuration

At this point, all possible configuration changes are

staged, and the environment is ready for switchover

8/14/2019 Sun Oracle Maa 060407

22/65

22

Switchover to

Single Instance RAC

Be sure you are up to date with redo apply

Shut down the apps

[0:43] Switch to the local standby

[0:01] Enable flashback

[0:05] Open the new primary database instance [0:02] Remove the old application topology

[1:34] Run AutoConfig on the database server

[0:02] Bounce the DB listener to get the correct services

[2:50] Run AutoConfig on the middle tiers (in parallel) Start the application, pointing to your single-node RAC instance

Add the single instance to the Clusterware configuration

Single Node

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

23/65

23

Clients

NAS Storage

OracleE-Business

Suite


SAN


NewDatabase

Node


Single Node

RAC on ASM

8/14/2019 Sun Oracle Maa 060407

24/65

24

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow


Phase 2 Two Node RAC

Phase 3 Full MAA Platform


Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

25/65

25

Phase 2: Two Node RAC

Clients

NAS Storage

OracleE-BusinessSuite


SAN


NewDatabase

Node


Single Node

RAC on ASM

Clients

NAS Storage


Two Node RAC Configuration

SAN


Two Node

RAC on ASM

OracleDatabase

8/14/2019 Sun Oracle Maa 060407

26/65

26

Phase 2 Add Secondary RAC Instance Using the Original Node

New

Productio

n

DatabaseNod

e

Original

DatabaseN

ode

AppsNode

Clone

Database

Software

Rolling Apps

Restart to

Recognize

New Node

Add Nodeto RAC

Cluster

Prepare

New

Database

Instance

EstablishOracle

Clusterwareand ASM

Establish

SolarisCluster

and Shared

Storage

Two Node

8/14/2019 Sun Oracle Maa 060407

27/65

27

Prep Original Node for Cluster:

Hardware, OS, Storage

Add in any hardware required for clusteroperations

Apply OS patches as necessary

Change the server name to be cluster-friendly (e.g.,ha1db to ha1db02)

Install Solaris Cluster and add the node to the cluster

Configure access to shared disk

Add this node to the cluster for Oracle Clusterwareand ASM

Two Node

RAC Configuration

Two Node

8/14/2019 Sun Oracle Maa 060407

28/65

28


Clone, Configure DB Software

Clone the DB software from the productionRAC DB oracle_home to the original server

Start the new DB instance on the original server

Configure new DB instance using AutoConfig and theDB parameter include file

Run AutoConfig in the production RAC DB server toregenerate the TNS configuration there

Two Node

RAC Configuration

Two Node

8/14/2019 Sun Oracle Maa 060407

29/65

29


Configure Middle Tier

Using OAMs Context Editor, set Tools and iAStwo_task to point to values in the generatedTNS_NAMES.ora file: To load-balance Forms sessions: set Tools OH TWO_TASK

to point to the _806_balance alias To load-balance self-service connections: set iAS OH

TWO_TASK to point to the _balance alias

Run AutoConfig on the apps tier servers. Bouncethem when desired, to take advantage of the newdatabase instance

Two Node

RAC Configuration

Two Node

8/14/2019 Sun Oracle Maa 060407

30/65

30


Add Node to Clusterware

To be able to use srvctl to control the newcluster, add the resources to Clusterware via srvctl: Add the database

Add all database instances

Add listeners to Clusterware : Point to the Apps TNS_ADMIN directory in $OH/bin/racgwrap

Make sure the listener is running

Run netca, cluster configuration, choose local node. Run on

all nodes. Run AutoConfig again, to overwrite the listener.ora file

created by netca.

RAC Configuration

8/14/2019 Sun Oracle Maa 060407

31/65

31

Clients

NAS Storage

OracleE-Business

Suite


SAN


Two Node

RAC on ASM

OracleDatabase

8/14/2019 Sun Oracle Maa 060407

32/65

32

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow





Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

33/65

33

Phase 3: Full MAA Architecture

Clients

NAS Storage



SAN


Two Node

RAC on ASM

MAA Configuration

Clients

Primary

Site

Disaster

Recovery Site

SANSAN

NAS Storage Oracle E-Business Suite


Oracle RAC Database

NAS Storage

OracleDatabase

8/14/2019 Sun Oracle Maa 060407

34/65

34

Phase 3 Establish Disaster Recovery Site Utilizing Oracle Data Guard

Primary

Database

Nodes

DR

Database

Nodes

Prepare

New

Database

Instance

EstablishOracle

Clusterwareand ASM

EstablishSolarisCluster

and SharedStorage

DR

Apps

Nodes

Primary

AppsNodes

Establish

Standby

Database

Clone

Database

Software

Backup

Database

Prepare Apps

Software for

DR Database

Clone Apps

Software

MAAConfiguration

8/14/2019 Sun Oracle Maa 060407

35/65

35

Full MAA:

Establish Target Environment

Build DR site hardware platform(best: mimic production multiple middle tiers, RACdatabase server cluster)

Install the operating system

Install and configure Solaris Cluster

Configure shared storage

Install Oracle Clusterware and ASM

MAA Configuration

MAAConfiguration

8/14/2019 Sun Oracle Maa 060407

36/65

36

Full MAA:

Configure Prod Database

Add TNS entries for standbycommunications between sites Configure failover across nodes, not load balancing

Set database parameters for standby operations.

Same as for local standby, except: Use only permanent sites in log_archive_config Use TNS entries for FAL_CLIENT and FAL_SERVER

parameters

Use TNS entries for log_archive_dest_2

Assuming in place: standby redo logs, extra undotablespace(s), redo threads, cluster catalog indatabase, password files, SQL*Net access control,

MAA Configuration

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

37/65

37

Full MAA:

Clone Prod DB Software

Run the Apps pre-clone utility

Copy database ORACLE_HOME to DR databaseservers

Run adcfgclone.pl dbTechStackon each DRdatabase server

g

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

38/65

38

Full MAA:

Generate Core init.ora

Edit the context files to correct topologyinformation (incorrect, as the DB is not yet up) Instance_number

Instance_thread

Undo_tablespace Move / remove init.ora and

_APPS_BASE.ora so AutoConfig regenerates

Run AutoConfig

Adjust the database configuration for when thisenvironment is primary and when it is standby, and forRMAN

g

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

39/65

39

Full MAA:

Configure Standby TNS

TNS configuration Copy production _ifile.ora to standby

_ifile.ora, to add the failover services

Listener configuration

Add the ability to listen on the physical machine name to thelist of addresses, using include files

Bounce the listener on each node on the DR site

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

40/65

40

Full MAA:

Clone the Database

Using RMAN, back up the productiondatabase including archivelogs, and theproduction control file as standby

Using RMAN, restore the database to the DR

site using one of the configured instances Start managed recovery

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

41/65

41

Full MAA: Update

Clusterware With Standby DB

Update the Oracle Clusterwareconfiguration on the standby site: Add the database

Add all instances

Add listeners Run AutoConfig once more to restore the base

listener.ora files

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

42/65

42

Full MAA:

Clone Application Software

Run the pre-clone step, copy the software,run adclonectx.pland adcfgclone.plon each DRsite middle tier server Ignore the error when running adcfgclone.pl appsTier

which occurs due to no connection to database Edit the context file to point Tools OH TWO_TASK,

iAS OH TWO_TASK, and Apps JDBC ConnectAlias to the appropriate load balancing service

8/14/2019 Sun Oracle Maa 060407

43/65

43

At this point, all possible configuration changes are

staged, and the environment is ready for switchover

MAA Configuration

Clients

Primary

Site

Disaster

Recovery Site

SANSAN

NAS Storage Oracle E-Business Suite


Oracle RAC Database

NAS Storage

8/14/2019 Sun Oracle Maa 060407

44/65

44

MAA: Getting There

With Less Downtime

MAA

Philosophy and Flow





Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

45/65

45

Ongoing Switchover and Failover

Testing

Periodically verify viability of DR environment

Practice steps so the process flows easily ifdisaster strikes

Use the DR environment to provide application

services when performing platform or sitemaintenance

MAA Configuration

8/14/2019 Sun Oracle Maa 060407

46/65

46

Test Failover

Be sure you are up to date with redo apply

Shut down the app and all but 1 RAC instance oneach site

Switch the standby to primary, enable flashback,

open, start other instances

Run AutoConfig on database, then middle tier Do the topology dance on the DB tier first

Start the Apps

Use Flashback Database to start the originaldatabase as a standby of the new production server

Requires a Brief Outage

8/14/2019 Sun Oracle Maa 060407

47/65

47

DR Testing Procedure Using Flashback

Database

Create a database restore point on the DR standbydatabase

Open the standby database, complete theconfiguration

Perform testing

Flash the standby back to the restore point, resumerecovery as a standby

8/14/2019 Sun Oracle Maa 060407

48/65

48

MAA: Getting There

With Less Downtime

MAA Philosophy and Flow





Partnering with Sun

8/14/2019 Sun Oracle Maa 060407

49/65

49

Oracle Clusterware

In Oracle RAC 10g - Various Oracle resources areconfigured to be managed by Oracle Clusterware

ONS (Oracle Notification Service)

VIP (Virtual IP Address)

LISTENERS

DATABASE INSTANCES

SERVICES

8/14/2019 Sun Oracle Maa 060407

50/65

50

Oracle Clusterware Provides

VIP resource Provides Application VIPs

HA - Framework Extends Oracle Clusterware HA protection to application

HA - API Interface to allow customers to change - at run time -

how Oracle Clusterware manages customer application

O l Cl t ith ASM E bl

8/14/2019 Sun Oracle Maa 060407

51/65

51

Oracle Clusterware with ASM Enables

Consolidated Clustered Storage

Clustered

Pool of Storage

ASM

Instance

ASM

Instance

ASM

Instance

ASM

Instance

Clustered

Servers

RAC orSingle Instance

Databases ERP ERP CRM CRM

Disk GroupDisk Group

ASM

Instance

HR

P t i ith S

8/14/2019 Sun Oracle Maa 060407

52/65

52

Partnering with Sun

Part of MAA means proving and testingour best practices, and working closelywith Sun to ensure our joint solutions workwell together

S l i Cl t ith O l

8/14/2019 Sun Oracle Maa 060407

53/65

53

Solaris Cluster with Oracle

Oracle Clusterware and Solaris Cluster worktogether in providing a reliable joint HA solution forOracle 10G RAC on Sun platforms

Proven and mature Sun Cluster framework

I/O fencing and data integrity

Interconnect failover and application traffic striping Shared storage support, APIs, and more

End-to-end Sun technology stack for better integration

Integrated cluster file system and volume manager

Supports up to 8 node RAC configurations

SPARC and AMD x64

More choice with lower total cost of ownership

S l i Cl t Q d I/O

8/14/2019 Sun Oracle Maa 060407

54/65

54

Solaris Cluster: Quorum and I/O

Fencing for Data Integrity

Solid implementation of quorum algorithmto prevent split-brain

I/O fencing prevents access to sharedstorage by a node that is not part of thecluster

Guarantees no data corruption through

non-cluster nodes accessing shared data Node time synchronization

S l i Cl t H tb t

8/14/2019 Sun Oracle Maa 060407

55/65

55

Solaris Cluster : Heartbeats,

Interconnects and Traffic Striping

Implements cluster heartbeats ininterrupt context

Not subject to scheduling problems due tohigh load or resource starvation

All interconnect links are used withautomatic failover built-in

Up to six links supported Separate networks for each private

interconnect means redundancy evenat switch level

All traffic is striped over privateinterconnects, resulting in higher

throughput and lower latency

S l i 10 O ti S t

8/14/2019 Sun Oracle Maa 060407

56/65

56

Solaris 10 Operating System

Offers over 600 exciting new features Supports horizontal or vertical scaling

Provides relentless availability

Delivers extreme performance

Provides unparalleled security

Facilitates leveraging of low cost hardware

Enables standardization on a single OS

Offers interoperability with Linux, Windows

S Fi T2000 S ith

8/14/2019 Sun Oracle Maa 060407

57/65

57

Sun Fire T2000 Servers with

CoolThreads Technology

Used in the MAA application tier runningOracle EBS apps 11.5.10.

Designed for Web, application tier, andmultithreaded workloads

Utilize an innovative design Incorporate UltraSPARC T1 processors with

CoolThreads technology Deliver breakthrough performance Provide massive thread-level parallelism Increase application throughput

Offer dramatic space and power efficiency Configured with an 8 core, 1.2 GHz

UltraSPARC T1 processor, 32 GB memory,two 73 GB disk drives

S Fi X4200 S

8/14/2019 Sun Oracle Maa 060407

58/65

58

Sun Fire X4200 Servers

Used in the MAA database tier runningOracle RAC database on Solaris 10 x64. Support up to two single or dual-core

AMD Opteron processors Deliver fast network performance with

four Gigabit Ethernet ports, up to five

64-bit PCI-X slots Virtually eliminate I/O bottlenecks with

AMDs HyperTransport technology Provide redundant power supplies, fans,

hard disk drives Bring extreme performance and a new level

of energy efficiency to the x86 market Configured with 8 GB memory, two 73 GB

disk drives

S St T k 5320

8/14/2019 Sun Oracle Maa 060407

59/65

59

Sun StorageTek 5320

NAS Appliance

Used in the MAA application tier

Easy to deploy and manage

Scales to 336 TB

Maximizes security with a closed operating system Ensures regulatory compliance with the Sun StorageTek

Compliance Archiving software

Increases availability and reliability with dual redundant RAIDcontrollers, journalling file system, and checkpointing

Handles multiple protocols for UNIX and Windows clients

S St T k 6540 A

8/14/2019 Sun Oracle Maa 060407

60/65

60

Sun StorageTek 6540 Array

Used in the MAA database tier Provides online, data-in-place expansion Scales to 168 TB in a small footprint Uses a high availability architecture and

data protection software Enables configuration and management

over the network

Sun StorageTek Tape Storage

8/14/2019 Sun Oracle Maa 060407

61/65

61

Sun StorageTek Tape Storage

Manage and protect data with tape libraries Gain control of information and make it

manageable with tape virtualizationtechnology

Take advantage of price, capacity and

performance without straining budgetswith tape drives Centrally authorize, secure and manage

encryption keys with tape encryptiontechnology

Improve the efficiency and productivity ofautomated tape libraries with a full rangeof tape management software

Software Components from Oracle

8/14/2019 Sun Oracle Maa 060407

62/65

62

Software Components from Oracle

and Sun

Oracle E-Business Suite 11.5.10.2 Oracle RAC database, ASM and

Clusterware 10.2.0.2 Oracle Enterprise Manager 10g Grid

Control Solaris 10 Operating System

(Update 3) Solaris Cluster 3.2 Advanced Edition

for Oracle RAC Sun N1 System Manager software

For More Information

8/14/2019 Sun Oracle Maa 060407

63/65

63

For More Information

http://search.oracle.com

or

http://www.oracle.com/technology/deploy/availability/htdocs/maa.htm

maximum availability architecture

8/14/2019 Sun Oracle Maa 060407

64/65

64

8/14/2019 Sun Oracle Maa 060407

65/65

Sun Oracle Maa 060407

Documents

Transcript of Sun Oracle Maa 060407