Oracle RAC SIG - Implementing a dual site extended distance RAC Cluster.pps

8/13/2019 Oracle RAC SIG - Implementing a dual site extended distance RAC Cluster.pps

1/58

(c) Copyright DSP Global Limited. All rights reserved. 1

UKOUG 2007

Implementing a dual site extendeddistance RAC Cluster

Dev Nayak04/12/07

DSP Global Limited


2/58


Agenda

Historical Oracle platform incorporating HA and DR

The Next Generation Oracle platformLinux v HP/UX Solutions

What we built

Does it work?

Conclusion

Questions


3/58


Historical Oracle Platform incorporating HAand DR

Standard Oracle Platform


4/58



Standard Oracle Platform

The server is a key SPOF


5/58



Oracle RAC Platform

Removes server as SPOF


6/58



Oracle RAC Platform

Removes server as SPOFIs Active-Active


7/58



Oracle RAC Platform


Is Scaleable


8/58



Oracle RAC Platform


Is Scaleable

Is Single Site

The site is a key SPOF


9/58



Oracle RAC Platform


Is Scaleable

With Dataguard


10/58



Oracle RAC Platform


Is Scalable

With Dataguard

Removes site as SPOF


11/58



Oracle RAC Platform


Is Scalable

With Dataguard

Removes site as SPOFIs Active-Passive


12/58



Oracle RAC Platform


Is Scalable

With Dataguard

Removes site as SPOFIs Active-Passive

Scalability is costly

Especially as each node requires alicense

Regardless of it being passive


13/58



Oracle RAC Platform


Is Scalable

With DataguardRemoves site as SPOF

Is Active-Passive




Thus, typical DR site provides lowerservice level

l l l f


14/58



Oracle RAC Platform


Is Scalable

With DataguardRemoves site as SPOF

Is Active-Passive





Requires Failover

Failover manual and failover time ishigh

Hi i l O l Pl f i i HA


15/58



Oracle RAC Platform


Is Scalable

With Split-site ServiceGuardRemoves site as SPOF

Is Active-Passive




16/58


17/58


The Next Generation Oracle platform

Oracle Platform

Removes site as SPOFIs Active-Active

Is Scaleable


18/58



Oracle Platform


Is Scaleable

I.e. Oracle RAC Platform

that removes site as the SPOF

Answer: Stretch a RAC Platform!


19/58



Oracle Platform


Is Scaleable





20/58



Oracle Platform


Is Scaleable





21/58



Oracle Platform


Is Scaleable





22/58



Oracle Stretch RAC Platform Considerations


23/58




Inter-Site Connectivity


24/58




Inter-Site ConnectivityDisk Mirroring

Disk Mirroring


25/58





RAC Cache Fusion

Cache Fusion

Disk Mirroring


26/58





RAC Cache Fusion

Requires dedicated fibre

Cache Fusion

Disk Mirroring


27/58



Oracle Stretch RAC Platform

ConsiderationsCluster Voting Disks

Cache Fusion

Disk Mirroring


28/58


29/58





Redundant for resilience

Cache Fusion

Disk Mirroring


30/58


31/58






Odd number

Say we lose a site?

Cache Fusion

Disk Mirroring


32/58






Odd number

Say we lose a site?

A node must be able toaccess at least half of thevoting disks to remain amember

Cache Fusion

Disk Mirroring

Oops!


33/58






Odd number

Say we lose a site?


The 3rdvoting disk needs an

independent site

Cache Fusion

Disk Mirroring

3RDSite


34/58






Odd number

Say we lose a site?


The 3rdvoting disk needs anindependent site

Cache Fusion

Disk Mirroring

3RDSite


35/58






Odd number

Say we lose a site?



The 3rdSite acts as an arbiter

Cache Fusion

Disk Mirroring

3RDSite


36/58






Odd number

Say we lose a site?




Cache Fusion

Disk Mirroring

3RDSite


37/58






Odd number

Say we lose a site?




Only Requires NFS

Cache Fusion

Disk Mirroring

3RDSite


38/58






Odd number

Say we lose a site?




Only Requires NFS

Doesnt have to be highlyavailable

Cache Fusion

Disk Mirroring

3RDSite


39/58


Agenda


The Next Generation Oracle platformLinux v HP/UX

What we built

Does it work?

Conclusion

Questions


40/58


Linux v HP-UX

Red Hat Linux HP-UX

RDBMS Oracle 10gR2 RAC Oracle 9i/10gR2RAC

Cluster Software Oracle Clusterware HP ServiceGuard

Software Mirroring Oracle ASM MirrorDisk UX orVeritas CVM

Network Teaming Linux bonding HP ServiceGuard

Distance < 100km 10km

16 < 10km


41/58


Cost Comparison

Costs

Linux is 42% CheaperOracle Licensing on HP/UX/Itanium is the significant factor

Linux/Opteron

HP-UX/Itanium

Total CapEx 175,879 282,014

Total 3-year OpEx 79,465 159,688

Total 3-year Cap and Op Ex 255,343 441,703


42/58


Agenda



What we built

Does it work?

Conclusion

Questions


43/58


44/58


What We Built

Item Extended Distance RAC

RDBMS Oracle 10gR2 release 10.2.0.3

Cluster Software Oracle 10gR2 Clusterware release 10.2.0.3

Mirroring and

Volume Manager

Oracle 10gR2 ASM release 10.2.0.3

MultipathingSoftware

EMC PowerPath release 4.5.1

Operating System Red Hat Enterprise Linux AS release 4update 4

NIC teaming Linux Channel bonding module


45/58


Cluster Interconnect

RAC Cache Fusion

No SPOF

Redundant NICs are bonded using

Linux bondingRedundant Switches at each site

Switch spanning preservesnetwork path between NICs duringswitch failure

lononl11

Eth1

Eth3

Bond1Private

IP:10.10

0.48.33

lononl12

Site A Site B

Eth1

Eth3

Bond1

Private

IP:10.

100.

48.

33


46/58


SAN

Separate SAN Fabric at each site

Redundant HBAs in each node

Redundant Storage controllers ineach Array

EMC PowerPath manages loadbalancing and failover of multiple

OS paths to each LUN

lononl11

HBA 0 HBA 1

ASM disks VotingDisk

OCR Disk

DATA diskgroup

RECO diskgroup

Storage Array


OCR Disk

DATA diskgroup

RECO diskgroup

Storage Array

lononl12

Site A Site B

HBA 0 HBA 1


47/58


Voting Disks

lononl11

Eth0

Eth2

Bond0

Public IP: 10.100.14.46

VIP: 10.100.14.64

HBA 0 HBA 1

NFS Voting Disk Server

NFS Vote Disk

NFS Voting DiskServer

VotingDiskStorage Array

VotingDiskStorage Array

Eth1

lononl12

Site A Site B

IndependentSite

Eth0

Eth2

Bond0

Public IP:10.100.14.48

VIP:10.100.48.65

HBA 0 HBA 1

/dev/raw/raw

1

/dev/raw/ra

w2


48/58


Database Mirroring

ASM Requires 1 failure group per siteto preserve inter-site mirroring

ASM doesnt use a Dirty Region Logand so doesnt provide partialresilvering

Oracle Software is on local storage

Facilitates rolling upgrades

lononl11

HBA 0 HBA 1


OCR Disk

DATA diskgroup

RECO diskgroup

Storage Array


OCR Disk

DATA diskgroup

RECO diskgroup

Storage Array

lononl12

Site A Site B

HBA 0 HBA 1


49/58


What We Built



What we built

Does it work?

Conclusion

Questions


50/58

Is It Highly Available and Disaster


51/58


g yTolerant?

Simulated loss of a node by

rebooting it

Result:Remaining node continues

Database cache remasters

60 second brown outThis can be tuned

Cache Fusion



52/58


g yTolerant?

Simulated loss of the cluster

interconnect

Result:One node is evicted from the

cluster

One node remains


60 second brown out

This can be tuned

Cache Fusion



53/58


g yTolerant?

Simulated Data Centre loss by halting the node andremoving SAN access to the array

Result:Remaining nodes continues


60 second brown outThis can be tuned

ASM mirrors require rebuilding

Cache Fusion


54/58


What is the Performance Like?

40% better performance thana similar extended cluster built

for another customer

3700 transactions per second

Cluster Interconnect Ping time of 0.8ms

Blocks were being passed betweeninstances with a latency of < 2ms

Meets the objective of being fasterthan a disk access

Throughput exceeded 500mbps

Performance test was boundedby CPU The inter-site components, IO and

the cluster interconnect, were notprimary bottlenecks

Interconnect Throughput

0

100

200

300

400

500

600

700

14:13:51

14:14:31

14:16:23

14:17:03

14:17:43

14:18:23

14:19:04

14:19:44

14:20:24

14:21:04

14:21:45

14:22:25

14:23:05

14:23:45

14:24:26

14:25:06

14:25:47

14:26:27

14:27:07

14:27:48

14:28:28

14:29:09

14:29:49

14:30:29

14:31:10

Time

Mbps

Mbps


55/58


What We Built



What we built

Does it work?

Conclusion

Questions


56/58


Conclusion

Extended distance RAC using Oracle Clusterware/Linux/AMD

Works!Platform is Disaster Tolerant and Highly Available

Failover requires no administrator intervention

Recovery from site failure in 60 seconds

Full utilisation of all resources

It is a high performance platform


57/58


Most Challenging 2 Node Cluster Ever

Oracle RAC SIG - Implementing a dual site extended distance RAC Cluster.pps

Documents

Transcript of Oracle RAC SIG - Implementing a dual site extended distance RAC Cluster.pps