RDS and Oracle 10g RAC Update

Paul Tsien, Oracle

2

Agenda

Oracle Database10g RACWhat is RDS (Reliable Datagram

Sockets)?Open Source RDS for LinuxBeta Customer Experience

3

Oracle Database 10g RAC

Oracle Database 10g Real Application Clusters (RAC) provides the ability to build an application platform from multiple systems that are clustered together

Allows applications to become Highly scaleable Highly available

Chosen to avoid a single node failure, causing application downtime Eliminates a node as single

point of failure

Shared DatabaseShared Database

OracleOracleInstanceInstance

OracleOracleInstanceInstance

OracleOracleInstanceInstance

4

Oracle Database 10g RAC

World’s best Scalability with Cache Fusion

Cache-to-cache data shipping

Scales off-the-shelf applications with no changes

World’s best Availability with Fast-Start Fault Recovery

Node failure is transparent to applications

Recovers from node failure in 17 seconds - workload independent

Pre-warmed cache speeds restart

Easily add and delete nodes

Shared CacheShared Cache

Cache FusionCache FusionUsers

The Ultimate Parallel Architecture

5

Oracle Database 10g RAC

Existing AppsFinancials, MFG, HR

and CRMCollaboration Suite In house developedDSS ISV Apps

Easy Migration Improve Utilization

ERP CRM DW

Real Applications in a Real Grid!

6

Oracle Database 10g RAC

RAC IPCThousands of processes200K+ associations (not connections)64 nodes

Oracle IPC UsageNew database functionality will significantly

increase IPC utilizationApproach database I/O ratesVery large messages (8 MB +)

7

What is RDS?

VisionA low overhead, low latency, high

bandwidth, ultra reliable, supportable, IPC protocol and transport system

Which matches Oracles existing IPC models for RAC communication

Optimized for transfers from 200 bytes to 8meg

8

What is RDS?

Goal and ObjectiveSupport for a reliable datagram IPC in OpenIB

Based on Socket APIMinimal code change / testing for OracleFailover inter HCA and intra HCA portsRuns over IB, Ether, iWARP, etc2-6 month validation / certification for RAC

9

What is RDS?

Reliable Datagram IPCUDP – Oracle adds reliable delivery via user

mode wire protocol engineTwo sockets per process, thousands of messages

on wireSlow sends times (windowing,acks,retrans)Holds together but degenerates under CPU loadWell tested !

10

What is RDS?

Available OptionsuDAPL / itAPI – not supportingIPOIB – high CPU overhead, same unreliable

delivery (UDP)SDP – connection orientedWe want to take our existing well tested UDP

module, shutoff most of it to run over an O/S provided RD IPC

11

What is RDS?

RD – Reliable Datagram IPC over IB50% less CPU than IPOIB, UDP½ Latency of UDP (no user-mode acks)Within 5% of uDAPL thru-put using OracleMinimal code change – reduced our UDP module by

70% - removed windowing, acks, retransmissions, etc.

RDS driver ~ = 1k C lines (b-copy) Decoupled from user-mode CPU loadingPasses all Oracle regression tests in < 2 wks !!!!Supports fail-over across and within HCAs

12

What is RDS?

RDS IPC over IBUses IB reliable connection (RC)Node to Node level connection

User mode sockets share small pool of node to node RCs.

Formed either dynamically at send or at system startup

13

Oracle Block Service Rate

0

5000

10000

15000

20000

25000

1 2 3 4 6 8 10 12 14 16 20 24 28 32

rds

ipoib

ge

14

Service Response Time

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

1 2 3 4 6 8 10 12 14 16 20 24 28 32

rds

ipoib

ge

15

CPU Cost Per Block Served

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

1 2 3 4 6 8 10 12 14 16 20 24 28 32

rds

ipoib

ge

16

Open Source RDS for Linux

Open Source RDSOracle is building Open Source RDS for Linux

http://oss.oracle.com/projects/rds/

Oracle will support RDS on LinuxOracle RDS will be pulled into OFEDOracle RDS will support GigE, simple NICs

and RDMA NICs (IB/iWARP)

17

Open Source RDS for Linux

StatusOracle is testing Open Source RDS on GigERDS on IB is in developmentBeta testing with customers using SilverStorm

RDS/IB stackVery stableVery good performance

RDS is supported by major tier one system vendors

18

Open Source RDS for Linux

Intel Confidential12

OpenFabrics Components and Architecture

RDMA NICR -NIC

Host Channel Adapter

HCA

User Direct Access Programming Lib

UDAPL

Reliable Datagram Service

RDS

iSCSI RDMA Protocol (Initiator)

iSER

SCSI RDMA Protocol (Initiator)

SRP

Sockets Direct Protocol

SDP

IP over InfiniBandIPoIB

Performance Manager Agent

PMA

Subnet Manager Agent

SMA

Management Datagram

MAD

Subnet Administrator

SA

Common

InfiniBand

iWARP

Key

InfiniBand HCAInfiniBand HCA iWARP RiWARP R --NICNIC

HardwareHardwareSpecific DriverSpecific Driver

Hardware SpecificHardware SpecificDriverDriver

ConnectionConnectionManagerManager

MADMAD

InfiniBand Verbs / APIInfiniBand Verbs / API

SA SA ClientClient

ConnectionConnectionManagerManager

Connection ManagerConnection ManagerAbstraction (CMA)Abstraction (CMA)

User Level User Level Verbs / APIVerbs / API

SDPSDPIPoIBIPoIB SRPSRP iSERiSER RDSRDS

UDAPLUDAPL

SDP SDP LibraryLibrary

User Level User Level MAD APIMAD API

Open Open SMSM

DiagDiagToolsTools

Hardware

Provider

Mid -Layer

Upper Layer Protocol

User APIs

Kernel Space

User Space

NFSNFS --RDMARDMA

RPCRPCClusterCluster

File SysFile Sys

Application Level

SMASMA

RR--NIC Driver APINIC Driver API

ClusteredDB Access

(Oracle10g RAC)

SocketsBased

Access(IBM DB2)

VariousMPIs

Access toFile

Systems

BlockStorageAccess

IP BasedApp

Access

Apps & Access

Methodsfor usingOF Stack

19

Beta Customer Experience

Customer RequirementsImprove application performance (throughput,

latency, etc.)Maintain data availabilityLower TCO through commodity hardware without

sacrificing performance or scalabilityWant to implement Grid and Utility computing

20

Beta Customer Experience

ResultsRDS/IB shows significant real world application

performance gains for certain workloads: DSS and mixed DSS/OLTP workloadsThroughput and latency

Customers are interested in unified fabric for cost and manageability reasonsReservation/QoS is a concern

Joint customer Oracle World presentation on RDS (Session S281216, 10/23/2006 11AM)