Hosted by

Storage Network Designs for OLTP Business Continuity

Marc FarleyPresident, Building Storage Networks, Inc.

Hosted by

Agenda

The Vendor Neutral Approach

Overview of OLTP &High Availability

I/O Redundancy Methods

Storage Network Technologies

Storage Networking for HA OLTP

Hosted by

Vendor Neutral Approach

Generic terms, not vendor terms

Assumed basic knowledge of SAN, NAS,

RAID

Hosted by

And now, for something completely different…..

Hosted by

OLTP Environments

Mission critical business applications• Business in real-time

Expensive equipment and software

Aggressive performance objectives

Highly skilled IT staff• Hands-on computing operations

Hosted by

OLTP Database SoftwareOracle,

• 8i Oracle Parallel Server (OPS)• 9i Real Application Cluster (RAC)

IBM • DB2 UDB• Informix

MS SQL Server

Sybase, My SQL, others

Hosted by

OLTP OS Platforms

IBM S/390 MVS

Unix Systems

Windows 2000+

HA Linux

Hosted by

OLTP Requirements

99.999% uptime

Non-degrading response time

High transaction rates

Seamless scalability

Cost relief

Hosted by

Database Storage Approaches

Raw parititions• Bypass OS I/O buffering

File system• Facilitates data management

NFS mounted• Offload DB server, NTAP + Oracle

Hosted by

Atomicity – No partial transactions

Consistency – All tables are in a consistent state before and after a completed transaction

Isolation – One transaction cannot contaminate other transactions

Durability – Transactions are complete only when the database updates are written to disk storage

ACID Properties of OLTP

Hosted by

Challenges of OLTP

Major systems integration effort• Intricate tuning and monitoring• Little tolerance for errors

Complex data structures & relationships

Time and sequence-sensitive processes• Must be adhered to for data integrity

Shifting workloads and bottlenecks

Hosted by

OLTP Database Files

Data files• Database data, tablespaces

Redo log files, archive log files• Reconstruct or rollback transactions

Control files• File layout information

Hosted by

OLTP Table Space Storage

Use many spindles to distribute hot spots

RAID 0+1 recommended

File system recommended over raw

partitions• Easier data management

Hosted by

Striping for Performance

RAID Controller (Microsecond performance)

DiskDrive

DiskDrive

DiskDrive

DiskDrive

DiskDrive

DiskDrive

Disk Drives (Millesecond performance)From rotational latency and seek time

Hosted by

My Personal Favorite, RAID 0+1

RAID Controller

DiskDrive

DiskDrive

DiskDrive

DiskDrive

1 2 3 4 5

DiskDrive

DiskDrive

DiskDrive

DiskDrive

DiskDrive

DiskDrive

Mirrored Pairs of Striped Members

Hosted by

OLTP Redo Log StorageRaw partitions recommended

• Sequential high speed writes

Separate mirror pairs per log file group

Capacity for 30 – 60 minutes of data

Goal is to limit disk contention for current and active log files

Hosted by

OLTP Archive Log Storage

File system or NFS mounting is required• NFS mounting is recommended

Mirroring or RAID

Goal is to have easy access in case they

are needed for reconstruction

Hosted by

High Availability

The ability for a system or application to immediately

continue its mission after loss or damage to system

components, systems, facilities and data

Hosted by

Availability Threats

Expected

• Scaling limitations

Processor

Storage capacity

Network

• Consolidations

• Product life cycles

Unexpected

• Failures

• Bugs

• Virus

• Operator errors

• Disasters

Hosted by

HA Engages All Elements

Systems• Application

Network connections• Network services

Storage and I/O subsystems

Hosted by

Scoping the Risks

System Network Storage

Component HBA Cable Disk drive

System Server Switch Subsystem

Pathological Virus attack

on platform

Service provider

outage

Environmental

media loss

Site Server rooms

gutted

All external

communications

Total data loss

Hosted by

Managing the Risks

Local copies of data • Immediate availability

(Remote) Nearby• Immediate availability to several hours

Remote Far away• One to several days availability

Hosted by

Disaster/Availability Radii

Local Remote Nearby

Remote Far Away

Hosted by

Nobody Expects…..

Weird things to happen to them

Disintegration of media

Underground flooding through tunnels

Fires in Telco switching centers

Hosted by

High Availability for OLTP

Duplication of functions• Without degrading performance• Without risking data integrity

Brute force techniques

Automation and efficiency

Cost is always an issue • And high availability DOES cost

Hosted by

Redundancy. Again!

A Long Time Ago in a Job Not So Far Away…………….

Remember Marc, there is only one

concept:REDUNDANCY!

Marc Skyfaller Farley

Marc Skyfaller Farley

Jedi Jim GastJedi Jim Gast

WhateverGot it Jim. Let’s Eat!

You must learn the to be a master of redundancy it if

you are going to be a storage geek.

Hosted by

Eventually, I Learned to Appreciate His Teachings……

•REDUNDANCYNSPoF (No Single Point of Failure)

•REDUNDANCYNSPoF (No Single Point of Failure)

Don’t get the giant spicy Polish for lunch – its too much for the digestion

Don’t get the giant spicy Polish for lunch – its too much for the digestion

Hosted by

OLTP HA Requires Complete Redundancy Protection

Client network

Server systems and components

Application modules

I/O Channels and Networks

Storage subsystems and components

Data

Hosted by

A Quick Look At Clustered Storage

Shared Nothing Shared Everything

Each server controls its own storage address space

Both servers share control of a common storage address space

Hosted by

Examples of OLTP Clusters

Microsoft SQL Server Oracle 9.1 RAC

Data is exchanged between servers

Failoverpaths only

Data is accessed directly from

storage

Hosted by

One more time, with subsystems…

Microsoft SQL Server Oracle 9.1 RAC

All storage is shared by all cluster nodes

Same subsystem but different address spaces

Hosted by

I/O Redundancy

Host to subsystem• Mirroring: Host to independent targets

• Multi-pathing: Host to a single target

Subsystem to subsystem• Store and forward:

Local

Remote

Hosted by

Disk Mirroring: Redundant storage targets

Independent, identically sized storage address spaces

One controller Two controllers

Hosted by

Disk Mirroring: I/Os to 2 Targets

“Brute force” redundancy: fast and simple

Both read and write I/Os• Overlapped reads for performance

Local connections

Limited capacity*

I/O Bottlenecks* for random I/O activity

• * if targets are disk drives

Hosted by

Disk Mirroring for Redo Log Files

Log files are a common bottleneck

Use raw partitions

Redundancy is required• Mirroring is adequate

Use highest RPM with lowest seek times

Put on a separate channel from database I/O

Use separate mirrored pairs per group

Hosted by

Mirroring to Storage Subsystems

Independent, identically sized storage address spaces

Two controllers

StorageSubsystem

StorageSubsystem

Hosted by

Mirroring to SubsystemsTargets are subsystems, not disks

• Separate address spaces

Capacity scales to subsystem max

Double level redundancy• Mirroring plus RAID

Multiple disk spindles reduces I/O bottlenecks

Hosted by

Disk Mirroring Datafiles from Host to Storage Subsystems

Disk mirroring + subsystem RAID

Excellent capacity scaling

Adjacent and across campus/town• One subsystem outside site radius

Requires longer distance cabling

Reads and writes both transmitted

Hosted by

Multi-Pathing: Redundant Paths Between a Host & Subsystem

Pathing software determines that a transmission error occurs & switches to a redundant path

Application data volume

Hosted by

Multi-pathing vs Mirroring

Mirroring assumes independent, but

similar storage targets

Multi-pathing assumes multiple paths to

the exact same target

Mirroring can use a single HBA, multi-

pathing needs two HBAs

Hosted by

Path Failures

1. HBA problem

2. Link, switch or network problem

3. Subsystem controller problem

Application data volume

Hosted by

Transmission failures recognized after SCSI timeouts are exceeded

I/O sent to storage

No ack received

The I/Os is retried and eventually an error is passed back to the process that issued the I/O

Hosted by

Path Failover for OLTP I/O

Redundant path resources take over activities for a failed

path to sustain operations without disrupting service or

risking data integrity

Hosted by

Store and Forward

Host

BA

Independent, identically sized storage address spaces

Hosted by

Store & Forward: One Host I/O and Two Copies of Data

Only real option for remote copies

Does not forward read I/Os

Proprietary protocols and methods• Standards are emerging ie. FC/IP

First step to storage snapshots

Hosted by

Store and Forward: Acknowledgements

A

I/O ACK

B

Forward

Asynchronous

A

I/O ACK

B

Synchronous

Forward

ACK

Hosted by

Trade-offs withAcknowledgement Handling

Synchronous• Always preferred• Slowest performance• State of copy is precise

Asynchronous: • Fastest performance• Least precise knowledge of copy status

Hosted by

Store & Forward: Local and Remote Copies

Local & nearby copy techniques

• Synchronous

• Fiber optic cabling, optical/DWDM services

Remote-far away copy techniques

• Asynchronous

• ATM gateways, OC-12 or less, FC/IP

Hosted by

Mirroring vs Synchronous Store and Forward for Local & Nearby Copies

Mirroring

• Async I/O

• Reads and writes

• No snapshot tie-in

• Uses more host slots

• Least costly

Store and Forward

• Async or Sync I/O

• Writes only

• Snapshot ready

• May conserve host I/O

slots

• Most costly

Hosted by

Combining Mirroring with Store and Forward

LocalNearby Remote Far

AwayMirroring Radius

Store and Forward Radius

Hosted by

Data Redundancy for OLTP

Backup

Snapshots

Delta (log files)

Hosted by

Backup for OLTP

A whole subject unto itself

Disaster recovery primarily

Cold? Who can afford to do that anymore?

Hot – put DB in backup mode

Backup snapshot image of data

Hosted by

Subsystem Snapshots for OLTP

DatabaseServer

DiskStorage

SubsystemA

DiskStorage

SubsystemB

DiskStorage

Subsystemc

1. Flush host buffers (sync, sync)

2. Create Snapshot

Hosted by

Logical Snapshots for OLTP

v

1. The address space is mapped

2. First updates

3. Secondupdates

Overwritten data locations are not returned to the free space pool. (Undelete)

Hosted by

Delta Redundancy with Log Files

Recording of all transaction activities

Roll forward, bring up to date

Roll Backward, go to known good state

Terrific tool for remote redundancy

Not HA

Process cannot have holes in it

Hosted by

Remote Redundancy w/ Log Files

-1

d(x) = f(x) – f(x-1) f(x-1) f(x)

Latest Redo Log File Previous Instance Current to Log File Switch Checkpoint

Hosted by

And now, some thoughts from our sponsor…..

How come I always end up doing all the work?

He never does anything

except eat and sleep

ManagingManagingRedundancy Redundancy is Hard Workis Hard Work

Redundancy is a way of life

Hosted by

SAN Considerations

Fabrics and SAN Islands

Zoning

Switches and directors

Multiplexing (oversubscribing)

Security

Hosted by

Fabrics ARE the SAN Environment

One size does not fit all applications

Larger fabrics carry more risks

VSANs are probably a good idea

Only use switches supporting hot, stateful

firmware upgrades

Hosted by

SAN Islands May be Best for OLTP

Most risk averse approach

Dual fabrics, one fabric per I/O path

Switch problems do not cascade

But, higher management costs

Hosted by

Zoning & OLTP

All ports defined to zones• No rogue ports and zombie zones

Restrict access to current servers• Need-to-access only

Hosted by

Switches and DirectorsRedundancy eats slots and ports

• Pathing, mirroring

• Separate channels for data and logs

Avoid traversing ISLs, if possible• Added latency and blocking potential

• Trunking must have NSPoF

Hosted by

Security

Admin security for an OLTP SAN should

be as strong as possible• No monkey business

No default passwords left

WAN encryption of log files

Hosted by

Recommendations:Determine OLTP availability needs

• Where copies should be, time to access

Match storage network implementation to

DB file types

Develop availability-driven policies• Equipment

• Processes