Tcoug Top 10 Asm Questions

<Insert Picture Here>

Top 10 Things You Always Wanted to Know About Automatic

Storage Management

But Were Afraid to Ask Nitin Vengurlekar

RAC/ASM Engineering

Grid Product Strategy


Agenda

• ASM Overview – 2 minute tour

• Top Ten Questions

• ASM Architecture

• ASM Configuration & Performance

• Backups

• ACFS

• Migration

• 3rd Party Software

• Summary

ASM Architecture - Review

• Automatic Storage Management (ASM) instance

• Instance that manages the diskgroup metadata

• Disk Groups

• Logical grouping of disks

• Determines file mirroring options

• ASM Disks

• LUNs presented to ASM

• ASM Files

• Files that are stored in ASM disk groups are called ASM files, this

includes database files

ASM ArchitectureA Brief Overview

Pool of Storage

ASM

Instance

Server

Non–RAC

Database

Oracle

DB Instance

Disk Group Disk Group

ASM Process ArchitectureA Brief Overview

Clustered

Pool of Storage

ASM

Instance

ASM

Instance

ASM

Instance

ASM

Instance

Clustered

Servers

RAC or Non–RAC

Databases Oracle

DB Instance

Oracle

DB Instance

Oracle

DB Instance

Oracle

DB Instance

Database

ASM

Instance

Oracle

DB Instance

DatabaseDatabase

Disk Group Disk Group


Top 10 ASM Questions

ASM Architecture


Q. What init.ora parameters does a user need to

configure for ASM instances?

A. The default parameter settings work perfectly for

ASM. The only parameters needed for 11g ASM:

• PROCESSES

• ASM_DISKSTRING*

• ASM_DISKGROUPS*

• INSTANCE_TYPE*


Q. How does the database interact with the ASM

instance and how do I make ASM go faster?

A. ASM is not in the I/O path so ASM does not impede

the database file access. Since the RDBMS instance

is performing raw I/O, the I/O is as fast as possible.

RDBMS and ASM Instance Interaction

Server

Operating System

DATABASE ASM

(1) Database opens file

(1A) OPEN

(1B) Extent Map

(2) Database Reads file

(2A) READ

(2B) I/O

Completes(3) Database Creates file

(3A) CREATE

(3B) Allocates file

(3C) Extent Map

(3D) Initializes file (3D) Commit


Q. Do I need to define the RDBMS

FILESYSTEMIO_OPTIONS parameter when I use ASM?

A. No. The RDBMS does I/O directly to the raw disk

devices, the FILESYSTEMIO_OPTIONS parameter is

only for filesystems.


Q. Don’t I lose all the advanced filesystem features when I

move to ASM; e.g., directIO, write coalescing, and pre-

fetch?

A. Yes, but that’s okay.

• Most of the filesystem features mentioned, though good for

general file data performance, interfere and fractionalize the

benefits inherently provided by the database; e.g.,

• DBWR & LGWR does write coalescing, and user processes do

sequential pre-fetches

• All IO capable processes do un-buffered IO (raw IO) because of ASM


Q. This is cool that ASM can now store Vote and OCR files.

But how does CSS and CRS startup in this configuration?

A. It just does ☺

• There are two keys processes and a lot of crafty coding to

get this to work in the correct startup sequence.


ASMCMD> pwd

+DATA/rst-cluster/OCRFILE

ASMCMD> ls -l

Type Redund Striped Time Sys Name

OCRFILE UNPROT COARSE JUN 25 11:00:00 Y REGISTRY.255.718984285

ASMCMD> lsdg

State Type Rebal Sector Block AU Total_MB Free_MB

Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name

MOUNTED EXTERN N 512 4096 1048576 203824 193028

0 193028 0 Y DATA/

CSS-CRS-ASM Instance Architecture

ASM

GPnP

1. Get ASM disk string

2. Do ASM disk discoveryto locate vote disk

ASM Disk

CRS

ASM Disk

DATA DG

SPFILE

Vote

OCR

3. Once Vote disk located, startup CSS

OHASD

4. ASM mounts DATA dg

Agents starts GPnP, CSS and ASM

CSS

5. CRS open OCR in well known ASM dg location



ASM Configuration


Q. Do I need 11gR2 Grid Infrastructure to use ASM

A. Yes. ASM is now part of Grid Infrastructure, which

includes, Clusterware, ASM and ACFS. So you’ll to

Install GI to use ASM

• In 11gR2 there are two options for install – GI for Standalone

Server (aka Oracle Restart) and GI for Clusterware


Q. We have a 16 TB database. I’m curious about the

number of disk groups we should use; e.g. 1 large

disk group, a couple of disk groups, or otherwise?

What about a database consolidation scenario

A. For VLDBs you will probably end up with different

storage tiers; e.g with some of our large customers

they have Tier1 (RAID10 FC), Tier2 (RAID5 FC), Tier3

(SATA), etc. Each one of these is mapped to a

diskgroup.


Q. What is the best LUN size for ASM

A. There is no best size! In most cases the storage

team will dictate based on their standardized LUN

size. The ASM admin merely has to communicate the

ASM Best Practices and application characteristics to

storage folks :

• Need equally sized / performance LUNs

• Minimum of 4 LUNs

• The capacity requirement

• The workload characteristic (random r/w, sequential r/w) &

any response time SLA

Using this info, and their standards, the storage folks should build

a nice LUN group set for you.


Q. In 11gR2 can my RDBMS and ASM instances run

different versions?

A. Yes. But since ASM is now part of GI stack, it must be

at the highest version. Keep in mind, there’s two

components of compatibility:

• Software compatibility

• Diskgroup compatibility attributes:

• compatible.asm

• compatible.rdbms

Disk Group Compatibility Example

• Need to have compatible.asm set to 11.2.0.1 for

OCR/Vote files and SPFILE in ASM

• ACFS also needs 11.2.0.1 compatible.asm

• Advance compatible.asm• ALTER DISKGROUP data

SET ATTRIBUTE

‘compatible.asm’ = ’11.2.0.1.0’


Q. Where do I run my database listener from; i.e., ASM

HOME or DB HOME?

A.

• For 11gR2, the SCAN listener is run from GI Home,

and database listener from DB HOME.

• For pre-11gR2, it is recommended to run the listener

from the ASM HOME. This is particularly important for

RAC env, since the listener is a node-level resource.

In this config, you can create additional [user] listeners

from the database homes as needed.



Backups


Q. How do I backup my ASM instance?

A. Not applicable! ASM has no files to backup


Q. When should I use RMAN and when should I use

ASMCMD copy?

A. RMAN is the recommended and most complete and

flexible method to backup and transport database files

in ASM.

ASMCMD copy is good for copying single files

• Supports all Oracle file types

• In some cases. can be used to instantiate a Data Guard

environment

• Does not update the controlfile

• Does not create OMF files

ASMCMD Copy

ASMCMD> ls

+fra/dumpsets/expdp_5_5.dat

ASMCMD> cp expdp_5_5.dat [email protected]:+DATA/dumpsets/ex

pdp_5_5.dat

source +fra/dumpsets/expdp_5_5.dat

target +DATA/dumpsets/expdp_5_5.dat

copying file(s)...

file, +DATA/dumpsets/expdp_5_5.dat,

copy committed.



Migration


Q. We are migrating to a new storage array. How do I

move my ASM database from storage A to storage B?

A.

• Given that the new and old storage are both visible to

ASM, simply add the new disks to the ASM disk group

and drop the old disks. ASM rebalance will migrate

data online.

• For pre-11gR2, See Note 428681.1, which covers how

to move OCR/Voting disks to the new storage array


ASM_SQL> alter diskgroup DATA

drop disk

data_legacy1, data_legacy2,

data_legacy3

add disk

‘/dev/sddb1’, ‘/dev/sddc1’,

‘/dev/sddd1’;

ASM Rebalancing

• Automatic online rebalance whenever

storage configuration changes

• Only move data proportional to storage

added

• No need for manual I/O tuning

Disk Group DATA (legacy disks)

ASM Rebalancing



• Online migration to new storage

Disk Group DATA

ASM Rebalancing



• Online migration to new storage

Disk Group DATA (new disks)


Q. Is it possible to unplug an ASM disk group from one

platform and plug into a server on another platform

(for example, from Solaris to Linux)?

A. No. Cross-platform disk group migration not

supported. To move datafiles between endian-ness

platforms, you need to use XTTS, Datapump or

Streams.



ACFS

46

3rd Party FSApplication

Extending ASM to Manage ALL Data

Automatic Storage Management (ASM)

ASM Instance Managing

Oracle DB Files

ASM Cluster & Single Node File System

(ACFS)

Database RAC

ACFS Snapshot

ASM Files

Dynamic Volume Manager

Dynamic Volumes

ACFS FSOracle Bin

3rd PartyFS

DB DatafilesDB DatafilesDB DatafilesDB DatafilesDB DatafilesDB Datafiles

OCR & Voting filesOCR & Voting files

ASM

Disk

Group

47

• General purpose scalable file system

• Journaling, extent based

• Single node and cluster

• POSIX, X/OPEN file system solution for UNIX/Linux

• Windows file system solution for Windows platforms

• Accessible through NAS protocols (NFS, CIFS)

• Leverages ASM technology

• Integrated with Oracle Clusterware for cluster support

• Multi OS platform (Linux and Windows at initial release)

• Integrated with Oracle system mgt tools

• Oracle installation and configuration

• Enterprise Manager and ASM Storage mgt tools

• Native OS File System Management tools

What is ASM Cluster File System

(ACFS)

48

ACFS Features• Provides filesystem snapshots (FCOW)

• File system integrity and fast recovery via ACFS metadata

checksums and journaling.

• ACFS designed as a peer to peer, multi-node, shared file

system model and delivers coherent data access

• ACFS file system is installed as a dynamically loadable OS

VFS driver

• Starting with RHEL5, Redhat now supports a 'white list' -

kernel APIs which they commit they will not change in updates

or patches. APIs used by ACFS-ADVM were added to their

'white list'.

• Customers should be able to install an update or patch to the

kernel and our drivers should not be impacted


Q. Is ACFS supported on other platforms besides Linux

A. Yes. Other platforms are forthcoming


Q. Can ACFS be used to store database datafiles?

What about archive logs?

A. No. Currently we will not support database file to be

stored in ACFS. This is due to performance reasons.

Though you can do this in test/Q&A environments

where performance is not essential


Q. Can I use ACFS to store BFILE data or other non-

database related data

A. Yes. ACFS is POSIX compliant filesystem, and thus

can store any file data type (besides database files ☺)


Q. Will ACFS support other Data services, such

advanced cloning, replication, de-dupe, etc..

A. Yes. ACFS Replication will be introduced in the next

patchset release. Other advanced features are part of

the roadmap.



3rd Party Software


Q. How does ASM work with multipathing software?

A. It works great! Multipathing software is at a layer

lower than ASM, and thus is transparent.

You may need to adjust ASM_DISKSTRING to specify

only the path to the multipathing pseudo devices.

ASM/RDBMS

/dev/<pseudo>

Multipath driver

/dev/sda1 /dev/sdb1

Controller Controller

Cache

Disk

IO cloud


Q. Is ASM constantly rebalancing to manage “hot

spots”?

A. No…No…Nope!!

I/O Distribution

• ASM spreads file extents evenly across all disks in disk group

• Since ASM distributes extents evenly, there are no hot spots

Average IOPS per disk during OLTP workload

0

50

100

150

200

250

300

Total

Disk

IOPS

FG1: - cciss/c0d2

FG1: - cciss/c0d3

FG1: - cciss/c0d4

FG1: - cciss/c0d5

FG1: - cciss/c0d6

FG2: - cciss/c0d2

FG2: - cciss/c0d3

FG2: - cciss/c0d4

FG2: - cciss/c0d5

FG2: - cciss/c0d6

FG3: - cciss/c0d2

FG3: - cciss/c0d3

FG3: - cciss/c0d4

FG3: - cciss/c0d5

FG3: - cciss/c0d6

FG4: - cciss/c0d2

FG4: - cciss/c0d3

FG4: - cciss/c0d4

FG4: - cciss/c0d5

FG4: - cciss/c0d6

Summary:

• ASM requires very few parameters to run

• ASM based databases inherently leverage raw disk

performance

• No additional database parameters needed to support

ASM

• Mixed ASM-database version support

• Facilitates online storage changes

• RMAN recommended for backing up ASM based

databases

• Spreads I/O evenly across all disks to maximize

performance and eliminates hot spots

ASM Collateral and Content

http://www.oracle.com/technology/asm

• ASM 11g New Features

• ASM Best Practices

• ASM vendor papers

• ASM-RAC Customer Case Studies



Extra credit questions


Q. Is ASMLIB required on Linux systems and are there

any benefits to using it?

A. ASMLIB is not required to run ASM, but it is certainly

recommended.

ASMLIB has following benefits:

• Simplified disk discovery

• Persistent disk names

• Efficient use of system resources


Q. Is it possible to do rolling upgrades on ASMLIB in a

RAC configuration

A. ASMLIB is independent of Oracle Clusterware and

Oracle Database, and thus can be upgraded on its

own

Tcoug Top 10 Asm Questions

Documents

Transcript of Tcoug Top 10 Asm Questions