Lun Alignment 163801

8/6/2019 Lun Alignment 163801

http://slidepdf.com/reader/full/lun-alignment-163801 1/16

An Oracle White PaperMay 2010

Detecting and Resolving Oracle Solaris LUNAlignment Problems



Oracle White Paper —Detecting and Resolving Oracle Solaris LUN Alignment Problems

`

1 Overview ........................................................................................ 1 2 LUN Alignment Challenges With Advanced Storage Devices ......... 2

3 Detecting and Resolving LUN Alignment Problems ........................ 3 4 Summary ........................................................................................ 8 5 Appendix: DTrace Code to Detect LUN Alignment Problems ......... 9 6 For More Information .................................................................... 12



Oracle White Paper — Detecting and Resolving Oracle Solaris LUN Alignment Problems

`

1 Overview

An advanced storage system, such as Oracle’s Sun Storage 7000 Unified Storage System, is a

traditional SCSI-accessed logical unit (LUN) to client systems. Although these devices may be

accessed with legacy 512B I/O transfers, internally these devices may be managed with

variable block sizes that are larger than the standard 512B sector size available on commercial

hard disk drives.

In practice, these advanced storage devices process data most effectively when the operating

system I/O request is aligned with the block size of the LUN presented by the storage device.

When I/O is not aligned to the block size of the LUN, response time may increase and

throughput may decrease compared to the aligned case.

To ensure optimal system performance with these types of storage devices, Oracle Solaris

provides the system administrator with advanced Oracle Solaris DTrace monitoring to detect

LUN alignment problems and the Oracle Solaris volume table of contents (VTOC) or EFI disklabel utility format to resolve LUN alignment problems.

This paper shows how to use DTrace to detect LUN alignment problems on an active system

and how to resolve those LUN alignment problems by tuning the EFI disk label.




`

2 LUN Alignment Challenges With Advanced Storage Devices

Legacy disk drives accessed with the SCSI protocol present a logical block architecture (LBA) to theoperating system that has historically relied on a 512B sector size for data storage. The defaultbehavior of operating system tools for formatting and accessing disk drives is to use 512B alignedrecords to store metadata about the drive geometry and as well as end-user data.

Although this process works well for simple disk drives, it can produce suboptimal results forintegrated storage systems that internally process data in larger allocation units. For example, if a disk label takes up the first 33 sectors of the LUN presented by the storage system and then stores userdata starting at sector 34, then an 8kB access from sector 34 through sector 49 will require the storagesystem to read two 8kB allocation units, sectors 32 – 47 and sectors 48 – 63, to process the requesteddata. Such misaligned I/O conditions reduce the efficiency with which the storage system can processI/O and consequently may increase response time at a fixed throughput or reduce throughput at afixed response time.

Illustration 1 shows an example of the I/O response time and throughput characteristics associated with misaligned and aligned I/O on the same storage system.

To resolve I/O alignment challenges presented by storage devices with allocation unit sizes greaterthan 512B requires the system administrator to begin storing user data on the first storage deviceallocation unit available after the disk label or VTOC. This action can be accomplished by configuring the operating system to leave enough empty space between the end of the disk label and the first byteof user data to ensure that the first byte of user data is written to the first byte of an allocation unit onthe storage device.

For example, the Oracle Solaris EFI disk label is stored on the first 33 sectors of the LUN presentedby a storage device, and the default partition table layout is to store user data starting at sector 34 or17kB into the LUN.

To align I/O on an 8kB allocation unit, the starting sector of the data partition may be advanced to

sector 48 (24kB), sector 64 (32kB), or any multiple of 16 sectors beyond sector 48. Larger allocationunit sizes can be accommodated by choosing an appropriately larger starting sector: 128 for 64kB or256 for 128kB.

Illustration 3 shows an example of how to use the Oracle Solaris format -e command to performthis alignment.




`

3 Detecting and Resolving LUN Alignment Problems

Detecting LUN alignment problems challenges the system administrator because of the complexity of the I/O processing stack between the application executing the I/O system call and the LUNprocessing the I/O. Complications include the presence of file systems and volume managers thatmay change the physical characteristics of the application-initiated I/O as well as knowledge of thepartition table on the LUN.

To simplify detection of LUN alignment problems in these complex environments, Oracle SolarisDTrace provides the function boundary testing (fbt) provider to track I/O operations at the SCSIdriver layer (sd).

The code shown in the Appendix is an example of how to monitor a running system to verify 8kBalignment of I/O at the LUN level. To provide a concrete example of the benefit of maintaining I/Oalignment between the host-side data access and the allocation unit size of the storage device, thissection shows an example of client-reported I/O performance measurements and correlated DTraceoutput from misaligned and aligned cases. It also shows an example of how to control data alignmentusing the Oracle Solaris format utility to tune an EFI disk label.

Illustration 1 shows storage system performance for 8kB random read operations using a default EFIdisk label that leads to client-side I/O being executed on 9kB boundaries. The workload shown in thetest case is 64 outstanding 8kB read requests, and performance is quantified by a 3.9 ms I/O responsetime at a throughput of 15,900 IOPS.

In practice, there is no obvious indication from this report that an I/O alignment problem is occurring except that I/O response time is longer and I/O throughput is lower than expected for the given

workload running on the specific storage system.

Illustration 2 shows the output of the supplied DTrace script that shows the root-cause of theperformance problem. In this example, we see that although the host is performing 8kB I/Ooperations consistent with the known allocation unit size of the LUN, the operations are executed on

9kB boundaries. Consequently, for each host-initiated I/O against the LUN the storage target needsto access 2 adjacent 8kB blocks: the first block for the first 7kB of the I/O and a second block for theremaining 1kB. This leads to inefficient processing of data, increases in response time, anddegradation in throughput.




`

Proper configuration of the starting location for data storage on the LUN can resolve the I/Oalignment issue. On the Oracle Solaris operating system, format is the tool for displaying, creating,and modifying partitions and labeling disks.

Illustration 3 shows the format output of the first available sector of an EFI disk label in OracleSolaris is at 34, which is 2 blocks beyond 32 kB we are using for this example. Illustration 3 alsoshows the steps to correct the 32kB block offset of the EFI label using format in expert mode withthe -e option.

The DTrace output shown in Illustration 4 verifies that correct I/O alignment is preserved betweenthe application and the storage device. In this example, all 8kB read I/O operations are conducted onmultiples of 8kB on the LUN presented by the storage system.

As a result of maintaining I/O alignment, system performance improves, as shown in Illustration 5.In this example, I/O response time drops to 3.3 ms and throughput increases to 20,200 IOPS. Thisshows a 20% gain in system performance for the tested workload. In practice, performance gains may be greater or less depending on the specific workload and the hardware configuration supporting the

workload.

Illustration 1: I/O Using Misaligned EFI Label




`

Illustration 2: DTrace Output Using Misaligned EFI Label




`

Illustration 3: Format Command




`

Illustration 4: DTrace Output Using Aligned EFI Label

Illustration 5: Improved System Performance




`

4 Summary

Detecting LUN alignment issues with advance storage devices can be achieved fairly quickly and easily using the built-in Oracle Solaris tools to monitor I/O with iostat and DTrace. Once detected,alignment issues can quickly be corrected by calculating the correct beginning block sector based onthe I/O block size and using the format command to write a proper EFI label.

The result of this tuning is more efficient data processing by the underlying storage system, areduction in I/O response time, and an increase in I/O throughput compared to untuned systems. This best practice should be followed by administrators seeking the most efficient use of storageresources.




`

5 Appendix: DTrace Code to Detect LUN Alignment Problems

#!/usr/sbin/dtrace -qs

BEGIN

{

start = timestamp;

}

int issued;

int misaligned;

int misalignedBy;

int nonmult;

int tot_issued;

int tot_misaligned;

int tot_misalignedBy;

int tot_nonmult;

int tot_elapsed;

struct buf *bp;

struct sd_xbuf *xp;

fbt:sd:sd_core_iostart:entry

{

issued++;

bp = (struct buf *) arg2;

xp = (struct sd_xbuf *)((bp)->b_private);

misaligned += (xp->xb_blkno % 16) ? 1 : 0;

misalignedBy += (xp->xb_blkno % 16 );

@misalignedByHist = quantize(xp->xb_blkno % 16 );

nonmult += (bp->b_bcount % 8192) ? 1 : 0;




`

@a[bp->b_bcount] = count();

}

END

{

elapsed = timestamp - start;

tot_elapsed += elapsed;

tot_issued += issued;

tot_misaligned += misaligned;

tot_nonmult += nonmult;

printf( "\n IO size Count\n" );

printa( @a);

printf( "\n IO misalignment Count\n" );

printa( @misalignedByHist);

printf( "%d.%09d seconds elapsed\n\n",

tot_elapsed / 1000000000,

tot_elapsed % 1000000000

);

printf( "%8d IOs issued\n%8d IOs misaligned, %8d block offset %8d byte offset\n",

issued,

misaligned,

misalignedBy/issued,

512*misalignedBy/issued

);

printf( "%8d IOs non-multiple of 8KB\n", tot_nonmult );

printf( "%8d Percent non-8k IOs\n", (tot_nonmult * 100 / tot_issued) );

}




`

tick-5sec

{

elapsed = timestamp - start;

printf( "%d.%09d seconds elapsed\n\n", elapsed/1000000000, elapsed%1000000000 );

printf( "%8d IOs issued\n%8d IOs misaligned, %8d block offset %8d byte offset\n",

issued,

misaligned,

misalignedBy / issued,

512 * misalignedBy / issued

);

printf( "%8d IOs non-multiple of 8KB\n", nonmult );

printf( "%8d Percent non-8k IOs\n\n", ( nonmult * 100 / issued ) );

printa( @misalignedByHist );

tot_elapsed += elapsed;

tot_issued += issued;

tot_misaligned += misaligned;

tot_nonmult += nonmult;

tot_misalignedBy += misalignedBy;

start = timestamp;

issued = 0;

misaligned = 0;

misalignedBy = 0;

nonmult = 0;

}




`

6 For More Information

Here are additional resources.

Oracle Solaris resources:

BigAdmin web applications and device lists:

o Hardware Compatibility Lists for Solaris OS:http://www.sun.com/bigadmin/hcl/data/sol/

o Hardware Certification Test Suite: http://www.sun.com/bigadmin/hcl/hcts/index.jsp

o Solaris 10 Applications Library: http://www.sun.com/bigadmin/apps/

o Sun Device Detection Tool: http://www.sun.com/bigadmin/hcl/hcts/device_detect.jsp

o Installation Check Tool: http://www.sun.com/bigadmin/hcl/hcts/install_check.jsp

o Device and Third-party Solaris Device Driver Reference Lists:http://www.sun.com/bigadmin/drivers

BigAdmin Oracle Solaris technology resource centers:

o Solaris Information Center: http://www.sun.com/bigadmin/hubs/documentation

o Solaris Patching Center: http://www.sun.com/bigadmin/patches/solaris/index.jsp

o Solaris 10 Upgrade Resources for System Administrators:http://www.sun.com/bigadmin/topics/upgrade/

o Solaris Containers (Zones): http://www.sun.com/bigadmin/content/zones/

o Logical Domains (LDoms): http://www.sun.com/bigadmin/hubs/ldoms/

o DTrace: http://www.sun.com/bigadmin/content/dtrace/index.jsp o ZFS: http://www.sun.com/bigadmin/topics/zfs/

o Predictive Self-Healing: http://www.sun.com/bigadmin/content/selfheal/

o Solaris 8 Vintage Support: http://www.sun.com/bigadmin/topics/vintagepatch/

BigAdmin Oracle Solaris resource collections (which include community submissions):

o Solaris resource collection: http://www.sun.com/bigadmin/collections/solaris.jsp

o Solaris 10 resource collection: http://www.sun.com/bigadmin/collections/solaris10.jsp



o Solaris on x86 resource collection:http://www.sun.com/bigadmin/collections/solarisx86.jsp

Discussions, such as the Solaris OS forums: http://forums.sun.com/category.jspa?categoryID=65

BigAdmin Operating System wiki page:http://wikis.sun.com/display/BigAdmin/Operating+System

http://www.sun.com/bigadmin/hcl/data/sol/


http://www.sun.com/bigadmin/hcl/hcts/index.jsp



http://www.sun.com/bigadmin/apps/



http://www.sun.com/bigadmin/hcl/hcts/device_detect.jsp



http://www.sun.com/bigadmin/hcl/hcts/install_check.jsp



http://www.sun.com/bigadmin/drivers


http://www.sun.com/bigadmin/hubs/documentation



http://www.sun.com/bigadmin/patches/solaris/index.jsp



http://www.sun.com/bigadmin/topics/upgrade/


http://www.sun.com/bigadmin/content/zones/



http://www.sun.com/bigadmin/hubs/ldoms/



http://www.sun.com/bigadmin/content/dtrace/index.jsp



http://www.sun.com/bigadmin/topics/zfs/



http://www.sun.com/bigadmin/content/selfheal/



http://www.sun.com/bigadmin/topics/vintagepatch/



http://www.sun.com/bigadmin/collections/solaris.jsp



http://www.sun.com/bigadmin/collections/solaris10.jsp









http://www.sun.com/bigadmin/collections/solarisx86.jsp


http://forums.sun.com/category.jspa?categoryID=65



http://wikis.sun.com/display/BigAdmin/Operating+System



























`

General links:

Sun download site: http://www.sun.com/download/

Oracle University web site: http://www.sun.com/training/

Discussions, such as Sun forums ( http://forums.sun.com/index.jspa ) and the BigAdminDiscussions collection ( http://www.sun.com/bigadmin/discussions/ )

Sun product documentation at http://docs.sun.com and the Sun Documentation Center( http://www.sun.com/documentation/ )

Sun wikis, such as the Sun BluePrints wiki( http://wikis.sun.com/display/BluePrints/Main )and theBigAdmin wiki( http://wikis.sun.com/display/BigAdmin/Home )

Support:

o Sun resources:

Register your gear: https://inventory.sun.com/inventory/ Sun Services: http://www.sun.com/service/index.jsp

SunSolve Online: http://sunsolve.sun.com

Community system administration experts:http://www.sun.com/bigadmin/content/communityexperts/

http://www.sun.com/download/



http://www.sun.com/training/



http://forums.sun.com/index.jspa



http://www.sun.com/bigadmin/discussions/



http://docs.sun.com/



http://www.sun.com/documentation/



http://wikis.sun.com/display/BluePrints/Main



http://wikis.sun.com/display/BigAdmin/Home



https://inventory.sun.com/inventory/



http://www.sun.com/service/index.jsp



http://sunsolve.sun.com/



http://www.sun.com/bigadmin/content/communityexperts/
















Detecting and Resolving Oracle Solaris LUN

Alignment Problems

May 2010

Author: Art Larking, AIE

Contributing Authors: Jeff Wright, AIE

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000Fax: +1.650.506.7200

oracle.com

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

This document is provided for information purposes only and the contents hereof are subject to change without notice. This

document is not warranted to be error-free, nor subject to any other warranties or conditi ons, whether expressed orally or implied in

law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any

liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This

document may not be reproduced or transmitted in any form or by anymeans, electronic or mechanical, for any purpose, without our prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective

owners.

AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices.

Intel and Intel Xeon are t rademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license

and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/OpenCompany, Ltd. 0110

Lun Alignment 163801

Documents

Transcript of Lun Alignment 163801