Discovering VIO relationships for EMC arrays

discovering vio relationships for emc arrays

EMC Proven Professional Knowledge Sharing 2011

Ewa Nowatkowska-GłowaczSenior Solutions [email protected]


Table of contents

Introduction ............................................................................................................... 3

Inquiry utility .............................................................................................................. 5

Regular hdisk ......................................................................................................... 6

PowerPath device (hdiskpower) ............................................................................. 6

Logical volume ..................................................................................................... 12

Summary ................................................................................................................. 21

Scripts and EMCgrab .............................................................................................. 23

Bibliography ............................................................................................................ 26

Appendix ................................................................................................................. 27

Disclaimer: The views, processes or methodologies published in this article are

those of the author. They do not necessarily reflect EMC Corporation’s views,

processes, or methodologies.


Introduction

This article describes a method for discovering VIO relationships for IBM p-based

systems connected to EMC Symmetrix® and CLARiiON® arrays. The Virtual I/O

Server (VIOS) provides virtual storage to client logical partitions by allowing a

physical adapter with attached disks on the VIOS partition to be shared by one or

more Virtual I/O Client (VIOC) partitions.

This type of configuration with use of VIOS/VIOC relationship becomes increasingly

popular among enterprise customers where IBM servers are installed. In large

migration projects, discovering and documenting relationships between servers and

their clients is a time consuming task.


Currently, there are no tools available which could perform (from the client point of

view) fully automated volume or device mapping between servers and clients. The

identification method shown in this article could be useful for verifying current

configuration for migration purposes or can be used for daily reporting. In this article,

a real implementation project will be mentioned where the EMC team worked out

methodology and scripts needed to discover current relationships in order to perform

migration of VIOS resources between EMC arrays (technology refresh). Along with

VIO servers resource migration, VIO clients potentially involved in any given

migration were also indicated. Configuration and commands’ outputs shown in this

article are based on documentation of a real environment and helps to understand

VIO configuration challenges. Sample scripts for reports automation for large

configuration will be shown at the end of this article.

The main idea of how to discover VIOC/VIOS relationship during a migration project

came from reading EMC document: emc157385. In this document, methodology

overview and individual steps are described and illustrated by CLI commands.

However, during migration projects, other possible configurations were discovered

which are also worth mentioning and which inspired the author to write this article.


Inquiry utility

Inquiry utility is required in this methodology to properly identify type of VIOC devices.

This utility can be downloaded from ftp://ftp.emc.com/pub/elab/inquiry/.

Here are examples of possible types of devices which could be identified in

miscellaneous configuration. The first five lines are typically directly connected

devices (both internal and external); the next three (line 6, 7, and 8) represent

devices seen by the VIO client.

---------------------------------------------------------------------

DEVICE :VEND :PROD :REV :SER NUM :CAP(kb)

---------------------------------------------------------------------

1 /dev/rhdisk6 :EMC :SYMMETRIX :5771 :8600cf3080 :3840000

2 /dev/rhdiskpower0 :EMC :SYMMETRIX :5771 :8600722080 :44193600

3 /dev/rhdisk1 :IBM :ST3300655LC :C806 :003185C :29296875

4 /dev/rhdisk2 :IBM H0:ST373455LC :C806 :0002F4A0 :71687000

5 /dev/rhdiskpower73 :DGC :RAID 5 :0326 :490000EC :1048576

6 /dev/rhdisk0 :AIX :VDASD : :hdisk66 :35354880

7 /dev/rhdisk4 :AIX :VDASD : :hdiskpow :35354880

8 /dev/rhdisk7 :AIX :VDASD : :001RT06H :13631488

Configuration shown in lines 6, 7, 8 will be described in detail later in this section.

In order to fully recognize the device behind VIO mapping, inquiry with option “-page0”

is needed against every client device involved in VIO configuration. In other words, if

client has three mapped devices shown above in lines 6, 7 and 8, inquiry should

query all these devices in the following syntax:

# inq.aix64_51 –r /dev/rhdisk0 –page0 (line 6)



ftp://ftp.emc.com/pub/elab/inquiry/


Regular hdisk

This is the simplest case which is described in detail in EMC document emc157385.

Inquiry utility in column “SER NUM” shows hdisk instance (hdisk66) of VIO server. Using

inquiry on VIO server we can discover more information about this disk. Detailed inquiry

(with option “-page0”) can also be used to find out more details about hdisk66.

---------------------------------------------------------------------


---------------------------------------------------------------------

/dev/rhdisk0 :AIX :VDASD : :hdisk66 :35354880

PowerPath device (hdiskpower)

Line 7 informs us that this is configuration where PowerPath® is installed on VIO server.

This type of configuration could be described by inquiry output shown below.

---------------------------------------------------------------------


---------------------------------------------------------------------

/dev/rhdisk4 :AIX :VDASD : :hdiskpow :35354880

Output from inquiry utility is not very helpful in this case because it doesn’t show full

hdiskpower instance; only 8 characters are shown (e.g. instance hdiskpower224 needs

13 characters to be a fully recognized device). Device hdiskpower is a pseudo device

created by EMC PowerPath software and can be EMC Symmetrix® symdev or EMC

CLARiiON® LUN. On the next two pages there are examples of outputs from inquiry

(inq.aix64_51 –r /dev/rhdiskX –page0) for devices from these two types of arrays.


Here is an example for a CLARiiON device. For successful match devices between VIOS

and VIOC, unique Clariion LUN’s UID is used. This UID is visible in powermt display

output as well as inquiry for specific hdisk. LUN UID is shown in lines which start with:

“0060”, “0070” and “0080”.


This is an example for Symmetrix symdev. In inquiry output (line “0060”) there is information about array serial number and symdev.


In dual Virtual I/O Servers configuration, virtual SCSI can be configured redundantly

in order to provide continuous access when a Virtual I/O Server has to be taken

offline for planned outages (software updates as a example). This configuration

allows one Virtual I/O Server to be rebooted without affecting the virtual I/O clients. A

simple picture showing this type of configuration is shown here.

The volume (symdev 124F) is located on an external storage subsystem (EMC

Symmetrix DMX4, serial number ends with “55”) and is assigned to both Virtual I/O

Servers (vios1 & vios2). The client partition (vioc) sees the disk through two paths

using MPIO (lspath verifies this configuration). Each of the paths is going through a

different Virtual I/O server. Inquiry doesn’t show dual VIOS configuration.

inq.aix64_51 -no_dots (VIOC)

---------------------------------------------------------------------


---------------------------------------------------------------------

/dev/rhdisk13 :AIX :VDASD : :hdiskpow : 13631488


From inquiry, we know that serial number of arrays ends with 55 and symdev number

is 124F. In order to check if there is dual VIOS configuration, lspath can be used to

verify that there are two paths for this hdisk13.

lspath (VIOC)

Enabled hdisk13 vscsi0



Uname command with option “-L” can be used to identify and verify VIO client

partition number.

uname –L (VIOC)

5 vioc


Logical volume

In inquiry output shown below there are four lines showing instances when logical

volume is used by VIO servers to map hdisk to the client. This is a quite complicated

configuration which also involves dual VIO servers and is shown in the picture on the

following page. Here is part of inquiry output for hdisks for which “SER NUM” column

is the same. Again, based on this output there is not enough information to recognize

the device (logical volume in this case).

inq.aix64_51 -no_dots ---------------------------------------------------------------------


---------------------------------------------------------------------

/dev/rhdisk0 :AIX :VDASD : :001MW06C : 2097152




uname –L 12 vioc

Client partition number is “12” (this is 0x0c in hexadecimal notation shown by lsmap

command in Client Partition ID column).

Every hdisk has only one available path but what was discovered later (outputs from

adequate LVM commands are not shown in this document), logical volumes created

on hdisk0 and hdisk5 were also mirrored at LVM level.

lspath Enabled hdisk0 vscsi0





lspv -l (VIOS1)

--------------------------------------------------------

lspv -l hdiskpower2 :-

hdiskpower2:

LV NAME LPs PPs DISTRIBUTION MOUNT POINT

001MW06CWO_lv01 20 20 00..00..00..10..10 N/A

001MW06CXP_lv01 13 13 11..00..00..00..02 N/A

001MW06CUP_lv01 1 1 01..00..00..00..00 N/A

001MW06CQP_lv02 24 24 00..12..11..01..00 N/A

--------------------------------------------------------


hdiskpower4:


001MW06CZP_lv01 16 16 00..00..00..00..16 N/A

001MW06CXP_lv01 3 3 03..00..00..00..00 N/A

001MW06CSP_lv01 72 72 00..23..23..23..03 N/A

--------------------------------------------------------


hdiskpower61:


001MW06CYP_lv02 24 24 00..03..21..00..00 N/A

001MW06CZP_lv02 10 10 02..00..05..03..00 N/A

001MW06CXP_lv02 24 24 24..00..00..00..00 N/A

001MW06CVP_lv01 3 3 00..03..00..00..00 N/A

001MW06CUP_lv01 39 39 00..00..00..24..15 N/A


lspv –l (VIOS2)

--------------------------------------------------------


hdiskpower32:


001MW06CYP_lv01 16 16 00..00..07..09..00 N/A

001MW06CVP_lv02 26 26 12..00..00..02..12 N/A

001MW06CXP_lv01 16 16 00..12..04..00..00 N/A

--------------------------------------------------------


hdiskpower62:


001MW06CXP_lv02 24 24 24..00..00..00..00 N/A

001MW06CYP_lv02 24 24 00..03..21..00..00 N/A

001MW06CZP_lv02 10 10 02..00..05..03..00 N/A

001MW06CVP_lv01 3 3 00..03..00..00..00 N/A

001MW06CUP_lv01 39 39 00..00..00..24..15 N/A


As it is shown in inquiry output on the previous page, there is additional information included in lines “0060-0080” which can be useful to double check which VIO server (one of two) mapped hdisk5 to the client.

It is worth mentioning that outputs from above commands are collected by EMCgrab

- “odmget_CuAt.txt” file in “host” directory.


Summary

The entire methodology which allows us to discover what physical devices are

behind hdisk seen on client side can be described in the following way:

First, use of “inquiry –no_dots” command to discover if any devices are

mapped from VIOS.

For every device which could potentialny be mapped by VIOS, inquiry with

option “-page0” should be invoked.

Output from “inquiry –page0” should be parsed in search of characteristic

strings from specific lines.

On the next page, “inquiry –page0” outputs from three configuration types described

earlier in this document are presented together in order to easily compare them.


Based on outputs from inquiry command (-page0) it is possible to discover and

differentiate VIOC devices:

line “0090:” can be used to verify if this is CLARiiON LUN

(substr($18,1,5)==”GCfcp”)

o if this is a CLARiiON device, concatenate strings in line “0060,

0070, 0080” in order to create LUN UID

o if this is a Symmetrix device, substr($18,5,2) gives array serial

number, substr($18,7,4) gives symdev


Scripts and EMCgrab

If the customer’s configuration consists of many VIO servers and clients, it is worth

considering the use of scripts which can help us to put devices in a given order and

match them between servers and clients. Here are a few examples of Shell scripts

(or procedures inside one script) which could automate this matching (real, used by

EMC script during migration project was much more complicated).

There is also a lot of information about server and client configuration available in the

form of files collected by EMCgrab. However, to fully match the client’s hdisk with

server hdiskpower (as an example), verification on the client side is needed (by using

“inquiry –page0” command) in order to discover this relationship.

Shell script vioc.sh shown in the Appendix identifies hdiskpower devices (both

Symmetrix and CLARiiON) shown in output from “inq.aix64_51 -no_dots” command

and for every identified hdiskpower device checks “backing device” using inquiry

utility with option “page0”. Here is an output from this script:

vioc|partition|size in GB|array serial|symdev or LUN UID|type

vioc|5|hdisk13|102.539|55|124F|SYMMETRIX

vioc|6|hdisk62|5||600601605A642200C4410D05FD9ADF11|CLARiiON

Please note that the fifth column is empty if the “backing device” is a CLARiiON array.

Next script “pp_config.sh” based on EMCgrab (file “powermt_display_dev=all.txt” in

“pp” directory) from VIO servers identifies all hdiskpower instances creating the

following output:

server|hdispower instance|array|symdev|

vios1|hdiskpower224|00029xxxxx55|124F|

vios2|hdiskpower224|00029xxxxx55|124F|

vios2|hdiskpower64|CK200xxxxxx25|LUN 77|

This script can be easly modified to show also LUN UID for CLARiiON LUNs.

Based on another collected by EMCgrab “_usr_ios_cli_ioscli_lsmap_-all.txt” file in

“host” directory, script check_lsmap.sh helps to sort what devices are presented to

which client partition.

vios|vioc|partition|vtd|backing device

vios1|vhost0|0x00000005|01DT100A5P_mp11|hdiskpower224

vios2|vhost0|0x00000005|01DT100A5P_mp11|hdiskpower224


Below is a simple diagram of how easily collection of “inquiry –page0” can be

automated. Instead of executing this command manually on every VIOC, a short

script can be executed sequentially on every VIOC and results transferred over open

connection between server and client. There are two examples below, one was

written in Perl and the other is Windows Batch.

#!perl

#

($min, $hour, $day, $month, $year)=(localtime) [1,2,3,4,5];

$dir="logs";

$logfile=sprintf("%04d.%02d.%02d_%02d%02d_log.txt",$year+1900,$month+1,$day,

$hour,$min);

$errorfile=sprintf("%04d.%02d.%02d_%02d%02d_error.txt",$year+1900,$month+1,$

day,$hour,$min);

print "Logfile is: $logfile\n";

open( STDOUT, "> $dir\\$logfile" );

open( STDERR, "> $dir\\$errorfile" );

use Socket;

$port=XXXXX;

$net="192.168.1.";

foreach $ip ($net.1,$net.2)

{

print "$ip \n";

socket(SERVER, PF_INET, SOCK_STREAM, getprotobyname('tcp'));

$te=sockaddr_in($port,inet_aton($ip));

connect(SERVER,$te)

or die "Cannot connect to $ip:$port \n";

while (1)

{

$buffer=<SERVER>;

if ($buffer ne "" ) {print "$buffer\n";}

else {last; }

}

close(SERVER);

}

close (STDOUT);

close (STDERR);

@ECHO OFF

SET output=O:\output.csv

SET log=O:\log.txt

SET port=XXXXX

ECHO Servers

ECHO -------

ECHO OFF > %output%

ECHO OFF > %log%

FOR /f "tokens=* delims=" %%h in (vioc.txt) DO (

ECHO %%h

ECHO %%h >> %log%

O:\plink1 -telnet %%h -P %port% >> %output% 2>> %log% )

1 plink is available is a part of famous PUTTY download.


Bibliography

The following documents were used during writing this document and can be read to

fully understand the topic:

EMC Host Connectivity Guide for AIX (chapter “Virtual I/O Server”)

Primus solutions: emc157385, emc156902, emc110159, emc109158

Release Notes for EMC Solution Enabler 7.0 contains the following information:

VMware and AIX VIO virtualized clients support has been added to Solutions Enabler.

The feature allows symrslv to resolve array devices behind the virtual disk. The host,

with HBAs and Symmetrix array, creates virtualized environments, or guests. The

host pools together disks and then can carve out virtual disks for the guests. Now,

Solutions Enabler can identify Symmetrix devices behind the virtual disk.

During the project, new Solution Enabler capabilities described above weren’t tested

(project started before release of this version). Potentially, exploring this way of

discovering VIO client devices could make the identification process even easier than

described in this article.


EMC believes the information in this publication is accurate as of its publication

date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC

CORPORATION MAKES NO RESPRESENTATIONS OR WARRANTIES OF ANY

KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND

SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR

FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication

requires an applicable software license.

Discovering VIO relationships for EMC arrays

Documents

Transcript of Discovering VIO relationships for EMC arrays