HP Virtual Connect With ISCSI Cookbook

5/21/2018 HP Virtual Connect With ISCSI Cookbook

1/182

HP Virtual Connect with iSCSI CookbookFebruary 2012, 3rdEdition

Technical white paper

Table of contentsPurpose ..............................................................................................................................................

Introduction .........................................................................................................................................

System requirements ............................................................................................................................. Virtual Connect with iSCSI Summary support ....................................................................................... Firmware and Software Support......................................................................................................... iSCSI Target support ......................................................................................................................... Unsupported configuration ................................................................................................................

Networking recommendations ............................................................................................................... Network considerations .................................................................................................................... Keep it simple and short ...................................................................................................................

Storage 'vendors' recommendations ................................................................................................... Flow Control ................................................................................................................................

Jumbo Frames (optional recommendation) ..................................................................................... 1iSCSI multipathing solutions ............................................................................................................. 1

Virtual Connect network scenarios ....................................................................................................... 1Scenario 1: iSCSI network physically separated ................................................................................ 1

Defining two iSCSI networks vNet ................................................................................................ 2Scenario 2: iSCSI network logically separated .................................................................................. 2

Defining a first Shared Uplink Set (VLAN-trunk-1) ............................................................................ 2Defining a second Shared Uplink Set (VLAN-trunk-2) ....................................................................... 2

Scenario 3: Direct-attached iSCSI Storage System .............................................................................. 2Direct-attached limitations ............................................................................................................ 2Scenario 3-A: Direct-attached iSCSI device with out-of-band management ......................................... 2Scenario 3-B: Direct-attached iSCSI device with in-band management .............................................. 2

Virtual Connect network configuration........................................................................................... 3Connecting the direct-attached iSCSI SAN to the VC Domain .......................................................... 3Preparing the network settings of the storage system ....................................................................... 3Configuring the Storage Management Server ................................................................................ 3Configuring the network interface bonds on the Storage System....................................................... 3Configuring the iSCSI Host .......................................................................................................... 3

Accelerated iSCSI .............................................................................................................................. 3Enabling Accelerated iSCSI on the server using Virtual Connect (also known as iSCSI Offload) .............. 4

Accelerated iSCSI with Microsoft Windows Server ............................................................................. 4Installing Emulex OneCommand Manager ..................................................................................... 4Configuring the IP addresses of the iSCSI ports .............................................................................. 4

Installing the Microsoft iSCSI Initiator ............................................................................................ 4Installing Microsoft MPIO ............................................................................................................ 5Installing the Device Specific Module (DSM) .................................................................................. 5Connecting volumes with MPIO .................................................................................................... 5Using the Microsoft iSCSI Software Initiator in conjunction with Accelerated iSCSI support .................. 6

Accelerated iSCSI with VMware ESX 4.1 .......................................................................................... 6Installing the NC551 / NC553 iSCSI drivers ................................................................................ 6Installing Emulex OneCommand Manager ..................................................................................... 6Configuring the IP addresses of the iSCSI ports .............................................................................. 6Configuring the iSCSI Volumes ..................................................................................................... 7Using the VMware iSCSI Software Initiator in conjunction with Accelerated iSCSI support................... 7

For more information ...................................................................................................................... 8

Boot from Accelerated iSCSI ............................................................................................................... 8


2/182

Creating an Accelerated iSCSI Boot Virtual Connect profile ................................................................ 8iSCSI Boot Image Creation Step-by-Step Guides ................................................................................ 9

Microsoft Windows 2008 R2 ...................................................................................................... 9VMware vSphere 5 .................................................................................................................... 9VMware vSphere 4.1 ............................................................................................................... 10Red Hat Enterprise Linux 5 Update 4 .......................................................................................... 11Suse Linux Enterprise Server 11 .................................................................................................. 11

Troubleshooting ............................................................................................................................... 12Emulex iSCSI Initiator BIOS Utility .................................................................................................. 12

Configuration checking ............................................................................................................. 12Emulex OneCommand Manager (OCM) ......................................................................................... 13Problems found with OneCommand Manager ................................................................................. 13Problems found during iSCSI Boot .................................................................................................. 13PXE booting problems ................................................................................................................... 13iSCSI boot install problems with Windows 2003 ............................................................................. 14

VCEM issues with Accelerated iSCSI Boot ....................................................................................... 14iSCSI issues with HP P4000 products ............................................................................................. 14

Appendix 1- iSCSI Boot Parameters ................................................................................................... 14Mandatory iSCSI Boot Parameters entries ....................................................................................... 14

iSCSI Initiator (iSCSI Boot Configuration) ..................................................................................... 14iSCSI Target (iSCSI Boot Configuration) ...................................................................................... 14

Initiator Network Configuration .................................................................................................. 14Optional iSCSI Boot Parameters entries .......................................................................................... 14

Secondary iSCSI Target Address ................................................................................................ 14Security enhancement using an authentication ............................................................................. 14

Appendix 2 - Dynamic configuration of the iSCSI Boot Parameters ........................................................ 15Windows 2008 DHCP server configuration .................................................................................... 15Linux DHCP server configuration .................................................................................................... 16Format of DHCP option 43 for NC551/NC553 CNA...................................................................... 16Examples .................................................................................................................................... 16

Appendix 3- How to monitor an iSCSI Network? ................................................................................. 16Monitoring Disk Throughput on the iSCSI Storage System ................................................................. 16Monitoring Network and Disk Throughput on the iSCSI Host ............................................................. 16

VMware vSphere ..................................................................................................................... 16Microsoft Windows Resource Monitor ......................................................................................... 16

Analyzing Network information from the Virtual Connect interface ..................................................... 17Analyzing Virtual Connect Network performance............................................................................. 17Wireshark ................................................................................................................................... 17

Software iSCSI analysis ............................................................................................................. 17iSCSI analysis for Accelerated iSCSI adapters ............................................................................. 17

For more information ........................................................................................................................ 18


3/182

Purpose 3

Purpose

This Virtual Connect iSCSI Cookbook provides users of Virtual Connect with a better understanding of theconcepts and steps required when using iSCSI with Virtual Connect Flex-10 or FlexFabric components. Thisdocument will help users answer some of the typical questions on iSCSI: What are the network

considerations to properly build an iSCSI network? What are the components supported by HP? How can Itroubleshoot my iSCSI environment?In addition, this document describes some typical iSCSI scenarios to provide the reader with some validexamples of how Virtual Connect Flex-10 or FlexFabric with iSCSI could be deployed within theirenvironments.Tips and some troubleshooting information for iSCSI boot and install are also provided.Detailed information regarding Emulex requirements is subject to change, and readers should always referto the documentation from the providers.

Introduction

The iSCSI standard implements the SCSI protocol over a TCP/IP network. While iSCSI can be implementedover any TCP/IP network, the most common implementation is over-1- and 10-Gigabit Ethernet (GbE). TheiSCSI protocol transports block-level storage requests over TCP connections. Using the iSCSI protocol,systems can connect to remote storage and use it as a physical disk, although the remote storage provideror target may actually be providing virtual physical disks. iSCSI serves the same purpose as Fibre Channelin building SANs, but iSCSI avoids the cost, complexity, and compatibility issues associated with FibreChannel SANs.Because iSCSI is a TCP/IP implementation, it is ideal for new field deployments where no FC SANinfrastructure exists. An iSCSI SAN is typically comprised of software or hardware initiators on the hostconnected to an isolated Ethernet network and some number of storage resources (targets). While the targetis usually a hard drive enclosure or another computer, it can also be any other storage device supportingthe iSCSI protocol, such as a tape drive. The iSCSI stack at both ends of the path is used to encapsulate

SCSI block commands into Ethernet Packets for transmission over IP networksiSCSI boot allows the c-class Blade to boot from a remote operating system image located on an Ethernetbased storage network.In addition, accelerated iSCSI enables the Converged Network Adapter on a ProLiant Blade server to runaccelerated iSCSI; it offloads the iSCSI function to the CNA rather than taxing the CPU of the server.


4/182

System requirements 4

System requirements

With Virtual Connect technology, only the following components support iSCSI Boot and Accelerated iSCSI:

Integrated NC553i Dual Port FlexFabric 10Gb Adapter (Intel-based BladeSystem G7 servers) Integrated NC551i Dual Port FlexFabric 10Gb Adapter (AMD-based BladeSystem G7 servers) HP NC551m Dual Port FlexFabric 10Gb Converged Network Adapter HP NC553m Dual Port FlexFabric 10Gb Converged Network Adapter HP Virtual Connect FlexFabric 10Gb/24-port Module HP Virtual Connect Flex-10 10Gb Ethernet Module

NOTE:iSCSI Boot is available as well on Virtual Connect with the QLogic QMH4062

1GbE iSCSI Adapter, but with some restrictions. The QMH4062 iSCSI settingscannot be managed by a Virtual Connect profile but they can be set manuallythrough the Qlogic Bios (CTRL+Q during Power-on Self-Test). The constraints toremember is that during a Virtual Connect profile move, the iSCSI boot settingswill not be saved and reconfigured on the target server.

NOTE:10Gb KR-based Ethernet switches (like Procurve 6120XG or Cisco 3120G) canbe used as well for Accelerated iSCSI boot, but this option is not covered in this

document.


5/182


Virtual Connect with iSCSI Summary support

Only the following combination of devices supports iSCSI Boot and Accelerated iSCSI when using VirtualConnect.

BladeSystem G7

+

Virtual Connect FlexFabricBladeSystem BLxx G7

with NC551i / NC553i Integrated CNA

+

Virtual Connect Flex-10Minimum VC 3.10 and above

BladeSystem BLxx G7with NC551i / NC553i Integrated CNA

BladeSystem G6

+ +

Virtual Connect FlexFabricNC551m / NC553m

mezzanineCNABladeSystem BLxx G6

Latest System BIOS

+ +

Virtual Connect Flex-10Minimum VC 3.10 and above NC551m / NC553mmezzanineCNA BladeSystem BLxx G6Latest System BIOS

NOTE:At the time of writing, HP BladeSystem c-Class Integrity Server Blades do notsupport Accelerated iSCSI and iSCSI Boot.


6/182


Firmware and Software Support

Setup of your iSCSI Solution with HP Virtual Connect Flex-10 or FlexFabric Module and FlexFabricAdapters (integrated and mezzanine cards) require using the HP BladeSystem Firmware Release Set2010.10 or later with updates for specific components of the solution.

For best results, use the pre-deployment planning steps in the following documents:

HP Virtual Connect for c-Class BladeSystem Setup and Installation Guidehttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=us

HP BladeSystem ProLiant Firmware Management Best Practices Implementer Guidehttp://h18004.www1.hp.com/products/servers/management/literature.html

Requirements for Accelerated iSCSI and iSCSI Boot:

VCM 3.10 (or above) One Command OS tool be2iSCSI driver [FlexFabric Adapters driver (integrated and mezzanine cards)] be2iSCSI Driver Update Disk for iSCSI boot installs iSCSI target DHCP server (optional)

NOTE:For the supported firmware, drivers, and software versions supported by HP, seethe HP Virtual Connect FlexFabric Solution Recipeat

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084and see theFlexFabric Adapter Firmware and FlexFabric Adapter Operating System Driverstables.

NOTE:If the VC firmware is downgraded to a version older than 3.10, the iSCSI bootparameter configuration is not supported and all iSCSI boot parameters arecleared.
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=ushttp://h18004.www1.hp.com/products/servers/management/literature.htmlhttp://h18004.www1.hp.com/products/servers/management/literature.htmlhttp://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=4144084http://h18004.www1.hp.com/products/servers/management/literature.htmlhttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3552696&taskId=135&prodTypeId=3709945&prodSeriesId=3552695&lang=en&cc=us


7/182


iSCSI Target support

Any target supporting the iSCSI protocol, for example, the HP P4300 LeftHand SAN Solution.

Unsupported configurationAll other Virtual Connect modules do not support Accelerated iSCSI and iSCSI Boot, including the following:

1/10Gb Virtual Connect Ethernet module 1/10Gb-F Virtual Connect Ethernet module

The following adapters will not support iSCSI Boot and Accelerated iSCSI with VC Flex-10 or VC FlexFabric:

HP NC532i/m Dual Port Flex-10 10GbE (Broadcom) HP NC522m Dual Port Flex-10 10GbE (NetXen) HP NC542m Dual Port Flex-10 10GbE (Mellanox) Not supported on 1 Gb NICs


8/182

Networking recommendations 8

Networking recommendations

Network considerations

When constructing an iSCSI SAN with Virtual Connect, some network considerations must be taken into

account.'Do not think of an iSCSI network as just another LAN flavor, IP storage needs the same sort of designthinking applied to FC infrastructure, particularly when critical infrastructure servers can boot from a remoteiSCSI data source.Network performance is one of the major factors contributing to the performance of the entire iSCSIenvironment. If the network environment is properly configured, the iSCSI components provide adequatethroughput and low enough latency for iSCSI initiators and targets. But if the network is congested, and links,switches, or routers are saturated, iSCSI performance suffers and might not be adequate for someenvironments.Following are some important tips and tricks to consider:

Keep it simple and short

With iSCSI, you can route packets between different networks and subnetworks, but keep in mind, everyroute and hop a packet must use, network latency will be added, tremendously affecting the performancebetween iSCSI initiator and iSCSI target.

A network switch will also add latency to the delivery time from the iSCSI packet, so HP recommendskeeping the distance short and avoiding any router or network switches in between the connection. Putsimply, it costs performance, reduces the IOP's per second, and increases the chances of storage trafficcompeting with other data traffic on congested inter-switch links.To avoid bottlenecks, inter-switch links should be sized properly, the use of 10-GbE uplinks and linkaggregation are highly recommended.

Networking considerations are as follows: Minimizing switch hops Maximizing the bandwidth on the inter-switch links, if present Using 10-GbE uplinks

Storage 'vendors' recommendationsStorage vendors usually have iSCSI SAN design recommendations. Following is a list of some of the mostimportant ones.

Flow ControlEthernet Flow Control is a mechanism used to manage the traffic flow between two directly connecteddevices and uses pause frames to notify the link-partner to stop sending traffic when congestion occurs. Ithelps resolve efficiently any imbalance in network traffic between sending and receiving devices.Enabling Flow Control is highly recommended by iSCSI storage vendors. It must be enabled globally acrossthe switches, the server adapter ports, and the NIC ports on the storage node.


9/182


Enabling Flow Control on iSCSI SAN Systems

Flow Control can usually be enabled on all iSCSI Storage Systems. For more specific information about

enabling Flow Control, see the Storage System's manufacturer documentation.

On an HP P4000, Flow Control can be set from the following TCP/IP settings page within the CMC

console:

Enabling Flow Control on the network switches

Flow Control should be enabled on each switch interface connected to the Storage device.See the 'switch's manufacturer documentation for more information about Flow Control.

NOTE:On ProCurve switches, if the port mode is set to autoand flow control is enabledon the P4000 port, the switch port will auto-negotiate flow control with the Storagedevice NIC.

Enabling Flow Control on Virtual Connect

Flow Control is enabled by default on all downlink ports. To enable Flow Control on all VC ports, includinguplink ports, enter the following command:

-> set advanced-networking FlowControl=on

BE CAREFUL:This command can result in data traffic disruption!


10/182


Enabling Flow Control on the iSCSI hostsBy default, Flow Control is enabled on all network interfaces when Accelerated iSCSI is enabled.For Software iSCSI, it might be necessary to enable Flow Control on the NIC/iSCSI initiator.

Jumbo Frames (optional recommendation)Jumbo Frames (MTU>=9000 bytes) are also frequently recommended by iSCSI storage vendors, as they cansignificantly increase the iSCSI performance.

Jumbo Frames have many benefits, particularly for iSCSI traffic, as they reduce the fragmentation overheadby immediately lowering CPU utilization. They also give more aggressive TCP dynamics, leading to greaterthroughput and better response to certain types of loss.

Jumbo Frames must be correctly configured end-to-end on the network, from the storage to the Ethernetswitches and up to the server ports.

NOTE:Using Jumbo Frames in some environments can cause more problems than howmuch it helps with performance. This is frequently due to misconfigured MTU sizes,but also because some devices support different maximum MTU sizes. If you areunsure about whether your routers and other devices support larger frame sizes,keep the frame size at the default setting.

Enabling Jumbo Frames on iSCSI Storage SystemsJumbo Frames can generally be used with all iSCSI Storage Systems and are usually enabled by setting theMTU size on an interface. The frame size on the storage system should correspond to the frame size oniSCSI Hosts (Windows and Linux application servers).For more specific information about how to enable Jumbo Frames, see the Storage System manufacturerdocumentation.


11/182


On an HP P4000, Jumbo Frames can be set from the following TCP/IP settings page within the CMCconsole.

NOTE:The maximum frame size supported by HP FlexFabric CNA (NC551 and 553) is8342 bytes.

NOTE:On the Storage System, set an MTU size above 8342 bytes.

Any Storage Systems configured with a frame size below 8342 bytes will result inan MTU negotiation failure with the CNA, causing the traffic to run at the defaultEthernet standard frame size (that is, 1518 bytes).

If 9000 bytes are needed for any specific reason, then Software iSCSI must beconfigured instead of Accelerated iSCSI.

Enabling Jumbo Frames on Network switchesJumbo Frames must be enabled across all ports of the iSCSI-dedicated VLAN or hardware infrastructure

(always end-to-end). For more specific information, see the switch's manufacturer documentation.

NOTE:Not all switches support both Jumbo Frames and Flow Control. If you must choosebetween the two, choose Flow Control.


12/182


Enabling Jumbo Frames on Virtual ConnectJumbo Frames are enabled by default on Virtual Connect; no configuration is required.

Enabling Jumbo Frames on the iSCSI hosts

There are two procedures for enabling Jumbo Frames on servers: one for Accelerated iSCSI (uses adedicated HBA port for iSCSI traffic) and one for Software iSCSI (uses a port of an existing NIC for iSCSItraffic):

Enabling Jumbo Frames on the iSCSI host with Accelerated iSCSI

Jumbo Frames are enabled by default on the FlexFabric 10Gb NC551 and NC553 CNA withAccelerated iSCSI; no configuration is required. The MTU size is auto-negotiated during the TCPconnection with the iSCSI target.For optimal performance, the Max MTU size supported by iSCSI Accelerated mode is limited to8,342 bytes and cannot be modified.

a. Checking MTU size under Windows:To see the MTU size that has been auto-negotiated under Windows, go to the EmulexOneCommand Manager (OCM), select the iSCSI target, and click on Sessions.

The TCPMSSvalue used for this connection is displayed in the Connection Negotiated LoginPropertiessection.


13/182


When TCPMSSdisplays 1436, the MTU negotiated size is 1514.When TCPMSSdisplays 8260, the MTU negotiated size is 8342.

b. Checking MTU size under VMware ESX:Like under MS Windows, MTU is automatically configured and the user has no control onthis setting. Under VMware, there is no way for the user to view the configured MTU/MSS.

Enabling Jumbo Frames on the iSCSI host with Software iSCSI

With Software iSCSI, Jumbo frames must be enabled under the Operating System on eachadapter/vSwitch running iSCSI.

a. Under MS Windows Server 2008:1. Right-click Networkin Start Menu and click Properties.2. Select the network adapter used for iSCSI and click Properties.3. Click Configure.4. Click theAdvancedtab.


14/182


5. Select Packet Sizeand change the MTU value.

6. Click OKto apply the changes.

To see the MTU value configured under Windows, go to OCM, then select the adapterused for iSCSI. The MTU is displayed under the Current MTUfield on the Portinformationtab.

For more information, refer to the Microsoft documentation.


15/182


b. Under VMware ESX:Enter the following command to set MTU for the vswitch:esxcfg-vswitch -m 9000 vSwitchEnter the following command to check the MTU configuration:esxcfg-vswitch l vSwitch

Switch Name Num Ports Used Ports Configured Ports MTU UplinksvSwitch2 128 3 128 9000 vmnic4,vmnic5

For more information, seeiSCSI and Jumbo Frames configuration on ESX 3.x and ESX 4.x(1007654).

Testing Jumbo FramesThe Jumbo Frames configuration can be tested by using the PINGcommand usually available on the iSCSIStorage System.

1. Test ping from the Storage System to the iSCSI host's interface using 8300 bytes for MTU.

2. The ping result should appear similar to the following:PING 192.168.5.14 (192.168.5.14) from 192.168.5.20 : 8300(8328) bytes of data.8308 bytes from 192.168.5.14: icmp_seq=5 ttl=64 time=47.7 ms
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654


16/182


iSCSI multipathing solutions

Using multipathing solutions is highly recommended for load balancing and failover to improve iSCSIperformance and availability.Multipathing solutions use redundant physical path componentsadapters, cables, and switchesto createlogical "paths" between the server and the storage device. If one or more of these components fails,

causing the path to fail, multipathing logic uses an alternate path for I/O so applications can still accesstheir data.For the operating system, this multipathing means needing an intelligent path manager called Multipath I/O(MPIO) to log in multiple sessions and to failover, if needed, among multiple iSCSI Host Bus Adapters(HBAs).MPIO is a key component to building a highly available, fault-tolerant iSCSI SAN solution. MPIOtechnologies provide for the following:

I/O path redundancy for fault tolerance I/O path failover for high availability I/O load balancing for optimal performance

For Microsoft Windows OS, storage vendors usually provide a vendor-specific Device Specific Module(DSM ) to optimize multipathing using the Microsoft MPIO framework. This vendor-specific module (DSM)for MPIO must be installed under the operating system. Consult your storage provider website for moreinformation.


17/182

Virtual Connect network scenarios 17

Virtual Connect network scenarios

For security and performance purposes, HP recommends separating the iSCSI network either logically(using different VLANs) or physically (using different physical switches) from the ordinary data network.Isolating the iSCSI traffic helps to improve response times, reliability, and prevent bottlenecks and

congestion. It also helps to address the TCP/IP overhead and flow control issues inherent in an Ethernetnetwork.

Another recommendation to maximize availability and optimal performance is to use an iSCSI-redundantnetwork path from Virtual Connect (and therefore from the server) to the storage system. This enables afailover mechanism in case of path failure among multiple iSCSI HBA. Using Multipath I/O softwarerunning under the OS (Windows, Linux, and VMware) is required to provide an automatic means ofpersisting I/O without disconnection.For a step-by-step typical scenario configuration, see the Virtual Connect Ethernet Cookbookhttp://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdf

Scenario 1: iSCSI network physically separatedThe iSCSI network is physically separated from the ordinary data network using a different switchinfrastructure.

ProsThis scenario is the best for performance, better latency, and is the recommended scenario. It maximizes thebandwidth availability. Here, the iSCSI traffic does not have to fight for bandwidth, as there is a dedicatedinfrastructure for the storage traffic.

ConsThis scenario uses more switches, more VC uplinks, therefore using more cabling. The solution cost is

increased.
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdfhttp://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdfhttp://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdf


18/182


Figure 1 - Physical view

EnclosureUID

Enclosure Interlink

PS

3

PS

2

PS

1

PS

6

PS

5

PS

4

OA1 OA2

Remove management modules before ejecting sleeve

FAN6

FAN10

FAN1

FAN5

21

3

5

7

4

6

8

UID

X1

HPVCFlex-10EnetModule X1 X 2 X 3 X 4 X 5 X 6

SHARED

X 7 X 8

SHARED:UPLINKor X-LINK

UID

X1


SHARED

X 7 X 8

SHARED:UPLINKor X-LINK

HP4GbVC-FCModule

UID

1 2 3 4

HP4GbVC-FCModule

UID

1 2 3 4

iL

O

UID

ActiveResetiLO

UID

ActiveReset

802.3ad LAG

802.1Q Trunk

LAN Switch 1 LAN Switch 2

PRODUCTION NETWORK IP STORAGE NETWORK

iSCSI network

Rear view

c7000 enclosure

VC Modules

PRODUCTION NETWORK

802.3ad LAG

802.1Q TrunkvNet-iSCSI-1 vNet-iSCSI-2

iSCSI Storage Device

Port 1 Port 2

Active

Virtual IP

Active

Prod-vNet-1 Prod-vNet-2Active Active


19/182


20/182


Defining two iSCSI networks vNet

Create a vNet namedvNet-iSCSI-1:1. On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet.2. Enter the Network NamevNet-iSCSI-1.3. Select Smart Link, but do not select any of the other options (such as, Private Networks, and so forth).4.

Select Add Port, then add one port from Bay 1.5. For Connection Mode, use Failover.

6. SelectApply.

Create a vNet namedvNet-iSCSI-2:1. On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet.2. Enter the Network NamevNet-iSCSI-2.3. Select Smart Link, but do not select any of the other options (such as, Private Networks, and so forth).4. Select Add Port, then add one port from Bay 2.5. For Connection Mode, use Failover.6. SelectApply.


21/182


NOTE:By creating two vNets, we have provided a redundant path to the network. Aseach uplink originates from a different VC module and vNet, both uplinks will beactive. This configuration provides the capability to lose an uplink cable, networkswitch, or depending on how the iSCSI ports are configured at the server (iSCSISoftware Initiator supporting failover), even a VC module.

NOTE:Smart LinkShould be enabled. It is used to turn off downlink ports within VirtualConnect if all available uplinks to a vNet or SUS are down. In this scenario, if anupstream switch or all cables to a vNet were to fail on a specific vNet, VC wouldturn off the downlink ports connected to that vNet, forcing the iSCSI SoftwareInitiator to fail-over to the alternate NIC.

Connection Mode FailoverShould be enabled here as only a single externaluplink port is used for this network. With multiple uplink ports, the connectionmode Auto can be used to enable the uplinks to attempt to form aggregationgroups using IEEE 802.3ad link aggregation control protocol. Aggregation groups

require multiple ports from a single VC-Enet module to be connected to a singleexternal switch supporting automatic formation of LACP aggregation groups, ormultiple external switches utilizing distributed link aggregation.


22/182


Scenario 2: iSCSI network logically separated

In this second scenario, we use the same switch infrastructure, but the iSCSI network is logically separatedfrom the ordinary data network through the use of 802.1Q VLAN trunking.Each Virtual Connect module is connected with more than one cable to the LAN Switches to increase thenetwork bandwidth and to provide a better redundancy.

ProsThis scenario uses less VC uplinks, therefore, less cabling. The solution cost is reduced.

ConsIn this scenario, the performance of iSCSI relies on the datacenter network performance.If the datacenter network is congested and saturated, iSCSI performance suffers and might not be adequatefor some environments.


EnclosureUID

Enclosure Interlink

PS3

PS2

PS1

PS6

PS5

PS4

OA1 OA2


FAN6

FAN10

FAN1

FAN5

21

3

5

7

4

6

8

UID

X1


SHARED

X 7 X 8

SHARED:UPLINK orX-LINK

UID

X1


SHARED

X 7 X 8

SHARED:UPLINK orX-LINK

HP4GbVC-FCModule

UID

1 2 3 4

HP4GbVC-FCModule

UID

1 2 3 4

iL

OUID

ActiveResetiLO

UID

ActiveReset

802.3ad LAG802.1Q Trunk

LAN Switch 1 LAN Switch 2iSCSI Storage Device


iSCSI network+

Production network

Rear viewc7000 enclosure

VC Modules

Port 1 Port 2

Virtual IP

PRODUCTIONNETWORK

IP STORAGE NETWORK

PRODUCTIONNETWORK

UplinkSet_1Active

UplinkSet_2Active

iSCSI network+

Production network


23/182


Figure 5 - Logical view of an iSCSI Vmware host

HP VC F lexFabr ic 10Gb/24-Por t Module

SHARED: UPLINK or X-LINKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

LOM 1 LOM 2

Console

Hypervisor Host

VM Guest VLAN

1Gb

1.5Gb

3.5Gb

4Gb

1Gb

1.5Gb

3.5Gb

4Gb

A

B

C

D

A

B

C

D

VMotion

10Gb 10Gb

vmnic0

vmnic2

vmnic1

vmnic3

vmnic4 vmnic5

iSCSI2 iSCSI3

VLAN

UT

101

103104

H

P VC F lexFabr ic 10Gb/24-Por t Module

SHARED: UPLINK or X-LINKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

UplinkSet_1Enc0:Bay1:X1,X2

Prod 802.1Q Trunk

(VLANs 101 through 105)

VC FlexFabric 10Gb Enc0:Bay1 VC FlexFabric 10Gb Enc0:Bay2

Virtual Connect Domain

NIC1

NIC2

UID

HPProLiant

BL460cG6

serialscsi

1port10k

146GB

serialscsi1port10k

146GB

UT

102

UplinkSet_2Enc0:Bay2:X1,X2

Prod 802.1Q Trunk

(VLANs 101 through 105)

iSCSI Network UT

105

LOM 2

2B FlexiSCSI

2C FlexNIC

2D FlexNIC

2A FlexNIC

LOM 1

1C FlexNIC

1D FlexNIC

1A FlexNIC

1B FlexiSCSI

VLAN_101-1 VLAN_102-1

VLAN_103-1

iSCSI_1

VLAN_104-1

VLAN_101-2 VLAN_102-2

VLAN_103-2

iSCSI_2

VLAN_104-2

Internal Stacking link

vSwitch

Defining a first Shared Uplink Set (VLAN-trunk-1)

Create an SUS named UplinkSet_1:1. On the Virtual Connect Home page, select Define, Shared Uplink Set.2. Insert Uplink Set Name as UplinkSet_1.3. SelectAdd Port, then add two ports from Bay 1.4. Add Networks as follows: (to add a network, right-click on the grey bar under the "Associate

Networks (VLAN)" header, then select ADD.)VLAN_101-1= VLAN ID = 101= CONSOLE

VLAN_102-1= VLAN ID = 102= VMOTION

VLAN_103-1= VLAN ID = 103= First VM Guest VLAN

VLAN_104-1= VLAN ID = 104= Second VM Guest VLAN(More VM Guest VLANs can be defined here)

iSCSI_1= VLAN ID=105

5. Enable SmartLink on all networks.6.

Leave Connection Mode asAuto(this will create an LCAP port channel if the upstream switch isproperly configured).

7. Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native toEnabled.

8. ClickApply.


24/182


Defining a second Shared Uplink Set (VLAN-trunk-2)

1. Create an SUS named UplinkSet_2.2. On the Virtual Connect home page, select Define, Shared Uplink Set.3.

Insert Uplink Set Name as UplinkSet_2.4. SelectAdd Port, then add two ports from Bay 2.

5. Add Networks as follows: (to add a network, right-click on the grey bar under the "AssociateNetworks (VLAN)" header, then select ADD.)

VLAN_101-2= VLAN ID = 101= CONSOLE

VLAN_102-2= VLAN ID = 102= VMOTION

VLAN_103-2= VLAN ID = 103= First VM Guest VLAN

VLAN_104-2= VLAN ID = 104= Second VM Guest VLAN(More VM Guest VLANs can be defined here)

iSCSI_2= VLAN ID=105

6. Enable SmartLink on all networks.7. Leave Connection Mode as Auto (this will create an LCAP port channel if the upstream switch is

properly configured).8. Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native to

Enabled.9. ClickApply.


25/182



26/182


Scenario 3: Direct-attached iSCSI Storage System

In this third scenario, an iSCSI device is directly connected to the Virtual Connect Domain without any switchinfrastructure. This scenario uses more VC uplinks than the second scenario, but no additional or dedicatedswitches are required like in scenario 1 to only connect a SCSI disk storage enclosure. This reduces the entiresolution cost and complexity.

ProsCost is greatly reduced, as no additional switch is required.

ConsThere are several limitations.

Direct-attached limitations

The following are important limitations the administrator must be made aware of:

When an iSCSI storage device is directly connected to a VC Domain, this iSCSI device is

only accessible to the servers belonging to this Virtual Connect Domain.

iSCSI Storage Systems sharing the same ports for both iSCSI host connectivity traffic andLAN management (also known as in-band management) can only be managed from the

Virtual Connect Domain.

The only network interface bond supported on the iSCSI Storage system is Active-Passive.

VC Active/Standby iSCSI vNet configuration is not supported.

iSCSI Storage systems can be divided into two categories: out-of-band management (using separate portsfor management and host traffic) and in-band management (using the same ports for management and hosttraffic). The direct-attached scenario will be divided into two subscenarios, according to the type of

management you may use:Scenario 3-A: Direct-attached iSCSI device with out-of-band management

Scenario 3-B: Direct-attached iSCSI device with in-band management


27/182


Scenario 3-A: Direct-attached iSCSI device with out-of-band management (using separate ports formanagement and host traffic)


iSCSI Hosts

UID

FLEX

1

FLEX2

HP ProLiantBL460c G7

SAS

6G DP 10k

300 GB

SAS6GDP 10k

300 GB

UID

FLEX1

FLEX

2


SAS

6G DP 10k

300 GB

SAS

6G DP 10k

300 GB

UID

FLEX

1

FLEX

2


SAS

6G DP 10k

300 GB

SAS

6G DP 10k

300 GB

EnclosureUID

Enclosure Interlink

PS3

PS2

PS1

PS6

PS5

PS4

OA1 OA2


FAN

6

FAN

10

FAN

1

FAN

5

21

3

5

7

4

6

8

UID

X1

HP VC Flex-10 Enet Module X1 X 2 X 3 X 4 X 5 X 6

SHARED

X 7 X 8

SHARED: UPLINK or X-LINK

UID

X1


SHARED

X 7 X 8


HP 4Gb VC-FC Module

UID

1 2 3 4

HP 4Gb VC-FC Module

UID

1 2 3 4

iL

O

UID

ActiveResetiLO

UID

ActiveReset



PRODUCTION NETWORK

Directly attached


Port 1Active

Port 2Passive

IP STORAGE NETWORK

iSCSI host network

Rear view

c7000 enclosure

VC Modules

PRODUCTION NETWORK


Management network

2533t

Storage

ManagementConsole

vNet- iSCSI-1 vNet- iSCSI-2

Active

Virtual IP

Active

MgtPort

In Scenario A, the iSCSI target is directly connected to the VC Domain using two Active/Active vNet (blueand red in the previous diagram) to provide iSCSI LAN access to the servers located inside the enclosure.The iSCSI storage device is using an out-of-band management network, which means dedicated portsseparated from the iSCSI host traffic are available for the management/configuration. Therefore, the iSCSIdevice can be managed from anywhere on the network.Both software and hardware iSCSI can be implemented on the iSCSI hosts without limitation.


28/182


29/182


Scenario 3-B: Direct-attached iSCSI device with in-band management (using the same ports formanagement and host traffic):


EnclosureUID

Enclosure Interlink

PS

3

PS

2

PS

1

PS

6

PS

5

PS

4

OA1 OA2


FAN6

FAN10

FAN1

FAN5

21

3

5

7

4

6

8

UID

X1


SHARED

X 7 X 8


UID

X1


SHARED

X 7 X 8


HP 4Gb VC-FC Module

UID

1 2 3 4

HP 4Gb VC-FC Module

UID

1 2 3 4

iL

O

UID

ActiveReset

iLO

UID

ActiveReset

802.3ad LAG

802.1Q Trunk


PRODUCTION NETWORK IP STORAGE NETWORK

iSCSI host network +

management network

Rear viewc7000 enclosure

VC Modules

PRODUCTION NETWORK


Storage

Management

ConsoleiSCSI Hosts

UID

FLEX

1

FLEX

2


SAS

6GDP 10k

300 GB

SAS

6G DP 10k

300 GB

UID

FLEX

1

FLEX

2


SAS6G DP 10k

300 GB

SAS

6G DP 10k

300 GB

UID

FLEX

1

FLEX

2

HP ProLiant

BL460c G7

SAS

6G DP 10k

300 GB

SAS

6G DP 10k

300 GB

UID

FLEX

1

FLEX

2

HP ProLiant

BL460c G7

SAS

6G DP 10k

300 GB

SAS

6G DP10k

300 GB

vNet-iSCSI-1 vNet-iSCSI-2

Directly attached


Port 1

Active

Port 2

Passive

Active

Virtual IP

Active

In scenario B, the iSCSI device is again connected directly to the VC Domain using two Active/Active vNet(blue and red in the previous diagram). But here, the iSCSI device is not using a dedicated interface formanagement, which means the same ports are used for both the management and the iSCSI host traffic(known as in-band management device).This implies, due to the Virtual Connect technologies, you can only manage and configure the iSCSI storagedevice from the Virtual Connect Domain (that is, only from a server located inside the enclosure andconnected to the iSCSI vNet).This means you need a dedicated server for the storage system management, and all SNMP trapnotifications for hardware diagnostics, events, and alarms will only be sent to this management console.Both software and hardware iSCSI can be implemented here without limitation on the iSCSI hosts.For more information about Accelerated iSCSI, see "Accelerated iSCSI".


30/182


Virtual Connect network configuration

The Virtual Connect network configuration is the same for Scenario 3-A and Scenario 3-B.You must define two Active/Active iSCSI Virtual Connect networks (vNets):vNet-iSCSI-1andvNet-iSCSI-2,like inScenario 1.These 2 vNets will be used exclusively for the direct attachment of the iSCSI storage

device.

NOTE:The direct-attached scenario is not supported with VC Active/Standby iSCSI vNet

configuration. A standby VC uplink is always seen as active by an upstreamdevice, so having no way to detect the standby state of the VC uplink, a storagesystem would incorrectly send traffic to the standby port, causing communicationissues.


31/182


Connecting the direct-attached iSCSI SAN to the VC Domain

Connect the VC uplink port of the two vNets to the network interfaces of the iSCSI device.

Figure 8: Out-of-band management configuration with a HP P4000 (Scenario 3-A)

Mgmt

2 1 UID

PORT1

10GbE

SFP

PORT2

L

A

L

A

T

XRXTXRX

NC550sp

HP VC FlexFabric 10Gb/24-PortModu le

S HA RE D:UP L INK or X -L INKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

H

P VC FlexFabric 10Gb/24-PortModu le

S HA RE D:UP L INKorX -L INKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

NIC

1

NIC

2

UID

HP ProLiant

BL460c

CNA 1 CNA 2

Prod-vNet-1 Prod-vNet-2

HP LeftHand P4300 G2

NIC

1

NIC

2

UID

HP ProLiant

BL460c

CNA 1 CNA 2NIC

1

NIC

2

UID

HP ProLiantBL460c

CNA 1 CNA 2NIC

1

NIC

2

UID

HP ProLiant

BL460c

CNA 1 CNA 2

iSCSI HostsBlade Servers

vNet-iSCSI-1 vNet-iSCSI-2ActiveActive

Directly attached iSCSI device

using out-of-band management

HP VC FlexFabric

Management network


32/182


Figure 9: In-band management configuration with an HP P4000 (Scenario 3-B)

Mgmt

2 1 UID

HP VC FlexFabric10Gb/24-Port Modu le

S HA RE D:UP L INK orX -L INKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

H

P VC FlexFabric10Gb/24-Port Modu le

S HA RE D:UP L INK or X -L INKX3 X4X1 X 2 X5 X 6 X 7 X 8

UID

NIC

1

NIC

2

UID

HPProLiant

BL460c

CNA 1 CNA 2

Prod-vNet-1 Prod-vNet-2

HP LeftHand P4300 G2

NIC

1

NIC

2

UID

HPProLiant

BL460c

CNA 1 CNA 2NIC

1

NIC

2

UID

HPProLiant

BL460c

CNA 1 CNA 2NIC

1

NIC

2

UID

HPProLiant

BL460c

CNA 1 CNA 2

iSCSI HostsBlade Servers

vNet-iSCSI-1 vNet-iSCSI-2ActiveActive

Directly attached iSCSI device

using in-band management

HP VC FlexFabric

Preparing the network settings of the storage system

The following describes a basic network configuration of an HP P4300:1. Open a remote console on the P4300 using the P4300 iLO or connect a keyboard/monitor to the

P4300.2. Type start, and press Enterat the log in prompt.3. Press Enterto log in or type the Username andpasswordif already configured.4. When the session is connected to the storage system, the Configuration Interface window displays.5. On the Configuration Interface main menu, tab to Network TCP/IP Settings, and press Enter.6. Tab to select the first network interface and press Enter.7.

Enter the host name, and tab to the next section to configure the network settings.8. Enter a Private IP address like 192.168.5.20/ 255.255.255.0with 0.0.0.0for the Gateway address.


33/182


9. Tab to OK, and press Enterto complete the network configuration.10.Press Enteron the confirmation window.

A window opens listing the assigned IP address that will be used later on to configure and managethe P4300.

Configuring the Storage Management Server

Scenario 3-AAn iSCSI device with an out-of-band management does not require a specific configuration, as themanagement console can reside anywhere on the network.

Scenario 3-B

An iSCSI device with an in-band management requires a little bit more attention, particularly when networkinterface bonding is enabled on the iSCSI Storage System.The following configuration can be followed for an in-band management iSCSI Storage System:

1. Create a Virtual Connect server profile for the Management Storage server.2. Assign NIC1 and NIC2 to the management network.3. Assign NIC3 and NIC4 to the iSCSI direct-attached VC networksvNet-iSCSI-1andvNet-iSCSI-2.


34/182


Figure 10: Example of a VC profile used by the Storage Management Server

4. Start the server and install Windows Server 2003 or 2008.5. At the end of the installation, assign a private IP address to NIC3(such as 192.168.5.150) and to

NIC4(such as 192.168.5.151) Ensure you use the same IP subnet addresses as the one set in the Storage System.

NOTE:Two IP addresses are used in this configuration to ensure the management serverstays connected to the iSCSI device regardless of the iSCSI device bondingconfiguration (no bonding or Active-Passive bond enabled).

Despite using the Active/Active iSCSI vNet configuration, NIC teaming cannot beused on the management server; otherwise the network connection to the iSCSIdevice will fail. This is due to the direct-attached design and using NIC bonding onthe iSCSI device. There is no such limitation with Scenario 3-A.


35/182


6. Install the Storage Management software (for example, HP P4000 Centralized Management Console;this is the utility to configure and manage the HP P4000 SAN Solution).

7. Open the Centralized Management Console (CMC) and locate the storage system using the Findfunction. Enter the address assigned previously on the first interface of the P4300 (which is192.168.5.20).

8. Select the Auto Discover by Broadcast check box.9. When you have found the storage system, it appears in the Available systems pool in the navigation

window.

Configuring the network interface bonds on the Storage System

When configuring an iSCSI SAN directly attached to VC, one point is worth considering: the networkinterface bonding.Network interface bonding is generally used on iSCSI SAN devices to provide high availability, faulttolerance, load balancing, and/or bandwidth aggregation. Depending on your storage system hardware,you can generally bond NIC's in one of the following methods:

Active - Passive Link Aggregation (802.3 ad) Adaptive Load Balancing

Related to the direct-attached scenario, when a bond is configured, multiple interfaces are used. but due tothe lack of Ethernet switches between the VC domain and the iSCSI SAN device, the support of NICbonding is very limited; only Active/Passive bonding methods are supported.

NOTE:Link Aggregation (802.3ad) is not supported because you cannot create an

aggregation group across different VC modules. Adaptive Load Balancing is notsupported either, as it requires the two interfaces to be connected to the samenetwork, which is not the case here, as we have two vNet for iSCSI (Active/Activescenario).


36/182


To configure Active/Passive bonding on a HP P4300:1. From the CMC, select the storage system and open the tree below it.2. Select TCP/IP Networkcategory, and click the TCP/IPtab.

3. Select the two interfaces from the list for which you want to configure the bonding. Right-click andselect New Bond.

4. Select theActive Passivebond type.

5. Click OK.6. After a warning message, click OKto rediscover the Storage system; the IP address is normally

preserved.7. After a few seconds, the new bond setting shows the Active-Passive configuration:

8. The Storage System is now ready to be used with fault tolerance enabled.


37/182


Configuring the iSCSI Host

For the iSCSI Hosts configuration, the Hardware and/or Software iSCSI connection can be used.

Multipathing and path checking

With the direct-attached scenario and its Active Passive iSCSI SAN NIC bonding, HP recommends

checking the second path to validate the entire configuration. After the iSCSI volume has been discoveredfrom the first path, log in to the iSCSI Storage System and trigger an NIC bonding fail -over; this will activatethe second interface. You will then have the ability to validate the second iSCSI path.


38/182

Accelerated iSCSI 38

Accelerated iSCSI

Traditional software-based iSCSI initiators generate more processing overhead for the server CPU. Anaccelerated iSCSI-capable card, also known as Hardware iSCSI, offloads the TCP/IP operations from theserver processor, freeing up CPU cycles for the main applications.

The main benefits are as follows:

Processing work offloaded to the NIC to free CPU cores for data-intensive workloads. Increased server and IP storage application performance. Increased iSCSI performance.

Software-based iSCSI initiators are supported by all Virtual Connect models (1/10Gb, 1/10Gb-F, Flex-10,and FlexFabric), but 10Gb accelerated iSCSI (Hardware-based iSCSI initiator) is only provided byConverged Network Adapters (that is, NC551i/m or NC553i/m) with only Virtual Connect Flex-10 and

Virtual Connect FlexFabric.The QLogic QMH4062 1GbE iSCSI Adapter is another adapter supporting accelerated iSCSI, but it is notsupported by VC Flex-10 and VC FlexFabric.

NOTE:Under the OS, Accelerated iSCSI differs from Software iSCSI in that it provides anHBA type of interface and not a network interface card (NIC). Consequently,additional drivers, software, and settings are sometimes required.


39/182


NOTE:The selection between Software iSCSI and Accelerated iSCSI is done under theVirtual Connect profile.

Figure 11 Example of a server profile with Software iSCSI enabled

Figure 12 Example of a server profile with Accelerated iSCSI enabled


40/182


The following steps are required to connect a server to an iSCSI target using accelerated iSCSI:1. Enable Accelerated iSCSI on the server using Virtual Connect.2. Install and configure a hardware iSCSI initiator under the OS.3. Use MPIO software to manage the redundant iSCSI connection.

This document provides the steps to enable Accelerated iSCSI on a BladeSystem server under Microsoft

Windows Server 2003/2008 and VMware vSphere Server.

Enabling Accelerated iSCSI on the server using Virtual Connect(also known as iSCSI Offload)

The first step is the same for both MS Windows and VMware vSphere servers: create a Virtual Connectserver profile enabling Accelerated iSCSI.

1. To create a new VC profile, open Virtual Connect manager.2. From the Definemenu, select Server Profile.

VCM assigns FCoE connections by default.

NOTE:A server with one FlexFabric adapter can be configured with a unique personality,

either all Ethernet, or Ethernet/iSCSI, or Ethernet/FCoE. Therefore, 'it is notpossible to enable at the same time both FCoE and iSCSI connections.

A server with multiple FlexFabric Adapters can be configured with both iSCSI andFCoE connections.


41/182


3. If you have a unique FlexFabric Adapter, delete the two FCOE connections; otherwise, go to the nextstep.

4. In the iSCSI HBA Connections section, clickAdd.

5. In the Network Namecolumn, click Select a network.

6. Select your iSCSI-dedicated VC network, thenclick OK.

7. In the Port Speedcolumn, you can adjust the speed setting to Auto, Preferred, or Custom.8. In the Boot Settingcolumn, leave as DISABLED.


42/182


NOTE:Boot setting disabled means Accelerated iSCSI is enabled, but iSCSI Boot isunavailable.

The disable mode offloads the iSCSI protocol processing from the OS to the NIC.

In addition to offloading TCP/IP protocol processing, it also offloads iSCSIprotocol processing.

The multiple network feature (that is, when using 802.1Q VLAN tagging) is notsupported for iSCSI connections.

9. Optionally, create a second iSCSI Connection for multipathing configuration.

NOTE:Allowing more than one iSCSI application server to connect to a volumeconcurrently without cluster-aware applications or without an iSCSI initiator withMPIO software could result in data corruption.

10.Configure the additional VC Ethernet Network connections that may be needed on the other FlexNIC.


43/182


11.When done, you can assign the profile to a server with an Ethernet adapter supporting AcceleratediSCSI.

12.ClickApplyto save the profile.13.The server can now be powered on (using either the OA, the iLO, or the Power button).


44/182


Accelerated iSCSI with Microsoft Windows Server

After creating a VC profile with Accelerated iSCSI enabled, proceed with the following steps underMicrosoft Windows Server:

1. Installing the Emulex hardware iSCSI initiator (that is, OneCommand Manager).2. Configuring the IP addresses of the iSCSI ports.3. Installing the Microsoft iSCSI Initiator.4. Installing the Microsoft MPIO.5. Installing the Device Specific Module (DSM):

a. Using the Microsoft DSM that comes with Microsoft MPIO.b. Using the Storage vendor DSM for MPIO provided by the Storage System vendor for better

performance and latency.6. Connecting to the iSCSI target using the iSCSI initiator.

Installing Emulex OneCommand Manager

OneCommand Manager is the Emulex utility managing the NC551and NC553 10Gb FlexFabric Converged

Network Adapters. Among other things, it provides comprehensive control of the iSCSI network includingdiscovery, reporting, and settings.For accelerated iSCSI under MS Windows server, the Emulex OneCommand Manager is mandatory, as itprovides the only way to configure the IP settings of the iSCSI HBA ports required to connect the iSCSIvolume.For more information about OneCommand Manager, see the User Manual of the Emulex OneCommandManagerhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdf

NOTE:Do not confuse hardware and software iSCSI initiators. Accelerated iSCSI alwaysuses a specific port (such as host bus adapters) and requires a utility from the HBAvendor (such as Emulex OneConnect Manager or Emulex Drivers be2iscsi).

Storage iSCSIAdapter Ports

Network Interface Ports

Just the opposite, software initiators use the standard server NIC port and areusually included in the operating system (such as Microsoft iSCSI initiator).
http://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdf


45/182


OneCommand Manager can be installed by HP System Update Manager (HP SUM) but it can as well bedownloaded from the web:

1. From the Support & Drivers hp.com webpage, enter a Blade model, for example BL460 G7.

2. Select the operating system.


46/182


3. Click Utility FC HBA.

4. Click Downloadto download the OneCommand Manager Application Kit

NOTE:The HP OneCommand Manager Application Kit contains a GUI and a CLI.

HbaCmd.exe (the CLI) is located by default in C:\ProgramFiles\Emulex\Util\OCManager.

5. Install the application.


47/182


6. Launch the OneCommand utility from Windows Start menuAll ProgramsEmulexOCManager.

7. OneCommand Manager shows the iSCSI ports detected on your server.


48/182


Configuring the IP addresses of the iSCSI ports

To log into the iSCSI target, an IP address must be assigned to each iSCSI HBA ports, as follows:1. From the OneCommand Manager interface, select the first iSCSI port, and click Modify.

2. Enter a static IP address or check DHCP Enabled.

This IP address must be in the same subnet as the one configured in the iSCSI Storage System.

3. Click OK.4. Select the second iSCSI port, and click Modify.


49/182


5. Enter a second static IP address or check DHCP Enabled.

6. Click OK.

Installing the Microsoft iSCSI Initiator

The Microsoft iSCSI Software Initiator enables connection of a Windows host to an external iSCSI storagearray using Ethernet NICs.For more information about the Microsoft initiator, seehttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspx.

NOTE:The Microsoft iSCSI Initiator utility is a software initiator (using Ethernet NICs) but it canalso be used to manage the iSCSI accelerated connectivity (using the iSCSI HBAs).

For Windows 2003:Download and install the latest version of the Microsoft iSCSI Initiator software.You must select the Microsoft MPIO Multipathing Support for iSCSIoption when you install the MicrosoftiSCSI Initiator.

For Windows 2008:The Microsoft iSCSI Initiator comes installed with both the Windows Server 2008 and the Server Coreinstallation.
http://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspx


50/182


Installing Microsoft MPIO

MPIO solutions are needed to logically manage the iSCSI redundant connections and ensure the iSCSIconnection is available at all times. MPIO provides fault tolerance against single point of failure in hardwarecomponents, but can also provide load balancing of I/O traffic, thereby improving system and applicationperformance.

For Windows 2003:MPIO is installed during the Microsoft iSCSI initiator installation. You must select the Microsoft MPIOMultipathing Support for iSCSIoption when installing the Microsoft iSCSI Initiator.

For Windows 2008:MPIO is an optional component in all versions of Windows 2008 Server so it must be installed.Go to Server Manager, thenselect Features Add Features Multipath I/O to install it.

Installing the Device Specific Module (DSM)

MPIO requires the installation of device-specific modules (DSM) to provide support for using multiple data

paths to a storage device. These modules are a server-side plug-in to the Microsoft MPIO framework.

A native Microsoft DSM provided by Microsoft, comes by default with Microsoft MPIO software, but storageproviders can develop their own DSM containing the hardware-specific information needed to optimize theconnectivity with their storage arrays.

Vendor-specific DSM usually provides better write performance, lower latency, and so forth, while theMicrosoft DSM offers more basic functions.

Installing the Microsoft DSM

The Microsoft DSM comes with the Microsoft iSCSI Initiator for Windows 2003 Server and is an optionalcomponent for Windows 2008 Server that is not installed by default.

For Windows 2003:The Microsoft DSM is installed along with the iSCSI Initiator.

For Windows 2008:To use the Microsoft DSM with all versions of Windows 2008 Server, you must install MPIO and enable it.

Additional details for some steps are in the white paper available fromhttp://www.microsoft.com/mpio1. Connect to a volume using the iSCSI initiator.2. In the MPIO Control Panel applet, click the Discover Multi-Pathstab, select theAdd support for iSCSI

devicescheck box, then clickAdd.
http://www.microsoft.com/mpiohttp://www.microsoft.com/mpiohttp://www.microsoft.com/mpiohttp://www.microsoft.com/mpio


51/182


The check box does not become active unless you are logged in to an iSCSI session.

3. The system asks to reboot for the policy to take effect.


52/182


4. After the reboot, the MPIO Devices tab shows the addition of MSFT2005iSCSIBusType_0x9.

5. Connect to a volume using the iSCI Initiator and select the Enable multi-pathcheck box.


53/182


6. When you connect to the volume, set your MPIO load balancing.

Installing the Storage vendor DSM

1. Download the specific DSM for your iSCSI storage array from your storage 'vendor's website.

NOTE:

The HP P4000 DSM for MPIOsoftware is notsupported with Accelerated iSCSI

HBA under Windows Server 2003 and Windows Server 2008.

2. Install the DSM for MPIO. A reboot may be required.3. Once the DSM for MPIO is installed, there are no additional settings. All iSCSI volume connections

made to an iSCSI Storage System will attempt to connect with the storage 'vendor's DSM for MPIO.


54/182


Connecting volumes with MPIO

1. Open the Microsoft iSCSI Initiator from the Control Panel.For Windows Server 2008, enter iSCSI in the Search tab.


55/182


2. On the Discoverytab, select Discover Portal.

3. Enter the IP address of the iSCSI Storage System, then click OK.

NOTE:iSCSI SAN vendors usually recommend using a Virtual IP address. See your iSCSISAN documentation for additional information.

4. On the Targetstab, you should discover a new iSCSI target if a LUN has been correctly presented tothe server.


56/182


5. Select the volume to log on to, then click Connect.

6. Select the Enable multi-pathcheck box.

NOTE:Do not select the Enable Multi-path checkbox if your iSCSI Storage System is notsupporting load-balanced iSCSI access.

7. Click OKto finish.


57/182


Multipath Verification

1. You can verify the DSM for MPIO operations. From the Targetstab, select the volume (nowconnected!), then click Properties.

2. On the Sessions tab, you should see multiple sessions of this target, as the DSM for MPIOautomatically builds a data path to each storage node in the storage cluster.

If after a few minutes only one session is available (you can click on refresh several times), it may benecessary to configure the multipathing manually.


58/182


Manual Multipath configuration

1. ClickAdd session.

2. Select the Enable multi-pathcheck box.

3. ClickAdvancedto open the Advanced Settings window.4. Configure the Advanced Settings as follows:

For Local adapter, select the first Emulex OneConnect OCe11100, iSCSI Storport.

For Initiator IP, select the IP address of the iSCSI HBA to connect to the volume.


59/182


For Target portal IP, select the IP of the iSCSI Storage System containing the volume.

5. Click OKto close the Advanced Settings dialog.6. Click OKto finish logging on.7. Repeat step 1 to step 6, using this time the second Emulex adapter.

For Source IP, select the IP address of the iSCSI HBA to connect to the volume.

For Target portal, select the IP of the iSCSI Storage System containing the volume.

8. Click OKto close the Advanced Settings dialog.


60/182


9. Click OKagain to finish logging on. You will see the second session using the second path.

NOTE:HP recommends testing the failover to validate the multipath configuration.


61/182


Using the Microsoft iSCSI Software Initiator in conjunction with Accelerated iSCSI support

Besides the Accelerated iSCSI attached disk, it is possible to use the iSCSI Software to connect additionalvolumes using one of the NIC adapters. This section describes the different configuration steps required.

Virtual Connect Profile Configuration

From the Windows 'server's VC Profile, ensure at least one server NIC is connected to an iSCSInetwork or to a network where an iSCSI device relies.

Enabling jumbo frames on the NIC adapters

1. From the HP Network configuration Utility, select the port connected to the iSCSI-1 VC Network.2. Click Properties.3. Select theAdvanced Settingstab, then clickJumbo Packet.

Software iSCSI

Accelerated iSCSI


62/182


4. Select the MTU size supported by the network infrastructure and the iSCSI Storage System.

5. Click OKto close.6. Repeat step 1 to step 5 on the second adapter connected to the iSCSI-2 VC Network.

Installing the iSCSI Storage DSM for MPIO

A vendor-specific DSM for the iSCSI storage array can be installed for better performance. If not available,the Microsoft DSM can be used.

Microsoft iSCSI Software configuration with multiple NICs

1. Open the Microsoft iSCSI initiator.2. Select the Discoverytab.3. Click Discover Portal.


63/182


4. Enter the IP Address of the iSCSI target system connected to one of the server NIC networks (forexample, iSCSI-1 and iSCSI-2).

5. Click OK.6. Select the Targettab.7. A new discovered iSCSI target should appear in the list if a LUN has been correctly presented to the

server.

8. Select the new target (the Status will be Inactive), then click Properties.9. ClickAdd session.


64/182


10.Select Multipath,then clickAdvanced.

11.Configure the Advanced Settings as follows: For Local adapter, select Microsoft iSCSI Initiator.

For Initiator IP, select the IP address of the first NIC connected to iSCSI-1 VC Network.

For Target portal IP, select the IP of the iSCSI Storage System containing the volume.


65/182


12.Click OKto close the Advanced Settings dialog.13.Click OKto finish logging on.14.Repeat step 11 to step 15, this time using the second NIC adapter connected to iSCSI-2 VC Network

and using the corresponding IP address.

15.On the Sessions tab, you should see several sessions using the first and second paths.

16.Click OKto close the Properties window.

Two iSCSI targets are now available through the Microsoft iSCSI initiator; one is using the iSCSI

Accelerated ports (iSCSI HBAs) and the other is using the NIC adapters with software iSCSI.


66/182


Accelerated iSCSI with VMware ESX 4.1

After creating a VC profile with Accelerated iSCSI enabled, proceed with the following steps underVMware ESX 4.1:

1- Installing the NC551 / NC553 iSCSI drivers.2- Installing the Emulex OneCommand Manager. (This step is optional.)3- Configuring the iSCSI HBA ports.4- Connecting to the iSCSI volumes.

Installing the NC551 / NC553 iSCSI driversEnsure the latest Emulex iSCSI drivers for VMware ESX have been installed. Visit the following hp.com linksto download to latest be2iscsi drivers:

HP NC551m Dual Port FlexFabric

10Gb Converged Network Adapter

http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=us

HP NC551i Dual Port FlexFabric10Gb Converged Network Adapter

http://h20000.www2.hp.com/bizsupport/TechSupport/Driv

erDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135

HP NC553i 10Gb 2-port FlexFabric

Server Adapter

http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135

NOTE:Hardware and software iSCSI can be managed by vSphere without any additionalutility. Accelerated iSCSI uses a specific port (that is, vmhba) and requires theinstallation of HBA drivers included in the be2iscsi package.

If the CNA is properly configured, you can view the vmhba on the list of initiators available forconfiguration:

1. Log in to the vSphere Client, and select your host from the inventory panel.2. Click the Configurationtab, then click StorageAdaptersin the Hardware panel. The Emulex

OneConnect vmhba appear in the list of storage adapters.
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4194638&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4194735&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=4132827&lang=en&cc=us&prodTypeId=3709945&prodSeriesId=4132949&taskId=135http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4145269&taskId=135&prodTypeId=3709945&prodSeriesId=4145106&lang=en&cc=us


67/182


Installing Emulex OneCommand Manager

OneCommand Manager (OCM) is the Emulex utility to manage the NC551and NC553 10Gb FlexFabricConverged Network Adapters.

OCM is not required to connect a VMware server to an iSCSI volume but it delivers lots of advanced

management features, configuration, status monitoring and online maintenance.OCM for VMware can be divided into the following installations: OneCommand Manager for VMware vCenter OCM application agent in VMware ESX Server

OneCommand Manager for VMware vCenter

For more information, see the OneCommand Manager for VMware vCenter User Manualhttp://www-dl.emulex.com/support/vmware/vcenter/100/vcenter_user_manual.pdf.See the Installing and Uninstalling the OCM for VMware vCenter Components p

HP Virtual Connect With ISCSI Cookbook

Documents

Transcript of HP Virtual Connect With ISCSI Cookbook