Iscsi Cookbook

download Iscsi Cookbook

of 185

Transcript of Iscsi Cookbook

  • 7/18/2019 Iscsi Cookbook

    1/185

    1

    Technical white paper

    HP Virtual Connect with iSCSI

    Cookbook

    (4thEdition)

    Table of contents

    Purpose 3Introduction 3System requirements 4

    Converged Network Adapters Support 4Virtual Connect Modules Support 4Firmware and Software Support 6iSCSI Storage Target Support 6

    Networking recommendations 7Network considerations 7Keep it simple and short 7Flow Control recommendations 8Jumbo Frames (optional recommendation) 10iSCSI multipathing solutions 16

    Virtual Connect network scenarios 17Scenario 1: iSCSI network physically separated 18Scenario 2: iSCSI network logically separated 22Scenario 3: Direct-attached iSCSI Storage System 26

    Accelerated iSCSI 38Creating a Virtual Connect profile with AcceleratediSCSI 40Accelerated iSCSI with Microsoft Windows Server 43Accelerated iSCSI with VMware vSphere Server 70For more information about iSCSI 85

    Boot from iSCSI 86Boot from iSCSI : Creating a Virtual Connect profile 86Boot from iSCSI: Installing Microsoft Windows Server2012 95Boot from iSCSI: Installing Microsoft Windows Server2008 100

  • 7/18/2019 Iscsi Cookbook

    2/185

    2

    Boot from iSCSI: Installing VMware vSphere 5 106Boot from iSCSI: Installing VMware vSphere 4.1 110Boot from iSCSI: Installing Red Hat Enterprise Linux 5Update 4 119Boot from iSCSI: Installing Suse Linux Enterprise

    Server 11 123Troubleshooting 130

    Emulex iSCSI Initiator BIOS Utility 130Emulex OneCommand Manager (OCM) 135Problems found with OneCommand Manager 138Problems found during iSCSI Boot 139PXE booting problems 144iSCSI boot install problems with Windows Server 145VCEM issues with Accelerated iSCSI Boot 146iSCSI issues with HP StoreVirtual 4000 Storage 146

    Appendix 1- iSCSI Boot Parameters 148Mandatory iSCSI Boot Parameters entries 148Optional iSCSI Boot Parameters entries 152

    Appendix 2 - Dynamic configuration of the iSCSI BootParameters 155

    Windows 2008 DHCP server configuration 156Linux DHCP server configuration 164Format of DHCP option 43 for the Emulex FlexFabricAdapters 165Examples 166

    Appendix 3- How to monitor an iSCSI Network? 167Monitoring Disk Throughput on the iSCSI StorageSystem 167Monitoring Network and Disk Throughput on theiSCSI Host 168Analyzing Network information from the VirtualConnect interface 171Analyzing Virtual Connect Network performance 175Wireshark 177

    Acronyms and abbreviations 182Support and Other Resources 184

    Contacting HP 184Related documentation 185

  • 7/18/2019 Iscsi Cookbook

    3/185

    3

    Purpose

    The purpose of this Virtual Connect iSCSI Cookbook is to provide users of Virtual Connect with a better understanding ofconcepts and steps required when using iSCSI with Virtual Connect components. This document will help users to answersome of the typical questions on iSCSI: What are the network considerations to properly build an iSCSI network? Whatare the components supported by HP? How can I troubleshoot my iSCSI environment?

    In addition, this document describes some typical iSCSI scenarios to provide the reader with some valid examples of howHP Virtual Connect with iSCSI could be deployed within their environments.

    Tips and some troubleshooting information for iSCSI boot and install are also provided.

    Detailed information regarding Emulex requirements is subject to change, and readers should always refer to thedocumentation from the providers.

    Introduction

    The iSCSI standard implements the SCSI protocol over a TCP/IP network. While iSCSI can be implemented over any TCP/IPnetwork, the most common implementation is over 1 and 10 Gigabit Ethernet (GbE). The iSCSI protocol transports block-level storage requests over TCP connections. Using the iSCSI protocol, systems can connect to remote storage and use itas a physical disk although the remote storage provider or target may actually be providing virtual physical disks. iSCSIserves the same purpose as Fibre Channel in building SANs, but iSCSI avoids the cost, complexity, and compatibilityissues associated with Fibre Channel SANs.

    Because iSCSI is a TCP/IP implementation, it is ideal for new field deployments where no FC SAN infrastructure exists. AniSCSI SAN is typically comprised of software or hardware initiators on the host connected to an isolated Ethernetnetwork and some number of storage resources (targets). While the target is usually a hard drive enclosure or anothercomputer, it can also be any other storage device that supports the iSCSI protocol, such as a tape drive. The iSCSI stackat both ends of the path is used to encapsulate SCSI block commands into Ethernet Packets for transmission over IPnetworks.

    An interresting technology of iSCSI is the iSCSI boot, it allows servers to boot from an operating system image located ona remote iSCSI target. iSCSI Boot enables organizations to purchase less expensive diskless server, to provide rapiddisaster recovery and a more efficient usage of the storage, etc.

    Another nice feature is the accelerated iSCSI that can also be enabled with the HP FlexFabric adapters, it offloads theiSCSI function to the Converged Network Adapter rather than taxing the CPU of the server.

  • 7/18/2019 Iscsi Cookbook

    4/185

    4

    System requirements

    When using HP Virtual Connect technology, iSCSI Boot and Accelerated iSCSI is only supported with the followingcomponents:

    Converged Network Adapters Support

    Integrated NC553iDual Port FlexFabric 10Gb Adapter (Intel based BladeSystem G7 servers) Integrated NC551iDual Port FlexFabric 10Gb Adapter (AMD based BladeSystem G7 servers) HP NC551mDual Port FlexFabric 10Gb Converged Network Adapter HP NC553mDual Port FlexFabric 10Gb Converged Network Adapter HP FlexFabric 10Gb 2-port 554FLBAdapter HP FlexFabric 10Gb 2-port 554MAdapter

    NOTE: iSCSI Boot is available as well on Virtual Connect with the QLogic QMH4062 1GbE iSCSI Adapter but with somerestrictions. The QMH4062 iSCSI settings cannot be managed by a Virtual Connect profile but they can manually be setthrough the Qlogic Bios (CTRL+Q during Power-on Self-Test). The constraints to remember is that during a VirtualConnect profile move, the iSCSI boot settings will not be saved and reconfigured on the target server.

    Virtual Connect Modules Support

    HP Virtual Connect FlexFabric 10Gb/24-port Module HP Virtual Connect Flex-10 10Gb Ethernet Module HP Virtual Connect Flex-10/10D Module

    NOTE: 10Gb KR-based Ethernet switches (like Procurve 6120XG, Cisco 3120G) can be used as well for Accelerated iSCSIboot but this is not covered in this document

    NC553m NC551m

    554FLB 554M

  • 7/18/2019 Iscsi Cookbook

    5/185

    5

    Virtual Connect with iSCSI Summary support

    When using HP Virtual Connect technology, iSCSI Boot and Accelerated iSCSI is only supported with the followingcombination of devices:

    BladeSystem Gen8

    +

    Virtual Connect FlexFabricBladeSystem Gen8

    with FlexFabric 10Gb 2-port 554FLB orFlexFabric 10Gb 2-port 554M

    +

    Virtual Connect Flex-10or Flex-10/10DMinimum VC 3.10 and above

    BladeSystem Gen8with FlexFabric 10Gb 2-port 554FLB or

    FlexFabric 10Gb 2-port 554M

    BladeSystem G7

    +

    Virtual Connect FlexFabricBladeSystem G7

    with NC551i / NC553i IntegratedCNA orNC551m/NC553m FlexFabric Adapter

    +

    Virtual Connect Flex-10or Flex-10/10DMinimum VC 3.10 and above

    BladeSystem G7with NC551i / NC553i IntegratedCAN or

    NC551m/NC553m FlexFabric Adapter

  • 7/18/2019 Iscsi Cookbook

    6/185

    6

    BladeSystem G6

    + +

    Virtual Connect FlexFabricNC551m / NC553m

    10Gb 2-port FlexFabricAdapter

    BladeSystem G6Latest System BIOS

    + +

    Virtual Connect Flex-10or Flex-10/10DMinimum VC 3.10 and above

    NC551m / NC553m10Gb 2-port FlexFabric

    AdapterBladeSystem G6Latest System BIOS

    NOTE: HP BladeSystem c-Class Integrity Server Blades do not support Accelerated iSCSI and iSCSI boot with VirtualConnect.

    Firmware and Software Support

    HP recommends to use the latest Service Pack for Proliant, for more information, seehttp://h18004.www1.hp.com/products/servers/service_packs/en/index.html

    Requirements for Accelerated iSCSI and iSCSI boot:

    VCM 3.10 (or above) is required in order to have Accelerated iSCSI and iSCSI boot support. One Command OS tool. (recommended) be2iSCSI driver (FlexFabric Adapters drivers). be2iSCSI Driver Update Disk for iSCSI boot installs. iSCSI target. DHCP server (optional).NOTE: If the VC firmware is downgraded to a version older than 3.10, the iSCSI boot parameter configuration is notsupported and all iSCSI boot parameters are cleared.

    iSCSI Storage Target Support

    Any storage system with iSCSI host ports is supported (e.g. the HP 3PAR StoreServ, HP EVA 6000 Storage, HPStoreVirtual 4000 Storage, HP MSA P2000 Storage, etc.).

    http://h18004.www1.hp.com/products/servers/service_packs/en/index.htmlhttp://h18004.www1.hp.com/products/servers/service_packs/en/index.htmlhttp://h18004.www1.hp.com/products/servers/service_packs/en/index.html
  • 7/18/2019 Iscsi Cookbook

    7/185

    7

    Networking recommendations

    Network considerations

    When constructing an iSCSI SAN with HP Virtual Connect, some network considerations must be taken into account.

    Dont think of iSCSI network as just another LAN flavor, IP storage needs the same sort of design thinking applied to FCinfrastructure particularly when critical infrastructure servers can boot from a remote iSCSI data source.

    Network performance is one of the major factors contributing to the performance of the entire iSCSI environment. If thenetwork environment is properly configured, the iSCSI components provide adequate throughput and low enoughlatency for iSCSI initiators and targets. But if the network is congested and links, switches or routers are saturated, iSCSIperformance suffers and might not be adequate for some environments.

    Here are some important tips and tricks to think about:

    Keep it simple and short

    With iSCSI, it is possible to route packets between different networks and sub networks but keep in mind every route and

    hop a packet must use, network latency will be added and affect tremendously the performance between iSCSI initiatorand iSCSI target.

    A network switch will also add latency to the delivery time from the iSCSI packet, so we recommend keeping the distanceshort and ovoid any router or network switches in between the connection. It simply costs performance, reduces theIOPs per secondand the chances of storage traffic competing with other data traffic on congested inter switch links.

    To avoid bottlenecks, inter switch links should be sized properly and use stacking cables, 10-Gigabit Ethernet uplinks, orlink aggregation or port trunking.

    Networking considerations include:

    Minimizing switch hops Maximizing the bandwidth on the inter-switch links if present. Use of 10-Gigabit Ethernet uplinks

  • 7/18/2019 Iscsi Cookbook

    8/185

    8

    Flow Control recommendations

    Storage vendors usually have iSCSI SAN design recommendations; enabling Flow Control is one of the most importantones.

    Ethernet Flow control is a mechanism to manage the traffic flow between two directly connected devices and makes useof pause frames in order to notify the link-partner to stop sending traffic when congestion occurs. It helps resolve in anefficient manner any imbalance in network traffic between sending and receiving devices.

    Enabling Flow Control is highly recommended by iSCSI storage vendors. It must be enabled globally across all switches,server adapter ports and NIC ports on the storage node.

    Enabling Flow Control on iSCSI Storage Systems

    Flow Control can usually be enabled on all iSCSI Storage Systems. For more specific information about how to enableFlow Control, see the Storage Systems manufacturer documentation.

    On aHP StoreVirtual 4000 Storage,Flow Control can be set from the TCP/IP settings page within the CMC console:

    Enabling Flow Control on Network Switches:

    Flow Control should be enabled on every switch interfaces connected to the Storage device.

    See the switchs manufacturer documentationfor more information about Flow Control.

    NOTE: On ProCurve switches, if the port mode is set to "auto" and flow control is enabled on the HP StoreVirtual 4000Storageport, the switch port will auto-negotiate flow control with the Storage device NIC.

    http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659
  • 7/18/2019 Iscsi Cookbook

    9/185

    9

    Enabling Flow Control on Virtual Connect:

    Flow Control is enabled by default on all downlink ports. To enable Flow control on all VC ports, including uplink ports,enter:

    ->set advanced-networking FlowControl=on

    NOTE: Be careful, this command can result in data traffic disruption!

    Enabling Flow Control on iSCSI Hosts

    By default, Flow Control is enabled on every network interfaces when Accelerated iSCSI is enabled.

    For Software iSCSI, it might be necessary to enable Flow control at the NIC/iSCSI initiator level.

  • 7/18/2019 Iscsi Cookbook

    10/185

    10

    Jumbo Frames (optional recommendation)

    Jumbo frames (MTU>=9000 bytes) are also frequently recommended by iSCSI storage vendors as it can significantlyincrease the iSCSI performance.

    Jumbo Frames have many benefits particularly for iSCSI traffic as it reduces the fragmentation overhead whichtranslates straight away to lower CPU utilization. It gives as well more aggressive TCP dynamics, leading to greaterthroughput and better response to certain types of loss.

    Jumbo frames must be correctly configured end-to-end on the network, from the storage to the Ethernet switches andup to the server ports.

    NOTE: Usage of the Jumbo frames in some environment can cause more problems than helps with performance. This isfrequently due to misconfigured MTU sizes but also because some devices support different max MTU sizes. So if you areunsure about whether your routers and other devices support larger frame sizes, keep the frame size at the defaultsetting.

    Enabling Jumbo Frames on iSCSI Storage Systems

    Jumbo frames can usually be used with all iSCSI Storage Systems and are mostly of the time enabled by setting the MTU

    size on an interface.The frame size on the storage system should correspond to the frame size on iSCSI Hosts (Windows and Linuxapplication servers).

    For more specific information about how to enable Jumbo frames, see the Storage Systems manufacturerdocumentation.

    OnHP StoreVirtual 4000 Storage,jumbo frames can be set from the TCP/IP settings page within the CMC console:

    NOTE: On the storage System, make sure to set a MTU size above 8342 bytes.

    Any Storage Systems configured with frames size below 8342 bytes will result in a MTU negotiation failure with the CNA,causing the traffic to run at the default Ethernet standard frame size (i.e. 1518 bytes).

    If 9000 bytes is needed by any specific reason then Software iSCSI must be configured instead of Accelerated iSCSI.

    http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659
  • 7/18/2019 Iscsi Cookbook

    11/185

    11

    Enabling Jumbo Frames on Network Switches

    Jumbo frames must be enabled across all ports of the iSCSI dedicated VLAN or hardware infrastructure (always end-to-end). See the switchs manufacturer documentation.

    NOTE: Not all switches support both Jumbo Frames and Flow Control if you have to pick between the two, choose FlowControl.

    Enabling Jumbo Frames on Virtual Connect

    Jumbo Frames are enabled by default on Virtual Connect, there is no configuration required.

    Enabling Jumbo Frames on iSCSI Hosts

    There are two different procedures for enabling Jumbo Frames on servers; one for Accelerated iSCSI (uses a dedicatedHBA port for iSCSI traffic) and one for Software iSCSI (uses a port of an existing NIC for iSCSI traffic):

    With Accelerated iSCSIJumbo Frames are enabled by default on the FlexFabric 10Gb NC551 and NC553 CNA with Accelerated iSCSI, there is noconfiguration required. The MTU size is auto-negotiated during the TCP connection with the iSCSI target.

    For optimal performance, the Max MTU size supported by iSCSI Accelerated mode is limited to 8342 bytes and cannot bemodified.

    Checking MTU size under Windows:To see the MTU size that has been auto-negotiated under Windows, it is necessary to install the Emulex OneCommand

    Manager (OCM)utility and when launched, to select the iSCSI target and to click on Sessions

  • 7/18/2019 Iscsi Cookbook

    12/185

    12

    The TCPMSSvalue used for this connection displayed in the Connection Negotiated Login Properties section, showsindirectly the MTU that has been negotiated:

    When TCPMSSdisplays 1436, the MTU negotiated size is 1514.When TCPMSSdisplays 8260, the MTU negotiated size is 8342.

    Checking MTU size under VMware ESX:Like under MS Windows, MTU is automatically configured and the user has no control on this setting. Note that withVMware, there is no way at this time to view the configured MTU/MSS.

  • 7/18/2019 Iscsi Cookbook

    13/185

    13

    With Software iSCSI

    With Software iSCSI, Jumbo frames must be enabled under the Operating System on each adapter/vSwitch running iSCSI:

    Under MS Windows Server 2012:1.In Control Panel, select View network status and tasksin the Network and Internet section.2.In the View your active networks section, click on the network adapter used for iSCSI and click Properties.3.Click Configure.4.Click the Advancedtab5.Select Jumbo Mtuand change the MTU value.

  • 7/18/2019 Iscsi Cookbook

    14/185

    14

    Under MS Windows Server 2008:1.Right-click Networkin Start Menu and click Properties.2.Select the network adapter used for iSCSI and click Properties.3.Click Configure.4.Click the Advancedtab5.Select Packet Sizeand change the MTU value

    6.Click OKto apply the changes.7.To see the MTU value configured under Windows, go to OCM, select the adapter used for iSCSI, the MTU is displayed

    under the Current MTUfield in the Port informationtab:

    For more information, refer to the OS Vendors documentation.

  • 7/18/2019 Iscsi Cookbook

    15/185

    15

    Under ESX:Run this command to set MTU for the vswitch:esxcfg-vswitch -m 9000 vSwitch

    To check the MTU configuration, enter:esxcfg-vswitch l vSwitch

    Switch Name Num Ports Used Ports Configured Ports MTU UplinksvSwitch2 128 3 128 9000 vmnic4,vmnic5

    For more information, seeiSCSI and Jumbo Frames configuration on ESX/ESXi (KB: 1007654).http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007654

    Testing Jumbo Frames:The Jumbo frames configuration can be tested by using the PING command frequently available on iSCSI StorageSystems:

    Test ping from the Storage System to the iSCSI hostsinterface using 8300 bytes for MTU:

    The Ping result should appear similar to:PING 192.168.5.14 (192.168.5.14) from192.168.5.20 : 8300(8328) bytes of data.

    8308 bytes from 192.168.5.14: icmp_seq=5 ttl=64 time=47.7 ms

    http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007654http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1007654
  • 7/18/2019 Iscsi Cookbook

    16/185

    16

    iSCSI multipathing solutions

    The use of Multipathing solutions is highly recommended for load balancing and failover to improve iSCSI performanceand availability.

    Multipathing solutions use redundant physical path componentsadapters, cables, and switchesto create logical"paths" between the server and the storage device. In the event that one or more of these components fails, causing thepath to fail, multipathing logic uses an alternate path for I/O so that applications can still access their data.

    For the Operating System, this multipathing means the need of an intelligent path manager called Multipath I/O (alsoknown as MPIO) to log in multiple sessions and to failover if needed among multiple iSCSI Host Bus Adapters (HBAs).

    MPIO is a key component to building a highly available, fault tolerant iSCSI SAN solution. MPIO technologies provide forthe following:

    I/O path redundancy for fault tolerance I/O path failover for high availability I/O load balancing for optimal performanceFor Microsoft Windows OS, storage vendors usually provide a vendor-specific DSM (Device Specific Module) to optimizemultipathing using the Microsoft MPIO framework. This Vendor-specific module (DSM) for multi-path I/O must beinstalled under the Operating System, consult your Storage provider web site for more information.

  • 7/18/2019 Iscsi Cookbook

    17/185

    17

    Virtual Connect network scenarios

    For security and performance purposes, it is recommended that the iSCSI network be separated either logically (usingdifferent VLANs) or physically (using different physical switches) from the ordinary data network. Isolating the iSCSItraffic helps to improve response times, reliability and prevent bottlenecks and congestion, it also helps to address theTCP/IP overhead and flow control issues inherent in an Ethernet network.

    Another recommendation to maximize availability and optimal performance is to use an iSCSI redundant network pathfrom Virtual Connect (and therefore from the server) to the storage system. This enables failover mechanism in case ofpath failure among multiple iSCSI HBA. The use of Multipath I/O software running under the OS (Windows, Linux andVMware) is required to provide an automatic means of persisting I/O without disconnection.

    For a more complete step by step typical scenario configuration, please refer to the Virtual Connect FlexFabric Cookbookhttp://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdf

    http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdfhttp://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdfhttp://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02616817/c02616817.pdf
  • 7/18/2019 Iscsi Cookbook

    18/185

    18

    Scenario 1: iSCSI network physically separated

    In this first scenario, the iSCSI network is physically separated from the ordinary data network using a different switchinfrastructure.

    PROS

    This scenario is the best for performance, for better latency and is the recommended scenario. It maximizes thebandwidth availability, here the iSCSI traffic do not have to fight for bandwidth as we have a dedicated infrastructure forthe storage traffic.

    CONS

    This scenario uses more switches, more VC uplinks and thus more cabling. The solution cost is increased

    Figure 1:Virtual Connect scenario using a separated iSCSI network

    EnclosureUID

    Enclosure Interlink

    PS3

    PS2

    PS1

    PS6

    PS5

    PS4

    OA1 OA2

    Remove management modules before ejecting sleeve

    FAN6

    FAN10

    FAN1

    FAN5

    21

    3

    5

    7

    4

    6

    8

    UID

    X1

    HPVCFlex-10EnetModule X1 X 2 X 3 X 4 X 5 X 6

    SHARED

    X 7 X 8

    SHARED:UPLINKorX-LINK

    UID

    X1

    HPVCFlex-10EnetModule X1 X 2 X 3 X 4 X 5 X 6

    SHARED

    X 7 X 8

    SHARED:UPLINKorX-LINK

    HP4GbVC-FCModule

    UID

    1 2 3 4

    HP4GbVC-FCModule

    UID

    1 2 3 4

    iLO

    UID

    ActiveReset

    iLO

    UID

    ActiveReset

    802.3ad LAG

    802.1Q Trunk

    LAN Switch 1 LAN Switch 2

    PRODUCTION NETWORK IP STORAGE NETWORK

    iSCSI network

    Rear view

    c7000 enclosure

    VC Modules

    PRODUCTION NETWORK

    802.3ad LAG

    802.1Q TrunkvN et-iSCSI-1 vNet-iSCSI-2

    iSCSI Storage Device

    Port 1 Port 2

    Active

    Virtual IP

    Active

    Prod-vN et-1 Prod-vN et-2

    Active Active

  • 7/18/2019 Iscsi Cookbook

    19/185

  • 7/18/2019 Iscsi Cookbook

    20/185

    20

    Defining two iSCSI networks vNet

    Create a vNet and name it vNet-iSCSI-1

    On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet Ether the Network Name of vNet-iSCSI-1 Select Smart Link, but, do NOT select any of the other options (i.e.; Private Networks etc.) Select Add Port, then add one port from Bay 1 For Connection Mode, use Failover Select Apply

  • 7/18/2019 Iscsi Cookbook

    21/185

    21

    Create a vNet and name it vNet-iSCSI-2

    On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet Ether the Network Name of vNet-iSCSI-2 Select Smart Link, but, do NOT select any of the other options (ie; Private Networks etc.) Select Add Port, then add one port from Bay 2

    For Connection Mode, use Failover Select Apply

    NOTE: By creating TWO vNets we have provided a redundant path to the network. As each uplink originates from adifferent VC module and vNet both, uplinks will be active. This configuration provides the ability to lose an uplink cable,network switch or depending on how the iSCSI ports are configured at the server ( iSCSI Software Initiator supportingfailover), even a VC module.

    Smart LinkIn this configuration Smartlink SHOULD be enabled. Smartlink is used to turn off downlink ports withinVirtual Connect if ALL available uplinks to a vNet or SUS are down. In this scenario if an upstream switch or all cables to avNet were to fail on a specific vNet, VC would turn off the downlink ports connect to that vNet, which would then forcethe iSCSI Software Initiator to fail-over to the alternate NIC.

    Connection Mode: Failovershould be enabled here as only a single external uplink port is used for this network. Withmultiple uplink ports, the connection mode Auto can be used to enable the uplinks to attempt to form aggregation

    groups using IEEE 802.3ad link aggregation control protocol. Aggregation groups require multiple ports from a singleVC-Enet module to be connected to a single external switch that supports automatic formation of LACP aggregationgroups, or multiple external switches that utilize distributed link aggregation.

  • 7/18/2019 Iscsi Cookbook

    22/185

  • 7/18/2019 Iscsi Cookbook

    23/185

    23

    Figure 5:Virtual Connect logical view of an iSCSI ESX host

    HP VCF l ex F abr i c10G b/24- P or tM odul e

    SHARED: U P L IN K or X - L IN KX3 X4X1 X2 X5 X6 X7 X8

    UID

    LOM 1 LOM 2

    Console

    Hypervisor Host

    VM Guest VLAN

    1Gb

    1.5Gb

    3.5Gb

    4Gb

    1Gb

    1.5Gb

    3.5Gb

    4Gb

    A

    B

    C

    D

    A

    B

    C

    D

    VMotion

    10Gb 10Gb

    vmnic0

    vmnic2

    vmnic1

    vmnic3

    vmnic4 vmnic5

    iSCSI2 iSCSI3

    VLAN

    UT

    101

    103

    104

    HPVCF l ex F abr i c10G b/24- P or tM odul e

    SHARED: U P L IN K or X - L IN KX3 X4X1 X2 X5 X6 X7 X8

    UID

    UplinkSet_1Enc0:Bay1:X1,X2

    Prod 802.1Q Trunk(VLANs 101 through 105)

    VC FlexFabric 10Gb Enc0:Bay1 VC FlexFabric 10Gb Enc0:Bay2

    Virtual Connect Domain

    NIC

    1

    NIC

    2

    UID

    HPProLiant

    BL460cG6

    serialscsi

    1por t10k

    146GB

    serialscsi

    1por t10k

    146GB

    UT

    102

    UplinkSet_2Enc0:Bay2:X1,X2

    Prod 802.1Q Trunk(VLANs 101 through 105)

    iSCSI Network UT

    105

    LOM 2

    2B FlexiSCSI

    2C FlexNIC

    2D FlexNIC

    2A FlexNIC

    LOM 1

    1C FlexNIC

    1D FlexNIC

    1A FlexNIC

    1B FlexiSCSI

    VLAN_101-1 VLAN_102-1

    VLAN_103-1

    iSCSI_1

    VLAN_104-1

    VLAN_101-2 VLAN_102-2

    VLAN_103-2

    iSCSI_2

    VLAN_104-2

    Internal Stacking link

    vSwitch

  • 7/18/2019 Iscsi Cookbook

    24/185

    24

    Defining a first Shared Uplink Set (VLAN-trunk-1)

    Create a SUS named UplinkSet_1

    On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as UplinkSet_1 Select Add Port, then add two ports from Bay 1 Add Networks as follows (to add a network, right click on the grey bar under the Associate Networks (VLAN) header,

    the select ADD;

    o VLAN_101-1= VLAN ID = 101= CONSOLEo VLAN_102-1= VLAN ID = 102= VMOTIONo VLAN_103-1= VLAN ID = 103= First VM Guest VLANo VLAN_104-1= VLAN ID = 104= Second VM Guest VLAN

    (More VM Guest VLANs can be defined here)

    o iSCSI_1= VLAN ID=105 Enable SmartLinkon ALL networks Leave Connection Mode as Auto(this will create a LCAP port channel if the upstream switch is properly configured) Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native to Enabled Click Apply

  • 7/18/2019 Iscsi Cookbook

    25/185

    25

    Defining a second Shared Uplink Set (VLAN-trunk-2)

    Create a SUS named UplinkSet_2

    On the Virtual Connect Home page, select Define, Shared Uplink Set Insert Uplink Set Name as UplinkSet_2 Select Add Port, then add two ports from Bay 2 Add Networks as follows (to add a network, right click on the grey bar under the Associate Networks (VLAN) header,

    the select ADD;

    o VLAN_101-2= VLAN ID = 101= CONSOLEo VLAN_102-2= VLAN ID = 102= VMOTIONo VLAN_103-2= VLAN ID = 103= First VM Guest VLANo VLAN_104-2= VLAN ID = 104= Second VM Guest VLAN

    (More VM Guest VLANs can be defined here)

    o iSCSI_2= VLAN ID=105 Enable SmartLinkon ALL networks Leave Connection Mode as Auto(this will create a LCAP port channel if the upstream switch is properly configured) Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set Native to Enabled Click Apply

  • 7/18/2019 Iscsi Cookbook

    26/185

    26

    Scenario 3: Direct-attached iSCSI Storage System

    In this third scenario, an iSCSI device is directly connected to the Virtual Connect Domain without any switchinfrastructure. This scenario uses more VC uplinks than the second scenario but no additional or dedicated switches arerequired like in scenario 1 to only connect a SCSI disk storage enclosure. This reduces the entire solution cost andcomplexity.

    PROS

    Cost is greatly reduced as no additional switch is required.

    CONS

    There are several limitations.

    Direct-attached Limitations

    There are three important limitations that administrator must be aware of:

    When an iSCSI storage device is directly connected to a VC Domain, this iSCSI device is ONLY accessible to the serversbelonging to this Virtual Connect Domain.

    iSCSI Storage Systems that are sharing the same ports for both iSCSI host connectivity traffic and LAN management(also known as in-band management) can ONLY be managed from the Virtual Connect Domain.

    The only network interface bond supported on the iSCSI Storage system is Active-Passive. VC Active/Standby iSCSI vNet configuration is not supported.iSCSI Storage systems can be divided into two categories, the ones with out-of-band management (separate ports areused for management and host traffic) and the ones with in-band management (use the same ports for managementand host traffic). So the direct-attached scenario will be divided into two sub-scenarios, according to the type ofmanagement you may use:

    Scenario 3-A: Direct-attached iSCSI device with out-of-band management (using separate ports for management andhost traffic).

    Scenario 3-B: Direct-attached iSCSI device with in-band management (using the same ports for management and hosttraffic).

    Scenario 3-C: Low cost Direct-attached iSCSI device with a c3000 enclosure.

  • 7/18/2019 Iscsi Cookbook

    27/185

    27

    Scenario 3-A: Direct-attached iSCSI device with out-of-band management (using separate ports for managementand host traffic)

    Figure 6:Virtual Connect direct-attached iSCSI target scenario with out-of-band management

    iSCSI Hosts

    802.3ad LAG

    802.1Q Trunk

    LAN Switch 1 LAN Switch 2

    PRODUCTION NETWORK

    Directly attached

    iSCSI Storage Device

    Port 1

    Active

    Port 2

    Passive

    IP STORAGE NETWORK

    iSCSI host network

    Rear view

    c7000 e nclosure

    VC Modules

    PRODUCTION NETWORK

    802.3ad LAG

    802.1Q Trunk

    Management network

    Storage

    Management

    Console

    vNet-iSCSI-1 vNet-iSCSI-2

    Active

    Virtual IP

    Active

    Mgt

    Port

    In this scenario A, the iSCSI target is connected directly to the VC Domain using two Active/Active vNet (blue and red inthe above diagram) to provide iSCSI LAN access to the servers located inside the enclosure (only).

    The iSCSI storage device is using an out-of-band management network; this means that dedicated port(s) separatedfrom the iSCSI host traffic, are available for the management/configuration. Therefore the iSCSI device can be managedfrom anywhere on the network.

    Both software and hardware iSCSI can be implemented on the iSCSI hosts without limitation.

  • 7/18/2019 Iscsi Cookbook

    28/185

    28

    NOTE: The direct-attached scenario supports only one iSCSI device per VC Active/Active network. To support moredirect-attached iSCSI devices, its mandatory to create for each iSCSI device an Active/Active VC network.

    LAN Switch LAN Switch

    iSCSI

    Target 1

    Direct-attached iSCSI devices

    iSCSI

    Target 2

    vNet1-iSCSI-1

    vNet1-iSCSI-2

    vNet2-iSCSI-1

    vNet2-iSCSI-2

    Rear view

    c7000 e nclosure

    VC Modules

    Production Network

    This limitation of one device per VC Active/Active network implies that you cannot directly attach multipleHPStoreVirtual 4000 Storage nodes to the same VC domain because in a multi node environment, StoreVirtual nodes needto talk to each other and this requirement is not possible as VC d ont switch traffic between different VC networks.

    LAN Switch LAN Switch

    HP P4000 series

    Node 1

    Direct-attached HP LeftHand P4000 series

    Rear view

    c7000 enclosure

    VC Modules

    Production Network

    HP P4000 series

    Node 2

    http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659
  • 7/18/2019 Iscsi Cookbook

    29/185

    29

    Scenario 3-B: Direct-attached iSCSI device with in-band management (using the same ports for management andhost traffic):

    Figure 7:Virtual Connect direct-attached iSCSI target scenario with in-band management

    802.3ad LAG

    802.1Q Trunk

    LAN Switch 1 LAN Switch 2

    PRODUCTION NETWORK IP STORAGE NETWORK

    iSCSI host network +management network

    Rear view

    c7000 enclosure

    VC Modules

    PRODUCTION NETWORK

    802.3ad LAG

    802.1Q Trunk

    StorageManagement

    ConsoleiSCSI Hosts

    vNet-iSCSI-1 vNet-iSCSI-2

    Directly attached

    iSCSI Storage Device

    Port 1

    Active

    Port 2

    Passive

    Active

    Virtual IP

    Active

    In this scenario B, the iSCSI device is again connected directly to the VC Domain using two Active/Active vNet (blue and

    red in the above diagram) but here the iSCSI device is not using a dedicated interface for management which means thatthe same ports are used for both the management and the iSCSI host traffic (known as in-band management device).

    This implies due to the Virtual Connect technologies that you can only manage and configure the iSCSI storage devicefrom the Virtual Connect Domain (i.e. only from a server located inside the enclosure and connected to the iSCSI vNet).

    This means that you need a dedicated server for the storage system management and that all SNMP trap notificationsfor hardware diagnostics, events and alarms will be sent only to this management console.

    Both software and hardware iSCSI can be implemented here without limitation on the iSCSI hosts.

    For more information about Accelerated iSCSI, please refer to theAccelerated iSCSIsection.

  • 7/18/2019 Iscsi Cookbook

    30/185

    30

    Scenario 3-C: Low cost Direct-attached iSCSI device with or without out-of-band management

    In this scenario, the iSCSI Storage device is directly connected to a c3000 enclosure filled with only one Virtual Connectmodule installed into interconnect Bay 1. By design, interconnect Bay 1 in c3000 enclosures always map to all embeddedNICs/CNAs, so all embedded server ports are all connected through the same interconnect module.

    This scenario can be proposed to customers looking for cost reduction and not necessarily for high availability.

    PROS

    Cost is even more greatly reduced as only one VC module is required.

    CONS

    Risk of network failure. In addition to the direct-attached limitations described before, this scenario does not offer afully redundant design: the VC module is a single point of failure, if it fails, the entire network communication will stopand all blades will go offline.

    Figure 8:Virtual Connect direct-attached iSCSI target scenario using a single VC module

    PS5

    PS4FAN6

    PS6FAN3FAN1PS3

    PS2

    PS1 FAN4

    OA1/iLOEnc.Interlink Encl osureUIDOA2/iLO

    iSCSI Hosts

    802.1Q Trunk

    LAN Switch 1 LAN Switch 2

    PRODUCTION NETWORK

    Directly attached

    iSCSI Storage Device

    Port 1

    Active

    Port 2

    Passive

    IP STORAGE N ETWORK

    Rear view

    c3000 enclosurefilled with only one VC module

    VC Flex-10/10D Module

    in Bay 1

    PRODUCTION NETWORK

    802.1Q TrunkvNet-iSCSI-1 vN et-iSCSI-2

    Active

    Virtual IP

    Active

    Optional

    out-of-band

    managementMgt

    Port

  • 7/18/2019 Iscsi Cookbook

    31/185

    31

    Virtual Connect Network configuration

    Scenario 3-A and Scenario 3-B

    The Virtual Connect network configuration is the same for Scenario 3-A and Scenario 3-B.

    You must define two Active/Active iSCSI Virtual Connect networks (vNet): vNet-iSCSI-1 and vNet-iSCSI-2 like inScenario 1.

    These two vNets can exclusively be used for the direct attachment of the iSCSI storage device. Each vNet can only support one VC uplink. Additional vNets are required for the non-iSCSI traffic.

    Scenario 3-C

    You must define two Active/Active iSCSI Virtual Connect networks (vNet): vNet-iSCSI-1 and vNet-iSCSI-2 like inScenario 1.The only difference being that uplink ports from the same VC Module in Bay 1 are used for both vNets.

    These two vNets can exclusively be used for the direct attachment of the iSCSI storage device. Each vNet can only support one VC uplink. Each vNet is configured with one VC uplink port from the module Bay 1. Additional vNets are required for the non-iSCSI traffic.

    Note: The Direct-attached scenario is not supported with VC Active/Standby iSCSI vNet configuration. A standby VCuplink is always seen as active by an upstream device so having no way to detect the standby state of the VC uplink, astorage system would incorrectly send traffic to the standby port, causing communication issues.

  • 7/18/2019 Iscsi Cookbook

    32/185

    32

    Connecting the Direct-Attached iSCSI SAN to the VC Domain

    Connect the VC uplink port of the two vNet to the network interfaces of the iSCSI device.

    Figure 9: Out-of-band management configuration with a HP StoreVirtual 4000 Storage Solution (Scenario 3-A)

    Mgmt

    2 1 UID

    P

    ORT1

    10GbE

    SFP

    P

    ORT2

    LA

    LA

    TXRXTXRX

    NC550sp

    HP VCFlexFabric 10Gb/24-Port Module

    S H A R E D : U P L I N K or X - L I N KX3 X4X1 X2 X5 X6 X7 X8

    UID

    HPVCFlexFabric 10Gb/24-Port Module

    S H A R E D : U P L I N K or X - L I N KX3 X4X1 X2 X5 X6 X7 X8

    UID

    NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2

    Prod-vNet-1 Prod-vNet-2

    HP StoreVirtual 4000 Storage

    NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2

    iSCSI Hosts

    Blade Servers

    vNet-iSCSI-1 vNet-iSCSI-2ActiveActive

    Directly attached iSCSI device

    using out-of-band management

    HP VC FlexFabric

    Management network

    Figure 10: In-band management configuration with a HP StoreVirtual 4000 Storage Solution (Scenario 3-B)

    Mgmt

    2 1 UID

    HP VC FlexFabric 10Gb/24-Port Module

    S H A R E D : U P L I N K or X - L I N KX3 X4X1 X2 X5 X6 X7 X8

    UID

    HP VC F l e x F a b ri c 1 0 G b / 2 4 -P o rt M o d u l e

    S H A R E D : U P L I N K or X - L I N KX3 X4X1 X2 X5 X6 X7 X8

    UID

    NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2

    Prod-vNet-1 Prod-vNet-2

    HP StoreVirtual 4000 Storage

    NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2NIC

    1

    NIC

    2

    UID

    HPProLiant

    BL460c

    CNA 1 CNA 2NIC

    1

    NIC

    2

    UID

    HP ProLiant

    BL460c

    CNA 1 CNA 2

    iSCSI Hosts

    Blade Servers

    vNet-iSCSI-1 vNet-iSCSI-2ActiveActive

    Directly attached iSCSI device

    using in-band management

    HP VC FlexFabric

  • 7/18/2019 Iscsi Cookbook

    33/185

    33

    Preparing the network settings of the storage system

    The following describes a basic network configuration of a HP StoreVirtual 4000 Storage Solution:

    Open a remote console on the StoreVirtual using the StoreVirtual iLO or connect a keyboard/monitor. Type start, and press Enterat the log in prompt. Press Enterto log in or type the Usernameand passwordif already configured. When the session is connected to the storage system, the Configuration Interface window opens. On the Configuration Interfacemain menu, tab to Network TCP/IP Settings, and press Enter. Tab to select the first network interface and press Enter. Enter the host name, and tab to the next section to configure the network settings. Enter a Private IP address like 192.168.5.20/ 255.255.255.0with 0.0.0.0for the Gateway address:

    Tab to OK, and press Enterto complete the network configuration. Press Enteron the confirmation window. A window opens listing the assigned IP address that will be used later on to

    configure and manage the P4300.

    Configuring the Storage Management Server

    Out-of-band management iSCSI device (Scenario 3-A and 3-C)An iSCSI device with an out-of-band management does not require any particular configuration as the managementconsole can reside anywhere on the network.

    In-band management iSCSI device (Scenario 3-B and 3-C)

    An iSCSI device with an in-band management requires a little bit more attention and particularly when network interfacebonding is enabled on the iSCSI Storage System.

    The following configuration can be followed for an in-band management iSCSI Storage System:

    Create a Virtual Connect server profile for the Management Storage server:o Assign NIC1 and NIC2 to the management network.o Assign NIC3 and NIC4 to the iSCSI direct-attached VC networks vNet-iSCSI-1 and vNet-iSCSI-2.

  • 7/18/2019 Iscsi Cookbook

    34/185

    34

    Figure 11: Example of a VC profile used by the Storage Management Server

    Start the server and install Windows Server 2003 or 2008. At the end of the installation, assign a private IP address to NIC3(e.g. 192.168.5.150) and toNIC4(e.g. 192.168.5.151)

    NOTE: Make sure to use here the same IP subnet addresses as the one set in the Storage System!

    NOTE: Two IP addresses are used in this configuration to ensure that the management server stays always connectedto the iSCSI device regardless of the iSCSI device bonding configuration (no bonding or Active-Passive bond enabled).

    Despite the use of the Active/Active iSCSI vNet configuration, NIC teaming cannot be used on the management serverotherwise the network connection to the iSCSI device will fail. This is due to the direct-attached design and the use ofNIC bonding on the iSCSI device. There is no such limitation with Scenario 3-A.

    Install the Storage Management software (e.g. HP StoreVirtual Management Software, this is the software used toconfigure and manageHP StoreVirtual Storage).

    http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659
  • 7/18/2019 Iscsi Cookbook

    35/185

    35

    Open the Centralized Management Console (CMC) and locate the storage system using the Find function, enter theaddress assigned previously on the first interface of the P4300 (e.g. 192.168.5.20).

    Select the Auto Discover by Broadcastcheck box. When you have found the storage system, it appears in the Available systems pool in the navigation window.

    Configuring the network interface bonds on the Storage System

    When configuring an iSCSI SAN directly attached to VC, one point is worth considering: the network interface bonding.

    Network interface bonding is generally used on iSCSI SAN devices to provide high availability, fault tolerance, loadbalancing and/or bandwidth aggregation. Depending on your storage system hardware, you can generally bond NICs inone of the following methods:

    Active Passive Link Aggregation (802.3 ad) Adaptive Load BalancingRelated to the direct-attached scenario, when a bond is configured, multiple interfaces are used but due to lack of

    Ethernet switches between the VC domain and the iSCSI SAN device, the support of NIC bonding is very limited: only theActive/Passive bonding method is supported.

    NOTE: Link Aggregation (802.3ad) is not supported because you cannot create an aggregation group across different VCmodules. Adaptive Load Balancing is not supported either as it requires the two interfaces to be connected to the samenetwork which is not the case here as we have two vNet for iSCSI (Active/Active scenario).

    To configure Active/Passive bonding on a HP StoreVirtual 4000 Storage Solution:

    From CMC, select the storage system, and open the tree below it.

  • 7/18/2019 Iscsi Cookbook

    36/185

    36

    Select TCP/IP Networkcategory, and click the TCP/IPtab.

    Select the two interfaces from the list for which you want to configure the bonding, right-click and select New Bond Select the Active Passivebond type

    Click OK After a warning message, click OKto rediscover the Storage system, the IP address is normally preserved. After a few seconds, the new bond setting shows the Active-Passive configuration:

    The Storage System is now ready to be used with fault tolerance enabled.

  • 7/18/2019 Iscsi Cookbook

    37/185

    37

    Configuring the iSCSI Host

    For the iSCSI Hosts configuration, Hardware or/and Software iSCSI connection can be used.

    Multipathing and path checkingWith the Direct-attached scenario and its Active Passive iSCSI SAN NIC bonding, it is recommended to check the secondpath to validate the entire configuration. After the iSCSI volume has been discovered from the first path, log in to theiSCSI Storage System and trigger a NIC bonding fail-over, this will activate the second interface, you will then have theability to validate the second iSCSI path.

  • 7/18/2019 Iscsi Cookbook

    38/185

    38

    Accelerated iSCSI

    Traditional software-based iSCSI initiators generate more processing overhead for the server CPU. An accelerated iSCSIcapable card, also known as Hardware iSCSI, offloads the TCP/IP operations from the server processor freeing up CPUcycles for the main applications.

    The main benefits are:

    Processing work offloaded to the NIC to free CPU cores for data-intensive workloads. Increased server and IP storage application performance. Increased iSCSI performance.Software-based iSCSI initiators are of course supported by all Virtual Connect models (1/10Gb, 1/10Gb-F, Flex-10 andFlexFabric) but 10Gb accelerated iSCSI (Hardware-based iSCSI initiator) is only provided today by Converged NetworkAdapters (i.e. NC551i/m or NC553i/m) with only Virtual Connect Flex-10 and Virtual Connect FlexFabric.

    The QLogic QMH4062 1GbE iSCSI Adapter is another adapter which also supports accelerated iSCSI but it is not supportedby VC Flex-10 and VC FlexFabric.

    NOTE: Under the OS, Accelerated iSCSI defers with Software iSCSI in the sense that it provides a HBA type of interface

    and not a Network interface card (NIC). Consequently, additional drivers, software and settings are sometimes required.NOTE: The selection between Software iSCSI and Accelerated iSCSI is done under the Virtual Connect profile:

    o Example of a server profile with Software iSCSI enabled:

  • 7/18/2019 Iscsi Cookbook

    39/185

    39

    o Example of a server profile with Accelerated iSCSI enabled:

    Several steps are required to connect a server to an iSCSI target using accelerated iSCSI:

    Accelerated iSCSI must be enabled on the server using Virtual Connect. A hardware iSCSI initiator must be installed and configured under the OS. A Multipath IO software must be used to manage the redundant iSCSI connection.This document will provide the steps to enable Accelerated iSCSI on a BladeSystem server under Microsoft WindowsServer 2003/2008 and VMware vSphere Server.

  • 7/18/2019 Iscsi Cookbook

    40/185

    40

    Creating a Virtual Connect profile with Accelerated iSCSI

    The first step is the same for both MS Windows and VMware vSphere servers; we have to create a Virtual Connect serverprofile that enables Accelerated iSCSI:

    Open Virtual Connect manager. From the Menu Define, select Server Profilein order to create a new VC profile.

    Note that VCM assigns FCoE connections by default!

    NOTE: A server with one FlexFabric adapter can be configured with a unique personality, either all Ethernet, orEthernet/iSCSI, or Ethernet/FCoE. Therefore its not possible to enable at the same time both FCoE and iSCSIconnections. Only a server with multiple FlexFabric Adapters can be configured with both iSCSI and FCoE connections.

    If you have a unique FlexFabric Adapter, delete the two FCOE connections, otherwise jump to the next bullet

    In the iSCSI HBA Connections section, click Add

  • 7/18/2019 Iscsi Cookbook

    41/185

    41

    In the Network Namecolumn, click on Select a network

    Select your iSCSI dedicated VC network and click OK

    In the Port Speedcolumn, you can adjust the speed settings (Auto Preferred Custom) In the Boot settingcolumn, leave DISABLED

    NOTE: Boot setting disabled means Accelerated iSCSI is enabled but iSCSI Boot is unavailable.The disable mode offloads the iSCSI protocol processing from the OS to the NIC. In addition to offloading TCP/IP protocolprocessing, it also offloads iSCSI protocol processing.

    NOTE: The multiple networks feature (i.e. when using 802.1Q VLAN tagging) is not supported for iSCSI HBA connections.

  • 7/18/2019 Iscsi Cookbook

    42/185

    42

    Optionally create a second iSCSI Connection for multipathing configuration

    NOTE: Allowing more than one iSCSI application server to connect to a volume concurrently without cluster-awareapplications or without an iSCSI initiator with Multipath I/O software could result in data corruption.

    Configure the additional VC Ethernet Network connections that may be needed on the other FlexNIC:

    When done, you can assign the profile to a server with an Ethernet adapter that supports Accelerated iSCSI:

    Click Applyto save the profile The server can now be powered on (using either the OA, the iLO, or the Power button)

  • 7/18/2019 Iscsi Cookbook

    43/185

    43

    Accelerated iSCSI with Microsoft Windows Server

    After the creation of a VC profile with Accelerated iSCSI enabled, we need to proceed with the following steps underMicrosoft Windows Server:

    Installation of the Emulex hardware iSCSI initiator (i.e. OneCommand Manager). IP Configuration of the iSCSI HBA ports. Installation of the Microsoft iSCSI initiator. Installation of Microsoft MPIO. Installation of the Device Specific Module (DSM)

    o Using the Microsoft DSM that comes with Microsoft MPIOo Using the Storage vendor DSM for MPIO provided by the Storage System vendor for better performance and

    latency.

    Connection to the iSCSI target using the iSCSI initiator.

    Installing Emulex OneCommand Manager

    OneCommand Manager is the Emulex utility to manage the NC55x 10Gb FlexFabric Converged Network Adapters. Amongother things, it provides comprehensive control of the iSCSI network including discovery, reporting and settings.

    For accelerated iSCSI under MS Windows server, the Emulex OneCommand Manager is mandatory as it provides the onlyway to configure the IP settings of the iSCSI HBA ports required to connect the iSCSI volume.

    For more information about OneCommand Manager, refer to:

    The User Manual of the Emulex OneCommand Manager:http://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdf

    The OneCommand Manager section further below in this document.NOTE: Do not confuse between hardware and software iSCSI initiators. Accelerated iSCSI always uses a specific port (i.e.host bus adapters) and require a utility from the HBA vendor ( i.e. Emulex OneConnect Manager, Emulex Drivers be2iscsi).

    Storage iSCSI

    Adapter Ports

    Network Interface Ports

    On the opposite, software initiators use the standard server NIC port and are usually included in the Operating System(i.e. Microsoft iSCSI initiator).

    http://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/supportmanual/c02018556/c02018556.pdf
  • 7/18/2019 Iscsi Cookbook

    44/185

    44

    OneCommand Manager can be downloaded from the Emulex web site:http://www.emulex.com/downloads/emulex.html

    Select the Windows version:

    Select the Managementtab

    http://www.emulex.com/downloads/emulex.htmlhttp://www.emulex.com/downloads/emulex.htmlhttp://www.emulex.com/downloads/emulex.html
  • 7/18/2019 Iscsi Cookbook

    45/185

    45

    Select the Enterprise Kit:

    NOTE: The HP OneCommand Enterprise Kit contains a graphical User Interface (GUI) and a Command Line Interface(CLI). HbaCmd.exe(the CLI) is located by default in C:\Program Files\Emulex\Util\OCManager

    Launch the OneCommand utility:o For Windows 2012: press the Windows logo key to open the Start screen and click on OCManager

  • 7/18/2019 Iscsi Cookbook

    46/185

    46

    o For Windows 2008: from the start menu, select All Programs/ Emulex/ OCManager

    OneCommand Manager shows the iSCSI ports detected on your server:

    Configuring the IP addresses of the iSCSI ports

    In order to log into the iSCSI target, an IP address must be assigned to each iSCSI HBA ports:

    From the OneCommand Manager interface, select the first iSCSI port, and click Modify.

    Enter a static IP address or check DHCP Enabled

  • 7/18/2019 Iscsi Cookbook

    47/185

    47

    This IP address must be in the same subnet as the one configured in the iSCSI Storage System.

    Note:The multiple networks feature being not supported for iSCSI HBA connections (seeCreating a Virtual Connect

    profile with Accelerated iSCSI)the VLAN Enabled option must remain unchecked for all scenarios presented in thiscookbook (i.e.scenario 1,scenario 2andscenario 3).

    The VLAN Enabled option can be required when Virtual Connect is tunneling the iSCSI traffic (i.e. when the iSCSInetwork under VCM has the Enable VLAN Tunneling checkbox selected). The use of a Tunnel network for the iSCSItraffic is not presented in this cookbook because the most common scenarios are using the iSCSI vNet in VLANmapping mode (i.e. not using the VLAN tunneling mode).

    Click OK. Select the second iSCSI port, and click Modify. Enter a second static IP address or enable DHCP

    Click OK.

  • 7/18/2019 Iscsi Cookbook

    48/185

    48

    Installation of the Microsoft iSCSI initiator

    The Microsoft iSCSI Software Initiator enables connection of a Windows host to an external iSCSI storage array usingEthernet NICs.

    For more information about the Microsoft initiator, seehttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspx

    NOTE: The Microsoft iSCSI Initiator utility is a software initiator (using Ethernet NICs) but it can as well be used tomanage the iSCSI accelerated connectivity (using the iSCSI HBAs).

    For Windows 2012:The Microsoft iSCSI initiator comes installed by default.

    To launch the Microsoft iSCSI initiator:

    Go to Server Manager Then select Tools/ iSCSI Initiator

    The first time the iSCSI Initiator is launched, the iSCSI service is required to be started, click Yes

    http://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspxhttp://technet.microsoft.com/en-us/library/ee338476%28WS.10%29.aspx
  • 7/18/2019 Iscsi Cookbook

    49/185

    49

    For Windows 2008:

    The Microsoft iSCSI initiator comes installed with both the Windows Server 2008, and the Server Core installation.

    For Windows 2003:

    Download and install the latest version of the Microsoft iSCSI Initiator software.You must select the Microsoft MPIO Multipathing Support for iSCSI option when you install the Microsoft iSCSI

    initiator.

    Installation of Microsoft MPIO

    MPIO solutions are needed to logically manage the iSCSI redundant connections and ensure that the iSCSI connection isavailable at all times. MPIO provides fault tolerance against single point of failure in hardware components but can alsoprovide load balancing of I/O traffic, thereby improving system and application performance.

    For Windows 2012:

    MPIO is an optional component in all versions of Windows 2012 Server so it must be installed.

    Go to Server Manager Then select Manage ThenAdd Roles and Features In the Featuressection, select Multipath I/O

    ClickNext andInstall

    For more information, see Microsoft Multipath I/O (MPIO) Users Guide for Windows Server 2012 athttp://www.microsoft.com/en-us/download/details.aspx?id=30450

    http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450
  • 7/18/2019 Iscsi Cookbook

    50/185

    50

    For Windows 2008:

    MPIO is an optional component in all versions of Windows 2008 Server so it must be installed.

    Go to Server Manager Then select Features Then AddFeatures Select Multipath I/O

    For Windows 2003:

    MPIO is installed during the Microsoft iSCSI initiator installation. You must select the Microsoft MPIO MultipathingSupport for iSCSI option when installing the Microsoft iSCSI Initiator.

    Installation of the Device Specific Module (DSM)

    MPIO requires the installation of device specific modules (DSM) in order to provide support for using multiple data pathsto a storage device. These modules are a server-side plug-in to the Microsoft MPIO framework.

    A native Microsoft DSM provided by Microsoft, comes by default with Microsoft MPIO software but storage providers candevelop their own DSM that contain the hardware specific information needed to optimize the connectivity with theirstorage arrays.

    Vendor specific DSM usually provides better write performance, lower latency, etc. while the Microsoft DSM offers morebasic functions.

    Installation of the Microsoft DSM

    The Microsoft DSM comes with the Microsoft iSCSI Initiator for Windows 2003 Server and is an optional component forWindows 2008 Server that is not installed by default.

    For Windows 2012:See Microsoft Multipath I/O (MPIO) Users Guide for Windows Server 2012 athttp://www.microsoft.com/en-us/download/details.aspx?id=30450

    http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450http://www.microsoft.com/en-us/download/details.aspx?id=30450
  • 7/18/2019 Iscsi Cookbook

    51/185

    51

    For Windows 2008:

    To use the Microsoft DSM with all versions of Windows 2008 Server, you must install MPIO and enable it. Additionaldetails for some steps are in the white paper available from http://www.microsoft.com/mpio

    Connect to a volume using the iSCSI initiator. In the MPIO Control Panel applet, click the Discover Multi-Paths tab, select the Add support for iSCSI devices check

    box, and click Add.

    The check box does not become active unless you are logged in to an iSCSI session.

    The system asks to reboot for the policy to take effect.

    http://www.microsoft.com/mpiohttp://www.microsoft.com/mpio
  • 7/18/2019 Iscsi Cookbook

    52/185

    52

    After the reboot, the MPIO-ed Devices tab shows the addition of MSFT2005iSCSIBusType_0x9.

    Connect to a volume using the iSCI Initiator and select the Enable multi-path check box.

  • 7/18/2019 Iscsi Cookbook

    53/185

    53

    When you connect to the volume, set your MPIO load balancing.

    For Windows 2003:

    The Microsoft DSM is installed along with the iSCSI Initiator.

  • 7/18/2019 Iscsi Cookbook

    54/185

    54

    Installation of the Storage vendor DSM

    Download the specific DSM for your iSCSI storage array from your storage vendors website.NOTE: TheHP StoreVirtual 4000 Storage DSMfor MPIO software is not supported with Accelerated iSCSI HBA underWindows Server 2003 and Windows Server 2008. To get more information about theHP StoreVirtual 4000 StorageDSM, seehttp://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03041928/c03041928.pdf

    Install the DSM for MPIO. A reboot might be required. Once the DSM for MPIO is installed, there is no additional settings, all iSCSI volume connections made to an iSCSI

    Storage System will attempt to connect with the Storage vendors DSM for MPIO.

    http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03041928/c03041928.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03041928/c03041928.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03041928/c03041928.pdfhttp://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03041928/c03041928.pdfhttp://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=4118659
  • 7/18/2019 Iscsi Cookbook

    55/185

    55

    Connecting volumes with MPIO

    Open the Microsoft iSCSI Initiator:o For Windows 2012, open the Server Managerconsole, go to Toolsthen select iSCSI Inititator.

    o For Windows Server 2008, open the Control Panelthen enter iSCSIin the search tab

  • 7/18/2019 Iscsi Cookbook

    56/185

    56

    On the Discoverytab, select Discover Portal

    Then enter the IP address of the iSCSI Storage System then click OK.

    NOTE: iSCSI SAN vendors usually recommend using a Virtual IP address; refer to the iSCSI SAN vendor documentation.

    On the Targetstab, you should discover a new iSCSI target if a LUN has been correctly presented to the server.

    Select the volume to log on to and click Connect

  • 7/18/2019 Iscsi Cookbook

    57/185

    57

    Select the Enable multi-pathcheck box.

    NOTE: Do not select the Enable Multi-path checkbox if your iSCSI Storage System is not supporting load balanced iSCSIaccess.

    Click OKto finish.

  • 7/18/2019 Iscsi Cookbook

    58/185

    58

    Multipath Verification

    You can verify the DSM for MPIO operations, from the Targettab, select the volume (now connected!) and click onProperties:

    You should see here the multiple sessions of this target as the DSM for MPIO automatically builds a data path to eachstorage node in the storage cluster.

    If after a few minutes only one session is available (you can click on refresh several times) it could be necessary toconfigure manually the multipathing.

  • 7/18/2019 Iscsi Cookbook

    59/185

    59

    Manual Multipath configuration

    Click on Add session

    Select the Enable multi-pathcheck box.

    Click Advancedto open the Advanced Settings window. Configure the Advanced Settings as follows:

    o For Local adaptor, select the first Emulex OneConnect OCe11100, iSCSI Storport...

    o For Source IP, select the IP address of the iSCSI HBA to connect to the volume.

  • 7/18/2019 Iscsi Cookbook

    60/185

    60

    o For Target portal, select the IP of the iSCSI Storage System containing the volume.

    Click OKto close the Advanced Settings dialog. Click OKto finish logging on Repeat the same steps using this time to the second Emulex adapter

    o For Source IP, select the IP address of the iSCSI HBA to connect to the volume.o For Target portal, select the IP of the iSCSI Storage System containing the volume.

    Click OKto close the Advanced Settings dialog.

  • 7/18/2019 Iscsi Cookbook

    61/185

    61

    Click OKagain to finish logging on. You should see now the second session using the second path:

    NOTE: It is recommended to test the failover to validate the multipath configuration.

  • 7/18/2019 Iscsi Cookbook

    62/185

    62

    Using the Microsoft iSCSI Software Initiator in conjunction with Accelerated iSCSI support

    Besides the Accelerated iSCSI attached disk, it is possible to use as well the iSCSI Software to connect additional volumesusing one of the NIC adapter(s). This section describes the different configuration steps required.

    Virtual Connect Profile Configuration

    From the Windows servers VC Profile, make sure that at least one server NIC is connected to an iSCSI network or to a

    network where an iSCSI device relies:

    Software iSCSI

    Accelerated iSCSI

  • 7/18/2019 Iscsi Cookbook

    63/185

    63

    Enabling jumbo frames on the NIC adapters under Windows 2012

    Windows PowerShell can be used using Set-NetAdapterAdvancedProperty -Name Adapters name -RegistryKeyword *JumboPacket -Registryvalue 9014

    OR

    Go to the Adapter Properties

    Select Configure

    Click on the Advancedtab

  • 7/18/2019 Iscsi Cookbook

    64/185

    64

    Then Jumbo Packet

    Select the MTU size supported by the network infrastructure and the iSCSI Storage SystemEnabling jumbo frames on the NIC adapters under Windows 2008/2003

    From the HP Network configuration Utility, select the port connected to the iSCSI-1 VC Network Click Properties Select the Advanced Settingstab then click on Jumbo Packet

  • 7/18/2019 Iscsi Cookbook

    65/185

    65

    Select the MTU size supported by the network infrastructure and the iSCSI Storage System

    Click OKto close Do the same on the second adapter connected to the iSCSI-2 VC Network

  • 7/18/2019 Iscsi Cookbook

    66/185

    66

    Installation of the iSCSI Storage DSM for MPIO

    A vendor specific DSM for the iSCSI storage array can be installed for better performance. If not available, the MicrosoftDSM can be used.

    Microsoft iSCSI Software configuration with multiple NICs

    Open the Microsoft iSCSI initiator Select Discoverytab Click on Discover Portal

    Enter the IP Address of the iSCSI target system connected to one of the server NIC network (e.g. iSCSI-1 and iSCSI-2)

    Click OK

  • 7/18/2019 Iscsi Cookbook

    67/185

    67

    Select the TargettabA new discovered iSCSI target should appear in the list if a LUN has been correctly presented to the server.

    Select the new target (inactive) and click Properties Click Add session Select Enablemulti-paththen click Advanced

  • 7/18/2019 Iscsi Cookbook

    68/185

    68

    Configure the Advanced Settings as follows:o For Local adaptor, select Microsoft iSCSI Initiator

    o For Source IP, select the IP address of the first NIC connected to iSCSI-1 VC Network

    o For Target portal, select the IP of the iSCSI Storage System containing the volume.

    Click OKto close the Advanced Settings dialog. Click OKto finish logging on

  • 7/18/2019 Iscsi Cookbook

    69/185

  • 7/18/2019 Iscsi Cookbook

    70/185

    70

    Accelerated iSCSI with VMware vSphere Server

    After the creation of a VC profile with Accelerated iSCSI enabled, we need to proceed with the following steps undervSphere:

    Installation of the Emulex iSCSI drivers Installation of the Emulex OneCommand Manager. (this is optional) IP Configuration of the iSCSI HBA ports. Connection to the iSCSI volumesInstalling the Emulex iSCSI drivers

    Make sure that the latest Emulex iSCSI drivers for VMware ESX have been installed; visit the following hp.com links todownload to latest be2iscsi drivers:

    To obtain these drivers, click below on the web link corresponding to the Emulex card you have in the server:

    HP NC551i Dual Port FlexFabric10Gb Converged Network Adapter

    http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=us

    HP NC553i 10Gb 2-port FlexFabricServer Adapter

    http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=us

    HP FlexFabric 10Gb 2-port 554FLBAdapter

    http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=us

    HP FlexFabric 10Gb 2-port 554MAdapter

    http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=us

    Then select your VMware version and download the HP Emulex 10GbE iSCSI Driver.

    NOTE: Hardware and software iSCSI can be managed by vSphere without any additional utility. Accelerated iSCSI uses aspecific port (i.e. vmhba) and requires the installation of HBA drivers included in the be2iscsi package.

    If the CNA is properly configured, you can view the vmhba on the list of initiators available for configuration:

    Log in to the vSphere Client, and select your host from the inventory panel. Select the Configurationtab and click on Storage Adaptersin the Hardware panel. The Emulex OneConnect vmhba

    appear in the list of storage adapters.

    http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215390&taskId=135&prodTypeId=3709945&prodSeriesId=5215389&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5215362&taskId=135&prodTypeId=3709945&prodSeriesId=5215333&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=4324854&taskId=135&prodTypeId=3709945&prodSeriesId=4296125&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=ushttp://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=5033634&taskId=135&prodTypeId=329290&prodSeriesId=5033632&lang=en&cc=us
  • 7/18/2019 Iscsi Cookbook

    71/185

    71

    Installing Emulex OneCommand Manager

    OneCommand Manager (OCM) is the Emulex utility to manage the NC55x 10Gb FlexFabric Converged Network Adapters.

    OCM is not required to connect a VMware server to an iSCSI volume but it delivers lots of advanced managementfeatures, configuration, status monitoring and online maintenance.

    OneCommand Manager (OCM) for VMware can be divided into two installations:

    OneCommand Manager for VMware vCenter OCM application agent in VMware ESX ServerOneCommand Manager for VMware vCenter

    For more information, see the OneCommand Manager for VMware vCenter User Manual:http://www-dl.emulex.com/support/vmware/vcenter/110/vcenter_user_manual.pdf See the Installing and Uninstalling OCM for VMware vCenter Components page 7.

    The OneCommand Manager for VMware vCenter can be downloaded from:http://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.html

    OCM application agent in VMware ESX Server

    For more information, see the OneCommand Manager Application User Manualhttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_gui_manual_elx.pdf

    Installation in a VMware ESX Server

    To install the OneCommand Manager Application agent in VMware ESX Server:

    Log into the ESX Server COS. Download the Application Kit (OCM core application) from

    http://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.html

    Copy the elxocmcore-esx-..rpmfile to a directory on the install machine CD to the directory to which you copied the rpm file. Install the rpm. Type:

    rpm -Uvh elxocmcore-esx-..rpm

    http://www-dl.emulex.com/support/vmware/vcenter/110/vcenter_user_manual.pdfhttp://www-dl.emulex.com/support/vmware/vcenter/110/vcenter_user_manual.pdfhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_gui_manual_elx.pdfhttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_gui_manual_elx.pdfhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_gui_manual_elx.pdfhttp://www.emulex.com/downloads/emulex/vmware/vsphere-51/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.htmlhttp://www.emulex.com/downloads/emulex/vmware/vsphere-41/management.htmlhttp://www-dl.emulex.com/support/vmware/vcenter/110/vcenter_user_manual.pdf
  • 7/18/2019 Iscsi Cookbook

    72/185

  • 7/18/2019 Iscsi Cookbook

    73/185

    73

    NOTE: Only OneCommand Manager for VMware vCenter 1.1.0 or later can discover and view iSCSI port information.

  • 7/18/2019 Iscsi Cookbook

    74/185

    74

    Configuring the IP addresses of the iSCSI ports

    In order to log into the iSCSI target, an IP address must be assigned to each vmhba iSCSI ports:

    From the vSphere client, select the host and click the Configurationtab / Storage Adapters Select the first vmhba and click Properties

    Then Configure

    Enter a static IP address

    NOTE: This IP address must be in the same subnet as the one configured in the iSCSI Storage System.

    Click OKto save the changes Repeat the same configuration for vmhba2

    Click OK

  • 7/18/2019 Iscsi Cookbook

    75/185

    75

    Configuring the iSCSI Volumes

    Configuring iSCSI Software target discovery address

    Select vmhba1and click Properties Select the Dynamic Discoverytab

    Click Add Enter the IP address of the iSCSI Storage System then click OK.

    NOTE: iSCSI SAN vendors usually recommend using a Virtual IP address; please refer to your iSCSI Storage Systemdocumentation.

  • 7/18/2019 Iscsi Cookbook

    76/185

    76

    Click Closethen accept the rescan message

    Newly discovered iSCSI targets should appear in the list if a LUN has been correctly presented to the server. Repeat the same configuration on vmhba2

    At the end the iSCSI target should be discovered on both ports

  • 7/18/2019 Iscsi Cookbook

    77/185

    77

    Adding iSCSI Software Datastore

    From the Hardware panel, click Storage Click Add Storage Select the Disk/LUNstorage type and click Next Select the iSCSI device freshly discovered to use for your datastore and click Next Click Next Enter a name for the Accelerated iSCSI datastore Adjust the file system values if needed then click Next Click Finish

    Multipath configuration checking

    To check the multipathing configuration:

    Select Storagefrom the Hardware panel From the Datastoresview, select the iSCSI datastore

    Make sure the number of paths displayed is more than one:

    You can click Propertiesthen Manage Paths

  • 7/18/2019 Iscsi Cookbook

    78/185

    78

    Details about the multipath settings is displayed:

  • 7/18/2019 Iscsi Cookbook

    79/185

    79

    Using the VMware iSCSI Software Initiator in conjunction with Accelerated iSCSI support

    The use of an iSCSI Software initiator in conjunction with the Emulex iSCSI hardware initiator is possible under VMwarewith Virtual Connect. This section describes the different configuration steps required.

    Virtual Connect Profile Configuration

    From the ESX servers VC Profile, make sure that at least one server NIC is connected to an iSCSI network or to a networkwhere an iSCSI device is available:

    vSphere iSCSI Software configuration with multiple NICs

    Open the vSphere Client, go to the Configurationtab then Networking Click on Add Networking Select VMkerneland click Next Select Create a virtual switch Select the two vmnic connected to the iSCSI network (e.g. iSCSI-1 and iSCSI-2) Click Next Enter a network label (e.g. iSCSI-1) then click Next Specify the IP Settings (Static or DHCP) then click Next Click Finish

    Software iSCSI

    Accelerated iSCSI

  • 7/18/2019 Iscsi Cookbook

    80/185

    80

    Select the vSwitch1just created and click Properties Under the Portstab, click Add Select VMkerneland click Next Enter a network label (e.g. iSCSI-2) then click Next Specify the IP Settings (Static or DHCP) then click Next Click FinishMapping each iSCSI port to just one active NIC

    Select iSCSI-1on the Portstab, then click Edit Click the NICTeamingtab and select Override vSwitch failover order. Move the second vmnic to the Unused Adapters

    Repeat the same steps for the second iSCSI port (iSCSI-2) but this time using the first vmnic as the Unused Adapter:

    Review the configuration then click Close

    Note that the iSCSI-1VMkernel interface network is linked to the vmk1port name and iSCSI-2is linked to vmk2.

  • 7/18/2019 Iscsi Cookbook

    81/185

    81

    Binding iSCSI Ports to the iSCSI Adapters

    From the vSphere CLI command, enter:

    #esxcli swiscsi nic add -n vmk1-d vmhba32 #esxcli swiscsi nic add -n vmk2-d vmhba32 #esxcli swiscsi nic list -d vmhba32

    Enable Jumbo Frames on vSwitch

    For a MTU=9000, enter on the vSphere CLI:

    # esxcfg-vswitch -m 9000 vSwitch1 # esxcfg-vswitch -l

  • 7/18/2019 Iscsi Cookbook

    82/185

    82

    Enable Jumbo frames on the VMkernel interface

    For a MTU=9000, enter on the vSphere CLI:

    #esxcfg-vmknic -m 9000 iSCSI-1 #esxcfg-vmknic -m 9000 iSCSI-2 # esxcli swiscsi nic list -d vmhba32

    Configuring iSCSI Software target discovery address

    Log in to the vSphere Client and select your server from the inventory panel. Click the Configurationtab and click Storage Adaptersin the Hardware panel

  • 7/18/2019 Iscsi Cookbook

    83/185

    83

    Select the iSCSI Software Adapter then click Properties

    Click on Configurethen check the Enabledoption Enter the same iSCSI name than the one already configured on the Accelerated iSCSI HBAs:

    NOTE: The host assigns a default iSCSI name to the initiator that must be changed in order to follow the iSCSI RFC-3720,where it is stated to use the same IQN for all connections for a given host and NOT associate different ones to differentadapters.

    Click OK Go to the Dynamic Discoverytab and click Add Enter the iSCSI Storage System IP address then press OKthen Close Accept the rescan of the freshly configured iSCSI Software HBA

  • 7/18/2019 Iscsi Co