XenServer Storage Integration Deep Dive

XenServerStorage Integration Deep DiveXenServerStorage Integration Deep Dive

2© 2009 Citrix Systems, Inc. — All rights reserved

Agenda

XenServer 5.5 Storage Architecture

Multipathing

Vendor Integration

StorageLink


EnterpriseEdition

Platinum Edition

FreeEdition

XenServer

Essentials

Shared Storage (iSCSI, FC, NFS)

XenCenter Management

64bit, Windows, Linux Workloads

Live Migration (XenMotion)

No socket restriction

High Availability

StorageLinkTM

Provisioning Services (p+v)

Lab Management

Performance Monitoring

Citrix XenServer & Essentials 5.5 Family

Active Directory Integration

Generic Storage Snapshotting

Provisioning Services (virtual)

Stage Management

Workload Balancing

Workflow Studio Orchestration

NEW

NEW

NEW

XenServer 5.5 Storage ArchitectureXenServer 5.5 Storage Architecture


Expanded Backup Support


Storage Technologies

Storage Repository

LUN

LVM Volume Group

LVMLogicalVolume

LVMLogicalVolume

Storage Repository

Filesystem

.VHDfile

.VHDfile

Storage Repository

LUN

LVM Volume Group

LVMLogicalVolume

LVMLogicalVolume

VHD header VHD header

XenServer 5.0 / 5.5NFS / EXT3

XenServer 5.0 iSCSI / FC


VM virtual disk


LVM XenServer 5.5 (LVHD)

• Replaces LVM for SRs

• Hosts VHD files directly on LVM volumes

• Best of both worlds• Features of VHD• Performance of LVM

• Adds Advanced Storage features• Fast Cloning• Snapshots

• Fast and simple upgrade

• Backwards compatible

MultipathingMultipathing


Why using Multipathing?

• Path redundancy to storage

• Performance increase by load sharing algorithms

• Many fibre channel environments by default have multiple paths

FCHBA 2

FCHBA 1

XenServer FC Switches Storage Subsystem

LUN 1

LUN 1

Storage controller 1

Storage controller 2

LUN 1

LUN 1

LUN 1


Enabling Multipathing

• xe host-param-set other-config:multipathing=true uuid=host_uuid

• xe host-param-set other-config:multipathhandle=dmp uuid=host_uuid

• Note: Do not enable multipathing by other ways (e.g. CLI)!!!


XenServer supports 2 multipathing technologies

Device Mapper Multipathing (DMP) RDAC MPP (mppVhba)

Default yes no

XenServer version >= 5.04.1 CLI

>= 5.0 Update 2

Management by XenCenter yes no

Support wide storage range only LSI controller based storage

Driver / Daemon multipathd mppVhba driver

CLI path check multipath –ll mpputil

Configuration /etc/multipath-enabled.conf /etc/mpp.conf(requires execution of /opt/xensource/bin/update-initrd)

See details: http://support.citrix.com/article/ctx118791

http://support.citrix.com/article/ctx118791


DMP vs RDAC MPP

• Check if RDAC MPP is running• lsmod | grep mppVhba

• „multipath –ll“ would show MD device as output (if DMP is active)

• Use only 1 technology• When RDAP MPP is running use it

• Otherwise use DMP


MPP RDAC: Path check

• mpputilLun #0 - WWN: 600a0b80001fdf0800001d9c49b0caa1 ---------------- LunObject: present DevState: OPTIMAL

Controller 'A' Path -------------------- Path #1: LunPathDevice: present DevState: OPTIMAL

Path #2: LunPathDevice: present DevState: OPTIMAL

Controller 'B' Path -------------------- Path #1: LunPathDevice: present DevState: OPTIMAL

Path #2: LunPathDevice: present DevState: OPTIMAL


DMP: Path check

• Monitoring using XenCenter

• Monitoring using CLI• Command: multipath -ll

Multipathing & Software iSCSIMultipathing & Software iSCSI


iSCSI with Software Initiator

• IP addressing to be done by XenServer Dom-0

• Multipathing also to be done by XenServer Dom-0

• Dom-0 IP configuration is essential

NIC 1

XenServer Switches Storage Subsystem

LUN 1

Storage LAN Ctrl 1

IP

LUN 1

XenServerDom-0

IPLUN

1LUN

1IP


Best practice configuration: iSCSI storage with multipathing

• Separation of subnets also on IP base

NIC 2

NIC 1

XenServer FC Switches Storage Subsystem

LUN 1

Storage LAN

Adapter 1

Storage LAN

Adapter 2

Port 1 IP:192.168.1.201

Port 2 IP:192.168.2.201

Port 1 IP:192.168.1.202

Port 2 IP:192.168.2.202

NIC 1 IP:192.168.1.10

NIC 2 IP:192.168.2.10

Subnet:255.255.255.0

Subnet 1

Subnet 2


Subnet 1

Not recommended configurations for multipathing and iSCSI:

NIC 2

NIC 1

XenServer

NIC 1 IP:192.168.1.10

NIC 2 IP:192.168.1.11

Both server NICs in same subnet

Subnet 1

NIC 2

NIC 1

XenServer

Team IP:192.168.1.10

Mixing of NIC teaming and multipathing


Multipathing with Software InitiatorXenServer 5

• XenServer 5 supports Multipathing with iSCSI software initiator

• Prerequisites are:• iSCSI target uses same IQN on all ports

• iSCSI target ports operate in portal mode

• Multipathing reliability has been enhanced massively in XenServer 5.5


How to check if iSCSI target operates in portal mode?

• Execute iscsiadm -m discovery --type sendtargets --portal <ip address of 1 target>

• Output must show alls IPs of the target ports with identical IQN• Example:

192.168.0.161:3260,1 iqn.strawberry:litchie192.168.0.204:3260,2 iqn.strawberry:litchie

• When connecting to iSCSI target using XenCenter Storage Repository Wizard, also all target IPs should show up after Discovery

NetApp IntegrationNetApp Integration


NetApp Storage

• NetApp Storage supports Multipathing

• For configuring NetApp storage and modification of multipath.conf see whitepaperhttp://support.citrix.com/article/CTX118842

• NetApp typically supports portal mode for iSCSI multipathing for iSCSI SW Initiator is supported

• Especially for low-end NetApp storage (e.g. FAS2020) with limited LAN adapters special considerations take place

http://support.citrix.com/article/CTX118842


NetApp low-end storage (iSCSI)

• Often limited by NIC configuration

• Example: 2 NICs per head

• 1 aggregate / LUN is represented by 1 head at a time (other head for fault tolerance)

• Thus: 2 NICs effectively can be used for storage connection

• Typically Filer delivers non-block-based protocols (e.g. CIFS) which also require redundancy as well as block based protocols (e.g. iSCSI)


iSCSI Network

NIC 1

NIC 0

NIC 0

NIC 1

Controller 0(active)

CIFS Network

NIC 0

NIC 1

Controller 1(fault tolerance)

Example FAS2020: Scenario 1no network reduncancy for iSCSI and CIFSseparation of networks


Example FAS2020: Scenario 2network redundancy for iSCSI and CIFSno separation of networks

NIC 1

NIC 0

NIC 0

NIC 1

Controller 0(active)

CIFS & iSCSI Network

NIC Bondvif /

bond

NIC 0

NIC 1


vif / bond


Example FAS2020: Scenario 3network redundancy for iSCSI (multipathing) and CIFSseparation of networks

NIC 1

NIC 0

NIC 0

NIC 1

Ctrl 1(active)

NIC Bond

Vif /bond


Same configuration

CIFS VLAN

iSCSI VLAN

NIC 3

NIC 2

Multipathing

CIFS VLAN

iSCSI VLAN

Dell / Equalogic IntegrationDell / Equalogic Integration


Dell Equalogic Support

• XenServer 5.5 includes Adapter (min. firmware 4.0.1 required)

• Redundant path configuration does not depend on using adapter or not

• All PS series are supported as running same OS

• StorageLink Gateway support planned


Dell / Equalogic

• See whitepaper for Dell / Equalogic storage http://support.citrix.com/article/CTX118841

• Each Equalogic has two controllers

• Only 1 controller is active

• Uses „Group ID“ address on storage side(similar to bonding / teaming on server side)

• Only connection over group ID, no direct connection to the iSCSI ports possible

• Therefore multipathing cannot be used bonding on XenServer side

http://support.citrix.com/article/CTX118841

Datacore IntegrationDatacore Integration


Multipathing architecture with Datacore

• Different IQNs for targets – no portal mode possible!!

NIC 2

NIC 1

XenServer Switches

LUN 1

Storage Controller

1

Storage Controller

2

Port 1 IP:192.168.1.201

Port 2 IP:192.168.2.202

NIC 1 IP:192.168.1.10

NIC 2 IP:192.168.2.10

Subnet:255.255.255.0

Subnet 1

Subnet 2

IQN 1

IQN 2


Datacore hints

• Special attention for software iSCSI

• Follow Datacore technical bulletin: TB15ftp://support.datacore.com/psp/tech_bulletins/TechBulletinsAll/TB15b_Citrix%20XenServer_config_501.pdf

• Datacore in VM• O.k. when not using HA

• Configuration possible, but take care about booting the whole environment

• Take care when updating XenServer

ftp://support.datacore.com/psp/tech_bulletins/TechBulletinsAll/TB15b_Citrix%20XenServer_config_501.pdf



StorageLinkStorageLink


Logical advancement of XenServer integrated storage adapters

Netapp & EqualogicStorage adapter


Citrix StorageLink Overview (XenServer)

Guest

Snap-infor XenServer

XenServer

StorageLink

Data Path

Control Path

iSCSI / FC


Leveraging the best of virtualization and storage


StorageLink OverviewXenServer

Netapp EqualLogicVSM

Bridge

Hyper-V

Hyper-V

VDS

Storage Repository

LUNLUN LUN

Storage Repository

LUNLUN LUN

Storage Repository

LUNLUN LUN

SAN/NAS

Virtual Storage Manager

ParentPartition

DOM0

XenServer

Data PathData Path

Control Path


StorageLink Gateway Overview

Vendor-specific VSM Storage Adapters run in separate processes

SMI-S is the preferred method of integration as it requires no custom development work

Hyper-V

Storage Modules

Fabric(SMIS)

Hypervisor Modules

Proxy

Network Modules

NetAppSMIS

activity

status

power

NetApp FAS980

activity

status

power

NetApp FAS980

NetApp

UID

HPStorageWorks

1 4 7 10

12963

HP EVA series

Citrix StorageLink Gateway Core

Web Services Interface (SOAP/.NET)

UID

HPStorageWorksMSA1500cs

HP StorageWorks

Modular Smart Array 1500

UID


HP StorageWorks


UID


HP StorageWorks


Vendor Y Storage Adapter

Vendor’sSMI-S

provider

VendorSMI-S

CIMOM

VendorSMI-S

CIMOM

Vendor X Storage Adapter

DellDell

XenServer


Storage Technologies

Storage Repository

LUN

LVM Volume Group

LVMLogicalVolume

LVMLogicalVolume

VHD header VHD header


VM virtual disk

LUN

Storage Repository

XenServer 5.5 iSCSI / FC +

LUN


Snapshot types

XenServer (free) Essentials Enterprise

Snapshot type Software based snapshot Hardware based snapshot(also software based snapshot possible

when not using StorageLink)

LUN access model LVM (1 LUN=x times VDI) LUN-per-VDI (1 LUN = 1 VDI)

Performance Good Superior

Utilization On XenServer host On storage subsystem

Overhead Low lowest


StorageLink: Microsoft look-and-feel


Essentials: Example usage scenarioEffective creation of VMs from template

VMTemplate

LUN

VM clone 1x

1. Copy of LUN

2. Modification of Zoning

3. Creation of VM

4. Assignment of LUN to VM

VM clone 3x

1. Copy of LUN


3. Creation of VM


1. Copy of LUN


3. Creation of VM


1. Copy of LUN


3. Creation of VM


Effectiveness:

• Fast-Cloning using Storage Snapshots• Fully automated storage and SAN configuration• for FC and iSCSI


StorageLink: Supported Storages

• StorageLink HCL http://hcl.vmd.citrix.com/SLG-HCLHome.aspx

http://hcl.vmd.citrix.com/SLG-HCLHome.aspx

XenServer Storage Integration Deep Dive

Documents

Transcript of XenServer Storage Integration Deep Dive