1 © 2004 Cisco Systems, Inc. All rights reserved. iSCSI Overview IP Storage Networking FCIP/iSCSI...

1© 2004 Cisco Systems, Inc. All rights reserved.iSCSI Overview

IP Storage NetworkingFCIP/iSCSI

Steve TegelerStorage Networking TeamNorthwest Territory425/[email protected]

222© 2004 Cisco Systems, Inc. All rights reserved.

OPT-20539761_05_2004_c2

Agenda

• Storage Networking Technology Review

• IP Storage Networking

• FCIPWrite Acceleration, Compression, IPSec, SAN Extension Tuner

• FCIP Wizard

• iSCSIWhat, Why, How

• Performance

• iSCSI Terminology and Topology

• Design considerations when deploying iSCSI

• Summary


OPT-20539761_05_2004_c2

Agenda


• iSCSI and IP Storage Networking

What, Why, How

• Performance



• Summary


OPT-20539761_05_2004_c2

The Old Storage Environment

• Direct Attached Storage (DAS)

• Storage is captive ‘behind’ the server

• Server CPU must handle user I/O requests, but also:

User-database inquiries

User file/print serving

Data-integrity checking

Communication with other devices

• Data access is file system and platform dependant

• Costly to scale; complex to manage

FC

Clients

SCSIFC

Direct-Attached Storage (DAS)

Servers

Win2k Linux Win2k Linux Unix

IP Network


OPT-20539761_05_2004_c2

The SCSI I/O Channel

• SCSI is the dominant protocol used to communicate between servers and storage devices in open system

• SCSI I/O channel is a half-duplex pipe for SCSI CDBs and data

• Parallel bus evolutionBus width: 8, 16 bitsBus speed: 5–80 MhzThroughput: 5–320 MBpsDevices/bus: 2–16 devicesCable length: 1.5m–25m

• A network approach can scale the I/O channel in many areas (length, devices, speed)

SCSI CDB: SCSI Command Descriptor Block Used to Relay SCSI Commands, Parameters, and Status between SCSI Initiators and SCSI Targets; Typically 6, 10, or 12 Byte Block

SCSI AdapterSCSI Adapter

Applications

File SystemFile System

Block DeviceBlock Device

SCSI GenericSCSI Generic

TCP/IPStack

TCP/IPStack

NICDriverNIC

Driver Adapter DriverAdapter Driver

Half-DuplexSCSII/O Channel

SCSIInitiator

SCSITarget

SCSI

RawRaw

EthernetNIC

EthernetNIC

Ethernet


OPT-20539761_05_2004_c2

Networking the I/O Channel

• Same SCSI protocol (SCSI-3) carried over a network transport layer via serial implementation

• Transport must not jeopardize SCSI payload (security, integrity, latency)

• Two primary transports to choose from today: Fibre Channel and IP

• A networked I/O channel allows for multiple improvements:

Distance limitations greatly increased

High number of addressable devices

Initiator

Target and LUNsN

etw

ork

edI/

O C

han

nel Channel

Controller

SCSI

Host System

Network


OPT-20539761_05_2004_c2

Fibre Channel Networking

• Very common method for networking SCSI

• Fibre Channel provides high-speed transport for SCSI payload

• Fibre Channel SAN overcomes many shortcomings of DAS including:

Addressing for up to 16-million nodes (24 bits)

Loop (shared) and Fabric (switched) transport

Speeds of 100 or 200 MBps (1 or 2 Gbps)

Distance of up to 10km (without extenders)

Can utilize CWDM or DWDM for over 10km

Support for multiple protocols

• Combines best attributes of a channel and a network

FibreChannel HBA

Fibre Channel Fabric

Initiator

SCSI

Host System

Target


OPT-20539761_05_2004_c2

IP: An Alternate I/O Transport

• Viable transport for I/O traffic

• Not necessarily for long-haul I/O only

• Similar characteristics to Fibre Channel:

Addressing for close to 4 billion nodes (IPv4)

Primarily a switched transport (with routing)

Ethernet speeds of 10/100 Mbps or 1/10 Gbps or various WAN speeds

Support for multiple high-level protocols

• Cost and manageability advantages with IP

• IP knowledge base widespread in industry

IP “ChannelAdapter”

Target

SCSI

Host System

Initiator

IP Network


OPT-20539761_05_2004_c2

IP Storage Networking

• IP storage networking provides solution to carry storage traffic within IP

• Uses TCP: a reliable transport for delivery• Applicable to local data center and long-haul applications• Two primary protocols:

iSCSI—Internet-SCSI—used to transport SCSI CDBs and data within TCP/IP connections

FCIP—Fibre-Channel-over-IP—used to transport Fibre Channel frames within TCP/IP connections—any FC frame—not just SCSI

IP TCPTCP FCIPFCIP FCFC SCSI Data

IP TCPTCP iSCSIiSCSI SCSI Data


OPT-20539761_05_2004_c2

FCIP – Extending your FC SAN

101010© 2004, Cisco Systems, Inc. All rights reserved.


OPT-20539761_05_2004_c2

Fibre Channel over IP (FCIP)Point to Point

FCIP – Fibre Channel over Internet Protocol

The encapsulation of Fibre Channel frames into IP packets and tunneling through an existing TCP/IP network infrastructure, in order to connect geographically distant

islands

LAN/MAN/WAN

FCIP Tunnel SessionFCIP Tunnel Session

FC DiskFC Disk

SAN SAN

Ethernet Catalyst Switches & Routers

Optical Extension Metro DWDM

& CWDM

IPSIPS

Sync or AsyncReplication

E-port E-port

FCIP tunnels can be thought of as

ISL’s with Latency


OPT-20539761_05_2004_c2

iSCSI

121212© 2004, Cisco Systems, Inc. All rights reserved.


OPT-20539761_05_2004_c2

Audience Poll

• Who has a FC Network today?

• Who has deployed iSCSI,

• Array based, or gateway based?


OPT-20539761_05_2004_c2

What is iSCSI?

• A SCSI transport protocol that operates over TCP/IPEncapsulates SCSI CDBs (operational commands: e.g. read or write) and data into TCP/IP byte streams

Allows IP hosts to access IP-based SCSI targets (either natively or via iSCSI to FC Gateways)

• Standards statusRFC 3720 on iSCSI

Collection of RFCs describing iSCSI

RFC 3347—iSCSI Requirements

RFC 3721—iSCSI Naming and Discover

RFC 3723—iSCSI Security

• Broad industry supportServer vendors now publishing own supported iSCSI drivers

Native iSCSI storage arrays now appearing


OPT-20539761_05_2004_c2

SCSI Block CommandsSCSI Block CommandsSCSI StreamSCSI StreamCommandsCommands

Parallel Parallel SCSI TransportSCSI Transport

SCSI Applications (File Systems, Databases)

Parallel SCSIParallel SCSIInterfacesInterfaces

SCSIDevice-TypeCommands

SCSIGeneric

Commands

SCSITransportProtocols

Layer 3 Network

Transport

Layer 2Network Fibre ChannelFibre Channel Ethernet, PPP, HDLC…Ethernet, PPP, HDLC…

Other SCSI CommandsOther SCSI Commands

IPIP

TCPTCP

SCSI Commands, Data, and Status

Recap SCSI Architectural Model Transports

FCPFCPSCSI over FCSCSI over FC

iSCSIiSCSISCSI over TCP/IPSCSI over TCP/IP


OPT-20539761_05_2004_c2

Why - iSCSI vs. FC It’s all about the $$$

• Leverage IP infrastructure for storage connectivity

• Low-cost complement to FC SAN, provides additional resource consolidation

1000BaseT NIC + GigE Port: Roughly $100 + $300 = $400

FC HBA+ FC Port: Roughly $1000 + $1000 = $2000

• Secure connectivity via CHAP-based authentication

• Transparent iSCSI routing gives iSCSI hosts a pWWNUses controller-based LUN masking or MDS-based virtual

targets for resource provisioning

Uses zoning for device connectivity

• iSCSI driver (free) works with any Ethernet NICTOE only necessary with processor-bound servers

• iSCSI is an industry-supported IETF standard

• Many O/S vendors providing iSCSI initiator (MS, HP, Novell, Linux), others provided by Cisco

iSCSI-enabled Hosts

FC Disk Array

CatalystEthernetSwitch

IPNetwork

Cisco MDS 9000 with

IP Services Module

iSCSI

iSCSI

iSCSI

FC Servers

FC Tape Library


OPT-20539761_05_2004_c2

iSCSI for Storage Consolidation

• IP access to open systems iSCSI and Fibre Channel storage

• iSCSI driver is loaded onto hosts on Ethernet network

• Able to consolidate servers via iSCSI onto existing storage arrays

• Able to build Ethernet-based SANs using iSCSI arrays

• Storage assigned on a LUN-by-LUN basis at iSCSI router

iSCSI-EnabledHosts (Initiators)

iSCSI Array

(Target)

StoragePool (Target)

iSCSIGateway

IPNetwork

FCFabric

FCFabric

FC HBA-Attached

Host (Initiator)

iSCSI

iSCSI

Logical Unit Number (LUN): A Field within SCSI Containing up to 64 Bits that Identifies the Logically Addressable Unit within a Target SCSI Device

iSCSIiSCSI


OPT-20539761_05_2004_c2

iSCSI for Remote Block Access

• Block access to remote storage over IP

• Application must tolerate latency for long distances

• Metro Ethernet services offer lower-latency transport alternative

• Remote backup over IP WAN

• Centralized management from centralized storage

iSCSI-EnabledHost

Remote Mirrors

IPWAN

StoragePool

FCFabric

FCFabric

iSCSI Device

Site A

Site B

iSCSI


OPT-20539761_05_2004_c2

How - iSCSI Architecture: Software Driver

iSCSI GW Module

SCSI DriverSCSI Driver TCP/IP DriverTCP/IP Driver

FC HBAFC HBA GigE NICGigE NIC

NICNIC SCSI AdapterSCSI Adapter




TCP/IPStack

TCP/IPStack

NICDriverNIC

Driver Adapter DriverAdapter Driver

iSCSI

iSCSI GW Device

Host

iSCSI Host Driver

Conventional SCSI Path

iSCSI Path

IP Network

Fibre ChannelFibre Channel

Applications


OPT-20539761_05_2004_c2

OS Support

• Many operating systems supported via Cisco drivers and/or from OS vendor

Cisco provides full-driver suite

Solaris 2.6 (EOL),7,8,9

Linux-based on 2.6 kernel

Win 2000 with SP2 or later

Windows XP Pro

WinNT 4.0 with SP6A

HP/UX 10.2, 11.0

AIX 4.3.3, 5.1, 5.2

OS vendors support native iSCSI drivers

Windows *native* Win 2000, XP, 2003 support

HP *native* HP/UX 11i support

IBM *native* AIX 5.x support

Novell Netware *native* support

Solaris 10 (March 2005)

Linux (RedHat Suse)

NIC AdapterNIC Adapter SCSI Adapter

SCSI Adapter

Adapter Driver

Adapter Driver

iSCSI


Applications



TCP/IP StackTCP/IP Stack

NIC DriverNIC Driver

iSCSISoftware

Driver


OPT-20539761_05_2004_c2

iSCSI HBAs and TCP Offload Engines (TOEs)

• Offloads TCP and, optionally, iSCSI processing into hardware

• Relieves host CPU from:

TCP processing—16-bit checksum per packet

iSCSI—optional 32-bit header and data digests (CRC32C)

TCP Offload

iSCSI and TCP Offload




HBADriverHBA

Driver

TCP/IPStack

TCP/IPStack

NICDriverNIC

Driver

iSCSIDriver

TOEDriverTOE

Driver

SCSI AdapterSCSI Adapter

Adapter Adapter DriverDriver

TCP/IPStack

TCP/IPStack

iSCSITCP/IPStack

TCP/IPStack

Applications


OPT-20539761_05_2004_c2

Agenda



What, Why, How

• Performance



• Summary


OPT-20539761_05_2004_c2

Example performance impact on CPU util % FC vs. iSCSI TOE vs. iSCSI SW Driver

CP

U %

Throughput MB/s

FC HBA

iSCSI TOE

iSCSI SW Driver

15-35MB/sInflection point determined by

system resources (CPU/Memory)

?


OPT-20539761_05_2004_c2

Agenda



What, Why, How

• Performance



• Summary


OPT-20539761_05_2004_c2

iSCSI Naming

• Initiator and target require iSCSI names

Name is location independent

iSCSI node name = SCSI device name of iSCSI device

Associated with iSCSI nodes, NOT adapters

Up to 255-byte displayable/human readable string (UTF-8 encoding)

Use SLP (Service Location Protocol) V2, iSNS, or query target for names (SendTargets)

• Two iSCSI name types:

iqn—iSCSI qualified name

eui—Extended Unique Identifier (IEEE EUI-64—also used for FC WWNs)


OPT-20539761_05_2004_c2

–– –

iSCSI Name Structure

– Unique String

iqn.1987-05.com.cisco.1234abcdef987601267da232.bettyiqn.2001-04.com.acme.storage.tape.sys1.xyz

Type DateOrganization

Naming AuthoritySubgroup Naming Authority or

String Defined by Organization Naming Authorityiqn

eui

Date = yyyy-mm When Domain Acquired

Reversed Domain Name

–Type EUI-64 Identifier (ASCII Encoded Hexadecimal)

eui.02004567a425678d

Type


OPT-20539761_05_2004_c2

iSCSI Connectivity

• iSCSI Initiator knows IP and IQN

• FC Target knows WWN and FCID

iSCSIHBA

HBA

iSCSIHBA

HBA FC

Fibre ChannelFabric

iqn.host-3IP-10.1.1.4


pWWN – P6nWWN –N6

FCID – XXXX06

pWWN – P5nWWN – N5

FCID –XXXX05

IP-10.1.1.1IPNetwork

iSCSIHBA

HBA



OPT-20539761_05_2004_c2

iSCSI Mapping to a WWN

• Each iSCSI Initiator gets a unique WWN and FCID

iSCSIHBA

HBA

iSCSIHBA

HBA FC

Fibre ChannelFabric



pWWN – P6nWWN –N6

FCID – XXXX06


FCID –XXXX05

IP-10.1.1.1

iSCSIHBA

HBA


pWWN – P2nWWN- N2

FCID XXXX02

pWWN – P4nWWN- N4

FCID XXXX04


FCID XXXX03


OPT-20539761_05_2004_c2

Agenda



What, Why, How

• Performance



• Summary


OPT-20539761_05_2004_c2

Considerations when building an iSCSI Fabric

• iSCSI fabric topologyEthernet fabric topology

• iSCSI fabric scalabilityTrunking

Port channeling

• iSCSI fabric availabilityVRRP

• iSCSI fabric securityAuthentication and binding

• iSCSI fabric manageabilityiSCSI identity and management

ScalabilityAvailability

SecurityManageability

iSCSIClients

END

TO

END

END

TO

END

Shared Storage Pool

iSCSI iSCSI

iSCSI iSCSI iSCSI

iSCSI iSCSI

iSCSI iSCSI iSCSI

IPSIPSIPSIPS


OPT-20539761_05_2004_c2

Dedicated IP Storage Network

• Separate logical IP network but not necessarily separate physical network

• Can use a VLAN of existing Ethernet network

• Recommend use of dedicated NIC on host for iSCSI

• Minimized potential for bandwidth contention

iSCSI-EnabledHosts

Storage Pool

iSCSIRouters

CatalystSwitches

DedicatedIP Storage Network

FCFabric

FCFabric

Clients

Front-Side IP Network

FC-Attached Hosts with HBAs

iSCSI iSCSI iSCSI iSCSI


OPT-20539761_05_2004_c2

IP Network Security Techniques

• FirewallStandalone or intelligent firewall service module

Allow well-known TCP port 3260 for iSCSI

• IPSec VPN VPN tunnel for iSCSI remote access

• Access Control List (ACL)

• VLAN and PVLANSubinterface implementation on iSCSI

Separated VLAN for iSCSI

• Port securityAllow, block, or restrain access to Ethernet based on MAC address


OPT-20539761_05_2004_c2

What is iSNS?

Internet Storage Name Service (iSNS) is a name registration service for IP storage devices:

Analogous to FCNS and DNS

Provides centralized management capabilities

iSNS supports:

Target device discovery

Discovery Domains (similar to zones)

Authentication

State change notification

Supports iSCSI and iFCP


OPT-20539761_05_2004_c2

What is iSNS? (cont.)

IP

FC

iFCPGateway

IP

IP

iSNS server

FC

FC

iSCSI

iSCSI

iSCSI

iSCSI

iSCSI

IP

IP

iSCSI

iSCSI

iSCSI

iSCSI

iSCSI

iSCSIiSCSI

FC

iSNSiSNS


OPT-20539761_05_2004_c2

Agenda



What, Why, How

• Performance



• Summary


OPT-20539761_05_2004_c2

Summary

• Leverages the existing IP infrastructure

Hence the intelligence, capacity, and best practice design can be leveraged in the iscsi infrastructure

• Complementary to FC yet represents a low-cost transport choice

• Midrange applications connectivity

• Midrange server connectivity with blade server integration as new system candidate

• Potential long-distance SAN transport


OPT-20539761_05_2004_c2

Reference Materials

• http://www.t10.org/

• http://www.t11.org/index.htm

• http://www.ietf.org/rfc.html on RFC 3720

• http://www.cisco.com/en/US/partner/products/hw/ps4159/index.html

• http://www.lightreading.com/webinar_archive_home.asp?webinar_id=27003

1 © 2004 Cisco Systems, Inc. All rights reserved. iSCSI Overview IP Storage Networking FCIP/iSCSI...

Documents

Transcript of 1 © 2004 Cisco Systems, Inc. All rights reserved. iSCSI Overview IP Storage Networking FCIP/iSCSI...