iSCSI and CLEAR-Flow

Extreme Networks White Paper

© 2011 Extreme Networks, Inc. All rights reserved. Do not reproduce.

Make Your Network Mobile

Abstract

This white paper is targeted to enterprise and service pro-vider IT managers who may be evaluating the ability of iSCSI SANs to satisfy the performance requirements of their most demanding storage applications. Storage application perfor-mance is dependent on more than just the performance of the storage array. The full data path and the integration with the other layers of the data center architecture must be taken into consideration. Optimizing the full data path from the server, through the network to the storage can contribute to significant improvements in performance and service levels. This white paper and the accompanying configuration guide details the work of Intel, Extreme Networks® and NetApp to demonstrate the impact on iSCSI performance in real world environments using Extreme Networks CLEAR-Flow technol-ogy to help you achieve optimal iSCSI performance.

The output of the collaborative testing is presented in two separate documents:

• White Paper: provides an overview of the latest advance-ments around iSCSI SAN capabilities and performance, including real world, end-to-end performance results with CLEAR-Flow.

• Configuration Guide: provides a sample reference archi-tecture with step-by-step configuration details for all the components and describes the end-to-end solution performance across a number of configurations.

High-Performance iSCSI SAN Architecture— A Solution’s View from Extreme Networks, Intel and NetApp

2

Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp

© 2011 Extreme Networks, Inc. All rights reserved.

Executive Summary

Recent technological innovations are changing the way data centers are engineered and operated. Three technology trends driving new economics and perfor-mance in the data center include server virtualization, unified storage based upon Ethernet storage protocols, and high-performance Ethernet networking.

Specifically, as related to storage infrastructure, we have seen a significant increase in the market acceptance of Ethernet-based storage solutions, such as iSCSI SANs and NAS. IDC estimates that by 2014, Ethernet storage revenue will surpass that of legacy Fibre Channel-based solutions.1 These trends signal a significant change in storage architectures to support business critical and high-demand storage applications that have traditionally been deployed using dedicated Fibre Channel networks to satisfy the desired performance and quality of service requirements.

Historically, Ethernet-based storage solutions have not been deployed for many business critical storage applications. Ethernet, by its very nature, is a best efforts transport mechanism. This fact has deterred many data center managers from deploying Ethernet storage in favor of Fibre Channel-based solutions. Fibre Channel was architected as a lossless networking protocol to address the requirements of high-performance storage networks. In order to increase application of iSCSI in more business critical storage applications, Extreme Networks has enabled lossless networking technology at the switch to provide a robust and cost-effective 1 GbE and 10 GbE solution with increased performance and quality of service. These capabilities work at the solution level, to provide 1 GbE and 10 GbE raw performance with lossless quality of service capa-bilities previously only found in Fibre Channel-based storage systems. These benefits are delivered through a technology called CLEAR-Flow.

The performance requirements for some data center applications can be sufficiently met with 1 GbE network speeds. However for those applications that require greater bandwidth and reduced latency, 10 GbE prod-ucts are also available. In fact, the adoption of 10 GbE

is rapidly increasing due to a number of factors. First, the deployment of server virtualization significantly increases the network I/O density on each server. Instead of servicing the I/O load of only one applica-tion, servers today are managing the I/O load of many applications running on the same hardware. As a result, network bandwidth requirements have increased at the server and edge switches. Ten Gigabit Ethernet can meet the increased I/O demands with fewer ports and cables.

Adoption of 10 GbE in the data center will accelerate as next generation server motherboards transition to embedded 10 GbE LAN on motherboard (LOM) ports and both LOM and NICs offer the addition of more cost-effective 10GBase-T (10 GbE copper) interconnects.

1 Gens, Frank. IDC Predictions 2011: Welcome to the New Mainstream. IDC. December 2010. (IDC #225878, Volume: 1)

Figure 1: Multiple GbE connections in typical virtualized servers.

Figure 2: Simplified server connectivity using 10 GbE.

3



Increasing the Performance of iSCSI Traffic

Lower Cost, Easier to Administer, ConvergediSCSI is popular because of its “built-in” stack, which is basically a storage layer on top of a well understood and ubiquitously implemented TCP/IP stack. The ease of administration of the iSCSI stack is afforded largely because it is included in all major server operating systems. No special drivers are needed with use of a standard Ethernet server adapter, which can reduce risk of component incompatibility. Native iSCSI initiators now support advanced adapter features and platform enhancements that deliver high performance to satisfy the demands of enterprise applications. Native iSCSI allows for easier server provisioning, which can contrib-ute to lower operational expenditures.

The performance of an iSCSI storage network is depen-dent upon the host adapter, the storage system and the network infrastructure. From the network perspective, iSCSI traffic must be protected and prioritized from other traffic flowing through the infrastructure. Previously, this was accomplished by leveraging two separate networks – one for storage (traditionally a Fibre Channel SAN) and another for the LAN traffic. A new set of 10 GbE standards designed to provide lossless characteristics and improved Quality of Service (QoS) is Data Center Bridging (DCB). Combined with the Fibre Channel over Ethernet (FCoE) standard, customers can now converge traditional FC traffic onto a shared 10 GbE network. Support for IP traffic isolation using DCB is still under development, but is forthcoming. However, alternatives are available today that can offer improved performance for iSCSI traffic in congested environments.

The Power of CLEAR-FlowExtreme Networks has developed a simple yet powerful approach to data center network storage convergence without having to compromise protection or perfor-mance guarantees. CLEAR-Flow deploys a dynamic policy that automatically identifies iSCSI traffic on the network. Once identified, the switch prioritizes the iSCSI traffic and creates a lossless environment that is protected from non-iSCSI traffic on the network. The implementation of dynamic iSCSI provisioning on the network switch is accomplished in two steps. The first step implements the policy at a network level, by installing a global policy file on each switch in the network, which then uses CLEAR-Flow to identify iSCSI traffic. The second step dynamically applies a group of parameters, in an automated fashion, at the port level once iSCSI traffic is detected.

Many networks today rely on QoS alone to resolve congestion issues and prioritize different traffic types on a per port basis. Extreme Networks can augment the benefits of QoS with its CLEAR-Flow engine to process iSCSI traffic identified on the network. CLEAR-Flow is a feature that allows Extreme Networks switches to make forwarding decisions based on traffic type. Instead of simply looking at the source and destination of the traffic and forwarding it along the appropriate Layer 2 or Layer 3 path, CLEAR-Flow takes things a step further by allowing network administrators to specify certain types of traffic that require more attention. In this case, the switch identifies traffic on source and destination TCP ports 3260. Once certain criteria for this traffic are met (100 iSCSI frames per second), the switch performs the following three actions:

1. Leverages QoS to assign the iSCSI traffic to a high priority queue.

2. Enables jumbo frames on the identified ports.3. Submits a log entry indicating that iSCSI traffic has

been identified and protected.

Once the iSCSI traffic has been identified and prioritized, it can be protected against other types of traffic on the network, including broadcast storms and other traffic flow types that might compromise the performance of iSCSI. This is made possible by placing all lower priority traffic into lower priority queues where they can be prioritized effectively under congested network condi-tions. In extreme situations, lower priority packets may be completely discarded in favor of iSCSI traffic.

This CLEAR-Flow enabled policy is configured as an access control list on the switch, which in this case permits all traffic and places iSCSI traffic into a higher priority queue.

If the iSCSI traffic were to stop at any time, the switch’s CLEAR-Flow engine dynamically identifies that the traffic has completed and removes the associated dynamic access control list, allowing hands-off admin-istration. The iSCSI targets and initiators can be moved anywhere throughout the network infrastructure without the need for further administration of the switch.

This simple, yet effective CLEAR-Flow policy provides superior dynamic prioritization and protection for all iSCSI traffic on the network. Furthermore, the policy can be deployed across any number of connected devices where the traffic will be identified on all ports, includ-ing edge, uplink and core. This ensures that the traffic is protected and prioritized along the entire path the packets take through the network.

4



Test Configuration OverviewThis test environment has four basic components. These components include the storage system, the server(s), Ethernet server adapters, and the network switches. The purpose of the test is to demonstrate how consistent performance for iSCSI traffic can be maintained even during times of network congestion, so that storage traffic is not impacted by other traffic on the network.

The test configuration shown in Figure 3 consists of five Intel Xeon processor 5600 series-based servers running Microsoft Windows Server 2008 R2. The servers are equipped with Intel Ethernet Server Adapter X520 series 10 GbE NICs using the native iSCSI initiator in Windows Server 2008 R2. These NICs are connected to an Extreme Networks Summit® X650 data center switch, which is in turn connected via a 10 GbE port to a NetApp® FAS6280 storage system. To create contention, a second Extreme Networks Summit X650 switch was connected in line to the first switch, with the storage system connected to the second switch. The connection between the two switches is another 10 GbE port on each switch. Two test servers were then connected to each switch to create contention on the link between the switches.

The hardware and software used to execute the func-tional and performance testing consists of:

• Intel white box dual-processor server

– Two Intel Xeon processor 5680 CPUs

• 6 cores/12 threads each

• 3.33 GHz clock speed

– Intel Ethernet Server Adapter X520 series 10 GbE adapters

– 24G Memory

– 2x146G HDD RAID 0

– Windows Server 2008 R2

• NetApp FAS6280 storage system

– Data ONTAP® 8.0.1 operating system

– 2T Flash Cache (intelligent read cache)

• Extreme Networks Summit X650 10 GbE Switch

– ExtremeXOS® v. 12.5

• Microsoft Windows Server 2008 R2

– Native OS FCoE and iSCSI initiators

• Iometer version 2006.07.27

Test ResultsThe test results show that CLEAR-Flow clearly helps to protect and prioritize iSCSI traffic. Each graph shows four basic tests performed using Iometer.

1. No Contention with CLEAR-Flow – Shows what would happen on a normal link with no other traffic on the network.

2. Contention with CLEAR-Flow – In this case, there is contention but CLEAR-Flow protects the iSCSI traffic and ensures it is lossless.

3. No Contention and No CLEAR-Flow – The normal case for iSCSI on a normal network, but a network with no other traffic.

4. Contention and No CLEAR-Flow – The worst case—no protection for the iSCSI traffic and contention on the network. Throughput is severely limited.

5582-01

W2K8(iSCSI with Intel)







NetAppFAS6280

Extreme Networks®Summit® X650

Extreme NetworksSummit X650

10 GbE 10 GbE

10 GbE

10 GbE10 GbE 10 GbE10 GbE

10 GbE

10 GbE

Figure 3: Test Configuration Architecture

5



The graphs are also divided between four different server configurations:

1. One server and one iSCSI session – a single server with a single stream

2. One server and eight iSCSI sessions – a single server with eight TCP streams

5583-01

IOmeter – 1 Server and 1 iSCSI Session/Server(with/without CLEAR-Flow)

2,500

2,250

2,000

1,750

1,500

1,250

1,000

750

500

250

No ContentionCLEAR-Flow

ContentionCLEAR-Flow

No ContentionNo CLEAR-Flow

ContentionNo CLEAR-Flow

0

100

90

80

70

60

50

40

30

20

10

0

Thro

ughp

ut (

MB

ps)

Ave

rage

CPU

Util

(%

)

512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB

MBps % CPU

5584-01

IOmeter – 1 Server and 8 iSCSI Session/Server(with/without CLEAR-Flow)

2,500

2,250

2,000

1,750

1,500

1,250

1,000

750

500

250





0

100

90

80

70

60

50

40

30

20

10

0

Thro

ughp

ut (

MB

ps)

Ave

rage

CPU

Util

(%

)


MBps % CPU

6



3. Five servers and one iSCSI session – five servers (enough to oversaturate the 10 GbE link) and a single stream per server

4. Five servers and eight iSCSI sessions - five servers and eight streams, this being the most taxing test

5585-01

IOmeter – 5 Servers and 1 iSCSI Session/Server(with/without CLEAR-Flow)

2,500

2,250

2,000

1,750

1,500

1,250

1,000

750

500

250





0

100

90

80

70

60

50

40

30

20

10

0

Thro

ughp

ut (

MB

ps)

Ave

rage

CPU

Util

(%

)


MBps % CPU

5586-01

IOmeter – 5 Servers and 8 iSCSI Session/Server(with/without CLEAR-Flow)

2,500

2,250

2,000

1,750

1,500

1,250

1,000

750

500

250





0

100

90

80

70

60

50

40

30

20

10

0

Thro

ughp

ut (

MB

ps)

Ave

rage

CPU

Util

(%

)


MBps % CPU

Make Your Network Mobile

Extreme Networks White Paper: Innovation With ExtremeXOS

© 2011 Extreme Networks, Inc. All rights reserved. Extreme Networks, the Extreme Networks Logo, ExtremeXOS and Summit are either registered trademarks or trademarks of Extreme Networks, Inc. in the United States and/or other countries. NetApp, the NetApp logo, Go further, faster, and Data ONTAP are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other trademarks are the trademarks of their respective owners. Specifications are subject to change without notice. 1772_01 03/11

www.extremenetworks.com

Corporateand North AmericaExtreme Networks, Inc.3585 Monroe Street Santa Clara, CA 95051 USAPhone +1 408 579 2800

Europe, Middle East, Africaand South AmericaPhone +31 30 800 5100

Asia PacificPhone +65 6836 5437

JapanPhone +81 3 5842 4011

Conclusion

Market acceptance of Ethernet-based storage solutions, such as iSCSI and NAS, is increasing as companies of all sizes recognize economic and operational benefits of converged data center networks based on Ethernet technology. Trends, such as server virtualization and service-based infrastructures, signal a significant change in storage architectures to support business critical and high-demand storage applications that have traditionally been deployed using dedicated single purpose networks.

Ethernet network throughput can be significantly affected by the level of overall congestion in the network, which can be a challenge for block-based storage applications. By enabling CLEAR-Flow technology in both 1 GbE and 10 GbE network environments, iSCSI storage traffic can be protected, and offers lossless storage performance along the data path. As servers become more virtualized and network utilization rises, this ability to prioritize data traffic is critical in order to provide predictable iSCSI SAN performance.

iSCSI and CLEAR-Flow

Technology

Transcript of iSCSI and CLEAR-Flow