Implementing Iscsi in Storage Area Networks - THIC iSCSI in Storage Area Networks ... Storage Area...
Transcript of Implementing Iscsi in Storage Area Networks - THIC iSCSI in Storage Area Networks ... Storage Area...
Implementing iSCSI in Storage Area Networks
The Advantages of a Software Initiator Solution
Wei-Li LiuADNET Systems, Inc.
2 Brookcrest Court, Potomac MD 20854-5501Phone: +1-301-770-4850 • Fax: +1-301-770-4828
E-mail: [email protected]
Presented at the THIC Meeting at the Raytheon ITS Auditorium 1616 McCormick Drive, Upper Marlboro MD 20774-5301
October 26-27, 2004
2THIC Meeting, 10/26/2004
Introduction
ADNET Systems, Inc.NASA Goddard Space Flight CenterTask HECN – High End Computing NetworkProject Manager – J. Pat Gary
3THIC Meeting, 10/26/2004
Outline
Network StorageImplementation of Storage Area NetworkImplementation of iSCSIThe iSCSI Test
4THIC Meeting, 10/26/2004
Network Storage
Direct Attached Storage (DAS)
Network Attached Storage (NAS)Advantages = multiple users; platform independence; expandability; fault tolerance
Storage Area Network (SAN)Advantage = prevents normal data backup over SAN from polluting regular network traffic
5THIC Meeting, 10/26/2004
Implementation of SAN
Fibre Channel iSCSI
AdvantagesSpeedMarket DominanceData Only Traffic
Cost Ethernet Ubiquities
DisadvantagesCostAdditional Training
SpeedMixed Network Traffic
7THIC Meeting, 10/26/2004
Implementation of iSCSI (Continued)
iSCSI Initiator Selections, e.g.
Software Initiator
Hardware Initiator
Adaptec TOE NAC
7711C
Adaptec 7211C
8THIC Meeting, 10/26/2004
Implementation of iSCSI (Continued)
Off Load Scheme SelectionsNo Off-Load (Software
Initiator)TCP Off-load Engine
(TOE)iSCSI Host Bus Adapter
(HBA)
Cisco’s iSCSI Driver w/ NIC
CPU
CARD
SOFTWARE INITIATOR
Adaptec 7211C iSCSI adapter
card
CPU
CARD
HARDWARE INITIATOR
CARD
CPU
Adaptec TOE NAC 7711C
adapter
HARDWARE INITIATOR
iSCSITCP/IP
TCP/IP
iSCSI
iSCSITCP/IP
10THIC Meeting, 10/26/2004
Test Goal
Compare performance of iSCSI data transfer using different offload methods
Do different operating systems affect the data transfer speed and efficiency?
Do different PC configurations affect the data transfer speed and efficiency?
11THIC Meeting, 10/26/2004
Test Variables
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711CAdaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
12THIC Meeting, 10/26/2004
Test Variables
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C Adaptec TOE NAC 7711C
13THIC Meeting, 10/26/2004
Test Variables
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
High End PC2 GHz Xeon with 100 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711CAdaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
31 2
14THIC Meeting, 10/26/2004
Test Configuration
Brocade 3800
Brocade 3800
Brocade 3800
Nishan
Building 28
Building 32
High End PC
Low End PC
GSFC SAN FC FABRIC
APPLE RAIDTarget LUN01 TB Apple Xserve Raid
Fibre Channel Switch
Brocade 3800
NASA GSFC SAN Fibre Channel FabriciSCSI Gateway
Nishan 3000
15THIC Meeting, 10/26/2004
Test Configuration (Continued)
1
2
34
56
SPEED TEST — IOZONEFile System Benchmark ToolVersatile in Simulating Actual AppsWrite / Rewrite / Read / Re-read
8 44 2 222 1 111
1 thread with a file size of 8 GB
2 thread with a file size of 4 GB
4 thread with a file size of 2 GB
8 thread with a file size of 1 GB
16THIC Meeting, 10/26/2004
Test Configuration (Continued)
EFFICIENCY TEST — CPU Usage
Windows—Task Manager Linux—TOP
17THIC Meeting, 10/26/2004
Test Results
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C Adaptec TOE NAC 7711C
18THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711CAdaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
1
19THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC Running Red Hat 9Performance Comparison
0.00
20,000.00
40,000.00
60,000.00
80,000.00
100,000.00
120,000.00
1 th
read
2th
read
s
4th
read
s
8th
read
s
1 th
read
2th
read
s
4th
read
s
8th
read
s
Linux w/ Gig E NIC & Cisco sortwareInitiator on a HE PC
Linux w/ Adaptec TOE on a HE PC
Threads
KB
/S
Initial write Rewrite Read Re-read
20THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC Running Red Hat 9CPU Usage Comparison
0.00
10.00
20.00
30.00
40.00
50.00
60.00
1threads
2threads
4threads
8threads
1threads
2threads
4threads
8threads
Linux w/ Gig E NIC & Cisco SW initiator ona HE PC
Linux w/ Adaptec TOE on a HE PC
Threads
Perc
enta
ges
Linux overall write Linux overall read
21THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C Adaptec TOE NAC 7711C
22THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711CAdaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
2
23THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC Running Windows 2000 ProPerformance Comparison
0.00
10,000.00
20,000.00
30,000.00
40,000.00
50,000.00
60,000.00
70,000.00
80,000.00
90,000.00
1 th
read
2th
read
s
4th
read
s
8th
read
s
1 th
read
2th
read
s
4th
read
s
8th
read
s
Win2K w/ Gig E NIC & Cisco softwareinitiator on a HE PC
Win2K w/ Adaptec iSCSI Card on a HE PC
Threads
KB
/S
Initial write Rewrite Read Re-read
24THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC Running Windows 2000 ProCPU Usage Comparison
0.00
20.00
40.00
60.00
80.00
100.00
120.00
1th
read
s
2th
read
s
4th
read
s
8th
read
s
1th
read
s
2th
read
s
4th
read
s
8th
read
s
Win2K w/ Gig E NIC & CISCO softwareinitiator on a HE PC
Win2K w/ Adaptec iSCSI Card on a HE PC
Threads
Perc
enta
ges
Initial write Rewrite Read Re-read
25THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C Adaptec TOE NAC 7711C
26THIC Meeting, 10/26/2004
Test Results (Continued)
High End PC2 GHz Xeon with 100 MHz PCI
Low End PC1.4 GHz Pentium IV with 33 MHz PCI
Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711CAdaptec TOE NAC 7711C
Cisco iSCSI Driver Cisco iSCSI Driver
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
Adaptec 7211 iSCSI adapter card
Adaptec TOE NAC 7711C
3
27THIC Meeting, 10/26/2004
Test Results (Continued)
Low End PC Running Windows 2000 ProPerformance Comparison
0.00
10,000.00
20,000.00
30,000.00
40,000.00
50,000.00
60,000.00
1 th
read
2th
read
s
4th
read
s
8th
read
s
1 th
read
2th
read
s
4th
read
s
8th
read
s
Win2K w/ Gig E NIC & Cisco softwareinitiator on a LE PC
Win2K w/ Adaptec iSCSI Card on a LEPC
Threads
KB
/S
Initial write Rewrite Read Re-read
28THIC Meeting, 10/26/2004
Test Results (Continued)
Low End PC Running Windows 2000 ProCPU Usage Comparison
0.00
20.00
40.00
60.00
80.00
100.00
120.00
1threads
2threads
4threads
8threads
1threads
2threads
4threads
8threads
Win2K w/ Gig E NIC & CISCO softwareinitiator on a LE PC
Win2K w/ Adaptec iSCSI Card on a LE PC
Threads
Perc
enta
ges
Initial write Rewrite Read Re-read
29THIC Meeting, 10/26/2004
Analysis
Performance & CPU Usage ComparisonWindows vs. Linux EnvironmentHigh End vs. Low End PC
30THIC Meeting, 10/26/2004
Analysis (Continued)
High End PC Running Red Hat 9
Performance CPU Usage
Cisco iSCSI Driver
Adaptec TOE NAC 7711C
Cisco iSCSI Driver
Adaptec TOE NAC 7711C
Overall Write 100% 110% 100% 68%
Overall Read 100% 98% 100% 67%
31THIC Meeting, 10/26/2004
Analysis (Continued)
High End PC Running Windows 2000 Pro
Performance CPU Usage
Cisco iSCSI Driver
Adaptec 7211 iSCSI
Cisco iSCSI Driver
Adaptec 7211 iSCSI
Overall Write 100% 103% 100% 79%
Overall Read 100% 80% 100% 47%
32THIC Meeting, 10/26/2004
Analysis (Continued)
Low End PC Running Windows 2000 Pro
Performance CPU Usage
Cisco iSCSI Driver
Adaptec 7211 iSCSI
Cisco iSCSI Driver
Adaptec 7211 iSCSI
Overall Write 100% 122% 100% 74%
Overall Read 100% 127% 100% 67%
33THIC Meeting, 10/26/2004
Analysis (Continued)
Generally, hardware initiator out performs software initiator in both data writing and reading
Exception – data reading with Windows 2000 on the high end PC
Generally, more threads yield better write/read performance
Exception – data writing with LinuxException – data writing/reading with Windows 2000 on the low end PC when implementing software initiator
CPU usage is always greater when using the software initiatorMore threads consume more CPU processing power
Exception – data reading with Windows 2000 on the low end PC when implementing software initiator
36THIC Meeting, 10/26/2004
Analysis (Continued)
Windows and Linux Performance Comparison
Software Initiator Hardware Initiator
Windows 2000 Pro
Linux Red Hat 9
Windows 2000 Pro
Linux Red Hat 9
Overall Write 44% 100% 53% 100%
Overall Read 81% 100% 66% 100%
38THIC Meeting, 10/26/2004
Analysis (Continued)
High End PC & Low End PC Performance Comparison
Software Initiator Hardware Initiator
Low End PC High End PC Low End PC High End PC
Overall Write 69% 100% 71% 100%
Overall Read 57% 100% 85% 100%
39THIC Meeting, 10/26/2004
Analysis (Continued)
High End PC & Low End PC CPU Usage Comparison
Software Initiator Hardware Initiator
Low End PC High End PC Low End PC High End PC
Overall Write 216% 100% 165% 100%
Overall Read 189% 100% 226% 100%
40THIC Meeting, 10/26/2004
Conclusion
CPU offloading works wellLinux outperforms WindowsHardware initiator is justified better in the case of a Low End PC
41THIC Meeting, 10/26/2004
Acknowledgements
Hoot Thompson (Patuxent Technology Partners)Mary Shugrue (BPX) Pat Gary (NASA)Ben Kobler (NASA)Bill Fink (NASA)Eduardo Takamura (ADNET)Aruna Muppalla (ADNET)Paul Lang (ADNET)Rich Bono (Adaptec)Jeff Martz (CSC)Rich Isley (GMRS)