iSCSI Performance in Integrated LAN/SAN Environment

Li Yin

U.C. Berkeley

Outline

Motivations Network Congestion and Disk Congestio

n differentiation iSCSI ns-2 Simulator Conclusion Future work

Motivations

Storage Area Network• Hosts/Servers are connected to storage devices through

network

• Advantages:

• High availability

• Better resource sharing

• Easy to manage Transport SCSI over IP Networks

• Less cost

• Longer operating distance

• Leverage existing technology and management tools

Motivations (cont.) iSCSI is one of the most promising protocols to carry

SCSI traffic over IP networks. SAN/LAN integration is one step towards SAN/WAN

integration• Utilize existing Ethernet to carry iSCSI traffic

• Problem:• iSCSI performance degrades dramatically with cross traf

fic

• Challenge:• How to sustain high I/O throughput even with competing

traffic?

Why Performance Degrades?

Resources are saturated• Cross traffic will compete for

network bandwidth, disk, CPU and etc.

First step: where is the congestion point?• Our hypothesis: Congestion

caused by network and disk behave differently

Second Step: how to avoid it?

Test bed Setup Topology:

Cisco iSCSI initiator Target machine runs IBM iSCSI target software driver A SCSI disk is attached to the target machine as the target

disk Nistnet running on the software router to control delay, pack

et drop rate and etc.

Software Router

TargetInitiators

Experimental Setup iSCSI target disk is attached as raw device at the i

nitiator• To filter out file system effect

16384 64KB random requests (1G data in total)• To reduce the cache effect

Initiator: response time seen by the initiator

Tif: Initiator request finish time; Tir: Initiator request receive time

Target: response time seen by the target iscsi driver

Ttf: Target request finish time; Ttr: Target request receive time

trtft TTR

irifi TTR

Read

Write

Ri

Ri

Rt

Rt

Case 1: Network congestion

Use packet drops to reflect network congestion

Nistnet introduces uniform random packet drops

The target disk performs no other disk operations

Experimental Results

initiator

target

read write

No drop

1% drop

5% drop

Case 2: Disk Competition

Target machine continuously issue random I/O requests to the target disk

No network interference is introduced

Experimental Results

initiator

target

read write

No drop, No compete

4KB Competing Request

4MB Competing Requests

Result Analysis

read write

initiator

target

No drop, No Compete

4KB Competing Request

1% Drop

Congestion Caused by Network

Target: How to improve performance when the degradation is caused by network?

Our methodology: ns-2 based iSCSI simulator• Simulation can help us alleviate the limitation

of available testbed configuration • Delay

• Bandwidth

Ns-2 iSCSI simulator iSCSI is implemented as a TCP applicati

on.• all iSCSI messages are encapsulated as an

application message transmitted using TCP Start from a simple disk models

• Parameters: • cache hit ratio

• mean, variance, minimum processing time and maximum processing time for various request size

Simulator Verification 16384 64KB random requests (1GB in total)

READ WRITE

Throughput

Delay

Simulation

Testbed

Testbed

Impact of TCP Parameters Impact of minimum timeout value (minrto)

• Real RTT is roughly 1.5ms

A more precise configuration of timeout value in LAN is useful, and is possible

1s minrto 100ms minrto 10ms minrto 1ms minrto

Throughput # of Fast Retransmission # of Timeout

Conclusions

iSCSI performance in LAN/SAN integrated environment is very important

Performance degradation caused by network and disk is possible to be distinguished

Ns-2 based simulator can match the real curve closely

TCP parameter configuration have big impact on the performance

Future Works

Congestion point detection• better monitoring point?

• more complicated cross traffic Simulation:

• More verification on write operations

• Bursty Drop Model

• More realistic system and disk model

• More analysis on how TCP will affect the performance?

Future Work (Cont.) Design scheme to improve performance when congestion

happens in network• Can we avoid packet drop?

• Resource management?• Shift to another target?

• Read: Multiple replica may exist• Write: write to some target temporarily, re-organize data later

• QoS management• Avoid iSCSI drop rate below some threshold

• If we have to, which packet to drop?• Leverage application layer information

• Application importance• iSCSI header packet vs. iSCSI data packet?• Packets belong to same iSCSI data unit or multiple data units?