Download - Building a 10 Gigabit Enabled Linux iSCSI Target

Transcript
Page 1: Building a 10 Gigabit Enabled Linux iSCSI Target

Page1

BuildingaHighPerformanceActiveDirectoryIntegrated10GBELinuxStorageServerforuseoverSingleModeFiberNetworks

(SATAVersion)

By:IanEvans

Page 2: Building a 10 Gigabit Enabled Linux iSCSI Target

Page2

Hardwareandsoftwareperquisites:

• (1)RedHat/CentOSx_86_64bitoperatingsystem(CentOSrecommended)

• (1)CiscoSMF10GBXENPAK(XENPAK‐10GB‐LR=)• (1)Supermicrohighdensitychassis(SC846TQ‐R900B)• (1)SupermicroATXservermotherboard(X7DCL‐3)

• (1)IntelXeonQuadCore1.6GHzProcessor(BX80563E5310P)• (2)ATP‐Samsung2GBECCFBDIMM’s(AP56K72U4BHE7S)• (1)3Ware24‐PortRAIDcard(9650SE‐24M8)

• (1)3Ware4‐PortRAIDcard(9650SE‐4LPML)• (2)3WareBatteryBackupUnits(BBU‐MODULE‐03)• (19)WesternDigitalSATA‐II750GBenterprisedrives(WD7500AYPS)

• (4)WesternDigitalSATA74GB10Kenterprisedrives(WD740ADFD)• (1)10GBsinglemodefiberuplink(1350nm)• (2)Singlemodepatchcables

• (1)IntelPro10GBESingleModeLRServerAdapter(PXLA8591LR)

• Antistaticgroundingmatandantistaticwriststrap.Thesecomponentscanbepurchasedfrommostelectronic

stores.

Theinstallationisfairlystraightforward.Aswithanyhardwareconfiguration,itisimportanttoreadallofthemanualspriortocomponentinstallation.Thefollowinginstructionswillguideyouthroughtheentireprocess:

HardwareInstallation

1) UnpacktheSupermicrochassis.Removethefans,drivebays,andplasticshrouds.Ensurethechassisiswellgroundedduringtheentireinstallationprocess.Installcomponents.

2) InstallthecontrollersandEthernetcardexactlyasshownbelow:

Page 3: Building a 10 Gigabit Enabled Linux iSCSI Target

Page3

3) Connectthemultilanecablesintothechassisbackplane.Thefirst4ports(0‐3)willgotothe4‐portcardandtheremainingconnectors(4‐23)willgotothe24‐Portcard:

4) Installtheharddrives.PlacetheWDRaptorsinSAS/SATA#0‐#3andtheWD750GBdrivesinSAS/SATA#4‐#23.

5) Installthesystemmemory,processorandheatsinkandairshroud.

6) Disableunnecessarydevices(e.g.serialandparallelports)intheBIOS.7) <ALT‐3>intothe4‐port3Warecard’sBIOS.SelectalloftheavailableWDRaptordrivesandcreateaRAID1+0

arraycalled“OS”.SettheStorSaveprofileto“Balanced”andwritecacheto“enabled”.Select<F8>tosave

andexit.8) <ALT‐3>intothe24‐port3Warecard’sBIOS.Selectoneofthe750GBdrivesandpress<s>totoggleitasahot

swapspare.Selecttheremaining18750GBdrivesandcreateaRAID6arraycalled“Data”.SettheStorSave

profileto“Performance”andthewritecacheto“enabled”.Select<F8>tosaveandexit.AllowtheRAID6arraytoinitialize(thismaytakeseveralhours).Uponreboot,youwillseetwocardswithfunctionalarrays.

SoftwareInstallationandConfiguration

1) Powerthesystemoffandunplugthe24‐port3Warecard.Setitofftotheside.2) Update3WarecardsandsystemBIOSandreboot.

3) StarttheRedHat/CentOSinstallationandgothroughallofthestandardinstallationoptions.LetRedHat/CentOSinstallthedefaultpartitionandLVMconfigurationonthe4‐port3Warecontroller.Installall

ofthedefaultpackageselectionsandpowerthesystemoffwhentheinstallationiscomplete.4) Installthe24‐port3Warecontrollerbackintothesystemandboottosu‐5) Updatethesystemandreboot:

a. Yum–yupdate&&reboot6) Installthepackagesnecessarytogetthesystemupandrunning:

a. Yum–yinstallsambagcckernel‐headerskernel‐develautconfautomakeopenssl*gcc‐c++

Page 4: Building a 10 Gigabit Enabled Linux iSCSI Target

Page4

7) Downloadandinstallthelatest10GbenetworkcarddriversfromIntel.comandinstall:

8) Modifynetworkstartupscriptfor10Gbecard:

9) RunGNUpartedtocreateaGPTdisk.

a. Parted/dev/sdab. mklabelgpt

c. Mkpartprimaryext30‐<maxpartitionsize>10) CreateEXT3filesystemandmount.**THISSTEPMAYTAKESEVERALHOURS**

a. Mkfs.ext3–m0/dev/sda1

b. Quitc. Mkdir/datad. Mount–text3/dev/sda1/data

e. Verifywith:Df–hT/data

Page 5: Building a 10 Gigabit Enabled Linux iSCSI Target

Page5

11) Addquotastogrub.conf:

12) Setfilesystemperformanceoptions:

13) Addthefollowingperformanceadjustmentsto/etc/sysctl.conf:

Page 6: Building a 10 Gigabit Enabled Linux iSCSI Target

Page6

14) Addadjustmentsto/etc/rc.local:

15) Downloadbonnie++andbenchmarkfilesystem:

a. Yum–yinstallbonnie++

16) Adjustfstabtoreflectthenewfilesystemandcheckmounts:

17) Edit/etc/nsswitch.conf:

18) SetstartupscriptssoWinbindandSambastartonboot:

19) InstalliSCSItargetandclientutilities:

a. svncheckoutsvn://svn.berlios.de/iscsitarget/trunk i. make&&makeinstall

b. yuminstall‐yiscsi*

20) ConfigureiSCSItargetandaccess:a. ChangetheTargetID:

i. Target2008‐1.com.domain:storage.disk0.sys.com

b. Changeincomingusertoreflectyouridentifier:i. IncomingUseriqn.2008‐04.com.remotebox:host(password)

c. ChangeLUNpathstoreflectyourpartitionconfiguration:

i. Lun0Path=/dev/vg0/data,Type=fileiod. ChangetheAliastoreflecttheproperservername:

i. AliasCarbon

Page 7: Building a 10 Gigabit Enabled Linux iSCSI Target

Page7

21) StartiSCSI‐Targetonboot:

22) Checkiscsiconnectionfromremotenode:

a. iscsiadm–mdiscovery–tst–p192.168.1.2

23) Edit/etc/krb5.conftoreflectyourdomainsetup

Page 8: Building a 10 Gigabit Enabled Linux iSCSI Target

Page8

24) ConfigureSambatouseADviathe/etc/samba/smb.conffile:

Page 9: Building a 10 Gigabit Enabled Linux iSCSI Target

Page9

25) Addsharesto/etc/samba/smb.confasnecessary:

26) Addtheserverintothedomainandtesttheconnection

a. netadsjoin–U<adminaccount>‐S<domaincontroller>b. Testdomainmembership:netadstestjoin.c. Testuserenumeration:wbinfo‐u.

d. Testgroupenumeration:wbinfo–g.

✗✗✗ AnaccurateNTPserverisrequiredtoensureproperdomaintimesynchronization.

Page 10: Building a 10 Gigabit Enabled Linux iSCSI Target

Page10

27) AdjusttherulesinIptablesasnecessary:

28) Rebootandcheckthenetworkanddiskconfigs:dmesg|less: