Blade Center Open Fabric Manager-techdoc-V1
Transcript of Blade Center Open Fabric Manager-techdoc-V1
IBM® BladeCenter® Open Fabric
Manager Implementation
Khalid Ansari [email protected]
IBM Advanced Technical Support
Mike Nolterieke [email protected]
IBM Customer Experience
Dave Pierson [email protected]
IBM Network Transformation Center
IBM BladeCenter Open Fabric Manager Implementation
Revision History 1.0 – June 4, 2008
Page 2 of 58
Initial Version
Notices: This paper is intended to provide information regarding IBM BladeCenter Open Fabric Manager. It discusses findings based on configurations that were created and tested under laboratory conditions. These findings may not be realized in all customer environments, and implementation in such environments may require additional steps, configurations, and performance analysis. The information herein is provided “AS IS” with no warranties, express or implied. This information does not constitute a specification or form part of the warranty for any IBM or non-IBM products. Information in this document was developed in conjunction with the use of the equipment specified, and is limited in application to those specific hardware and software products and levels. The information contained in this document has not been submitted to any formal IBM test and is distributed as is. The use of this information or the implementation of these techniques is a customer responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk. IBM may not officially support techniques mentioned in this document. For questions regarding officially supported techniques, please refer to the product documentation, announcement letters, or contact the IBM Support Line at 1-800-IBM-SERV. This document makes references to vendor-acquired applications or utilities. It is the customer responsibility to obtain licenses of these utilities prior to their usage. © Copyright International Business Machines Corporation 2008. All rights reserved. U.S. Government Users Restricted Rights – Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
IBM BladeCenter Open Fabric Manager Implementation
Table of Contents Table of Contents..........................................................................................................................................3 Executive Overview.......................................................................................................................................4 Technical Analysis ........................................................................................................................................5
Hardware ...................................................................................................................................................5 Firmware Prerequisites .............................................................................................................................5 Configure Blade to Boot from SAN............................................................................................................5
Disable the internal hard drive (IDE/SCSI/SATA/SAS) .........................................................................5 Configure the HBA to boot from SAN........................................................................................................7
Implementing IO Virtualization with BladeCenter Open Fabric Manager ...................................................10 Initial BOFM Setup ..................................................................................................................................10
Method 1: Using AMM .........................................................................................................................10 Method 2: Using the Advanced BOFM GUI.........................................................................................15
Verifying the Initial BOFM configuration..................................................................................................19 Implementing Automatic Failover Function with Advanced BladeCenter Open Fabric Manager...........21 SAN Zoning .............................................................................................................................................34 Configure Storage Subsystem ................................................................................................................39 OS Installation .........................................................................................................................................45 Testing Automatic Failover......................................................................................................................46
Advanced BladeCenter Open Fabric Manager Console GUI Simplification .......................................46 Simple Failover Test without Event Action Plan ..................................................................................47 Failover Test with an Event Action Plan (EAP) ...................................................................................50
Appendix .....................................................................................................................................................55 Table of Abbreviations.............................................................................................................................55 References ..............................................................................................................................................56 Trademarks .............................................................................................................................................57 Notices.....................................................................................................................................................58
Page 3 of 58
IBM BladeCenter Open Fabric Manager Implementation
Executive Overview BladeCenter Open Fabric Manager Benefits:
• Save time - preconfigure LAN and SAN connections once for each blade
• Manage growth and complexity - up to 100 chassis, 1400 blades and over 11,000 SAN and LAN addresses
• Remain open with infrastructure flexibility - works with all BladeCenter chassis, blades and switches
• Connect with ease - single, simple interface for server and switch management
• Manage risk to keep your business running - available automated I/O failover to standby blades
Overview: IBM BladeCenter Open Fabric Manager (BOFM) is designed to help you manage growth and complexity by making it easy to manage I/O and network interconnects for up to 100 BladeCenter chassis - up to 1400 blade servers. BladeCenter Open Fabric Manager helps make blade deployment EASY: once Installed, the utility is resident in the Advanced Management Module (AMM) so you can preconfigure LAN and SAN connections. Thus, I/O connections are made automatically when you plug in a blade. And no special tools or training is required; just manage with the easy-to-use GUI.
In contrast to typical competitive tools that only support a small number of proprietary blades and switches, BladeCenter Open Fabric Manager is OPEN and allows you to manage a wide range of industry Ethernet and Fibre Channel switches - Cisco, Nortel, Brocade, QLogic.
BladeCenter Open Fabric Manager is the RIGHT CHOICE now and into the future. The software can even be deployed on existing hardware through a firmware upgrade, helping you protect your current BladeCenter investments and giving you the flexibility to deploy precisely the solution you need to help your business realize innovation.
Manage risk with an optional upgrade which provides a standalone utility to enable automatic failover across chassis. It allows you to define action plans that can be automatically execute based on customer-defined trigger events. If the trigger event occurs, a blade in the predefined standby pool will automatically receive I/O parameters and take over for the original blade. This OS independent tool is an EASY and OPEN way to virtualize the Ethernet MAC & Fibre Channel WWPNs on your blade servers.
Page 4 of 58
IBM BladeCenter Open Fabric Manager Implementation
Technical Analysis
Implementing Boot from SAN on IBM BladeCenter Blades When working with IBM BladeCenter Server Blades a best practice is to use an external storage to boot from. Using external storage to boot from SAN not only can provide better I/O performance but also removes the component in the server that is most likely to physically fail. Additionally, true benefits of IBM Open Fabric Manager can be realized by using the storage external to the blade server. If a hardware failure is experienced on the blade, then the administrator only has to replace the failed component while the system image resides on the SAN attached storage. In the following sections we will cover the steps required to setup boot from SAN on x86 architecture BladeCenter blades.
Hardware • (2) HS21 XM – Model-Type 7995-AC1
o BIOS – 1.08 - BuildID: MJE124BUS o Blade Systems Management Processor – 1.12 – BuildID: MJBT19AUS o Qlogic 4Gb Fiber HBA Expansion (41Y8583) – 1.43
• BladeCenter H Chassis – Model-Type 8852-4XU o Advanced Management Module Firmware – 1.36H - Build ID: BPET36H
• Disk Storage System DS4300 – Model-Type 1722- o Firmware - 06.60.08.00
Firmware Prerequisites Before you get too far into setting up boot from SAN environment, you need to prepare the hardware to be compatible with BladeCenter Open Fabric Manager. It is highly recommended to update all of the BIOS and firmware to the latest version for compatibility with BladeCenter Open Fabric Manager. This ensures system compliance and enables IBM support to assist if there are any problems. It is a good practice to update the firmware on all components but those marked below with a * are critical to the operation of BladeCenter Open Fabric Manager.
• Blades o BIOS Firmware o BMC Firmware o HBA Firmware
• Chassis o Advanced Management Module Firmware o I/O Switch Firmware
• External Storage o Controller Firmware o HDD Firmware
Configure Blade to Boot from SAN There are some changes that need to be made at a firmware configuration level before a blade can boot from SAN attached storage.
Disable the internal hard drive (IDE/SCSI/SATA/SAS) This will vary by blade model but you will need to disable the on board disk from the blade BIOS utility.
Page 5 of 58
1. Press F1 at POST to enter the BIOS setup.
IBM BladeCenter Open Fabric Manager Implementation
2. Select Devices and I/O Ports as shown in Figure 1 below.
Figure 1 BIOS Setup Main Menu
Page 6 of 58
3. Select Planar SAS or whatever the on-board storage device is and set to Disabled as shown in Figure 2
IBM BladeCenter Open Fabric Manager Implementation
Figure 2 Disable on board storage
4. Save the settings and exit from the BIOS configuration. The blade will now reboot.
Configure the HBA to boot from SAN It is necessary to configure the HBA BIOS to boot from SAN. Follow the steps below to complete
this setup.
1. Enter the HBA BIOS setup utility by entering the proper keystroke when the HBA ROM loads. It is likely Ctrl+Q as shown in Figure 3 below.
Figure 3 HBA ROM prompt
2. Select the first port as shown in Figure 4. Press Enter.
Page 7 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 4 Port selection
3. Select Configuration Settings Adapter Settings. 4. Modify the Host Adapter BIOS setting to Enabled as shown in Figure 5. Take note of the World
Wide Port Name (Adapter Port Name) we will need it later for configuration.
Page 8 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 5 HBA BIOS settings
5. Save the settings and exit the HBA BIOS utility. 6. If setting up a multipath SAN perform the same steps on the second HBA port.
Page 9 of 58
IBM BladeCenter Open Fabric Manager Implementation
Implementing IO Virtualization with BladeCenter Open Fabric Manager Once the BOFM environment is prepared by updating the AMM firmware, blade server BIOS, BMC & HBA BIOS, the BOFM configuration can be created. There are two methods that can be used to setup BOFM.
o Method 1: Using the Advanced Management Module (AMM) web/cli interface o Method 2: Using the Advanced BOFM GUI
Note: Method 2 is only available if you have purchased the Advanced BOFM option. There are three major steps needed for I/O virtualization with BOFM.
• Create an Initial Configuration • Modifying a Server Profile
o Selecting Ethernet MAC address and FC WWPN range o Specify FCP Target WWPN, HBA Boor order and boot device priority
• Apply the Server Profile We will first cover the AMM method. If you would prefer to use the Advanced BOFM GUI skip to page 15, Method 2: Using the Advanced BOFM GUI.
Initial BOFM Setup
Method 1: Using AMM The Create an Initial Configuration option is accessed from the BladeCenter Advanced Management Module (AMM) Web interface. Note: These steps can be performed using the AMM CLI interface also, but we will refer to the Web interface in this example.
1. From the AMM Web interface Select the Blade Open Fabric Manager select the Create an Initial Configuration option as shown in Figure 6.
Page 10 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 6 Open Fabric Manager Configuration Menu
2. Specify the virtual Ethernet MAC address and Fibre Channel WWPN as shown in Figure 7.
Figure 7 Specify Virtual Addresses Menu
3. In the Advance Options menu, specify the Fibre Channel WWPN of the Boot device and the Boot LUN ID as shown in Figure 8.
Page 11 of 58
Important: The BOFM server profile GUI menu only allows configuring primary and secondary path to the boot LUN. Any additional paths must be manually configured from the HBA utility.
IBM BladeCenter Open Fabric Manager Implementation
Figure 8 Specify Virtual Addresses Advance Option menu
4. Specify the IP address of the BladeCenter chassis or select the default option as shown in Figure 9. Note: The default option will add entries for any remote chassis that have been discovered via Monitors Remote Chassis. By default there will be no discovered chassis unless you have clicked the Discover button in the Remote Chassis dialog.
Figure 9 Chassis to Include
5. If necessary, disable the Popup blocker, download and Save the configuration file as shown in Figure 10.
Page 12 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 10 File Download Option
6. Please power off any blades that are affected by the update and then select the “you can apply the configuration now” link as shown in Figure 11.
Figure 11 Apply Configuration
7. Click OK on the confirmation dialog shown in Figure 12.
Page 13 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 12 Confirmation Dialog
8. As shown in Figure 13, the AMM ensures that the Blades are powered off, the firmware on the Blades are BOFM compatible, and that there are no duplicate addresses.
Figure 13 Compatibility check is performed
9. The following Figure 14 shows that all the 14 blades in the chassis are ready to receive virtual addresses and BOFM will be enabled. Select Continue to Confirm BOFM Configuration Changes.
Page 14 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 14 Confirm BOFM Configuration Changes menu
10. Close the BOFM dialog shown in Figure 15. Power On the blade servers and confirm that the BOFM assigned virtual addresses are in use. You can confirm that the virtual addresses are in use by comparing the MAC/WWPN that the OS on the blade reports to the address that is in the BOFM configuration file saved in step 5.
Figure 15 The Configuration file was applied successfully
11. To confirm that the new addresses are in use, refer to “Verify Initial BOFM Configuration” section on Page 19.
Method 2: Using the Advanced BOFM GUI The initial configuration can also be configured using the BladeCenter Configuration Manger which is part of the Advanced BOFM option. Advanced BOFM is available as a standalone option and as an IBM Director plug-in. Either way it runs in IBM Director. You would order the Standalone version unless you have an existing IBM Director infrastructure to add the plug-in to.
1. From the Advanced BOFM/IBM Director console locate the chassis that we want to configure the virtual addresses on in the middle objects pane. Right click on the chassis and select BladeCenter Management BladeCenter Configuration Manager as shown in Figure 16 below.
Page 15 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 16 Selecting the chassis
2. Click Create a new profile. 3. Enter a name for the new profile in the dialog box and click OK. 4. At the Select components dialog select BladeCenter Open Fabric Manager – Blade Address
Configuration and click OK. 5. As shown in Figure 17 below, at the Configuration Manager Editor dialog expand the
Management category in the left pane and select BladeCenter Open Fabric Manager – Blade Address Configuration. Confirm that the chassis is selected in the Available Chassis list and click Add to add the chassis to the Domain Members. If you wish to create a configuration for multiple chassis you can add additional IP Addresses of additional chassis.
Page 16 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 17 Configuring the virtual addresses
6. Click Next. 7. Focus moves to the Enable tab. There is likely nothing to do here. You should see the chassis
listed under Domain Members. Click Next. 8. Focus moves to the Address Pool tab. Most likely the default values are fine here. If you are
spanning multiple chassis you will want to increase the Maximum Chassis In Domain count. Click Next.
9. Focus moves to the Boot Settings tab. Fill in the SAN WWPNs and Blade LUNs. These are modifiable on a per blade basis in the upcoming Manual Override tab. Click Next.
10. Focus moves to the Advanced Options tab. Most likely the default values are fine here. Click Next. Note: If you receive an error regarding “Ran out of MAC address” it is because you have multiple IPs of chassis defined and only a single chassis defined in the Address Pool tab.
Page 17 of 58
11. Focus moves to the Manual Override tab shown in Figure 18. Here you have much finer control over how each slot is treated in the chassis. In most cases the only changes here would be to disable certain slots. Make any alterations and click Next.
IBM BladeCenter Open Fabric Manager Implementation
Figure 18 Fine tuning the chassis
12. Review the summary tab and when you are satisfied click Export. This will bring up a file save dialog. You can save the file to whatever filename you like but please give it a filename extension of csv. For example, bofm.csv Save the csv file in a safe place as it could come in handy if you ever need to restore the virtual addresses to the chassis.
13. Save the profile that you have just created by clicking File Save Profile. 14. In order for the configuration file to transfer to the chassis, FTP must be enabled on the chassis.
This can be enabled in the AMM under MM Control Network Protocols File Transfer Protocol (FTP) as shown in Figure 19.
Page 18 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 19 Enable FTP
15. You can now apply the profile to the chassis by dropping the profile on the chassis as shown in
Figure 20 below.
Figure 20 Applying the profile
16. Select Execute Now from the dialog that appears.
Verifying the Initial BOFM configuration
1. Login to the BladeCenter Advanced Management module Web interface. 2. From the AMM GUI, select Blade Tasks Configuration Open Fabric Manager as shown in
Figure 21.
Page 19 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 21 Blade Configuration Open Fabric Manager menu
3. Selecting one of the blade servers with BOFM enabled will list the status of BOFM, virtual MAC and WWPN and Fibre Channel Target information as shown in Figures 22 and 23.
Figure 22 BOFM status and virtual MAC addresses in use for blade in bay10
Page 20 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 23 BOFM status and virtual FC WWPNs in use for blade in bay10
4. Select Close to exit the BOFM menu. This concludes the example of setting up BOFM.
Implementing Automatic Failover Function with Advanced BladeCenter Open Fabric Manager To setup an Automatic Failover function with Advanced BOFM, the following steps are needed:
• Create a profile for Standby blade pool and add the blade servers • Create an Event Action Plan(EAP)
o Define a Simple Event Filter o Add Event Filter o Customize an Action Task o Add Action to the Event Filter
• Apply the Standby blade Profile to the blade servers in Standby pool • Apply the EAP to the blade servers in Standby pool
1. Select and Open BladeCenter Configuration Manager as shown in Figure 24.
Page 21 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 24 Open BladeCenter Configuration Manager option
2. Create a Profile for Standby Pool as shown in Figure 25.
Page 22 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 25 Create a new profile
3. Select the BladeCenter chassis type and specify the name of the profile as shown in Figure 26.
Figure 26 select chassis type and specify the profile name
Page 23 of 58
4. Select the Standby Blade Pool Configuration option as shown in Figure 27.
IBM BladeCenter Open Fabric Manager Implementation
Figure 27 Select the Standby Blade Pool Configuration option
5. Select and Add blade servers to the Select the Standby Blade Pool Configuration option as shown in Figure 28.
Page 24 of 58
Figure 28 Add Blades to the Standby Blade Pool
IBM BladeCenter Open Fabric Manager Implementation
6. From the main menu, Select File Save Profile and exit as shown in Figure 29.
Figure 29 Save & exit the profile
7. Select the Event Action Plan Open as shown in Figure 30.
Figure 30 Event Action Plan menu
8. To create a new Event Action Plan(EAP), Right click Event Action Plans in the left most pane labeled Event Action Plans and select New option, as shown in Figure 31.
Page 25 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 31 Create a new EAP
9. Name the EAP as show in Figure 32.
Figure 32 Name the EAP
10. Create a Simple Event Filter by right clicking Simple Event Filter and then clicking New as shown in Figure 33.
Figure 33 New event filter
Page 26 of 58
11. This opens the Event Filter dialog shown in Figure 34.
IBM BladeCenter Open Fabric Manager Implementation
Figure 34 Event Filter dialog
12. Uncheck the Any box and select the MPA, Component, Server, Power, and Off boxes as shown in Figures 35 and 36.
Page 27 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 35 Event Filter dialog --> MPA
Page 28 of 58
IBM BladeCenter Open Fabric Manager Implementation
13. Save the Event by clicking File Save As as shown in Figure 36 and 37.
Figure 36 Save Event
Page 29 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 37 Name Event filter
14. Add the filter just created to the EAP by selecting the event filter created in the last step and then right click on the EAP just created in the Even Action Plans column and select Add Event Filter as shown in Figure 38.
Figure 38 Add event filter
Page 30 of 58
IBM BladeCenter Open Fabric Manager Implementation
15. Create a Task Action by right clicking Start Task on the “event” system and then selecting Customize as shown in Figure 39.
Figure 39 Create Task Action
16. Select [BladeCenter Management][BladeCenter Configuration Manager][<name of standby pool created in step 6>] as shown in Figure 40.
Figure 40 Select action
17. Save the Action by clicking File Save As, as shown in Figures 41 and 42.
Page 31 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 41 Save action
Figure 42 Name action
18. Add Action to the EAP created in step 9 as shown in Figure 43 by right clicking the filter and selecting Add action. Then select the action created in the last step.
Page 32 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 43 Add action to EAP
19. This completes the creation of the Event Action Plan as shown in Figure 44.
Figure 44 Event Action Plan complete
Page 33 of 58
IBM BladeCenter Open Fabric Manager Implementation
SAN Zoning When booting from a SAN we need to make sure that communication between blades and storage is isolated to the specific blades and storage partitions that we designate. To ensure this we will setup SAN zoning on the fibre channel switches.
1. Login to the GUI fibre channel switch management utility. The default username and password is likely USERID and PASSW0RD.
2. At the main GUI screen select Zoning Edit Zoning as shown in Figure 45.
Figure 45 Edit zoning
Page 34 of 58
3. At the main zoning configuration dialog click on the Zone Set icon as shown in Figure 46 to add a new zone set.
IBM BladeCenter Open Fabric Manager Implementation
Figure 46 Create new zone
4. Put the name of the new zone set in the dialog box and click OK as shown in Figure 47.
Figure 47 Name new zone
5. Select the ZoneSet created in step 4 in the left pane and click the Zone icon as shown in Figure 48 to add a new zone.
Page 35 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 48 Add zone
6. Put the name of the new zone in the dialog box and click OK as shown in Figure 49.
Figure 49 Name zone
Page 36 of 58
IBM BladeCenter Open Fabric Manager Implementation
7. Now that we have a new zone we need to add the World Wide Name (WWN) of both the HBA port and the Storage Controller port to the zone. Select the zone name created in step 6 in the left pane of the zoning GUI and the storage controller WWN in the right pane and click the Insert icon to add the Controller WWN to the zone as shown in Figure 50. Do the same steps to add the HBA WWN to the zone created in step 6. You will end up with a zone containing two world wide names as shown in Figure 51.
Page 37 of 58
Figure 50 Insert World Wide Names
IBM BladeCenter Open Fabric Manager Implementation
Figure 51 World Wide Names added
8. Click Apply and you will be prompted as shown in Figure 52. Here you can perform an Error check just to make sure everything is fine.
Figure 52 Applying zoning configuration
Page 38 of 58
9. Save the Zoning configuration by clicking Save Zoning and you will be prompted by the dialog shown in Figure 53. Click Yes to activate the zone set.
IBM BladeCenter Open Fabric Manager Implementation
Figure 53 Zone set activation
10. Select the set to be activated in the dialog displayed and click OK as shown in Figure 54. Click Close on the Save Zoning & Error Check dialog.
Figure 54 Select Zone Set
11. If you have a multipath SAN, complete these same steps for the second port on the HBA and second controller on the SAN storage. This will require logging into the fibre channel switch interface on the second fibre channel switch.
Configure Storage Subsystem The following steps will take us thru configuring the DS4300 storage unit.
1. Upon launch of the storage manager utility you will be presented with the dialog shown in Figure 55. If you are not presented with this dialog you can access it via View Task Assistant. Click on the icon next to Configure Storage Subsystem.
Page 39 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 55 Configuration wizard
2. Select Automatic configuration. Click OK. 3. Click Next at the Introduction dialog 4. Select RAID 5 as a suggested configuration and click Next as shown in Figure 56.
Page 40 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 56 RAID Configuration
5. Click Finish at the summary dialog. 6. It will take some time for the RAID setup to complete. You can continue on to defining hosts
while the RAID array is building. At this point you will probably be prompted by the Configuration Task dialog. Click Cancel on it to dismiss it.
7. Now you should be back at the Task Assistant shown in Figure 54. If you do not see the Task Assistant please launch it by clicking View Task Assistant. Click on the icon next to Define Hosts.
8. Click Next at the dialog shown in Figure 57.
Figure 57 Host definition
Page 41 of 58
9. You will now be prompted by the Host Name/HBA Attributes dialog shown in Figure 58. The dialog should have the HBA port identifiers listed in the Known HBA host port identifiers field. We are looking to add the host port identifiers that correspond to the HBA ports on the blade that we are booting from storage. To find these out we can either look at the CSV file created earlier or
IBM BladeCenter Open Fabric Manager Implementation
we can look in the Advanced Management module under Blade Tasks Configuration Open Fabric Manager Click on the blade name in the table to see details of the virtual MAC and WWPN addresses as shown in Figure 59. Take note of the fibre WWPNs.
Page 42 of 58
Figure 58 Host Name/HBA Attributes
IBM BladeCenter Open Fabric Manager Implementation
Figure 59 BOFM Virtual Addresses
10. Enter a host name in the Host Name field as shown in Figure 60. Add the HBA host ports to the Selected HBA host ports by selecting them in the left pane and clicking Add to add them to the right pane. Once they are in the right pane you will need to select them and click Edit in order to assign them an alias. I suggest an alias name of the blade name followed by p1 or p2 for port 1 and port 2. When complete the Host Name/HBA Attribute dialog should look something like Figure 60. Click Next.
Page 43 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 60 Adding HBA Host Ports
11. A summary dialog is displayed as shown in Figure 61. Take note of the host type message and take action if needed. Click Finish.
Page 44 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 61 HBA host summary
OS Installation
Page 45 of 58
1. I installed Redhat Enterprise Linux® 5.0 Server Update 1 on a HS21 blade. The key to installing to SAN storage is making sure that only one path to the disk storage is enabled. The easiest way to do this other than pulling the fibre channel connection is to disable the second HBA port at the fibre switch as shown in Figure 62. To do this select the port by click on it and then select Port Port Properties and select either Downed or Offline from the drop down list of Port States.
IBM BladeCenter Open Fabric Manager Implementation
Figure 62 Disable secondary path
2. After the OS installation is complete, boot the OS with only a single path enabled. 3. There are several options for configuring a multipath setup but the current preferred direction
seems to be to use RDAC found here: http://www.lsi.com/rdac . DM-MPIO looks to be the future multipath direction but is not quite as easy to implement yet.
4. Download and install the RDAC package as documented. Enable the second path. 5. Reboot the server.
Testing Automatic Failover
Advanced BladeCenter Open Fabric Manager Console GUI Simplification This section will outline some steps that can be taken at the console GUI in order to simplify what is displayed. This should help make failover simpler to see visually in this first round of testing.
1. Add a more descriptive name to the objects that we are working with. To do this, select All Managed Objects in the left pane. Locate the objects you wish to rename in the middle pane and right click on them and select Rename. Modify the name and click OK.
2. Create a new group by clicking Console New Group Static Group. This will bring up the static group editor shown in Figure 63. Add the chassis as well as the blade service processors and any physical blades that have operating systems by selecting them in the left pane and clicking Add. When all objects are added click File Save, enter a name for the group and click OK.
Page 46 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 63 Create new group
3. In order to have a view that shows the topology; we need to select the newly created group in the left pane and then click Associations Chassis Membership and you will get a better view of the chassis topology as shown in Figure 64.
Figure 64 BladeCenter topology view
Simple Failover Test without Event Action Plan
Page 47 of 58
The simplest failover test does not include any kind of trigger to initiate the failover. We simply apply the Standby Blade Pool profile to a service processor and initiate the failover as shown in the steps below.
IBM BladeCenter Open Fabric Manager Implementation
1. Confirm that the service processors of the blades that you want to perform the failover on are
listed in the middle pane. 2. In the right pane expand BladeCenter Management BladeCenter Configuration Manager to
reveal the blade standby pool created previously. Left click and drag this profile onto the blade that you want to failover from (powered on blade) as shown in Figure 65. Note: In order for failover to occur there must be at least one blade in the standby pool that is powered off.
Figure 65 Simple manual failover
3. Once you drop the profile on the blade you will see the dialog shown in Figure 66. Click Execute Now.
Figure 66 Execute failover
4. The dialog shown in Figure 67 is displayed stating the failover is in progress. If you expand out the In Progress drop down topology in the lower pane you can double click on the service processor listed in order to view more detail on the failover process as shown in Figure 68.
Page 48 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 67 Failover in progress
Figure 68 Detailed failover progress status
5. When the failover process is complete it will state that the “Profile completed successfully” in the detailed log and “Complete” in the profile execution history dialog. These dialog boxes can be closed if still open.
6. After a few minutes if you look at the console you will see that the physical blade/OS has moved the service processor that it sits under due to the fact that it has physically changed the blade that it is running on as shown in Figure 69.
Page 49 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 69 Blade failover complete
Failover Test with an Event Action Plan (EAP) In a more complex scenario we want to have an event or various events that trigger the blade to failover. We do this by associating an EAP with the service processor. The following steps show how to setup the blades with an event action plan that will trigger a failover when the power off command is received.
Page 50 of 58
1. Associate the blade failover EAP with the service processor of the blade that you wish to provide redundancy for. To do this, expand the Event Action Plans section in the Tasks pane of the Advanced BOFM console revealing the failover EAP that we created previously as shown in Figure 70. Drag the blade failover EAP to the service processor.
IBM BladeCenter Open Fabric Manager Implementation
Figure 70 EAP association
2. You will be prompted by the dialog shown in Figure 71. Click OK.
Figure 71 EAP added
3. In order to make it more clear as to what EAPs have been associated with what hardware, click Associations Event Action Plans. This will nest the action plans associated with hardware under the hardware it is associated with as shown in Figure 72.
Page 51 of 58
IBM BladeCenter Open Fabric Manager Implementation
Figure 72 EAP associations
4. Now we see that we have an EAP associated with a blade. In order to trigger the EAP we need to issue a power off command to the service processor. To do this, right click on the service processor and select Power Management Power Off Now as shown in Figure 73.
Figure 73 Power off blade
Page 52 of 58
5. Click Execute Now in the dialog box displayed.
IBM BladeCenter Open Fabric Manager Implementation
6. After a few minutes if you look at the console you will see that the physical blade/OS has moved the service processor that it sits under due to the fact that it has physically changed the blade that it is running on, as shown in Figure 74.
Figure 74 Failover complete
7. If you wish to fail the blade back to its original configuration you can simply drop the standby blade profile on the service processor of the running blade as shown in Figure 75.
Page 53 of 58
Figure 75 Fail back process
IBM BladeCenter Open Fabric Manager Implementation
8. After several minutes the blade will fail back to its original configuration. This concludes the example of setting up and exercising Advanced BOFM.
Page 54 of 58
IBM BladeCenter Open Fabric Manager Implementation
Appendix Table of Abbreviations
Page 55 of 58
AMM Advanced Management Module (BladeCenter) BIOS Basic Input-Output System BMC Baseboard Management Controller BOFM BladeCenter Open Fabric Manager CLI Command Line Interface EAP Event Action Plan (IBM Director) FTP File Transfer Protocol GUI Graphical User Interface HBA Host Bus Adapter HDD Hard Disk Drive IBM International Business Machines I/O Input/Output LAN Local Area Network LUN Logical Unit Number MAC Media Access Control OS Operating System POST Power-On Self-Test RAID Redundant Array of Independent Disks RDAC Redundant Disk Array Controller ROM Read Only Memory SAS Serial Attached SCSI SCSI Small Computer System Interface WWN World Wide Name WWPN World Wide Port Name
IBM BladeCenter Open Fabric Manager Implementation
References
1. IBM 4Gb SAN solution for IBM BladeCenter: http://www.redbooks.ibm.com/abstracts/sg247313.html?Open
2. Emulex HBA installation and management on BladeCenter white paper: http://www.emulex.com/white/hba/IBMBlade.pdf
Page 56 of 58
IBM BladeCenter Open Fabric Manager Implementation
Trademarks IBM, the IBM Logo, BladeCenter, ClusterProven, ServerProven terms are registered trademarks of International Business Machines Corporation in the United States, other countries, or both.
For a complete list of IBM Trademarks, see http://www.ibm.com/legal/copytrade.shtml.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Page 57 of 58
Other company, product, or service names may be trademarks or service marks of others.
IBM BladeCenter Open Fabric Manager Implementation
Notices (c) 2008 International Business Machines Corporation. All rights reserved.
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. For a copy of applicable product warranties, write to: Warranty Information, P.O. Box 12195, RTP, NC 27709, Attn: Dept. JDJA/B203. IBM makes no representation or warranty regarding third-party products or services including those designated as ServerProven® or ClusterProven®.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
MB, GB, and TB = 1,000,000, 1,000,000,000 and 1,000,000,000,000 bytes, respectively, when referring to storage capacity. Accessible capacity is less; up to 3GB is used in service partition. Actual storage capacity will vary based upon many factors and may be less than stated. Some numbers given for storage capacities give capacity in native mode followed by capacity using data compression technology. Maximum internal hard disk and memory capacities may require the replacement of any standard hard drives and/or memory and the population of all hard disk bays and memory slots with the largest currently supported drives available.
The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.
Page 58 of 58
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.