Best Practices for PureFlex systems implementation · PDF fileIBM Power Systems Technical...

Materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.3

© Copyright IBM Corporation 2013

2011IBM Power Systems Technical UniversityOctober 10-14 | Fontainebleau Miami Beach | Miami, FL

IBM

Best Practices for PureFlex systems implementation

pPS149

Arthur Torok, Luca Comparini

2© Copyright IBM Corporation 2013

IBM Power Systems Technical University Athens 2013

Agenda – Logical view

Project PlanningProject Planning

FSM Discovery and InventoryFSM Discovery and Inventory

Automation ManagerAutomation Manager

PureFlex upgradePureFlex upgradeInitializationInitialization

SMIA – Discover FabricsSMIA – Discover Fabrics

Using VMControlUsing VMControl

SAN IntegrationSAN IntegrationPureFlex PowerVM TuningPureFlex PowerVM Tuning FCoEFCoE

Network IntegrationNetwork Integration

Architecture / Project Management

BaU configuration – FSM not mandatory

FSM systems management

Private Cloud



Analytics on our past engagements: talk the walk

42 customers

77 chassis

43% has > 1 chassis

20% has IBMi

70% has AIX99% has x86 also

Most represented sectors:

23% Banking

23% IT Outsourcing (MSP)

17% GB




IBM

High Level Planningproject planning



Before we start – understanding Flex vs PureFlex



Project Planning

Project Planning

Ensure to do a project Kick-Off– Plan the right skill around the table– Don't assume that attendants already know; take the time to explain high level concepts– Define success criteria– Define time planning

When project is complex, better to take the time to produce a technical architecture document– To document expectations– To get stakeholders acceptance– Define responsibilities (RACI matrix)



Skill planning - Education




IBM

Network Planning



Network planning is all about choosing...



Understanding the chassis internal topology



Understanding I/O - Feature on Demand

Mezzanine Card

I/O Module 1 I/O Module 22 mezzanine per node16 lanes per node14 nodes224 total lanes



Understanding pnic vs vnic (x86)

Mezzanine Card

With 10Gb networking, each “lane” could be used as:- 1 physical nic- up to 4 virtual nics (with dynamic bandwidth allocation and QoS)

10Gb pnic

1Gbvnic

2.5Gbvnic

4Gbvnic

224 physical lanesUp to 896 virtual lanes

2.5Gbvnic



Looped and blocking design, no host NIC teaming

• Each I/O module in the Flex System Chassis is shown with two direct aggregations to a pair of upstream Top-of-Rack (ToR) switches

• This topology is suggested for environments in which hosts need network redundancy, but they are not themselves performing any NIC teaming.

• Offers complete network-level redundancy out of the chassis but if not balanced correctly, you lose half of the available links and bandwidth due to Spanning Tree Protocol (STP) blocking them



Non-looped, single upstream, host NIC teaming

Each I/O module in the Flex System Chassis connects directly to a single ToR switch through single or aggregated links.

Topology is highly suggested when compute nodes use some form of NIC teaming.

This architecture with NIC teaming on the host, and Layer 2 Failover on the I/O modules, provides high availability with no loopsNo wasted bandwidth to spanning-tree blocked links.



Trunk failover in action

Layer 2 failover is enabled on each Flex I/O Module



Fully redundant network configuration

Features• Uses aggregated links or port-channels.• Provides loop-free network design• Recommended for environments that wish

to leverage a best-practice implementation utilizing next generation networking features such as VSS (Virtual Switching System) or vPC (Virtual Port Channel) technology.

Advantages of this approach are:• Active/Active uplinks helps to avoid the

wasted bandwidth associated with links blocked by spanning tree

• Maximum redundancy and fault tolerance

Disadvantages of this approach are:• Requires upstream equipment that

supports virtualization features• More cabling and connections are

necessary• Careful implementation and planning to

ensure correct operation by a network architect / administrator



Double wide Power node



Double wide Power – slot assignment example

http://www.redbooks.ibm.com/technotes/tips0880.pdf

VIOS A: Slot 3: C36Port 3: L2-T1Port 4: L2-T2

VIOS A: Slot 1: C34Port 1: L1-T1Port 2: L1-T2

VIOS B: Slot 1: C34Port 3: L2-T1Port 4: L2-T2

VIOS B: Slot 3: C36Port 1: L1-T1Port 2: L1-T2

http://www.redbooks.ibm.com/technotes/tips0880.pdf



Double wide Power – Without ScSE Upgrade 1

SWITCH 1

P7-460 SLOT 5-6VIOS1

VIOS2

P1-C34-L1-T1-T2

P1-C36-L1-T1-T2

en0 S1_INT5A

en1 S2_INT5A

en2 S1_INT6A

en3 S2_INT6A

SWITCH 2

S1_INT5A

S1_INT6A

S2_INT5A

S2_INT6A

S1_EXT1

S1_EXT2

S1_EXT3

S1_EXT4

S2_EXT1

S2_EXT2

S2_EXT3

S2_EXT4



Double wide Power – With ScSE Upgrade 1

SWITCH 1


VIOS2

P1-C34-L1-T1-T2

P1-C36-L2-T1-T2

P1-C34-L2-T1-T2

P1-C36-L1-T1-T2

en0 S1_INT5A

en1 S2_INT5A

en2 S1_INT6B

en3 S2_INT6B

en0 S1_INT5B

en1 S2_INT5B

en2 S1_INT6A

en3 S2_INT6A

SWITCH 2

S1_INT5A

S1_INT5B

S1_INT6A

S1_INT6B

S2_INT5A

S2_INT5B

S2_INT6A

S2_INT6B

S1_EXT1

S1_EXT2

S1_EXT3

S1_EXT4

S2_EXT1

S2_EXT2

S2_EXT3

S2_EXT4



Double wide Power – Without ScSE Upgrade 1

SWITCH 1


P1-C34-L1-T1-T2

en0 S1_INT5A

en1 S2_INT5A

SWITCH 2

S1_INT5A

S2_INT5A

S1_EXT1

S1_EXT2

S1_EXT3

S1_EXT4

S2_EXT1

S2_EXT2

S2_EXT3

S2_EXT4 en4

SEA

PVID=1

Switch 1 Switch 2

Backbone Backbone

en4 =lnagg en0 + en1

MGT DEV



Double wide Power – With ScSE Upgrade 1

SWITCH 1


P1-C34-L1-T1-T2

P1-C36-L2-T1-T2

en0 S1_INT5A

en1 S2_INT5A

en2 S1_INT6B

en3 S2_INT6B

SWITCH 2

S1_INT5A

S1_INT6B

S2_INT5A

S2_INT6B

S1_EXT1

S1_EXT2

S1_EXT3

S1_EXT4

S2_EXT1

S2_EXT2

S2_EXT3

S2_EXT4

en4 en5

SEA SEA

PVID=1 PVID=2

SWITCH 1 SWITCH 2

Backbone Backbone

en4 =lnagg en0 + en3en5 =lnagg en1 + en2

MGT DEV



Network planning questions

Who is the current installed switch vendor?

What upstream networking devices will Flex switches be connected to?

Do you have any previous experience with IBM/BNT networking equipment?

What switch CLI does the customer prefer to use? IBM/BNT or Cisco?

How many uplinks and what cables are to be used (RJ45, SFP+, QSFP, DAC)?

Network Switch uplinks (x2 recommended per switch – minimum)

Use of shared uplinks reduces the number of uplink cables

Will ScSEs be used as Switches or Pass-thru ScSEs?

Number of physical management network connections

CMM, FSM, External Management Port on ScSE, Storwize V7000

Number of IPs required



Network planning questions

What are the performance characteristics of the Data network in use?

VLANs, load balancing, availability

Will vNICs be implemented? Virtual Fabric, Switch independent & Universal Fabric port modes

Each vNIC pair will need to be on it’s own separate VLAN

Assign VLAN designation at the OS level (Windows & VMWare)

Is FCoE / iSCSI being used?

Feature must be activated on the 2 port LOM or 4 port CN4054 mezzanine adapter

FoD key must be applied in the IMM, OneCommand Manager or FSM

Are there enough Switch Partitions? Switch partition upgrades determine internal and external usable switch ports.

NIC Teaming for high availability and/or bandwidth aggregation.




IBM

SAN Planning



Fiber Channel options for connectivity

Pass-Thru NPIV Mode (FC5022 or FC3171)Use when you already have an existing FC fabric in your environment and don’t want additional switches added to that fabricDownfall – You cannot directly connect devices to the chassis like a tape library

Full-Fabric Mode (FC5022 or FC3171)Full Fiber Channel switch that you have to zone and add to you fabric.You are able to directly connect devices like a tape libraryCan add complexity as another switch is added to the fabric

Breakdown of FC Switch VendorsFC5022 – Brocade (16GB)FC3171 – Qlogic (8GB)



How to set NPIV mode (Qlogic)



Fiber Channel port types & NPIV

N_Port ID Virtualization (NPIV) is a Fibre Channel protocol that facilitates sharing a single physical N_Port among multiple N_Port IDs.

Virtualization refers to the ability of a single physical N_Port, thereby allowing multiple, distinguishable entities on the same physical ports. In other words, it makes a single Fibre Channel port appear as multiple virtual ports, each having its own N-Port ID and virtual worldwide name (WWN).

The NPIV protocol requires an N_Port (typically a host bus adapter any device that acts as an NPIV gateway) and a fabric (generally an FC switch) so that the N_Port can request and acquired multiple addresses from the fabric.

N_Port, node port.

A host, HBA, or storage device port connected to the F_Port of the switch.

F_Port, fabric port.

A switch port that connect a host, HBA, or storage device to the SAN.

E_Port, known as Inter Switch Link (ISL).

A switch port that connect the switch to another switch directly.



Dual VIOS – vtSCSI without TOR

fcs0 fcs1

vhost0

SWITCH 3 SWITCH 4

hdisk0

mpio

CLIENT LPAR

VIOS A

V7000node2

V7000node1

vscsi0

hdisk0

fcs0 fcs1

vhost0VIOS B

vscsi0

hdisk0

mpio/sddpcm



Dual VIOS – NPIV without TOR

fcs0 fcs1

vfchost0 vfchost1

fcs0 fcs1

SWITCH 3 SWITCH 4

hdisk0

mpio/sddpcm

CLIENT LPAR

VIOS A

V7000node2

V7000node1

fcs0 fcs1

vfchost0 vfchost1VIOS B

fcs2 fcs3



Dual VIOS – NPIV with TOR (ex: with Passthrough)

fcs0 fcs1

vfchost0 vfchost1

fcs0 fcs1

SWITCH / PASST 3

SWITCH / PASST 4

TOR TOR

hdisk0

mpio/sddpcm

CLIENT LPAR

VIOS A

V7000node2

V7000node1

fcs0 fcs1

vfchost0 vfchost1VIOS B

fcs2 fcs3



Zoning considerations

vtSCSI� Zone VIOS with Storage

NPIV� Zone LPAR virtual WWPN with Storage� If you need LPM, ensure to zone both WWPNlssyscfg -r prof -m Server-7895-42X-xxxxxx -F virtual_fc_adapters --filter lpar_names=AIX

"""10/client/1/VIOS1/10/c050760586f60000,c050760586f60001/1"",""11/client/2/VIOS2/11/c050760586f60002,c050760586f60003/1"",""12/client/1/VIOS1/12/c050760586f60004,c050760586f60005/1"",""13/client/2/VIOS2/13/c050760586f60006,c050760586f60007/1"""

Number of paths� Keep a reasonable number of paths (4 or 8 ok; 16 max)




IBM

What to do to start



Configure the chassis components



Login into CMM to start initialization wizard

CMM IP address192.168.70.100 if Flex

192.168.93.100 if PureFlex



Set Component IP address



Systems Management

Network

SAN

Power nodes

Intel nodes

KVM

PowerVM(VIOS)

Storage

Chassis (CMM)

VMWare VCenter Win / Hyper-V

Linux

AIXIBMi

FSM



Understanding FSM networks

KVM host

FSM code

bond bond

eth0 eth1

ActiveCMM

Stand-byCMM

nodesnodesnodesIMM / FSP

nodesnodesI/O Modules INT-MGT

I/O Module 1

I/O Module 2

Hardware Management Network (GOLD)

Customer Data Network

(PURPLE / GREEN)



Initialize FSM – management network



Initialize FSM – management network

Eth0 - Private Management Network (In-band) Private and secure Gigabit Ethernet network. Used to complete management-related functions throughout the chassis including:

• Compute nodes• Switches• Chassis• Flex V7000 & Storwize V7000 Storage

Connects the Chassis Management Module (CMM), the IMM/FSP of the compute nodes, switches in the I/O bays, the Flex V7000 storage and the Flex System Manager (FSM)

Uses a special Broadcom 5718-based management network adapter

Eth1 - Data Network (Out of band) FSM bonded pair of NIC ports connecting to ports 1 of both switch 1 and switch 2 Connects to the Internet for product updates and other related management functions

Note: The Eth0 or CMM management networks in multiple chassis are connected together through the external ports of the CMMs in each chassis by using a GbE top-of-rack switch.



Select Chassis to be Managed




IBM

What to do with the FSM(discover, access, inventory,

manageV7000)



FSM Best Practices – step by step

After the Chassis has been managed it usually takes around 15 minutes for the inventory to be collected,meaning the FSM's database will be updated with all the information regarding the hardware elements present in the managed chassis. It's a good idea to wait for this process to finish, if you have issues with patience (like I do) go to the Plugins tab and click the active&scheduled jobs link, you should see the jobs running.




Usually access to the components should show OK, but just to make sure, have a look at the Plugins tab, you should see immediately under the Inventory link if there are any systems with No Access or Partial Access, in case there are make sure you request access to avoid any future inventory collection or update issues.




If using V7000 storage you should “discover&manage” it from the FSM. If you have an internal Flex System V7000 system you can use the Launch IBM Flex System V7000 EZSetup task and follow the wizard, alternatively you can use the quick link, on the Additional Setup tab, Manage System Storage. This will give you several possibilities to manage and monitor your storage subsystem, get an overview of the disk usage, pools, hardware alerts, create and delete volumes.




First you might have to generate a public and private pair of rsa keys ssh-keygen -t rsa -f $HOME/v7000_rsa, then using browse select the private key. After the task is complete inventory will automatically be collected.



As inventory is a central component of great importance from the FSM's usability perspective I recommend to do a complete collection after all the hardware has been discovered/managed, my preference is the command line from a terminal/putty ssh session: smcli collectinv -p “All Inventory”. Inventory collection is task that highly utilizes the database so I wouldn't recommend doing too many things during this process.





IBM

Upgrade best practices



Updating Best Practices – order of operations

There are a number of methods to update the component firmware of your Flex System. Below is the recommended order of operations for performing these depending on the hardware you have.

For the latest information on firmware updating best practices, please see the Firmware Update Guide at the link below.

https://www-947.ibm.com/support/entry/myportal/docdisplay?lndocid=MIGR-5091991

Chassis without a FSM :

CMM

X-Architecture compute nodes

Flex V7000

I/O Modules

Chassis with a FSM :

Flex System Manager Node

FSP on Power System Nodes

CMM

Adapters / Hard Drives on Power System Nodes

X-Architecture compute nodes

Flex V7000

I/O Modules

https://www-947.ibm.com/support/entry/myportal/docdisplay?lndocid=MIGR-5091991



Updating Best Practices – order of operations

Obtaining FirmwareIn addition to updating the firmware on the nodes, you should also update the CMM and all the switch modules you have in the chassis. To make it easy, IBM has a single page where you can easily access the latest firmware for all of the components.

http://www-01.ibm.com/software/brandcatalog/puresystems/centre/details?uid=S_PUREFLEX

Log in locally to each component and update the corresponding firmware.



Updating CMM

To update the Chassis Management Module (CMM), log in to the CMM, click on Mgt Module Management, and select Firmware.

If you have two CMM’s, you only need to update the firmware on your primary CMM. Once the update process is complete, and the CMM has rebooted, it will automatically update the secondary CMM.

When you download the firmware package, you will need to extract the zip file. The update process will require the file with the .uxp extension within that package.



Updating x86 nodes

Update Servers via Bootable Media CreatorDownload IBM Bootable Media Creator from the link below :

http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=TOOL-BOMC iSCSI

Once the download is complete, right click on the file and choose run as administrator.

Redhat and SLES version of the tool are also available and will run in a similar format.



Updating FC3171 Switch

To update the Qlogic FC3171 fiber channel switch, log in to (default user/pass is USERID/PASSW0RD) and click on Switch > Load Firmware. Select the folder where the .bin file firmware is located. You do not need to select the exact file. The switch will automatically access the firmware file in that directory.



Updating EN4093 Switch

The networking switch has two components that need to be updated. The Boot code and the Image code. Both the boot and image code are included in the single download package. Typically we recommend updating the boot code first and then one of the image banks. It is recommend that you keep one level back in one of the image banks.

Updating the firmware is as easy as clicking the browse button and then selecting the firmware package.



Updating via FSM

If you have the Flex System Manager, updating chassis components is a easy process by selecting the “Update Chassis Components” option and following the order provided. We always recommend checking the Firmware Update Guide for the latest information and any changes to the order listed in the Flex System Manager.




IBM

IBM SMIA step by step




To fully utilize the advanced features and capabilities of the FSM, like automatic workload deployments, you should be able to read and write zoning information on your SAN infrastructure.

Since most storage vendors developed their own “language” a need for commonality arose. The Storage Networking Industry Association developed a standard, Storage Management Initiative – Specification, shortly SMI-S, to enable interoperable management of storage systems.

If you have a Qlogic switch in full fabric mode all you need to do is discover the switch and configure SNMP access. Qlogic switches have an SMI-S provider built in their firmware that can be used to discover the zoning information. However due to current limitations this access is read only, meaning you will not be able to deploy NPIV based workloads as the FSM can not create new zones.

Brocade switches do not have this built in component thus an external tool, SMI-S provider is required to be able to manage a Brocade based SAN infrastructure from the FSM. IBM decided to preinstall this java based tool in the FSM, you have to to start and configure it as shown in the next slide:




From the applications tab click Start under SMIA Configuration Tool:




Tip: Make sure you have a dns entry or you have the name in your local host file, credentials are the same as for the FSM.




Click Options and configure https authentication:




After changing to https Stop and Start form the Applications tab:




Do a Fabric discovery:




Click add, put the IP address of the Brocade switch and USERID/PASSW0RD by default:




After this you have to use the Discover Storage System task as with the V7000 but this time select FC switch and provide the details:




The SMI-S IP will be the FSM itself:




After successfully configuring the IBM Fabric Manager, go to the Plugins tab, click on Resource Explorer and click on the “All Systems” dynamic group. Type Farm in the search box, this is a container object that represents all storage components known to the FSM, like the V7000 and SAN switches. Right click this object and select Collect Inventory. This should update the FSM's database with all the zoning and SNMP information from the fabric switches, and also with the information from the V7000. Since the fabric switches are now known to the FSM any alerts generated by these will be received and displayed in the FSM.




IBM

Tips for PowerVM(tuning)



Tips for PowerVM

If you have Power nodes in your Chassis I would assume you will also have Virtual I/O Servers, unless you want to use the Power nodes in full system partition mode, in which case more specific tuning will have to be considered.

For quite some time now VIOS and AIX have the so called Common Agent installed by default. Though it's not mandatory for basic management, I highly recommend making use of it's features, especially for more advanced functions, like automating deployments of Virtual Servers. To do this, after you have successfully installed and configured your VIOS, run a discovery on the IP address of the VIOS, authenticate (request access), then collect inventory. This will give the FSM access and visibility to the Operating System component of the Power node, specifically of the VIOS. The procedure is the same with AIX systems.



Tips for PowerVM

Tips for creating the Virtual Server profile for the VIOS:

- use 1 or 2 dedicated processors in donating mode, this way you ensure the best locality and availability of CPU and memory resources and also that the idle cycles can be used by the other partitions. If you plan to have high network traffic without jumbo frames and vSCSI storage you should have at least 2 CPU's, if you use jumbo frames and NPIV 1 CPU should be enough. Memory should be between 6-8Gb to start and 12Gb maximum in case you have to increase it later on the fly. Physical I/O can be split between 2 VIO servers depending on the physical hardware capabilities, in a p460 for example you would typically have 4 mezzanine adapters, thus you can split 2-2 for creating a dual VIOS configuration. The network adapters have 2 asics, thus even one adapter can be split in two, the 2 port fibre channel adapters have one while the 4 port adapters have 2 asiscs. In the new p270 node there is also a special adapter for splitting the on board SAS controller into 2 in which case you could potentially install 2 separate VIO servers on local disks. This is not possible with any other nodes as they only have 1 SAS controller.



Tips for PowerVM

Networking tips:

For high availability and redundancy reasons I suggest to consider creating etherchannel network adapters on the VIOS. As LACP is natively supported this is what I prefer to use. It does imply you will need VLAG and will have to create the LACP port groups on the chassis network switches. I usually put INTAx on IOM1 with INTAx on IOM2 in one LACP group and INTBx with INTBx in a different LACP group. If it's a full wide node with Dual VIOS I would use INTAy with INTAy and INTBy with INTBy on the second VIOS. This configuration will give you 2x20Gbs adapters per VIOS. Then you have to create the etherchannel adapter making sure you select type 802.3ad, and the en adapters corresponding to the ports you configured previously. In most cases they would be en0 with en1 and en2 with en3 but it's best to double check the location codes first.

Jumbo frames are very useful in reducing processing overhead and increasing throughput though it should be used extensively throughout the infrastructure. Good news is the PureSystems network switches all support this by default. Fine tuning for jumbo frames is quite an extensive topic that usually has it's own session, please check the agenda for more information.



Tips for PowerVM

Next step is to create the SEA adapters. If you haven't done so while creating the virtual server definition for the VIOS you should now create the required virtual network adapters, depending on how many etherchannel adapters you had created in the previous step.I suggest to have one virtual adapter for the management IP of the VIOS, then 1 or 2 for each Shared Ethernet Adapter. If you are using Dual VIOS you will need at least 2 adapters/SEA, one will be the “trunk” adapter, this one needs to have access to the external network, the other one will be the control channel (heartbeat) adapter used for communication between the 2 VIO servers. The trunk adapter can be IEEE 802.1Q compatible and it can manage up to 20 VLANs. If you need to serve more VLAN's you will have to create additional adapters for each bundle of 20.

This is an example command for creating the SEA:

mkvdev -sea ent9 -vadapter ent4 -default ent4 -defaultid 1 -attr ha_mode=auto ctl_chan=ent5 largesend=1 large_receive=yes

Ent9 is the physical adapter, in our case the new etherchannel adapter, ent4 is the virtual trunk adapter, the one that has the option to access external networks and ent5 is the control channel adapter. Note that both ctl_chan virtual adapters, on VIOS1 and VIOS2 should have the same unique PVID. If you plan to use jumbo frames, which I highly recommend also add this parameter to the command: jumbo_frames=yes



Tips for PowerVM

Useful parameters:

For fibre channel adapters you should use dynamic tracking of devices and fast fail error recovery, this will ensure that any missing paths will be noticed immediately and the multipath I/O driver will initiate a failover and prevent loss of data. It is also recommended to install the sdd-pcm driver that builds on the default mpio and adds extra functionality and usability.

chdev -dev fscsi0 -attr dyntrk=yes fc_err_recov=fast_fail -perm

Since the adapters are usually active these changes cannot be done on-line thus the “-P” parameter which also implies you will have to reboot the VIOS to activate the changes.

For specific applications you might have to change additional parameters.

For vScsi disks in a dual VIOS setup you have to make sure to change the reservation policy before assigning the disks to a partition:

chdev -dev hdisk# -attr reserve_policy=no_reserve




IBM

Tips for VMControl



Tips for VMControl

VMControl is the first step into a Power systems based Cloud environment on the FlexSystem platform, it provides the functionality to capture existing workloads and automatically deploy new systems. Deployment includes the creation of the Virtual Server, creation of required virtual network adapters, creation of required disks and their corresponding virtual adapters, all the mappings plus the OS installation.

Given everything has been set-up correctly this process will complete under 5 minutes after a few easy steps where you are required to provide the name, future IP address, hostname, dns servers, etc.

The reason for being so quick is because it's based on flashcopy technology, either when using V7000 for direct LUN allocation or VIOS Shared Storage Pools based virtual disks.



Tips for VMControl

For this to work there a few key factors that have to be considered:

- the VIOS Common Agent has to be discovered and access has to be OK.

- the correct repository subagent has to be installed on the VIOS, and if using Shared Storage Pools there is an additional subagent that you will have to install. This will ensure that you can create an image repository to store all the metadata required to successfully catalog and manage OS images that you will use in the future to deploy new workloads.

- the SAN switches and the V7000 have to be discovered and successfully accessed

- the system that you will capture has to have the Common Agent discovered and has to have the “activation engine” code installed and the activation engine has to be configured. The activation engine code resides on the FSM and can be copied using scp after which it has ot be installed on the host using specific java code:

opt/ibm/director/proddata/activation-engine/vmc.vsae.tar [email protected]:/tmp/

export JAVA_HOME=/usr/java5/jre

untar and execute the AE install script

After installing the activation engine and before trying to capture make sure you run an inventory collection on the FSM, Farm, VIOS, host system.



Tips for VMControl

After you customize the OS that you will capture you have to run the AE.sh script:

/opt/ibm/ae/AE.sh --reset

This will prepare the OS for capture and will also shut down the virtual server. After the system is shut down you can initiate the capture by right clicking the Virtual Server element in resource explorer or by following the VMControl links.

In case you want to make some changes and capture the server again you have to reset the activation engine by issuing these commands:

rm /opt/ibm/ae/AP/*

cp /opt/ibm/ae/AS/vmc-network-restore/resetenv /opt/ibm/ae/AP/ovf-env.xml

If you have done everything correctly, which I know you did, you will now have your customized image ready to deploy. Please note that the CPU, Memory, Network, Storage information is all captured form the original image, all new deployments will use this information as default, which of course you can change later.




IBM

Tips for automating actions(EAP, ESA, alerting...)



Tips for automation

The FSM comes with a very useful feature called Electronic Service Agent, also known as “call home”. I highly recommend enabling this, it will automatically create a service call with IBM Support in case of any hardware failures, it will also collect and upload the logs.You do need to have an active support agreement contract and you have to provide the customer number. It's also a good idea to check on the IBM Support website that the correct product is registered.



Tips for automation

Part of the advanced RAS features of the FSM include predictive hardware alerting which can be enabled by creating an Event Automation Plan that will filter out from the event log the specific alerts and will log them in the error log. This plan can also include an action to send and email or integrate into IBM Tivoli Monitoring, for example.



Tips for automation

Go to the Plugins tab, Automation Manager, Event Automation Plans, create new:



Tips for automation

Select All Systems as target, this will ensure that all predictive hardware alerts are captured from all systems:



Tips for automation

Change to Advanced Event Filters and select Hardware Predictive Failure Alerts:



Tips for automation

Next step is to select what action to do in case an alert is raised. If you don't see an appropriate action in the list click create:



Tips for automation

Select from the list send email, this way the recipient will be informed:



Tips for automation

Note you can only use one address for action, and you can customize what to include in the message using variables:



Tips for automation

Next step is to finish the creation and activate the plan. Since this is related to predictive hardware alerts it will be active 24/7, with other plans you have the flexibility to chose the timeframe. For example if your service agreement is only from 9 to 5 you on weekdays you can set the plan to be active only during this timeframe.

Another useful feature is monitoring, which can include filesystem usage, CPU, memory, process monitors etc.

For each monitor you can define a threshold with a severity and you can create automation plans for these as well.




IBM

FCoE



FCoE

Support for FCoE in Power nodes comes with the new 8 port, dual asic Converged Ethernet Adapter and the CN4093 10Gb converged switch. Due to current limitations of the available CN4093 switch only 6 of these can actually be used and if you plan to split the adapter for 2 VIO servers you will only be able to use 2 ports for each.

When used in converged mode each physical port is split into 2 providing Fibre Channel and Ethernet traffic.

The CN4093 converged switch can act as the primary SAN director when running in full fabric mode or as an NPV gateway to provide pass-through traffic to a top of rack SAN director. It does use a Qlogic fabric thus full fabric integration with Brocade is not possible, in this case I recommend using NPV. It has 12 omniports that can be configured in pairs only, and you can attach a SAN SFP+ for direct connection of storage systems or SAN directors.



FCoE

Useful commands to configure Converged Enhanced Ethernet on the CN4093:

#cee enable: this will enable FCoE forwarding

#fcoe fips enable: this enables snooping for FCoE frames

To change an omniport for fibre channel (note it has to be done in pairs)

#system port EXT6-EXT7 type fc

All internal ports where you want to use FCoE must have tagging enabled:

#interface port INTA2-INTA14

#tagging



FCoE

You have to define a new vlan for FCoE traffic only, the accepted industry default is 1002, and configure it accordingly:

#vlan 1002

#member INTA2-INTA14,EXT6-EXT7

You have to include all host ports and the omniports configured as FC in this vlan. Then if you plan to use the switch in pass through (NPV) mode:

#npv enable

#npv traffic-map external-interface EXT6-EXT7 (providing the FC ports that will be connected to your SAN Director switch)

Note that it is not supported to transport vlan 1002 through an ISL link.



FCoE

For full fabric management, the FCF mode has to be used:

#vlan 1002

#fcf enable

Then you have to create the aliases, zones and zoneset:

#fcalias V7K_N1_P1 wwn 50:05:07:68:02:35:da:f5

#fcalias Node5_2_P1 wwn 10:00:34:40:b5:ef:d0:19

#zone name Node5_2_v7k_F1

#member fcalias V7K_N1_P1

#member fcalias node5_2_P1

#zoneset name Fabric1

#member Node5_2_v7k_F1

#zoneset activate name Fabric1



FCoE

After doing the configuration I recommend to save it:

#write

Then do a reload.

To check the status:

#show fcoe database

#show fcoe fips fcoe

#show npv flogi-table



FcoE support using FC targets

Flex System Interoperability Guide: http://www.redbooks.ibm.com/abstracts/redpfsig.html

http://www.redbooks.ibm.com/abstracts/redpfsig.html

http://www.redbooks.ibm.com/abstracts/redpfsig.html



FCoE support using FCoE targets (end to end)

Hardware based iSCSI support



Win prizes by submitting

evaluations online. The more evalutions

submitted, the greater chance of

winning

ibmtechu.com/gr



Thank you for attending this session

Don’t miss these valuable opportunities:

Keep the momentum going

ibm.com/trainingNow is the time to explore your options for additional training. One stop shopping for all your technical training needs.

ibm.com/training/trainingpathsIBM Training Paths – These flowcharts map out the sequence of classes you need, to obtain a specific skill or professional certification. Get started today!

ibm.com/certifyStand out from the crowd when you earn a valuable credential of an IBM Certified Professional. Find certifications by product or solution.

We value your feedback

1. Submit Session Evaluations on ibmtechu.com and receive an entry to WIN prizes. The more evaluations submitted, the greater your chance of winning!

1. Submit your Overall Session Evaluation and receive a free gift at the registration desk.

Best Practices for PureFlex systems implementation · PDF fileIBM Power Systems Technical...

Documents

Transcript of Best Practices for PureFlex systems implementation · PDF fileIBM Power Systems Technical...