010 VIO Overview

7/22/2019 010 VIO Overview

1/40

Copyright IBM Corporation 2006 Course materials may not be reproduced in whole or in part without written permission.3.3

Virtual I/O Server Overview

QPS01 IBM System p APV - 010


2/40


3/40


4/40

Copyright IBM Corporation 2006

IBM System p

QPS01 010 VIO Overview 4

The IBM benchmarks results shown herein were derived using particular, well configured, development-level and generally-available computer systems. Buyersshould consult other sources of information to evaluate the performance of systems they are considering buying and should consider conducting applicationoriented testing. For additional information about the benchmarks, values and systems tested, contact your local IBM office or IBM authorized reseller or accessthe website of the benchmark consortium or benchmark vendor.

IBM benchmark results can be found in the IBM eServer p5, pSeries and IBM RS/6000 Performance Report athttp://www-1.ibm.com/servers/eserver/pseries/hardware/system_perf.html

Unless otherwise indicated for a system, the performance benchmarks were conducted using AIX V4.3 or AIX 5L. IBM C Set++ for AIX and IBM XLFORTRAN for AIX with optimization were the compilers used in the benchmark tests. The preprocessors used in some benchmark tests include KAP 3.2 forFORTRAN and KAP/C 1.4.2 from Kuck & Associates and VAST-2 v4.01X8 from Pacific-Sierra Research. The preprocessors were purchased separately fromthese vendors. Other software packages like IBM ESSL for AIX and MASS for AIX were also used in some benchmarks.

For a definition and explanation of each benchmark and the full list of detailed results, visit the web site of the benchmark consortium or benchmark vendor.

TPC http://www.tpc.org

SPEC http://www.spec.org

Linpack http://www.netlib.org/benchmark/performance.pdf

Pro/E http://www.proe.comGPC http://www.spec.org/gpc

NotesBench http://www.notesbench.org

VolanoMark http://www.volano.com

STREAM http://www.cs.virginia.edu/stream/

SAP http://www.sap.com/benchmark/

Oracle Applications http://www.oracle.com/apps_benchmark/

PeopleSoft - To get information on PeopleSoft benchmarks, contact PeopleSoft directly

Siebel http://www.siebel.com/crm/performance_benchmark/index.shtm

Baan http://www.ssaglobal.com

Microsoft Exchange http://www.microsoft.com/exchange/evaluation/performance/default.asp

Veritest http://www.veritest.com/clients/reports

Fluent http://www.fluent.com/software/fluent/fl5bench/fullres.htm

TOP500 Supercomputers http://www.top500.org/

Ideas International http://www.microsoft.com/exchange/evaluation/performance/default.asp

Storage Performance Council http://www.storageperformance.org/results

Notes on Benchmarks and Values

Revised August 26, 2003


5/40


IBM System p


rPerf

rPerf (Relative Performance) is an estimate of commercial processing performance relative to other pSeries systems. It is derived from an IBM

analytical model which uses characteristics from IBM internal workloads, TPC and SPEC benchmarks. The rPerf model is not intended to represent any

specific public benchmark results and should not be reasonably used in that way. The model simulates some of the system operations such as CPU,

cache and memory. However, the model does not simulate disk or network I/O operations.

rPerf estimates are calculated based on systems with the latest levels of AIX 5L and other pertinent software at the time of system announcement.Actual performance will vary based on application and configuration specifics. The IBM pSeries 640 is the baseline reference system and has

a value of 1.0. Although rPerf may be used to approximate relative IBM UNIX commercial processing performance, actual system performance may

vary and is dependent upon many factors including system hardware configuration and software design and configuration.

All performance estimates are provided "AS IS" and no warranties or guarantees are expressed or implied by IBM. Buyers should consult other sources

of information, including system benchmarks, and application sizing guides to evaluate the performance of a system they are considering buying. For

additional information about rPerf, contact your local IBM office or IBM authorized reseller.

Notes on Performance Estimates

Revised June 28, 2004


6/40


IBM System p


What is Advanced POWER Virtualization (APV)

APV the hardware feature code for POWER5 servers that enables:

Micro-partitioning fractional CPU entitlements from a shared pool of

processors, beginning at one-tenth of a CPU

Partition Load Manager (PLM) a policy-based, dynamic CPU and memory

reallocation tool

VIO Server (virtual SCSI and Shared Ethernet Adapter)

Physical disks can be shared as virtual disks to client partitions

Shared Ethernet Adapter (SEA) A physical adapter or EtherChannel in

a VIO Server can be shared by client partitions. Clients use virtual

Ethernet adapters

Virtual Ethernet a LPAR-to-LPAR Virtual LAN within a POWER5 Server

Does not require the APV feature code


7/40


IBM System p


Why Virtual I/O Server?

POWER5 systems will support more partitions than physical I/O slots available

Each partition still requires a boot disk and network connection, but now

they can be virtual instead of physical

VIO Server allows partitions to share disk and network adapter resources

The Fibre Channel or SCSI controllers in the VIO Server can be accessed

using Virtual SCSI controllers in the clients

A Shared Ethernet Adapter in the VIO Server can be a layer 2 bridge for

virtual Ethernet adapters in the clients

The VIO Server further enables on demand computing and server consolidation


8/40


IBM System p


Why Virtual I/O Server?Virtualizing IO saves:

Gbit Ethernet Adapters

2 Gbit Fibre Channel Adapters

PCI slots

Eventually, IO drawers

Server frames?

Floor space?

Electric, HVAC?

Ethernet switch ports

Fibre channel switch ports

Logistics, scheduling, delays of physical Ethernet, SAN attach

Some servers run 90% utilization all the time everyone knows which ones.

However,

Average utilization in the UNIX server farm is closer to 25%. They dont all

maximize their use of dedicated IO devices

VIO is departure from new project, new chassis mindset


9/40


IBM System p


Virtual I/O Server Characteristics

Requires AIX 5.3 and POWER5 hardware with APV feature

Installed as a special purpose, AIX-based logical partition

Uses a subset of the AIX Logical Volume Manager and attaches to

traditional storage subsystems

Inter-partition communication (client-server model) provided via the POWER

Hypervisor

Clients see virtual disks as traditional AIX SCSI hdisks, although they may be

a physical disk or logical volume on the VIO Server

One physical disk on a VIO server can provide logical volumes for several

client partitions


10/40


IBM System p


Creating the Virtual IO Server


11/40


IBM System p


Virtual I/O Server installation

VIO Server code is packaged and shipped as an AIX mksysb image on a VIO CD

Installation methods

CD install

HMC install - Open rshterm and type installios; follow the prompts

The Network Installation Manager (NIM)

VIO Server can support multiple client types

AIX 5.3

SUSE Linux Enterprise Server 9 for POWER

Red Hat Enterprise Linux AS for POWER Version 3 and 4


12/40


IBM System p


Virtual I/O Server Administration

The VIO server uses a command line interface running in a restricted shell no

smitty or GUI

There is no root login on the VIO Server

A special user padmin executes VIO server commands

First login after install, user padmin is prompted to change password

After that, padmin runs the command license accept

Slightly modified commands are used for managing devices, networks, code

installation and maintenance, etc.

The padmin user can start a root AIX shell for setting up third-party devices

using the command oem_setup_env


13/40


IBM System p


VIO Server/Client Overview


14/40


IBM System p


VIO Server Configuration with MPIO

MPIO in client LPAR

automatically configures

Client sees one hdisk with two

MPIO paths lspath l hdisk0

Paths are fail_over only. No

load balancing in client MPIO

hdisk1 in each VIO server

attached to vscsi server

adapter as a raw disk

Set reserve_policy attribute

on hdisk1 to no_reserve ineach VIO server. Also, with two fcs in

a VIO server, change algorithm to

round_robin for hdisk1. SDDPCM

driver installed in each VIO server

LUN appears in each VIOserver as hdisk1

Single RAID5 LUN carved in ESS,

zoned to 4 FC adapters in the two VIO

servers

SDDPCM SDDPCM


15/40


IBM System p


Checking for MPIO Capability

Run oem_setup_env to get to a root prompt

Run # odmget qattribute=unique_id CuAt

In the output, look for a field unique_id followed by a value field

lsattr rl hdisk0 a algorithm (both round_robin or fail_over)

With VIOS 1.3, MPIO is available with:

- IBM Enterprise Storage Server and DS6000 with SDDPCM (but notSDD)

- FAStT and DS4000 with RDAC- IBM System Storage N3700 and N5200

- EMC PowerPath

- HDS with HDLM


16/40


IBM System p


VIO Server Configuration with LVM Mirroring


17/40


IBM System p


Internals AIX

Virtual I/O Server AIX client

LVM

OEM

multipathing

disk DD

HBA DD

VSCSI

client

disk DD

LVM

MPIOPCM

MPIO

PCM

VSCSI

serverinternal orexternal storage

logical volumes

OEM device paths

physical volumes


18/40


IBM System p


Virtual I/O Server Resource Configuration

Add physical devices to the VIO Server

If serving LVs as hdisks, create a volume group on one or more disks with

mkvg

- mkvg [-f] [-vg VolumeGroup] PhysicalVolume

- mkvg f vg rootvg_clients hdisk2

rootvg_clients

Create logical volumes on the volume group

- mklv [-lv NewLogicalVolume | -prefix Prefix ]

VolumeGroup Size [PhysicalVolume ]- mklv lv aix_sq07 rootvg_clients 7G hdisk2


19/40

19 IBM Systems GroupPOWER5 VIO Server 2006 IBM Corporation

DLPAR add of VIO Server Virtual Adapter

List virtual adapter after added by DLPAR

lsdev virtualname status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter

Doesnt show up in VIO Server until after running cfgdev

cfgdev dev vio0

List virtual adapter after running cfgdev

lsdev virtualname status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervhost2 Available Virtual SCSI Server Adapter vsa0 Available LPAR Virtual Serial Adapter


20/40


IBM System p


Virtual IO Server Resource Mapping

Configuring virtual target device

mkvdev vdev aix_sq07 vadapter vhost0 dev vt_aix_sq07

mkvdev vdev hdisk7 vadapter vhost1 dev vt_hdisk7

$ lsdev -virtual

name status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)

vhost0 Available Virtual SCSI Server Adapter

vhost1 Available Virtual SCSI Server Adapter

vsa0 Available LPAR Virtual Serial Adapter

vt_aix_sq07 Available Virtual Target Device - Logical Volumevt_hdisk7 Available Virtual Target Device - Disk


21/40


IBM System p


Confirm what backs up each vhostList all vscsi server adapters with all flag

$ lsmap -all

SVSA Physloc Client Partition ID

--------------- ---------------------------------- ------------------

vhost0 U9111.520.10C1C1C-V3-C2 0x00000001

VTD vtscsi0

LUN 0x8100000000000000

Backing device aix_sq07

Physloc

SVSA Physloc Client Partition ID

--------------- ---------------------------------- ------------------

vhost1 U9111.520.10C1C1C-V3-C4 0x00000001

VTD vtscsi1

LUN 0x8100000000000000

Backing device hdisk7

Physloc U787A.001.DNZ00ZE-P1-C1-T1-L4-L0

List a single vscsi server adapter with vadapter flag

$ lsmap vadapter vhost0

slot #

Logical Volume

hdisk


22/40


IBM System p


More Use of lsmap

$for v in `ioscli lsdev -virtual | grep vhost | awk '{print$1}'`

do

ioscli lsmap -vadapter $v -fmt : | awk -F: '{ print $1" "$2""$4" "$6 $8 $10}'

done

vhost0 U9117.570.10C0EDC-V16-C7 virt_aix_sq07 aix_sq07

vhost1 U9117.570.10C0EDC-V16-C8 virt_suse_sq07 suse_sq07







vhost8 U9117.570.10C0EDC-V16-C15 virt_rhel_sq07 rhel_sq07





23/40


IBM System p


Client Virtual Disk Attributes

# lsdev -Cc disk

hdisk0 Available Virtual SCSI Disk Drive

# lscfg -vl hdisk0

hdisk0 U9117.570.10C0EDC-V7-C5-T1-L810000000000Virtual SCSI Disk Drive

# lsattr -El hdisk0

PCM PCM/friend/vscsiPath Control Module False

algorithm fail_over

Algorithm Falsemax_transfer 0x20000Maximum TRANSFER Size True

pvid 00cc0edc916c5bd80000000000000000Physical volume identifier False

queue_depth 3Queue DEPTH False

reserve_policy no_reserveReserve Policy False

# lscfg -vl vscsi1

vscsi1 U9117.570.10C0EDC-V7-C6-T1 Virtual SCSI ClientAdapter

Device Specific.(YL)........U9117.570.10C0EDC-V7-C6-T1


24/40


IBM System p


Virtual Ethernet

Virtual Ethernet

Enable inter-lpar communications without a physical adapter

IEEE-compliant Ethernet programming model

Implemented through inter-partition, in-memory communication

VLAN splits up groups of network users on a physical network onto

segments of logical networks

Virtual switch provides support for multiple (up to 4K) VLANs

Each partition can connect to multiple networks, through one or more

adapters

VIO server can add VLAN ID tag to the Ethernet frame as appropriate.

Ethernet switch restricts frames to ports that are authorized to receive frames

with specific VLAN ID

Virtual network can connect to physical network through routing"

partitions generally not recommended


25/40


IBM System p


SEA Configuration

$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)

ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)

$ mkvdev sea ent1 vadapter ent0 default ent0 defaultid 1ent2 Available

$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)

ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)

ent2 Available Shared Ethernet Adapter

$ lsdev dev ent2 -attrpvid 1 PVID to use for the SEA device

pvid_adapter ent0 Default virtual adapter to use for non-VLAN-tagged packetsreal_adapter ent1 Physical adapter associated with the SEA

thread 0 Thread mode enabled (1) or disabled (0)

virt_adapters ent0 List of virtual adapters associated with the SEA


26/40


IBM System p


Shared Ethernet Adapter setup

VIO Server after mksysb InstallPhysicalEthernet

Virtual

Ethernet

VIO Server after sea config

mkvdev sea ent1 vadapter ent0 default ent0 defaultid 1ent2 Available

VIO servers IP address may beput on shared adapter interface,or on separate, non-external

access virtual adapter. VIO IPaddress is NOT configured onphysical, nor on SEA virtual adapter.

ent1 ent0

9.19.126.98

Shared Ethernet ent2

ent0ent1


27/40


IBM System p


SEA with NIB in VIO client Partitions

At Power5 GA, only way to provide network redundancy

VIO servers each share adapter, each on different internal network (PortVLAN ID)

Client builds NIB EtherChannel across the Port VLAN IDs no support for VLAN tagging both VIOs, all clients same IP subnet


28/40


IBM System p


What is SEA Failover? (AIX 5.3 ML3, VIO 1.2.0.0)

Shared Ethernet Adapter Redundancy

- SEA adapters in two VIO servers can back each other up.

- Two VIO servers now allowed to put external accessvirtual adapters on same internal PVID, with differenttrunk priorities

- Administrator dictates which SEA will be the primary basedon an assigned Trunk Priority

- A standby mode is available to force the SEA failover tobackup server, allowing for regular updates or maintenance

- The backup server that became primary will switch to beinga backup when primary resumes operation


29/40


IBM System p


What is SEA Failover (AIX 5.3 ML3, VIO 1.2.0.0)

Current limitation is one backup SEA per primary SEA atthis time

The backup SEA is idle until a failure occurs on theprimary SEA

- This is transparent to client LPARs

Failure may be due to:

- Physical adapter failure

- VIOS crash/reboot

- Failure to reach a pre-defined remote address


30/40


IBM System p


SEA Failover

Inside P5, we now allowed more than one trunk adapter with the sameVLAN ID; These adapters get different Trunk Priority between 1-15

- A lower priority is the more favored SEA; primary

- If there is more than one virtual Ethernet on the same SEA,each must have the same priority

The SEA failover protocol helps determine which one should behave asprimary, and when a failover should occur


31/40


IBM System p



32/40


IBM System p


Shared Ethernet Adapter Failover

CLI command to configure SEA failover with a remotemkvdev sea ent0 vadapter ent1 default ent1 defaultid 10 \

attr ha_mode=auto ctl_chan=ent2

IBM S t G


33/40

33

IBM Systems Group

POWER5 VIO Server Overview - QPS01 010 2006 IBM Corporation


Recap of these adapters:ent0 physical adapter to be shared it could be an EtherChannel over several physicalsent1 VIO Server 1 virtual, PVID 10, Access External Checkbox, Trunk Priority lowent1 VIO Server 2 virtual, PVID 10, Access External Checkbox, Trunk Priority highent2 virtual, PVID 10, no Access External Checkbox, no Trunk Priority, VIO server IP addrent3 virtual, PVID 99, Control Channelent4 Shared adapter, created over ent0 and ent1

ent3

ent1

en

t4

ent2

vio

ip

addr

en

t4

ent0

ent0

ent1

ent2

vio

ip

addr

ent3

Enterprise network

Power5 internal PVID 10


VIO

Server1VIO

Server2

Client LPAR

ent0

Lower trunkpriority Higher trunk

priority

IBM S t G


34/40

34

IBM Systems Group



VIO command line to configure SEA failover in each VIO LPAR

mkvdev sea ent0 vadapter ent1 default ent1 defaultid 10 attrha_mode=auto ctl_chan=ent3

Some examples show setting ha_mode and ctl_chan with a separate chdevcommand. Dont.

ent3

ent1

en

t4

ent2

vio

ip

addr

en

t4

ent0

ent0

ent1

ent2

vio

ip

addr

ent3

Enterprise network



VIO

Server1VIO

Server2

Client LPAR

ent0

Lower trunkpriority-favored Higher trunk

priority

IBM Systems Group


35/40

35

IBM Systems Group



You may put VIO IP addr on en4 (supported). We find stability putting it on anothervirtual adapter en2. ent2 adapter is a virtual adapter on Port VLAN id 10, withoutAccess External Network checkbox

Remove and reconfig of SEA can be done without taking down VIO server ip, default router, etcNIM restore of either VIO thru SEA on the other VIO is more seamless for example,

nim install VIO Server1 into its ent2 adapter, with RECOVER_DEVICES=yes, thru SEA ofVIO Server2

ent3

ent1

en

t4

ent2

vio

ip

addr

en

t4

ent0

ent0

ent1

ent2

vio

ip

addr

ent3

Enterprise network



VIO

Server1VIO

Server2

Client LPAR

ent0

Lower trunkpriority Higher trunk

priority


36/40


IBM System p


SEA Failover Pros and Cons

Unexpected outages such as systems crashes are uncommon but may occur

- A backup server detecting that the primary server is down will make itself the primary andwill begin bridging traffic.

No EtherChannel requirement at VIO Client partition- Single virtual adapter in client partition, nim install thru it, keep the configured IP

interface after nim install and boot

- Client does not need rmtcpip ; EtherChannel config ; mktcpip after nim install

Allows use of Enterprise Network VLANs inside Power5 Server

SEA failover can support planned VIO maintenance

- Scheduled maintenance can now be performed with no loss of user connections

- Traffic is transparently sent over another SEApadmin chdev dev -attr ha_mode=standby

Cons?

- More adapters in VIO server a bit more complex to configure

- All client partitions drive IO thru the more favored VIO SEA, until failed over to lessfavored VIO no balancing of client LPARs between VIO servers. Perhaps EtherChannelof physical adapters in VIO is important here.


37/40


SEA Failover vs Network Interface Backup

Manual intervention required to force

affected (1/2) client partitions back to

primary VIO Server

Automatic fail-back to primary

VIO Server when network link

becomes available

Failback to Primary

Not SupportedSupported802.1Q VLAN Tagging

No idle connections as client partitions

can be split between the two VIO

Servers

Requires a backup idle

connection on opposite VIO

Server

Physical network connections

(historically limited to 1500)

> 1500 requires AIX V5.3 TL4*

>= 1500MTU Size

Configuration required on all of the

client partitions

Configuration required only on the

VIO Servers

Configuration

VIO Server V1.1VIO Server V1.2

GA6 Firmware/HMC

Software Pre-requisites

Network Interface BackupSEA Failover

Note: IY79375 (AIX V5.3 TL4) provides Jumbo Frame Support for EtherChannel with virtual adapters


38/40


Integrated Virtualization Manager (IVM) providesHMC functionality in a partitionat a much lower price point

AIX 5L

V5.3

Linux

SLES 9

Linux

RHEL 4

Virtual LAN

VIOS

IVM

Provides LPAR/virtualization support

without a physical HMC no HMC

hardware translates to lower entry point

Web-based, intuitive/user-friendly interface

Shipped with the Virtual I/O Server (VIOS)

provides tight integration of LPAR

management and virtual storage

configuration

Supports creating/management of I/O and

LPARs within a single physical server

All I/O is virtualized Virtual Console,

Storage, Ethernet, and Optical

Subset of HMC Service functionality

To be available for entry servers up to

p5-550Q (both System p5 and p5

families)

HMC within a partition

POWER Hypervisor

Integrated Virtualization Manager

Integrated Virtualization Mgr


39/40


IBM System p


Reference

InfoCenter http://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm

Virtualizing your compute environment

http://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/iphb2.pdf

VIO Server and PLM command line reference http://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/commands/commands.pdf

Redbook

Advanced POWER Virtualization on IBM System p5http://www.redbooks.ibm.com search on SG24-7940

http://www.redbooks.ibm.com/redpieces/pdfs/sg247940.pdf

VIO Server Page from Fix Central Overview, Download, Documentation

http://www14.software.ibm.com/webapp/set2/sas/f/vios/home.html


40/40


IBM System p


More references

http://techsupport.services.ibm.com/server/vios/documentation/configuring_mpio_for_the_virtual_client.pdf

http://www14.software.ibm.com/webapp/set2/sas/f/vios/documentation/faq.html

http://www14.software.ibm.com/webapp/set2/sas/f/vios/home.html

http://www-128.ibm.com/developerworks/eserver/library/es-pwr5-virtualvlan/?ca=dnp-439

http://www14.software.ibm.com/webapp/set2/sas/f/vios/documentation/home.html

010 VIO Overview

Documents

Transcript of 010 VIO Overview