VIO Updates - IBM · PDF file• ability to roll out updates to VIO servers without...

Technical University/Symposia materials may not be reproduced

in whole or in part without the prior written permission of IBM. 9.0

© Copyright IBM Corporation 2014

VIO UpdatesFeatures You May Have Missed

James Nash

Power Systems Strategic Initiatives

IBM


James Nash

[email protected] Systems Strategic Initiatives

(formerly known as ATS)

Special Thanks to

Benoit Creau

Steve Knudson

Steve Nasypany

Mark Olson

Steve Pittman

4

Session Objectives What’s relatively new


PowerVM Editions and Versions

VIO Server 2.2.3

― Simplified SEA failover configuration

― Shared Storage Pool enhancements

― Updates to ‘part’

― NPIV Statistics

― Useful but oft forgotten

Live Partition Mobility

Power Enterprise Pools

Power Integrated Facility for Linux

VIO Alternatives

―SR-IOV

―RoCE

―PowerKVM5

PowerVM Editions


Virtual I/O Server

VIO Server

POWER Hypervisor (PHYP)

Client

LPAR

Client

LPAR

Client

LPAR


VIO Versions

• 2.2.1.5-FP25 SP03 December 2012

• 2.2.1.6-FP25 SP04 April 2013

• 2.2.1.7-FP25 SP05 August 2013

• 2.2.1.8-FP25 SP08 February 2014

• 2.2.1.9-FP25 SP09 August 2014

• 2.2.2.0 December 2012

• 2.2.2.1-FP26 March 2013

• 2.2.2.2 FP26 SP01 February 2013

• 2.2.2.3 FP26 SP02 August 2013

• 2.2.2.4-FP26 SP04 February 2014

• 2.2.2.5-FP26 SP05 August 2014

• 2.2.2.6 January 2015

• 2.2.3.1-FP27 SP01 November 2013

• 2.2.3.2-FP27 SP02 April 2014

• 2.2.3.3-FP27 SP03 August 2014

• 2.2.3.4-FP27 SP04 November 2014


Latest Levels as of August 2014

Keep it current

• 2.2.1.9-FP25 SP09 August 2014

• 2.2.2.6 January 2015

• 2.2.3.4-FP27 SP04 November 2014


HMC 8.8.2 SP1 – New view


HMC 8 requires:

Rack Mount CR5 or newer

Desk-side C08 or newer

At least 2 GB of memory

10

NIB and SEA Failover


SEA Failover with Load Balance


Simplify SEA Failover

• No Need for a Control Channel

• Requirements

– VIO server 2.2.3 or higher

– HMC 7.7.8 or higher

– System Firmware 780

• Note – Function is not available on “B” model servers

– IBM Power 770 (9117-MMB)

– IBM Power 780 (9179-MHB)

– VLAN ID 4095 not in use



VIO Server 1

ent0

Ethernet

Switch 1

Client

LPARClient

LPAR

VIO Server 2

ent3

13

ent0

11

ent0ent1

11ent3

13

ent2

12ent1

11

ent2

12

ent0

12

Simplify SEA Failover – VIO 2.2.3

Ethernet

Switch 2

14


padmin@vio1$ entstat –all ent1 | grep –iE “Priority|Port VLAN ID”

Priority: 1 Active: False

Port VLAN ID: 11



Port VLAN ID: 12



Port VLAN ID: 13


• Verify VLAN IDs and Trunk Priorities



Port VLAN ID: 11



Port VLAN ID: 12



Port VLAN ID: 13

15


padmin@vio1$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3

–default ent1 –defaultid 11 –attr ha_mode=sharing

ent4 Available


• Create your Shared Ethernet Adapter

padmin@vio2$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3

–default ent1 –defaultid 11 –attr ha_mode=sharing

ent4 Available

ctl_chan=ent#

ctl_chan=ent#

16


padmin@vio1$ lsdev -dev ent4 –attr

attribute value description user_settable

accounting disabled Enable per-client accounting of network statistics True

adapter_reset yes Reset real adapter on HA takeover True

ctl_chan Control Channel adapter for SEA failover True

gvrp no Enable GARP VLAN Registration Protocol (GVRP) True

ha_mode sharing High Availability Mode True

hash_algo 0 Hash algorithm used to select a SEA thread True

jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True

large_receive yes Enable receive TCP segment aggregation True

largesend 1 Enable Hardware Transmit TCP Resegmentation True

lldpsvc no Enable IEEE 802.1qbg services True

netaddr 0 Address to ping True

nthreads 7 Number of SEA threads in Thread mode True

pvid 15 PVID to use for the SEA device True

pvid_adapter ent1 Default virtual adapter for non-VLAN-tagged packets True

qos_mode disabled N/A True

queue_size 8192 Queue size for a SEA thread True

real_adapter ent0 Physical adapter associated with the SEA True

send_RARP yes Transmit Reverse ARP after HA takeover True

thread 1 Thread mode enabled (1) or disabled (0) True

virt_adapters ent1,ent2,ent3 List of virtual adapters associated with the SEA True


• Verify VLAN IDs and Trunk Priorities

17


padmin@vio1$ entstat –all ent4 | grep –i “Control Channel PVID”

Control Channel PVID: 4095

padmin@vio1$ entstat –all ent4 | grep –i state

State: PRIMARY_SH


• Where is the Control Channel




State: BACKUP_SH

18


VIO Server 1

ent0

ent4

SEA

Ethernet

Switch 1

Client

LPARClient

LPAR

VIO Server 2

ent3

13

ent0

11

ent0ent1

11

ent4

SEA

ent3

13

ent2

12ent1

11

ent2

12

ent0

12

VLAN 4095


Ethernet

Switch 2

19

Remove Existing Control Channel



padmin@vio1$ oem_setup_env

root@vio1# rmdev –l ent6

ent6 Defined

or

padmin@vio1$ rmdev –dev ent6 -ucfg

ent6 Defined


• Requires both SEA adapters in a defined state (outage)

padmin@vio2$ oem_setup_env

root@vio2# rmdev –l ent6

ent6 Defined

or

padmin@vio2$ rmdev –dev ent6 -ucfg

ent6 Defined

21


root@vio1# lsattr –El ent6 –a ctl_chan

ctl_chan ent4 Control Channel adapter for SEA failover True

root@vio1# chdev –l ent6 –a ctl_chan=“”

ent6 changed

padmin@vio1$ lsdev –dev ent6 –attr ctl_chan


padmin@vio1$ chdev –dev ent6 –attr ctl_chan=“”

ent6 changed


• Blank out the control channel

root@vio2# lsattr –El ent6 –a ctl_chan


root@vio2# chdev –l ent6 –a ctl_chan=“”

ent6 changed

padmin@vio2$ lsdev –dev ent6 –attr ctl_chan


padmin@vio2$ chdev –dev ent6 –attr ctl_chan=“”

ent6 changed

22


root@vio1# mkdev –l ent6

ent6 Available

or

padmin@vio1$ cfgdev –dev ent6

ent6 Available


• Requires both SEA adapters in a defined state (outage)

root@vio2# mkdev –l ent6

ent6 Available

or

padmin@vio2$ cfgdev –dev ent6

ent6 Available

23





State: PRIMARY_SH


• Where is the Control Channel




State: BACKUP_SH

24


VIO Server 1

ent0

ent6

SEA

Ethernet

Switch 1

Client

LPARClient

LPAR

VIO Server 2

ent3

13

ent0

11

ent0ent1

11

ent6

SEA

ent3

13

ent2

12ent1

11

ent2

12

ent0

12

VLAN 4095


Ethernet

Switch 2

25

Simplification Summary


Simplified SEA Failover setup

If you do not specify a control channel (ctl_chan) when

running mkvdev –sea … The system will “discover” SEA

Failover “partner” adapter in the other VIO

Discovery protocol uses VLAN id 4095. If this is one of your

actual tagged VLANs, you must continue to use control

channel on both sides

Multiple SEA pairs in the machine can share VLAN 4095 for

discovery

This is still SEA failover, and we still set priority 1 or 2 on the

trunked virtual adapter in the SEA

VIO server 2.2.3, HMC 7.7.8, Firmware 780. Not supported on

MMB or MHB at this time.

Perhaps we stay consistent with our current Power7 practices,

and use this for new Power8 machines.

26

Shared Storage Pools

Shared Storage Pool

PHYP

VIOS VIOS

PHYP

VIOS VIOS

PHYP

VIOS VIOS

PHYP

VIOS VIOS


VIO 2.2.3


Shared Storage Pool Enhancements

– Pool Resiliency – mirror the storage pool (two failover groups)

– Pool shrink by removing physical volume

– Dynamic disk growth in the pool

– Scaling – more clients, larger physical volumes

– New lu, pv, failgrp commands

– Cluster wide operations performed concurrently

– VIO 2.2.2 added

• ability to roll out updates to VIO servers without stopping the cluster

• In case of failed repository disk, cluster remains active, no changes,

until disk is replaced

• VLAN tagging supported in the cluster

developerWorks articles to get started

28

Shared Storage Pool Limits

Feature Min Max

• Number of VIOS Nodes in Cluster 1 16

• Number of Physical Disks in Pool 1 1024

• Number of Virtual Disks (LUs) Mappings in Pool 1 8192

• Number of Client LPARs per VIOS node 1 200

• Capacity of Physical Disks in Pool 10GB 16TB

• Storage Capacity of Storage Pool 10GB 512TB

• Capacity of a Virtual Disk (LU) in Pool 1GB 4TB

• Number of Repository Disks 1 1

• Capacity of Repository Disk 512MB 1TB

Recommended VIO server requirements – per SSP node:

• Minimum CPU: 1 CPU of guaranteed entitlement

• Minimum memory: 4GB

• Storage requirements per SSP cluster (minimum): – One fiber-channel attached disk for cluster repository, 1 GB

– At least one fiber channel attached disk for data, 10GB


New command: lu

$ lu -list

POOL_NAME: test_pool

TIER_NAME: SYSTEM

LU_NAME SIZE(MB) UNUSED(MB) UDID

aix2-rootvg 10240 0 ad4f5d...

aix3-rootvg 20480 18460 242eac...

aix4-rootvg 102400 0 56e22b...

aix5-rootvg 10240 0 1a4758...

aix6-rootvg 40960 0 bfa955...

aix7-rootvg 51200 0 599ad4...


New command: pv

$ lspv

NAME PVID VG STATUS

hdisk0 00f629dbc192abd7 rootvg active

hdisk1 00f629dbbe759aa5 caavg_private active

hdisk2 00f629dbbe7c9b29 None

hdisk3 00f629dbbd739cbe None

$ pv -list

POOL_NAME: test_pool

TIER_NAME: SYSTEM

FG_NAME: Default

PV_NAME SIZE(MB) STATE UDID

hdisk2 1048576 ONLINE 3C231IBM FlashSystem


VIO Sizing Recommendations


http://www.redbooks.ibm.com/redbooks/pdfs/sg248062.pdf32

VIO Sizing – System Planning Tool


http://www-947.ibm.com/systems/support/tools/systemplanningtool/33

System Planning Tool


VIO Sizing – Workload Estimator

• Updated with information on POWER8 and PowerKVM


http://www-947.ibm.com/systems/support/tools/estimator/35

Workload Estimator


Other options

• IBM Sales Representative

• IBM Client Representative

• IBM Architect

• IBM Client Technical Specialist

• IBM Business Partner

1-866-883-8901

http://www-304.ibm.com/partnerworld/wps/bplocator/landing

• IBM Techline


http://www-304.ibm.com/partnerworld/wps/bplocator/landing

After you have the box…


Performance Advisors

• Run advisors on test or production systems.

• Advisors will evaluate the environment for performance

optimization opportunities

– Gives guidance on how to make the necessary changes.

• Three advisors available….

– PowerVM Virtualization

– Java (JPA)

– VIOS & Virtual Ethernet (built-in VIO with v2.2.2.1 via ‘part’)

• “Built in Smarts” to detect some of the most common

problems that are encountered

• Available on Developer Works

– FREE OF CHARGE

• https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power+Systems/page/Performance+Tools


Virtualization Performance Advisor

• Diagnoses performance issues within PowerVM partitions.

• Runs within an AIX partition (dedicated or shared)

• Analyzes performance characteristics

• Provides a health check report.

• Report includes system configuration (available resources, hardware & software levels, etc), partition (type, mode, available resources, s/w levels), partition settings and recommendations to improve performance (e.g. placement, SMT, processor folding, processor, and memory pool utilization).


Java Performance Advisor


VIO 2.2.2.1 – part command


Fibre Channel Statistics


Color-coded Icons


Severe Performance Problem


Investigate


part - Components Monitored

The following are the types of advisory reports that

are generated by the VIOS Performance Advisor

tool: System configuration advisory report

–CPU (central processing unit) advisory report

–Memory advisory report

–Disk advisory report

–Disk adapter advisory report

– I/O activities (disk and network) advisory report


https://www.ibm.com/developerworks/wikis/display/WikiPtype/VIOS+Advisor

An option for VIO servers below v2.2.2.1

47

https://www.ibm.com/developerworks/wikis/display/WikiPtype/VIOS+Advisor

Reminder – viosbr (added at v2.1.2)


$ viosbr -backup file backup -frequency daily numfiles 5

Creates a backup file once a day, prefixed with

“backup” in /home/padmin/cfgbackups

– backup.01.tar.gz





$ viosbr -restore file

/home/padmin/cfgbackups/backup.03.tar.gz

48

http://www.ibm.com/partnerw

orld/gsd/searchprofile.do?na

me=VIOS_Recognized_List

For VIOS Recognized applications:http://www.ibm.com/partnerworld/gsd

Click on "Search solutions" link on the left

side of the page

Click on "Validations" towards the top

middle right of the page

Type "VIOS Recognized" in the search field

and click “Search”

This will generate a current list of officially

tested applications that are supported on

the VIO Server.


Third party software - updated

49

http://www.ibm.com/partnerworld/gsd/searchprofile.do?name=VIOS_Recognized_List

http://www.ibm.com/partnerworld/gsd

Live Partition Mobility


Evacuate option

migrlapr –o m –m source_server –t target_server –all

Performance suggestions

– large_send, large_receive on physical adapters in VIO

(these SHOULD already be on)

– Suggesting tcp_sendspace, tcp_recvspace 524288 on

VIO interface

– Jumbo frames if your environment supports it

50



Power Enterprise Pools allow mobile activations of CPU and

memory to be shared among a group of pool of Power Systems

All previous activations are static and do not move between

servers

New systems can be added to the pool

Pool information can be viewed, including pool resource

assignments, compliance, history logs

Up to 75% of system memory can be mobile

In a 770 pool, up to 60 mobile activations allowed (feature EP22)

In a 780-795 pool, up to 120 mobile activations for 780, up to 192

for 795 (feature EP23)

For 770, 780, up to 30 mobile memory activations of 100GB

(EMA4)

For 795, up to 120 mobile memory activations of 100GB (EMA4)

51



HMC 7.7.8, with 2GB of memory required

770-780 D models (9117-MMD, 9179-MHD) and 795 (9119-FHB)

only

Firmware 780 only (FW780, AH780)

770, 780 must have at least 4 static core activations, 25% of

memory static activations

795 must have 24 static core activations, or 25% of installed

cores, whichever is higher

In all servers, 25% of memory must have static activations

GA Dec 6 2013 for 795

GA April 25 2014 for 770, 780

Two type of pool – 770 pool, and a high end pool (780, 795)

“Master” HMC, a redundant HMC acts as “backup master”

52



Configuration file obtained from IBM CoD to create a pool, add server to pool,

add mobile activations to pool

53

Power Integrated Facility for Linux (IFL)


One Power IFL feature is 4 processor, and 32GB

activations, with 4 PowerVM EE entitlement and SWMA,

feature ELJ0 (zero not O)

Lower price activation than for AIX or IBM i, for Linux

only

Power customer agrees to segregate AIX and IBM i

partitions into a separate virtual shared processor pool,

away from Power IFL cores.

Any IBM Software Group (SWG) products run on Linux

on Power are also discounted - priced at 70 Processor

Value Units (PVU)

770, 780 B, C, D models, and 795

Firmware 780, HMC 7.7.8

54

Power Integrated Facility for Linux (IFL)


How do we keep AIX and IBM i from running on the less

expensive Linux cores? AIX, i, and VIO go in a sub pool, where

we can set a maximum CPU limit. Lower cost Linux cores are

used in the default pool

55

o

System

Topology

Node

DRILL DOWN

Partition

DRILL DOWN

V1.1.2 support for POWER8

PowerVP Performance Monitor


Servers supported

― POWER7

― POWER7+

― POWER8

Firmware version 770, 780, 810 or higher.

System and partition-level agents

Minimum OS levels:

― AIX 6.1, AIX 7.1

― RedHat Enterprise Linux 6.4

― SUSE Linux Enterprise Server 11 SP3

― IBM i 7.1 with TR6





59

Do I need a VIO Server?


• RoCE or IB

• SR-IOV

• PowerKVM

• PowerNV

60

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&uact=8&docid=j_9K_QdzoAZjdM&tbnid=WeuoXrWWGfhc2M&ved=0CAgQjRw&url=https://design.ubuntu.com/brand/ubuntu-logo&ei=BrMMVP7dOI2MyATEo4LwCw&psig=AFQjCNGizicwXZ36mk2sUeIqIlwBqvJUfA&ust=1410204807014955

RDMA over Converged Ethernet

RoCE

• PCIe3 2-port 40 GbE NIC / RoCE QSFP+ Adapter

– FC EC3B; CCIN 57BD


http://www-01.ibm.com/support/knowledgecenter/POWER8/p8hcd/fcec3b.htm?cp=POWER8%2F4-3-10-1-1-1661

SR-IOV


NIC SR-IOV for POWER7+ 770/780 with

Specific 10GE Ethernet adapters #EN0K and #EN0H in Gen2 slots

Or with newest IMFC #EN10 and #EN11

62

SR-IOV

Single Root I/O Virtualization

Runs “closer to the silicon” potentially offering some

performance efficiencies

Doesn’t require VIOS as a pre-req and thus can do simple

virtualization under PowerVM without VIOS …. BUT …. VIOS

continues to offer many additional advanced functions

Architecturally can virtualize a resource like an Ethernet

adapter and allocate/provide a user-defined minimum level of

bandwidth to a partition … Quality of Service (QoS)

Ethernet NIC announced.

Could use VIOS & SR-IOV together

LPAR #2

LPAR #3

LPAR #1


Additional Linux Options


Power S822L8247-22L

Power S824L

Power S812L8247-12L

64

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&uact=8&docid=j_9K_QdzoAZjdM&tbnid=WeuoXrWWGfhc2M&ved=0CAgQjRw&url=https://design.ubuntu.com/brand/ubuntu-logo&ei=BrMMVP7dOI2MyATEo4LwCw&psig=AFQjCNGizicwXZ36mk2sUeIqIlwBqvJUfA&ust=1410204807014955

The Redbook


http://www.redbooks.ibm.com/abstracts/sg248198.html?Open

65

http://www.redbooks.ibm.com/abstracts/sg248198.html?Open


VIO Command Reference (v2.2.3.3)

http://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf

66



Appendix

PowerVM Shared Ethernet Adapter simplification:

Get rid of Control Channel Adapterhttp://chmod666.org/index.php/powervm-shared-ethernet-apdater-simplification-get-rid-of-control-channel-adapter/

PowerVM developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/PowerVM

Virtual IO and virtualization developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtual%20I

O%20and%20virtualization

Virtualization Best Practices developerWorks – wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtualizatio

n%20best%20practices

HMC 8.8.1 SP1https://www.ibm.com/developerworks/community/blogs/brian/entry/hmc_version_8_release_8_8_1_highlights?lang

=en

VIO and IVM Command Referencehttp://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf

VIO 2.2.3.3 Release Noteshttp://www-01.ibm.com/support/knowledgecenter/POWER8/p8eeo/p8eeo.pdf

67

http://chmod666.org/index.php/powervm-shared-ethernet-apdater-simplification-get-rid-of-control-channel-adapter/

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/PowerVM

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtual%20IO%20and%20virtualization

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtualization%20best%20practices

https://www.ibm.com/developerworks/community/blogs/brian/entry/hmc_version_8_release_8_8_1_highlights?lang=en


http://www-01.ibm.com/support/knowledgecenter/POWER8/p8eeo/p8eeo.pdf

James Nash

Power Systems

Strategic Initiatives

Formerly known as

ATS

1177 S. Beltline RoadCoppell, TX 75019

[email protected]


VIO Updates - IBM · PDF file• ability to roll out updates to VIO servers without...

Documents

Transcript of VIO Updates - IBM · PDF file• ability to roll out updates to VIO servers without...