IBM SAN Volume Controller DH8 Port Usagegalileosuite.com/wp-content/uploads/2015/12/... ·...

IBM SAN Volume Controller DH8 Port Usage

White Paper

Recommendations to address the flexibility and complexity of port usage using SVC DH8.

About the Author

Tim Conley

Co-founder of ATS Group and Galileo Performance Explorer™, and former IBM® systems architect and engineer

Tim Conley brings over 27 years of IT experience in

enterprise open systems and storage products supporting

banking, insurance, public, and private sectors, with

emphasis on Power Systems, IBM Storage, storage area

networks (SAN), and Enterprise wide Backup and Recovery.

IBM SAN Volume Controller DH8 Port Usage | GalileoSuite.com

The latest generation of IBM’s SAN Volume Controller (SVC) node hardware, denoted

2145-DH8, is an upgrade from previous SVC node hardware offerings. It offers more

central processing unit (CPU) compute capacity over its SVC CG8 predecessor—and

that’s obviously a good thing. Additionally, there are two times more input/output (I/O)

slots and more memory capacity.

It also allows for greater fiber channel (FC) connectivity than previous models. With

options for four, eight, or twelve 8 Gbit/s ports, the DH8 can introduce complexity as

well as flexibility.

This whitepaper will discuss IBM’s recommendation on port usage, along with The ATS

Group’s recommendation, which does vary from IBM’s in some cases. Please note that,

while the DH8 supports 16 Gbit/s FC ports, we will focus on 8 Gbit/s ports, which are

much more common in the field.

In all cases, we work toward the following goals:

• Follow IBM’s recommendation as a starting point.

• Deviate from IBM’s recommendation to enhance resiliency where appropriate.

• Deviate from IBM’s recommendation to minimize changes required on

node replacements.

Recommendations for a Four-Port DH8 ........................................................................... 2

Recommendations for an Eight-Port DH8 ...................................................................... 2

Recommendations for a Twelve-Port DH8 ..................................................................... 4

Table of Contents:

IBM SAN Volume Controller DH8 Port Usage | GalileoSuite.com 2

Recommendations for a Four-Port DH8

For a four-port DH8, there really isn’t any difference between this hardware and previous SVC node

hardware, which shipped with four FC ports by default. In this configuration, all ports are used for all traffic:

In a typical dual-fabric environment, the odd-numbered ports would generally be connected to

“fabric A” and the even-numbered ports would generally be connected to “fabric B.” Although

replication is not specifically listed in the table above, the best practice would be to choose one

port in fabric A and one port in fabric B, on each node, and use those for replication.

Port SAN FabricPort Usage – IBM Recommendation

Port Usage – ATS Group Recommendation

Slot 1 Port 1 (Port 1) A Host/Storage/Inter-Node Host/Storage/Inter-Node

Slot 1 Port 2 (Port 2) B Host/Storage/Inter-Node Host/Storage/Inter-Node

Slot 1 Port 3 (Port 3) A Host/Storage/Inter-Node Host/Storage/Inter-Node

Slot 1 Port 4 (Port 4) B Host/Storage/Inter-Node Host/Storage/Inter-Node



Slot 1 Port 1 (Port 1) A Host/Storage Inter-Node

Slot 1 Port 2 (Port 2) B Host/Storage Replication or Host/Storage

Slot 1 Port 3 (Port 3) A Host/Storage Host/Storage

Slot 1 Port 4 (Port 4) B Host/Storage Host/Storage

Slot 2 Port 1 (Port 5) A Inter-Node Host/Storage

Slot 2 Port 2 (Port 6) B Inter-Node Host/Storage

Slot 2 Port 3 (Port 7) A Replication or Host/Storage Replication or Host/Storage

Slot 2 Port 4 (Port 8) B Replication or Host/Storage Inter-Node

Recommendations for an Eight-Port DH8

Stepping up to an eight-port DH8, we have the flexibility to segregate inter-node traffic and

replication traffic. For an eight-port DH8 being installed as a new SVC cluster, our recommendation

differs from IBM’s a bit:








Slot 2 Port 1 (Port 5) A Inter-Node Inter-Node

Slot 2 Port 2 (Port 6) B Inter-Node Inter-Node


Slot 2 Port 4 (Port 8) B Replication or Host/Storage Replication or Host/Storage

We recommend spreading the two inter-node ports across the two cards, with one inter-node port

on slot 1 and one inter-node port on slot 2. IBM’s recommendation places both inter-node ports

on the same slot. IBM notes that their recommendation is based on taking the simplest approach,

which makes sense for many customers. However, we prefer to help our customers manage the

added complexity, in order to limit the loss of functionality if a card were to fail.

With IBM’s recommended configuration, a failure of the card in slot 2 would take all inter-node

communication offline, preventing the node from participating in the cluster. In addition, its

partner node would enter write-through mode, and no volumes serviced by this I/O group would

benefit from the SVC’s write cache.

In our configuration, a failure of either card would cause the node to reboot in an attempt to disable

the failed card. However, upon the restart of the node, our configuration leaves a degraded but

functioning node, with at least one port online for host, storage, inter-node, and replication traffic.

It’s not a likely scenario, but we think the additional resiliency is worth the added complexity.

For an eight-port DH8 being installed into an existing cluster, via node replacement of existing

four-port nodes, our recommendation aligns with IBM’s:


While we embrace the added complexity of splitting the inter-node ports on a new SVC cluster

installation, attempting to do so on an existing cluster would require zoning changes to hosts and

storage arrays (along with storage array port definition changes, host disk rescans, etc.). For this

reason, we don’t believe the additional resiliency is worth the additional work required, and we

stick with IBM’s simpler approach.

If a node replacement is performed to replace an eight-port CG8 with an eight-port DH8, we

recommend keeping the existing port usage for each of the eight ports. It may be possible to

segregate inter-node traffic, but that would require a study of the customer’s current configuration.

Recommendations for a Twelve-Port DH8

Stepping up to a twelve-port DH8 gives maximum flexibility (and, of course, the potential for

maximum complexity). Because of the high throughput potential of a twelve-port DH8, IBM

recommends two options, based on whether or not the customer expects the write workload to an

I/O group to exceed 3 GiB/s.

For a twelve-port DH8 installed in a new SVC cluster, where the I/O group write throughput is

expected to be less than 3 GiB/s:







Slot 2 Port 1 (Port 5) A Inter-Node Replication or Host/Storage






Slot 5 Port 3 (Port 11) A Replication or Host/Storage Host/Storage

Slot 5 Port 4 (Port 12) B Replication or Host/Storage Host/Storage


Again, our recommendation differs from IBM’s solely in that we prefer to spread the two inter-node

ports between two slots.

For a twelve-port DH8 installed in a new SVC cluster, where the I/O group write throughput is

expected to be greater than 3 GiB/s:

In this scenario, our recommendation matches IBM’s recommendation, since the four inter-node

ports are already spread over two slots.

Lastly, we will discuss the scenario of a twelve-port DH8 replacing a four-port node in an existing

cluster. Like the previous example of an eight-port node replacing a four-port node, we prefer to

not change the function of the existing four ports.
















Our recommendation differs a bit from IBM’s in order to spread the inter-node ports across two

slots. Both IBM’s recommendation and our recommendation keep ports 1 through 4 as host/

storage, to avoid unnecessary changes to existing zoning, storage arrays, and hosts.

For a twelve-port DH8 replacing a four-port node in an existing cluster, where the I/O group write

workload is expected to be greater than 3 GiBs:







Slot 2 Port 1 (Port 5) A Inter-Node Replication or Host/Storage






Slot 5 Port 3 (Port 11) A Replication or Host/Storage Host/Storage

Slot 5 Port 4 (Port 12) B Replication or Host/Storage Host/Storage



Slot 1 Port 1 (Port 1) A Inter-Node Host/Storage

Slot 1 Port 2 (Port 2) B Inter-Node Host/Storage





For a twelve-port DH8 replacing a four-port node in an existing cluster, where the I/O group write

workload is expected to be less than 3 GiB/s:


Solution OpportunityThe ATS Group, a premier IBM® business partner, provides a range IBM SVC

storage optimization IT applications designed to help your business best utilize

your business-critical stored data. In particular, Galileo Performance Explorer™

provides server and storage analytics and virtualization capabilities that can help

you make the most of your most important data. Galileo monitors and analyzes many

of the operations necessary for SVC DH8 storage solutions. For ways to improve

performance management across your storage

systems, contact ATS Group:

Phone: (484) 320-4302

E-mail: [email protected]

Website: GalileoSuite.com

Our recommendation differs slightly from IBM’s, in that we do not want to change the function of

ports 1 through 4 on an existing cluster.

The fiber channel configuration options on the DH8 SVC nodes allow for quite an increase in

flexibility compared to previous nodes. As is usually the case, that flexibility comes at the price of

increased complexity. Hopefully, this article helps to alleviate some of that complexity in your design.

Of course, as is also usually the case, there is no “one size fits all” solution; your results may vary.






Slot 5 Port 2 (Port 10) B Host/Storage Inter-Node



(Continued from previous table)

Galileo Performance Explorer is a trademark of the ATS Group. All other products and trademarks referred to are property of their respective owners.

© Copyright ATS Group. All rights reserved.

GalileoSuite.com

IBM SAN Volume Controller DH8 Port Usagegalileosuite.com/wp-content/uploads/2015/12/... ·...

Documents

Transcript of IBM SAN Volume Controller DH8 Port Usagegalileosuite.com/wp-content/uploads/2015/12/... ·...