IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for...

36
© 2011 IBM Corporation IBM Power Systems Virtual IP - Load Balancing, Fault Tolerance, & IOA Sharing Session ID: VT 445-4 Fant Steele [email protected] Certified I/T Specialist – IBM Lab Services © 2011 IBM Corporation IBM Power Systems 2 Description and Objectives This session covers the many uses of virtual networking available in TCP/IP on System i. Instructions for defining and using Virtual IP Addresses (VIPA) will be provided. We will also discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate with LPARS. Objectives Explain how Virtual IP Addresses (VIPA) may be used on the system to provide fault tolerance and application isolation Describe the techniques used to flow traffic between physical and virtual ethernet networks Describe load balancing using the built in functions of TCP/IP is i5/OS

Transcript of IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for...

Page 1: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

Virtual IP - Load Balancing, Fault Tolerance, & IOA SharingSession ID: VT 445-4

Fant Steele [email protected] I/T Specialist – IBM Lab Services

© 2011 IBM Corporation

IBM Power Systems

2

Description and Objectives

This session covers the many uses of virtual networking available in TCP/IP on System i. Instructions for defining and using Virtual IP Addresses (VIPA) will be provided. We will also discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate with LPARS.

Objectives

– Explain how Virtual IP Addresses (VIPA) may be used on the system to provide fault tolerance and application isolation

– Describe the techniques used to flow traffic between physical and virtual ethernetnetworks

– Describe load balancing using the built in functions of TCP/IP is i5/OS

Page 2: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

3

Agenda

What is Virtual IPWhat components are used to make virtual IP work–IP forwarding–Direct Routing–Transparent subnets and Proxy ARP–Network Address Translation (NAT)–Virtual IP address–Schowler Routes–Virtual Ethernet adapter–Integrated Virtual Ethernet (IVE / HEA)

Typical solutions–Multiple HTTP servers–Multiple Domino servers–Load Balancing across multiple physical adapters–Reducing points of failure to increase availability–Connecting between multiple logical partitions

© 2011 IBM Corporation

IBM Power Systems

4

What is Virtual IP ?

Virtual IP can be one or more of several components available inOS/400. These include:–Virtual IP addresses (V4R3)–TCP/IP over Virtual OptiConnect (V4R4)–Virtual Ethernet LAN adapters (V5R1)

These components are implemented by software and imitate hardware.They can be used to supplement Real physical LAN adapters on the system.Takes advantage of the fact that the iSeries and OS/400 TCP/IP implements the Weak Multi-homing model as per RFC1122

"Weak Multi-homing model: The adapter on which a packet is received is irrelevant

Page 3: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

5

Components

© 2011 IBM Corporation

IBM Power Systems

6

If IP datagram forwarding is set to *Yes then IP traffic will be routed through the system based on the route table entries.In this example the traffic between the two PC systems (10.1.1.11 and 10.1.2.10) routes through the iSeries system. The PCs have a route statement that sets the iSeries as their default route

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

10.1.2.10 10.1.2.11

10.1.2.12

10.1.2.1

IP Forwarding

Page 4: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

7

Virtual LAN 1 addrs - 10.1.2.1 - 10.1.2.254subnet mask 255.255.255.0

Route statements must be placed in the network routers– Static routes defined by hand– Advertised routes via RIP or others protocols

Routes point to the real adapter addressDNS entries point to the virtual addressesThe route table is then checked and the datagram is forwarded to the destinationAll traffic is routed through the I/O partition

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

subnet mask 255.255.255.0

Direct Routing

© 2011 IBM Corporation

IBM Power Systems

8

The target address is compared to the routing table using the network mask (subnet mask) of the TCP/IP interfacesAfter determining that the target host should be on a local segment of the network, TCP/IP broadcast an ARP (address resolution protocol) request to find the host adapter that has the IP address assignedIf the adapter with the address is active on the network then it replies with the MAC address assigned to the adapter Communications is then accomplished using the physical communications layer

10 11 12

1 14 TCP/IP Hosts

10.1.1.____

Finding a host on the network (ARP)

Page 5: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

9

Virtual LAN 1

There are several ways to get the traffic from the Virtual LAN Segment to the real network. These include:

– Proxy ARPIP Addresses DO NOT Change

– Network Address TranslationIP Addresses CHANGE

– Direct RoutingIP Addresses DO NOT Change

IP Forwarding must be enabled for any of these to workThe default route in the guest LPARS must point to the virtual IP address in the I/O partition as the gateway

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

subnet mask 255.255.255.0

Example:One physical LAN adapterThree partitions

– I5/OS– Suse Linux– RedHat Linux

All Partitions connected on LAN 1

But what about Virtual Adapters

© 2011 IBM Corporation

IBM Power Systems

10

Virtual LAN 1 addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240

Proxy ARP replies with the MAC address of the physical adapterAll traffic is routed through the proxying partitionDNS entries point to Virtual addressesProxy can be i5/OS, Linux, or AIXVirtual LAN address range must be a subset of the physical LAN addressesVLAN MTU must be =< Physical LAN

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

subnet mask 255.255.255.0

ProxyARP

Proxy ARP will reply to ARP request on the real network for addresses on the virtual segment of the network

Transparent Subnet (Proxy ARP)

Page 6: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

11

Virtual LAN 1 addrs - 10.1.2.1 - 10.1.2.254subnet mask 255.255.255.0

Addresses for each of the LPARS must be defined in the NATing partition

– Additional IP Interface on the REAL adapter– Virtual IP Addresses with proxy to the real adapters

(recommended)NAT LPAR can be i5/OS, Linux, or AIXStatic NAT rules must be created to MAP the public (Physical) address to the private (VLAN) addressDNS entries point to the addresses in the real segmentThe real adapter Replies to the ARP requestThe IP address in the IP header is rewritten (mangled) The route table is then checked and the datagram is forwarded to the destinationAll traffic is routed through the proxying partition

10.1.1.310.1.1.210.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

subnet mask 255.255.255.0

NAT rewrites the IP header and then Forwards the datagram

Public PrivateMap 10.1.1.2 <-> 10.1.2.2Map 10.1.1.3 <-> 10.1.2.3

Network Address Translation (NAT)

© 2011 IBM Corporation

IBM Power Systems

12

Network Address Translation (NAT)

I5/OS implements the following NAT types

Masquerade, or Hide, NAT

–Enables clients in internal network to access public network

Static, or Map, NAT

–Enables systems in the public network to access internal servers–Enables systems in the real LAN to access other logical partitions

Masquerade, or Hide “port-mapped”, NAT

–Enables systems in the public network to access internal servers. Conversation can be initiated from either side

Benefits of NAT

Saves public IP addresses

Transparent to the client

Simplifies routing in the internal network

Efficient (good performance)

Page 7: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

13

The Network Address Translation (NAT) function lets you translate internal IP addresses to external IP addresses.NAT is based on the fact that only a small number of the hosts in a private network are communicating outside of that network. If each host is assigned an IP address from the registered IP address pool only when they need to communicate, then only a small number of global addresses are required. NAT might be a solution for networks that have private address ranges or illegal addresses and want to communicate with hosts on the Internet. In fact, most of the time, this can also be achieved by implementing a firewall. Hence, clients that communicate with the Internet by using a proxy or SOCKS server do not expose their addresses to the Internet, so their addresses do not have to be translated anyway. However, for any reason, when proxy and SOCKS are not available or do not meet specific requirements, NAT might be used to manage the traffic between the internal and external network without advertising the internal host addresses.The native AS/400 system NAT, supports masquerading and static NAT.

I5/OS NAT implementationThe implementation of NAT on the AS/400 system takes three forms:

Masquerade, or Hide, NATStatic, or Map, NATMasquerade, or Hide “port-mapped”, NAT

Masquerade or Hide NAT, is primarily used to enable clients in your internal network with private IP addresses assigned, to access the public network. This is accomplished by translating the client’s private address (trusted address) to the public address of the AS/400 gateway (border address). Static, or Map, NAT is primarily used to enable systems in the public network to access servers in your internal network by translating the actual internal server address to a public address. This is a one-to-one mapping of IP address. There is no port translation. Masquerade, or Hide “port-mapped NAT, is used primarily to enable systems on the public network to access servers in your internal network. Both, IP address and port are translated. For example you could have an HTTP server on the internal network bound to IP address 10.1.1.1 and port 5000 being accessed from the public network using IP address 204.222.180.5 and port 80. The conversation can be initiated from either network therefore it also enables clients in the internal network to access systems in the public network. For detailed information on AS/400 NAT implementation refer to the article Networking Security - IP packet security at http://www.as400.ibm.com/infocenter and to the IBM redbook V4 TCP/IP for AS/400: More Cool Things Than Ever, SG24-5190.

© 2011 IBM Corporation

IBM Power Systems

14

10.1.1.10 193.20.1.1

1024 55336

Source Addr Dest. Addr. SP DP

10.1.1.10 192.10.1.5 1024 23

Source Addr Dest. Addr. SP DP

192.10.1.5 10.1.1.10 23 1024

Source Addr Dest. Addr. SP DP

193.20.1.1 192.10.1.5 55336 23

Source Addr Dest. Addr. SP DP

192.10.1.5 193.20.1.1 23 55336

Outbound traffic Inbound traffic

10.1.1.10

192.10.1.5

Public Interface

Masquerading Function

193.20.1.1

Dynamic Mapping

Masquerade or Hide NAT

TRUSTED BORDER UNTRUSTED

A private IP address or a range of IP addresses are hidden behind a single public IP address on the AS/400 gateway performing NAT.Only clients in the internal network can initiate the connection which improves security.Translation is done for outgoing packets and incoming packets are translated back and redirected to original destinationInternal port numbers are associated with random port numbers (address and port translation)Single public interface supports multiple simultaneous conversationsThree IP address type TRUSTED, BORDER, UNTRUSTEDCan be configured over leased, LAN, or PPP link

10.1.1.0

LAN or PPP link

Internet

Page 8: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

15

Masquerading is used to allow the private network to hide behind and be represented by the address bound to the public interface of the NAT machine. In most situations, this will be the address that has been assigned by an ISP which may be dynamic in the case of a PPP connection. This type of translation can only be used for connections originating within the private network destined for the outside public network. Each connection out, is maintained by using a different source (client) IP port number.The main characteristics of hide NAT are:

A private IP address or a range of IP addresses are hidden behind a single public IP address on the AS/400 gateway performing NAT.Only clients in the internal network can initiate the connection which improves security.Translation is done for outgoing packets and incoming packets are translated back and redirected to original destination.Internal port numbers are associated with random port numbers. This means that both, the address and the port number are hidden

form the public network.The registered address on the NAT machine is a usable interface outside of NAT.Single public interface supports multiple simultaneous conversations.

Address typesWhen using NAT there are three address types that you must configure in the Defined Addresses rules.

Trusted, used for internal or private addresses. These addresses are hidden from the public network.Untrusted, used for external or public addresses.Border, used for addresses that are public and that form a boundary between trusted and untrusted networks. This is the public address on the AS/400 gateway to

which the internal address or addresses are translated. Figure 10 on page 29 illustrates these concepts.

© 2011 IBM Corporation

IBM Power Systems

16

Virtual IP Address

Powerful tool for load balancing, fault tolerance, unnumbered interface anchor, etc.Can be viewed as "primary" or "external" IP address -- "IP address of the system"Externally accessible local IP address unbound to a single physical interfaceIf TCP/IP is UP then the Virtual IP address should be available to the applications using it VirtualIP interfaces : Not directly routable:

Reachable only via indirect route through "physical IP address" (IP address of physical interface)AS/400 will never answer ARP request to *VIRTUALIP address ****** option in V5R2 ******Allows same *VirtualIP address to exist on multiple hosts

VirtualIP is also supported by other IBM server platforms (AIX, MVS)VirtualIP interfaces advertised by RIPv2May be called Circuitless or Loopback interfaces

CorporateNetwork

10.1.1.x

NetworkDispatcher or Router

10.1.1.11

10.1.1.12

*VirtualIP = 10.250.1.1*VirtualIP = 10.1.1.1 CAN ARP in V5R2

10.1.1.13

Internet

Page 9: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

17

Virtual IP Address EvolutionV5R2 Added:

Proxy ARP automatic agent selection (based on first interface activated)

V5R3 Added:Agent selection based on highest speed available interfaceIf multiple VIPAs being proxied, spread across interfaces

V5R4 Added:Preferred Interface Selection

10.1.1.13

10.1.1.12

10.1.1.11

10.1.1.x

R1

R2

*VirtualIP10.1.1.1 Internet

DNS Entry10.1.1.1

© 2011 IBM Corporation

IBM Power Systems

18

Virtual IP Address

Page 10: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

19

Virtual IP Address

© 2011 IBM Corporation

IBM Power Systems

20

Virtual IP Address

Page 11: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

21

Virtual IP Address (V5R4)

© 2011 IBM Corporation

IBM Power Systems

22

Preferred Interface List for Virtual IP and Virtual Ethernet

Virtual IP Address (VIPA) enhancements introduced in V5R4 to have better control over VIPA proxy ARP agent selection

New preferred interface list available for virtual IP addresses

– is an ordered list of the interface addresses that will take over for the failed adapter

– allows you to manually select which adapters and IP addresses are to be the preferred interface for VIPA proxy ARP agent selection

Chart created by Thomas Barlen

i5/OS

VIPA10.1.1.15 / 32

TB TB

Intranet10.1.1.0 / 255.255.255.0

.11 .10

ProxyARPAgent

.12

Page 12: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

23

i5/OS provides proxy ARP support for VIPAs that are in a non-local subnet

– physical interfaces answer ARP requests for IP addresses that are not in the same subnet as the physical interface address

Provides IP mobility support for local area networks (LANs)

– allows IP address to be moved from a home network to a different network (migration)Cisco’s IOS Local Area Mobility (LAM) feature can exploit this V5R4 enhancement

Proxy ARP on behalf of VIPAs in a different subnet

OriginalRouter(home)

LAMRouter

10.10.10.20 /24

10.10.10.30 /24VIPA

10.10.10.75 / 32

Proxy ARP enabledIGP used

Routing table10.10.10.0 /2410.10.10.75 /32

10.10.10.1 /24

Proxy ARP + LAM enabledIGP used

Routing table10.10.10.0 /2410.10.10.75 /32

10.20.20.1 /24

10.20.20.30 /24

IBM i

Network A Network B

© 2011 IBM Corporation

IBM Power Systems

24

Notes:

In OS/400 V5R2 and i5/OS V5R3, proxy ARP is supported for virtual IP addresses (VIPAs) that are in the same subnet as the IP interface addresses on the physical LAN adapters. For instance, when configured for proxy ARP, an Ethernet IP interface 10.10.10.10 / 24 would answer ARP requests for a VIPA 10.10.10.20 /24, but it would not answer ARP requests for 10.20.20.5 /24.

With i5/OS V5R4, support was added to provide proxy ARP support for VIPAs that are not in the same subnet as local interfaces. This allows, for example, an Ethernet IP interface of 10.10.10.10 / 24 to answerARP requests for a VIPA 10.20.20.5 /24.

This enhancement can be leveraged by a technology that Cisco introduced for mobility in local area networks. The feature that isonly supported by Cisco routers is called Local Area Mobility (LAM) and is part of Cisco’s router operating system IOS.

Page 13: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

25

Notes:

What does LAM do? Let’s explore LAM based on the network diagram that is shown on the previous page. Assume that the iSeries system with its VIPA 10.10.10.75 /24 used to be installed in network A. When a client in network A (10.10.10.0 /24) wanted to communicate with the iSeries system, which was on the same subnet, the client would issue an address resolution protocol (ARP) request to obtain the MAC address for the iSeries interface. At a certain point in time it was necessary to move the iSeries system to a different building. This building had an IP subnet of 10.20.20.0 /24 (network B). However, many applications and clients had hardcoded the iSeries IP address 10.10.10.75, so a quick change of the iSeries IP address was not possible. The answer to this problem is LAM. LAM allows you to implement a migration scenario like the one described above. When LAM is used, the router (LAM router) where the iSeries system is now connected to will be enabled for proxy ARP and LAM (mobility) on the LAN interface. In addition, route redistribution has to be enabled on this router using one of the interior gateway protocols (IGPs), such as Open Shortest Path First (OSPF) or Enhanced Interior Gateway Routing Protocol (EIGRP). The host route for 10.10.10.75 is then distributed to other routers in the network. On the original router in network A, proxy ARP has also to be enabled. The original router’s routing table contains now an entry for network 10.10.10.0 /24 for its LAN interface and a more specific host route to 10.10.10.75 on the external interface. When a client now wants to communicate with 10.10.10.75 (VIPA), the original router answers the ARP request, thus pretending to be the interface the client wants to talk to and routes the packets to the LAM router, which in turn sends the packet to the network B. In order for the LAM router to send a packet to the iSeries, the router first sends an ARP request for 10.10.10.75 to the local network. This is where the V5R4 enhancement comes into the picture. The LAN interface on the iSeries will now answer the ARP request for the VIPAeven though the VIPA is in a different subnet than the LAN IP interface (10.20.20.30).For more information on Cisco’s LAM support, go to the Cisco home page at http://www.cisco.com and search for “Local Area Mobility”.

© 2011 IBM Corporation

IBM Power Systems

26

Schowler Routes

Extends Duplicate Route, Round Robin load balancing to local networksƒ DRRR - Based on "Duplicate Route Priority" and "Preferred Binding Interface" parmsƒ Problem: Neither parameter is available on *DIRECT routesƒ Solution: "Schowler" Routes

–Special indirect route that replaces a *DIRECT route–Same Route Destination, Subnet Mask & TOS as equivalent *DIRECT route–Next Hop and Preferred Binding Interface are set to the IP address of the equivalent local interface–Same local network connectivity as *DIRECT route but allows user to set Duplicate Route Priority and Preferred Binding Interface options for local network load balancing

–Requires PTFs (1Q00) for V4R3 or V4R4, is integrated into V4R5

ƒ Side Benefit: Host routes may be prioritized over *DIRECT routes

10.6.7.1

10.6.7.2

10.6.7.3

10.6.7.x

Private DNS Records:

MyServer 10.6.7.3

Schowler Route Replacements for *DIRECTs:Rte Dest. Subnet Mask Next Hop Preferred IFC Dup Rtr Pri10.6.7.0 255.255.255.0 10.6.7.1 10.6.7.1 610.6.7.0 255.255.255.0 10.6.7.2 10.6.7.2 610.6.7.0 255.255.255.0 10.6.7.3 10.6.7.3 6

Standard *DIRECT Routes:Rte Dest. Subnet Mask Next Hop10.6.7.0 255.255.255.0 *DIRECT10.6.7.0 255.255.255.0 *DIRECT

10.6.7.0 255.255.255.0 *DIRECT

Page 14: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

27

The Duplicate Route, Round Robin method of load balancing that was introduced in V4R2 was oriented towards remotely connected clients. This method of load balancing is based on two indirect route parameters:

- Duplicate Route Priority - Preferred Binding Interface.Configuring multiple duplicate routes with the same priority caused the routes to be selected in a round robin fashion.The problem was that these two parameters were not available for the *DIRECT routes that are automatically added when an interface is added. Thus, this form of load balancing did not work with locally connected hosts."Schowler" routes extend this load balancing capability to locally connected hosts. A Schowler route is functionally equivalent to the *DIRECT route that it replaces, but since it is added just like any other indirect route, the above two load balancing parameters can now be configured by the user. Schowler routes have two special characteristics:

–The same route destination, subnet mask and TOS setting as the equivalent *DIRECT route–The Next Hop and Preferred Binding Interface IP addresses are both set to the IP address of the associated local interface.When the Duplicate Route Priority is set greater than the default of 5, the equivalent Schowler routes are selected in a round robin fashion, identical to what can be done with other indirect routes.In the previous chart, we have 3 interfaces configured, connecting the AS/400 to the 10.6.7.x network, 10.6.7.1, 10.6.7.2 and 10.6.7.3. The first box in the lower left shows the standard *DIRECT routes that are automatically added with the interfaces. However, by adding 3 equivalent Schowler routes, shown in the lower box, the three *DIRECT routes disappear and are replaced by the Schowlers.One final use of Schowler routes is to reverse the default AS/400 TCP/IP routing logic that always prioritizes *DIRECT routes over any indirect routes, even *HOST routes. By replacing the *DIRECT routes with Schowler routes, no "highest priority" *DIRECT routes will be found during route lookup. All candidate routes are now indirect, and prioritized by subnet mask. Thus, a *HOST route, with a subnet mask of 255.255.255.255 will be considered the highest priority route.

© 2011 IBM Corporation

IBM Power Systems

28

Virtual Ethernet Adapter Support

Added in V5R1

Provides 1Gb "Ethernet" LAN connections across the system bus

Used for communications between logical partitions

Included in OS/400 as "no charge" item

One system will support up to 16 virtual LANs

Created when logical partitions are defined

May be configured and maintained using System Service Tools or Operations Navigator

Appear as type 268C adapter in WRKHDWRSC *CMN and are assigned a regular resource name (CMNxx)

Port number of resource matches to LAN id (0 - 15) 4096 in V5R3 with Power5 hypervsor

Linux references the resource as ethnn where nn is 0-15

A single virtual LAN can be used to connect 2 or more partitions

Can also be used to connect to Windows Servers on IXS, IXA and iSCSI

Page 15: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

29

Multiple Proxy ARP Agent Support with Virtual EthernetProxy ARP support on multiple interfaces in an LPARed system with virtual EthernetImproves availability when using transparent subnettingPreferred interface list defines priorities for proxy ARP agent selection V5R4 only – Additional PTFS also needed

– MF41339 5722999– SI27233 5722SS1 (requires endtcp and strtcp to activate)

Chart created by Thomas Barlen

Linuxi5/OS Linux i5/OS

•Web Appl

•Productionserver

TB TB

•File Server DevelopmentPartition

Virtual I/O

Intranet10.1.1.0 / 255.255.255.0

.10 .11 .12

10.1.1.32 / 255.255.255.248

.33 .34 .35 .36

ProxyARPAgent

Preferred Interface List

10.1.1.33

10.1.1.1010.1.1.1210.1.1.11

© 2011 IBM Corporation

IBM Power Systems

30

Virtual LAN 1 addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240

Provides fault tolerant proxy for virtual ethernetpartitionsReplaces use of associated local interface

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

ProxyARP

Proxy ARP will reply to ARP request on the real network for addresses on the virtual segment of the network

Preferred Interface List for Virtual Ethernet (transparent subnetting)

10.1.1.2

Page 16: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

31

Notes:

Proxy ARP replies with the MAC address of the physical adapter

All traffic is routed through the proxying partition

DNS entries point to Virtual addresses

Proxy can be i5/OS, Linux, or AIX (i5/OS in this example to provide fault tolerance)

Virtual LAN address range must be a subset of the physical LAN addresses

VLAN MTU must be =< Physical LAN

© 2011 IBM Corporation

IBM Power Systems

32

What’s in a Name – Integrated Virtual Ethernet (aka HEA)Integrated Virtual Ethernet (IVE) – External name in documentation

Host Ethernet Adapter (HEA) – Name used on user interfaces

New Hardware capability - Built into GX+ bus (P5IOC2) on most p6 systems

– Provides accelerated Ethernet connectivityEssentially, a system with a HEA has several integrated Ethernet adapters, called logical

ports. IVE can be used by multiple partitions.

– Integrated on most POWER6 systems

– Several variations of physical, external ports Dual 1 Gbit copper: supporting 10BASE-T, 100BASE-T, 1000BASE-TQuad 1 Gbit copper: supporting 10BASE-T, 100BASE-T, 1000BASE-T

Dual 10 Gbit fiber: supporting 10GBASE-SR or 10GBASE-LR

– Logical PortsUp to 32 logical ports, but can also be configured as 1, 2, 4, 8, 16 logical ports

Number of logical ports controlled by parameter called “Multi-Core Scaling Value”

– Several other configuration parametersAll based on tuning performance to match customer configuration and environment

e.g. Speed, frame size, duplex

Page 17: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

33

IVEAdvantages:

No POWER Hypervisor hits

Does not require a VIO server or hosting LPAR to be running

No configuration required on any VIO servers or hosting LPARs

Removes SW packet forwarding overhead from VIO server or hosting LPAR

Provides equivalent performance as a dedicated Ethernet adapter

Each LPAR owns an ethernet adapter and MAC address

Consideration:

Consider total amount of data and total bandwidth available

PH

YP

Linux i5/OS AIX

EthernetDriver

EthernetDriver

EthernetDriver

IVE

© 2011 IBM Corporation

IBM Power Systems

34

IVE - Integrated Virtual Ethernet

VPD card

2 x 1Gb Eth

Seria l 2

Seria l 1

VPD card

Serial 2

10Gb Eth

10Gb EthBase Offering: #5636

2 Serial, 2 1Gb Eth

10G b Upgrade O ffering: #5637 1 Serial, 2 10Gb Eth

VPD card

4 x 1Gb Eth

Serial 2

4 x 1G b Upgrade O ffering: #56391 Seria l, 4 1Gb Eth

The feature code number is dependant on the Machine type. These are for a 9117-MMA

Page 18: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

35

A config view of HEA (quad 1-Gb)

Resources

Port Groups

Physical Ports

Logical Ports (LPorts)

4 - Physical ports

2 - Port groups

32 - logical ports (max)

with a MCS value of 1Recommended Value

© 2011 IBM Corporation

IBM Power Systems

36

Dual 10-Gb

2 - Physical ports

2 - Port groups

32 - logical ports (max)

with a MCS value of 1

Resources

Logical Ports (LPorts)

Port Groups

Physical PortPhysical Port

Page 19: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

37

Dual 1-Gb

2 - Physical ports

1 - Port groups

16 - logical ports (max)

with a MCS value of 1Recommended Value

Resources

Physical Ports

Logical Ports (LPorts)

Port Group

© 2011 IBM Corporation

IBM Power Systems

38

Virtualization: HEA Logical Port Concept

Logical L2 switch

Physical Port

Logical Ports

Partition PartitionPartition

HEA

To a LPAR, a HEA logical port appears as a generic Ethernet interface

– With its own resources and MAC address

– Sharing bandwidth w/ other logical ports defined on same physical port

– OS sees the HEA Logical port as just another ethernet adapter and may be used exactly like any other ethernet adapter

Page 20: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

39

Logical Port to Physical Port Mapping

Logical Ports (LPorts)LPAR1 LPAR1 LPAR2 LPAR3

LPAR4 LPAR4 LPAR5 LPAR6

LPAR7 LPAR8 LPAR9 LPAR9

Physical Port Physical Port

•Logical ports are allocated to partitions•Each Logical Port can be owned by a separate LPAR•A Partition can own multiple Logical Ports•Only one Logical Port per Physical Port per partition•When a Logical port is assigned to an LPAR, it is also associated with a physical port in the port group

•One Logical Layer 2 switch per Physical port•Physical port looks like an “uplink” to the rest of the network from the port group

Port Group

© 2011 IBM Corporation

IBM Power Systems

40

Host Ethernet Adapter (HEA) - considerations

Up to one logical port per physical HEA port on each LPARPartition mobility of a partition with a directly configured HEA logical port is not supportedHEA devices consume more system memory than other Ethernet devices LHEA (l-hea) is the parent device of an LHEA Port (lp-hea) (AIX)An LHEA can contain 1-4 LHEA Ports Dependent on the type of daughter card used

V5R4 – Default - Logical ports report as available to the LPARS regardless of physical port status– As a result, Virtual IP fault tolerance using Proxy ARP does not work.

IBM i 6.1 - Default - Logical ports report physical port status, behavior can be changed

APAR - MA36089PTF List:

– Release 545 : MF44862 available 08/06/03 (8183)– Release 610 : MF44073 available 08/05/19 (8127)

Licensed internal code has been changed to allow the physical link state to be used for Host Ethernet Adapter ports. Contact support for assistance in changing this behavior.

Page 21: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

41

Added in V6R1

New routing protocol supported

– The i5/OS® operating system has been extended to support the Open Shortest Path First (OSPF) routing protocol. Open Shortest Path First (OSPF) is a link-state routing protocol in which routers or systems within the same area maintain an identical link-state database that describes the topology of the area.

Virtual IP enhancements that affect the TCP/IP routing and workload balancing topic collection are as follows:

– * Virtual IP address support has been extended to include IPv6 addresses.

– * A Point-to-Point Protocol (PPP) interface or a Layer Two Tunneling Protocol (L2TP) interface can use a virtual IP address as the local IP address to provide fault tolerance for remote connections.

– * You can configure virtual IP Proxy ARP while the virtual IP interface is active.

http://publib.boulder.ibm.com/infocenter/systems/scope/i5os/topic/rzajw/rzajwwhatnew.htm?tocNode=int_217851

© 2011 IBM Corporation

IBM Power Systems

42

TCP/IP Configuration

New alias names for IP interfaces

– CL commands (i.e. STRTCPIFC) can use name instead of IP addressADDTCPIFC ALIASNAME(LABNET1)STRTCPIFC ALIASNAME(LABNET1)ENDTCPIFC ALIASNAME(LABNET1)CHGTCPIFC ALIASNAME(LABNET1)RMVTCPIFC ALIASNAME(LABNET1)

Work with TCP/IP Interfaces System: RCHASM27

Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 9=Start 10=End

Internet Subnet Interface Alias Opt Address Mask Status Name

172.5.92.48 255.255.255.128 Active PROD 10.1.1.1 255.255.255.0 Active LABNET1 10.1.1.2 255.255.255.0 Active LABNET2 10.1.1.3 255.255.255.0 Active *NONE 10.1.1.50 255.255.255.255 Inactive VIPA1

127.0.0.1 255.0.0.0 Active LOCALHOST

Page 22: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

43

Notes:

A new parameter was added to TCP/IP interfaces in V5R4. The parameter Alias Name provides administrators with an option to define a name for an IP address. i5/OS commands and interfaces also support the alias name. For example, interfaces can be started or stopped via a name rather than an IP address. This is especially useful when dealing with IPv6 interface addresses.

The Convert Interface ID (QtocCvtIfcID) API can be used to retrieve the IP address of an interface when given the name or the name of an interface when given the IP address.

© 2011 IBM Corporation

IBM Power Systems

44

Typical Solutions

Page 23: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

45

Fault Tolerance using *VirtualIP

Interface 10.1.1.1 fails:

Any connections to 10.1.1.1 are lost, connections to 10.1.1.2, 10.1.1.3 remain active.

But connections to 10.2.1.1, the *Virtual IP address, remain active , system stays available

Use *VirtualIP to provide continuous availability even through an interface failure

ƒ What if, instead of an external router, an interface adapter fails?

ƒ Unbound routes automatically switched to active interface. (Routes explicitly bound to interface not moved)

ƒ But IP address of failed interface is still unavailable -- "system still appears down"

ƒ Solution: Use a "Virtual IP" address as the primary system address to which external users connect

Primary IP address of system remains active as long as system is active

System stays accessible so long as at least one physical interface remains active

10.1.1.3

10.1.1.2

10.1.1.1

10.1.1.x

R1

R2

*VirtualIP10.2.1.1

DNS Entry10.2.1.1

Internet

© 2011 IBM Corporation

IBM Power Systems

46

This chart demonstrates a powerful use of *VirtualIP addresses. Here, we define a *Virtual IP address as the primary address for the system. In the DNS, only the *VirtualIP address is defined. All external users access the system via the 10.2.1.1 *VirtualIP address.

If any of the local interfaces fail, the system remains accessible so long as at least one interface remains active. Connections can be transparently re-routed through any of the available interfaces as needed. The advantage of this is that because a *VirtualIP address is not tied to a hardware adapter, it remains active so long as TCP/IP is active.

Page 24: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

47

Fault Tolerance using *VirtualIP - V5R2

Interface 10.1.1.11 fails:

Any connections to 10.1.1.11 are lost, connections to 10.1.1.12, 10.1.1.13 remain active.

But connections to 10.1.1.1, the *Virtual IP address, remain active , system stays available

System TCP/IP moves the APR response to a new working adapter

Use *VirtualIP to provide continuous availability even through an interface failure

ƒ What if, instead of an external router, an interface adapter fails?

ƒ Unbound routes automatically switched to active interface. (Routes explicitly bound to interface not moved)

ƒ But IP address of failed interface is still unavailable -- "system still appears down"

ƒ Solution: Use a "Virtual IP" address as the primary system address to which external users connect

Primary IP address of system remains active as long as system is active

System stays accessible so long as at least one physical interface remains active

10.1.1.13

10.1.1.12

10.1.1.11

10.1.1.x

R1

R2

*VirtualIP10.1.1.1

DNS Entry10.1.1.1

Internet

© 2011 IBM Corporation

IBM Power Systems

48

IP Address Takeover using *VirtualIP

AS1 is taken down:

IP Address Takeover inactivates 10.2.1.1 *VirtualIP interface on AS1 and activates equivalent interface on AS3

RouteD on AS3 advertises that it can now reach 10.2.1.1

After route change is propagated, all traffic to 10.2.1.1 should be directed to AS3

ƒ What if entire system is taken down?

ƒ V4R4: IP Address Takeover -> Switch primary server address to physically different machine

If backup machine is on the same network, route switchover is automatic (via ARP)

But backup machine can even be on a totally different network:

Define Primary server address as a *VirtualIP interfaceWith RIPV2, movement of *VirtualIP address is advertised throughout the network

Note: Also requires V4R4 Clustering product be installed

10.1.1.xR1

*VirtualIP10.2.1.1

10.1.1.1

10.1.2.xR2

10.1.2.4

*VirtualIP10.2.1.1

AS1

AS3

10.1.1.3

AS2

Page 25: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

49

Finally, *VirtualIP addresses can improve system availability when used in conjunction with the V4R4 Clustering product. The Clustering application controls on which system is the *VirtualIP address active at any point in time. When that system is taken down, the same *VirtualIP address is activated on a backup system.

If the backup system is connected to the same network as the primary system, no special routing procedures are required. Consider AS1 as the primary system and AS2 the backup. When the takeover IP address comes active on AS2, it will broadcast an ARP packet to the rest of the local network, informing all other hosts that the IP address has moved to a new system,

But IP address takeover is not limited to both machines being on the same network. All we need is to define the takeover address as an address that is not directly accessible from either of the local networks -- in other words, a *VirtualIP address.

For example, consider the backup system being AS3, rather than AS2. In this case, we need to define the takeover address as a *VirtualIP address that is not part of either of the local networks to which the AS/400's are attached. That is why. on the previous page, the *VirtualIP address is defined as 10.2.1.1. This address is not part of either the 10.1.1.x or the 10.1.2.x networks.

When the 10.2.1.1 takeover address is moved from AS1 to AS3, RIPv2 will advertise to the rest of the network that 10.2.1.1 is now reachable by AS3. Assuming the intermediate routers are also running RIPv2, within a few minutes, the route tables throughout the rest of the network will be updated.

© 2011 IBM Corporation

IBM Power Systems

50

RouterX

Firewall10.1.1.1

10.1.1.2 10.1.2.1

Internet

Rest of the 10.0.0.0 Corporate Network

A 10.1.1.11*VIRTUALIP

10.250.250.1 SYSNAME10.250.250.2 HTTPSVR110.250.250.3 HTTPSVR210.250.250.11 DOM110.250.250.12 DOM210.250.250.13 DOM3

.

.

.10.250.250.20 DOM20

Router XRoute Directives

Next HopDestination Subnet Mask Gateway10.250.250.0 255.255.255.0 10.1.1.11

Multiple TCP/IP Servers using Virtual IP (separate network range)

10.1.1.101 Local PC

In this scenario we are hosting many servers on a single iSeries. Some of these are HTTP servers and some are Domino servers. We need multiple addresses so that each server can bind to a unique address and be accessed by the well known ports for the service it is providing (80, 443, and 1352 for Domino; 80 and 443 for HTTP). The traditional way to define multiple TCP/IP on the iSeries is to create multiple TCP/IP interfaces on the same line description. The applications will bind to the Virtual IP address and therefore will not notice if the physical adapter fails. This may prevent the restart of some servers.The benefits of using Virtual IP are not fully realized in this configuration because there is only one physical LAN adapter in this configuration.

Page 26: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

51

Building the configuration1Get an address for the physical LAN adapter (if it is not already created)

In our example we use 10.1.1.112Select a subnet to use for the Virtual IP addresses

This range of addresses should not be in use anywhere else in the network.In our example we use 10.250.250.x

3Create the Line Description for the LAN adapterCRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)

4Define the IP interface for the LAN adapterADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')

5Add the corporate and Internet route entries to the TCP/IP route tableADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2')ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1')

6Define the Virtual IP addresses (repeat as needed)ADDTCPIFC INTNETADR('10.250.250.1') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388) ADDTCPIFC INTNETADR('10.250.250.20') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388)

7Add route entries to the routers, firewall, and systems that need to point to the subnet 10.250.250.0Tell the network administrator that the iSeries looks like a router for that subnetIf the routers etc. support RIP2 you can start ROUTED server and let the system broadcast the net route8Add entries to your DNS to point to the virtual IP addresses9Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for

each serverNOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition

• Start the servers and test.

© 2011 IBM Corporation

IBM Power Systems

52

RouterX

Firewall10.1.1.1

10.1.1.2 10.1.2.1

Internet

Rest of the 10.0.0.0 Corporate Network

A 10.1.1.11*VIRTUALIP

10.1.1.21 SYSNAME10.1.1.22 HTTPSVR110.1.1.23 HTTPSVR210.1.1.24 DOM110.1.1.25 DOM210.1.1.26 DOM3

.

.

.10.1.1.43 DOM20

Multiple TCP/IP Servers using Virtual IP (with proxy ARP same network address)

10.1.1.101 Local PC

In this scenario we are hosting many servers on a single iSeries. Some of these are HTTP servers and some are Domino servers. We need multiple addresses so that each server can bind to a unique address and be accessed by the well known ports for the service it is providing (80, 443, and 1352 for Domino; 80 and 443 for HTTP). The traditional way to define multiple TCP/IP on the iSeries is to create multiple TCP/IP interfaces on the same line description. The applications will bind to the Virtual IP address and therefore will not notice if the physical adapter fails. This may prevent the restart of some servers.The benefits of using Virtual IP are not fully realized in this configuration because there is only one physical LAN adapter in this configuration. In this case we have used IP addresses in the same network as the real network and checked the proxy APR check box so that all the addresses will reply to APR request using the MAC address of the 10.1.1.11 adapter

No additional routes needed in the router configuration

Page 27: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

53

Building the configuration1Get an address for the physical LAN adapter (if it is not already created)

In our example we use 10.1.1.112Get addresses to use as virtual IP addresses. The addresses do NOT need to be consecutive. In our

example however they are.2Check the network equipment to make sure that the switches etc. support multiple IP addresses on a

single port.3Create the Line Description for the LAN adapter

CRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)4Define the IP interface for the LAN adapter

ADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')5Add the corporate and Internet route entries to the TCP/IP route table

ADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2')ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1')

6Define the Virtual IP addresses. For proxy APR you must use iSeries Navigator (repeat as needed)Refer back to page 19 for an example.

7No additional routes are needed in the routers or firewalls. 8Add entries to your DNS to point to the virtual IP addresses9Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for

each serverNOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition

• Start the servers and test.

© 2011 IBM Corporation

IBM Power Systems

54

BIND to an IP address - HTTP Servers

http://hostname:2001HTTP Config and AdminOriginal HTTP Server

ConfigurationBasic Settings

Select Bind server to host address

Apache ServerConfigurationGeneral Settings

Click Add and type IP addressSelect All and click Remove

Page 28: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

55

BIND to an IP address - Domino Servers

notes.ini

Server Document

© 2011 IBM Corporation

IBM Power Systems

56

Router

X

Firewall10.1.1.1

10.1.1.2 10.1.2.1

Internet

Rest of the 10.0.0.0 Corporate

Network

A

10.1.1.12

10.1.1.13

10.1.1.11

10.1.1.14

10.1.1.15

*VIRTUALIP10.250.250.1 SYSNAME10.250.250.2 HTTPSVR110.250.250.2 HTTPSVR210.250.250.11 DOM110.250.250.12 DOM210.250.250.13 DOM3

.

.

.10.250.250.20 DOM20

Router XRoute Directives

Next HopDestination Subnet Mask Gateway

10.250.250.0 255.255.255.0 10.1.1.1110.250.250.0 255.255.255.0 10.1.1.12

B

C

D

E

Interface A and B are setup as primary input interfaces. Interface C, D, and E are setup for output connection balancing. As connections are made TCP/IP will round-robin between C, D, and E. If all three of these become unavailable the TCP/IP will move to the next lower Priority (6) and use A and B for output as well as input.The router directives are set up to round-robin between interface A and B. This is a function of the router. Most routers will provide this type of support. In this example the load on each interface is NOT considered. The assumption is that the traffic load is similar for all connections.

TCP/IP inbound and outbound balancing with Virtual IP

OS/400 TCP/IP Route Entries

Preferred Dup.Route Subnet Binding Route

Dest. Mask Next Hop Interface Pri.10.1.1.0 255.255.255.0 10.1.1.11 10.1.1.11 6

10.1.1.0 255.255.255.0 10.1.1.12 10.1.1.12 610.1.1.0 255.255.255.0 10.1.1.13 10.1.1.13 710.1.1.0 255.255.255.0 10.1.1.14 10.1.1.14 7

10.1.1.0 255.255.255.0 10.1.1.15 10.1.1.15 7

10.0.0.0 255.0.0.0 10.1.1.2 10.1.1.11 610.0.0.0 255.0.0.0 10.1.1.2 10.1.1.12 6

10.0.0.0 255.0.0.0 10.1.1.2 10.1.1.13 710.0.0.0 255.0.0.0 10.1.1.2 10.1.1.14 710.0.0.0 255.0.0.0 10.1.1.2 10.1.1.15 7

*dftroute *none 10.1.1.1 10.1.1.11 6

*dftroute *none 10.1.1.1 10.1.1.12 6*dftroute *none 10.1.1.1 10.1.1.13 7*dftroute *none 10.1.1.1 10.1.1.14 7

*dftroute *none 10.1.1.1 10.1.1.15 7

X

Y

Z

10.1.1.101 Local PC

Page 29: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

57

Balancing example explained

This example takes full advantage of using virtual IP addresses. In addition to providing each application a unique address to bind to, it also provides support for :

–Inbound connection balancing–Outbound connection balancing–Some level of fault tolerance

Inbound connection balancing is provided by:–Virtual IP addresses defined on the system–External router, firewall, and/or switch with L3 routing built in

Outbound connection balancing is provided by the preferred binding interface and duplicate route priority parameters on the OS/400 TCP/IP route entries. The connection balance will round-robin between all the interfaces at the same duplicate route priority when this value is set greater than 5. If all the interfaces at one value become unavailable (7 in our example) the system will switch to a set at the next lower value (6 in our example). The entries are split into three groups in the example.

–Group X - provides connection balancing to the local segment of the LAN–Group Y - provides connection balancing to the rest of the corporate network using the router–Group Z - provides connection balancing to the Internet using the firewall

Fault tolerance:–When a connection such as TELNET is established it will occur between the remote host and the virtual IP address. The session on the iSeries will be bound to the virtual IP address. If the physical interface drops the session will stay active. The system will reroute the traffic over another existing outbound interface. The router will also reroute the traffic to a different interface. So long as the iSeries and the router can move the traffic the session will remain active.

© 2011 IBM Corporation

IBM Power Systems

58

1 Get an addressesfor the physical LAN adapters (if it is not already created)In our example we use 10.1.1.11 - 10.1.1.15

2 Select a subnet to use for the Virtual IP addressesThis range of addresses should not be in use anywhere else in the network.In our example we use 10.250.250.x

3 Create the Line Description for the LAN adapter for each adapterCRTLINETH LIND(ETHLAN1) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)

4 Define an IP interface for each LAN adapterADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')

5 Add the local, corporate and Internet route entries to the TCP/IP route table for each interface (see table in example)ADDTCPRTE RTEDEST('10.1.1.0') SUBNETMASK('255.255.255.0') NEXTHOP('10.1.1.11') BINDIFC('10.1.1.11') DUPRTEPTY(6ADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2') BINDIFC('10.1.1.11') DUPRTEPTY(6)ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1') BINDIFC('10.1.1.11') DUPRTEPTY(6)

6 Define the Virtual IP addresses (repeat as needed)ADDTCPIFC INTNETADR('10.250.250.1') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388) ADDTCPIFC INTNETADR('10.250.250.20') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388)

7 Add route entries to the routers, firewall, and systems that need to point to the subnet 10.250.250.0Tell the network administrator that the iSeries looks like a router for that subnetIf the routers etc. support RIP2 you can start ROUTED server and let the system broadcast the net route8 Add entries to your DNS to point to the virtual IP addresses9 Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for each server

NOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition

• Start the servers and test.

Creating the example

Page 30: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

59

Inbound load balancing revisited

•Can now limit proxy agent to the desired set of physical interfaces

Physical IP Address

10.1.1.11

10.1.1.12

Virtual IP Address

10.250.250.1

VIPA Preferred Interface List10.250.250.1 1) 10.1.1.11

2) 10.1.1.12

10.1.1.13

10.1.1.14

10.1.1.15

© 2011 IBM Corporation

IBM Power Systems

60

TCP/IP and LPAR

Page 31: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

61

The advent of LPAR provided yet another environment to apply the same routing concepts as previously discussed.

With LPAR, a single AS/400 is logically partitioned in multiple virtual machines. Each partition has its own address space its own instance of TCP/IP, and may have its own dedicated I/O adapters. To TCP/IP, each partition appears like a distinct AS/400

Moreover, TCP/IP communication between the different partitions is done via a virtual opticonnect bus or a virtual ethernet LAN (V5R1). The TCP/IP routing code sees the path to another LPAR partition no differently than the path to another system connected via a physical opticonnect bus. All of the concepts and configurations that were previously described for "TCP/IP over Opticonnect" environments apply equally well to "TCP/IP with LPAR"

© 2011 IBM Corporation

IBM Power Systems

62

LPAR Scenario - Using Virtual Ethernet LAN

10.1.1.1

10.1.1.10 10.1.1.11

10.1.1.12

i5/OSPar ID 1

LinuxPar ID 2

Linux2Par ID 3

Virtual LAN 1

Virtual LAN 5

Requirments:One physical LAN adapterThree partitions

–i5/OS–Linux–Linux

All Partitions connected on LAN 1Partition 1 and 3 connected on LAN 5

subnet mask 255.255.255.0

addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240

addrs - 10.1.1.233 - 10.1.1.238subnet mask 255.255.255.248

Page 32: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

63

Steps to Implement - After Partitioning is complete

1 Connect virtual LANs to the correct partitions2 Define physical LAN adapter (CRTLINETH)3 Define TCP/IP Interface over real LAN adapter4 Set IP Forwarding to *YES

5 Determine TCP/IP addresses to useSelect a contiguous range that is a natural subnet

6 Define TCP/IP Interfaces over virtual Ethernet adapters7 Vary on lines and start TCP/IP interfaces8 Build LAN configuration in other partitions• Test connectivity

© 2011 IBM Corporation

IBM Power Systems

64

Setting up Virtual Ethernet LAN (HMC LVL 6)

HMC is used to create, modify, and monitor logical partitions and their resources.

Select the virtual adapter type

Click Create

Page 33: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

65

Setting up Virtual Ethernet LAN (HMC LVL 6)

HMC is used to create, modify, and monitor logical partitions and their resources.

Slot in the virtual partition

VLAN id – all partitions on the VLAN are on the same segment of the network and can communicate

© 2011 IBM Corporation

IBM Power Systems

66

Display Resource Detail System: ATSI5P1

Resource name . . . . . . . : CMN09 Text . . . . . . . . . . . . : Ethernet Port Type-model . . . . . . . . . : 268C-001 Serial number . . . . . . . : 00-00000 Part number . . . . . . . . :

Location: U9406.520.10A965C-V1-C2-T1

Logical address: SPD bus: System bus 255 System board 0 System card 0 Communications:

More... Press Enter to continue.

F3=Exit F5=Refresh F6=Print F12=Cancel Display Resource Detail System: ATSI5P1

Resource name . . . . . . . : CMN28 Text . . . . . . . . . . . . : Ethernet Port Type-model . . . . . . . . . : 268C-001 Serial number . . . . . . . : 00-00000 Part number . . . . . . . . :

Location: U9406.520.10A965C-V1-C5-T1

Logical address: SPD bus: System bus 255 System board 0 System card 0 Communications: More...

Press Enter to continue.

F3=Exit F5=Refresh F6=Print F12=Cancel

Find the Resource - i5

Use Slot ID to find Virtual LAN ID

WRKHDWRSC *CMN

Page 34: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

67

ADDTCPIFC INTNETADR('10.1.1.1') LIND(ETHLAN) SUBNETMASK('255.255.255.0')

CRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*AUTO) DUPLEX(*AUTO)

Add the TCP/IP Interface to the real LAN

Set IP forwarding to *YES

© 2011 IBM Corporation

IBM Power Systems

68

Determine TCP/IP addresses to use

Once the TCP/IP interface has been created the interface should be varied on.

Proxy ARP basically consists of the establishment of a sub-network within the larger network that the i5/OS physical interface is connected to.

The IP address of the virtual network connection along with the subnet mask determines the network range that i5/OS will proxy for

A subnet-calculator can be useful in helping to determine the address range.

The TCP/IP attribute Datagram Forwarding has to be set to *Yes to allow network packets to flow between the two network interfaces.

Page 35: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

69

ADDTCPIFC INTNETADR('10.1.1.241') LIND(VIRTLAN) SUBNETMASK('255.255.255.240') LCLIFC('10.1.1.1')

CRTLINETH LIND(VIRTLAN) RSRCNAME(CMN11) LINESPEED(1G) DUPLEX(*FULL) TEXT('Virtual Ethernet to Other Partitions')

Add a TCP/IP Interface to the Virtual LAN

© 2011 IBM Corporation

IBM Power Systems

70

References

• http://www.redbooks.ibm.com/

• SG24-5190: " V4 TCP/IP for AS/400: More Cool Things Than Ever"

• SG24-6718 - iSeries IP Networks: Dynamic

• GC24-3376: "TCP/IP Tutorial Technical Overview",

•Chapter 11-- Availability, Scalability and Load Balancing

• SG24-5147: " AS/400 Autoconfiguration: DNS and DHCP Support

•Section 15.2: Transparent Subnet Masking

• SG24-6232: "Linux on iSeries: An Implementation Guide"

• http://www.as400.ibm.com/infocenter/

• Networking--> TCP/IP -->TCP/IP routing and workload balancing.

• IBM Network Dispatcher:

•http://www.software.ibm.com/network/dispatcher/

Page 36: IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate

© 2011 IBM Corporation

IBM Power Systems

71

8 IBM Corporation 1994-2008. All rights reserved.References in this document to IBM products or services do not imply that IBM intends to make them available in every country.

Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at http://www.ibm.com/legal/copytrade.shtml.

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registeredtrademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce.ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.UNIX is a registered trademark of The Open Group in the United States and other countries.Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.

Information is provided "AS IS" without warranty of any kind.

The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.

Prices are suggested U.S. list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Photographs shown may be engineering prototypes. Changes may be incorporated in production models.

Trademarks and Disclaimers