IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for...
Transcript of IBM Power Systems Virtual IP - Load Balancing, Fault ... Presentations/Fant...discuss techniques for...
© 2011 IBM Corporation
IBM Power Systems
Virtual IP - Load Balancing, Fault Tolerance, & IOA SharingSession ID: VT 445-4
Fant Steele [email protected] I/T Specialist – IBM Lab Services
© 2011 IBM Corporation
IBM Power Systems
2
Description and Objectives
This session covers the many uses of virtual networking available in TCP/IP on System i. Instructions for defining and using Virtual IP Addresses (VIPA) will be provided. We will also discuss techniques for providing load balancing and how to use virtual ethernet networks to communicate with LPARS.
Objectives
– Explain how Virtual IP Addresses (VIPA) may be used on the system to provide fault tolerance and application isolation
– Describe the techniques used to flow traffic between physical and virtual ethernetnetworks
– Describe load balancing using the built in functions of TCP/IP is i5/OS
© 2011 IBM Corporation
IBM Power Systems
3
Agenda
What is Virtual IPWhat components are used to make virtual IP work–IP forwarding–Direct Routing–Transparent subnets and Proxy ARP–Network Address Translation (NAT)–Virtual IP address–Schowler Routes–Virtual Ethernet adapter–Integrated Virtual Ethernet (IVE / HEA)
Typical solutions–Multiple HTTP servers–Multiple Domino servers–Load Balancing across multiple physical adapters–Reducing points of failure to increase availability–Connecting between multiple logical partitions
© 2011 IBM Corporation
IBM Power Systems
4
What is Virtual IP ?
Virtual IP can be one or more of several components available inOS/400. These include:–Virtual IP addresses (V4R3)–TCP/IP over Virtual OptiConnect (V4R4)–Virtual Ethernet LAN adapters (V5R1)
These components are implemented by software and imitate hardware.They can be used to supplement Real physical LAN adapters on the system.Takes advantage of the fact that the iSeries and OS/400 TCP/IP implements the Weak Multi-homing model as per RFC1122
"Weak Multi-homing model: The adapter on which a packet is received is irrelevant
© 2011 IBM Corporation
IBM Power Systems
5
Components
© 2011 IBM Corporation
IBM Power Systems
6
If IP datagram forwarding is set to *Yes then IP traffic will be routed through the system based on the route table entries.In this example the traffic between the two PC systems (10.1.1.11 and 10.1.2.10) routes through the iSeries system. The PCs have a route statement that sets the iSeries as their default route
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
10.1.2.10 10.1.2.11
10.1.2.12
10.1.2.1
IP Forwarding
© 2011 IBM Corporation
IBM Power Systems
7
Virtual LAN 1 addrs - 10.1.2.1 - 10.1.2.254subnet mask 255.255.255.0
Route statements must be placed in the network routers– Static routes defined by hand– Advertised routes via RIP or others protocols
Routes point to the real adapter addressDNS entries point to the virtual addressesThe route table is then checked and the datagram is forwarded to the destinationAll traffic is routed through the I/O partition
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
subnet mask 255.255.255.0
Direct Routing
© 2011 IBM Corporation
IBM Power Systems
8
The target address is compared to the routing table using the network mask (subnet mask) of the TCP/IP interfacesAfter determining that the target host should be on a local segment of the network, TCP/IP broadcast an ARP (address resolution protocol) request to find the host adapter that has the IP address assignedIf the adapter with the address is active on the network then it replies with the MAC address assigned to the adapter Communications is then accomplished using the physical communications layer
10 11 12
1 14 TCP/IP Hosts
10.1.1.____
Finding a host on the network (ARP)
© 2011 IBM Corporation
IBM Power Systems
9
Virtual LAN 1
There are several ways to get the traffic from the Virtual LAN Segment to the real network. These include:
– Proxy ARPIP Addresses DO NOT Change
– Network Address TranslationIP Addresses CHANGE
– Direct RoutingIP Addresses DO NOT Change
IP Forwarding must be enabled for any of these to workThe default route in the guest LPARS must point to the virtual IP address in the I/O partition as the gateway
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
subnet mask 255.255.255.0
Example:One physical LAN adapterThree partitions
– I5/OS– Suse Linux– RedHat Linux
All Partitions connected on LAN 1
But what about Virtual Adapters
© 2011 IBM Corporation
IBM Power Systems
10
Virtual LAN 1 addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240
Proxy ARP replies with the MAC address of the physical adapterAll traffic is routed through the proxying partitionDNS entries point to Virtual addressesProxy can be i5/OS, Linux, or AIXVirtual LAN address range must be a subset of the physical LAN addressesVLAN MTU must be =< Physical LAN
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
subnet mask 255.255.255.0
ProxyARP
Proxy ARP will reply to ARP request on the real network for addresses on the virtual segment of the network
Transparent Subnet (Proxy ARP)
© 2011 IBM Corporation
IBM Power Systems
11
Virtual LAN 1 addrs - 10.1.2.1 - 10.1.2.254subnet mask 255.255.255.0
Addresses for each of the LPARS must be defined in the NATing partition
– Additional IP Interface on the REAL adapter– Virtual IP Addresses with proxy to the real adapters
(recommended)NAT LPAR can be i5/OS, Linux, or AIXStatic NAT rules must be created to MAP the public (Physical) address to the private (VLAN) addressDNS entries point to the addresses in the real segmentThe real adapter Replies to the ARP requestThe IP address in the IP header is rewritten (mangled) The route table is then checked and the datagram is forwarded to the destinationAll traffic is routed through the proxying partition
10.1.1.310.1.1.210.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
subnet mask 255.255.255.0
NAT rewrites the IP header and then Forwards the datagram
Public PrivateMap 10.1.1.2 <-> 10.1.2.2Map 10.1.1.3 <-> 10.1.2.3
Network Address Translation (NAT)
© 2011 IBM Corporation
IBM Power Systems
12
Network Address Translation (NAT)
I5/OS implements the following NAT types
Masquerade, or Hide, NAT
–Enables clients in internal network to access public network
Static, or Map, NAT
–Enables systems in the public network to access internal servers–Enables systems in the real LAN to access other logical partitions
Masquerade, or Hide “port-mapped”, NAT
–Enables systems in the public network to access internal servers. Conversation can be initiated from either side
Benefits of NAT
Saves public IP addresses
Transparent to the client
Simplifies routing in the internal network
Efficient (good performance)
© 2011 IBM Corporation
IBM Power Systems
13
The Network Address Translation (NAT) function lets you translate internal IP addresses to external IP addresses.NAT is based on the fact that only a small number of the hosts in a private network are communicating outside of that network. If each host is assigned an IP address from the registered IP address pool only when they need to communicate, then only a small number of global addresses are required. NAT might be a solution for networks that have private address ranges or illegal addresses and want to communicate with hosts on the Internet. In fact, most of the time, this can also be achieved by implementing a firewall. Hence, clients that communicate with the Internet by using a proxy or SOCKS server do not expose their addresses to the Internet, so their addresses do not have to be translated anyway. However, for any reason, when proxy and SOCKS are not available or do not meet specific requirements, NAT might be used to manage the traffic between the internal and external network without advertising the internal host addresses.The native AS/400 system NAT, supports masquerading and static NAT.
I5/OS NAT implementationThe implementation of NAT on the AS/400 system takes three forms:
Masquerade, or Hide, NATStatic, or Map, NATMasquerade, or Hide “port-mapped”, NAT
Masquerade or Hide NAT, is primarily used to enable clients in your internal network with private IP addresses assigned, to access the public network. This is accomplished by translating the client’s private address (trusted address) to the public address of the AS/400 gateway (border address). Static, or Map, NAT is primarily used to enable systems in the public network to access servers in your internal network by translating the actual internal server address to a public address. This is a one-to-one mapping of IP address. There is no port translation. Masquerade, or Hide “port-mapped NAT, is used primarily to enable systems on the public network to access servers in your internal network. Both, IP address and port are translated. For example you could have an HTTP server on the internal network bound to IP address 10.1.1.1 and port 5000 being accessed from the public network using IP address 204.222.180.5 and port 80. The conversation can be initiated from either network therefore it also enables clients in the internal network to access systems in the public network. For detailed information on AS/400 NAT implementation refer to the article Networking Security - IP packet security at http://www.as400.ibm.com/infocenter and to the IBM redbook V4 TCP/IP for AS/400: More Cool Things Than Ever, SG24-5190.
© 2011 IBM Corporation
IBM Power Systems
14
10.1.1.10 193.20.1.1
1024 55336
Source Addr Dest. Addr. SP DP
10.1.1.10 192.10.1.5 1024 23
Source Addr Dest. Addr. SP DP
192.10.1.5 10.1.1.10 23 1024
Source Addr Dest. Addr. SP DP
193.20.1.1 192.10.1.5 55336 23
Source Addr Dest. Addr. SP DP
192.10.1.5 193.20.1.1 23 55336
Outbound traffic Inbound traffic
10.1.1.10
192.10.1.5
Public Interface
Masquerading Function
193.20.1.1
Dynamic Mapping
Masquerade or Hide NAT
TRUSTED BORDER UNTRUSTED
A private IP address or a range of IP addresses are hidden behind a single public IP address on the AS/400 gateway performing NAT.Only clients in the internal network can initiate the connection which improves security.Translation is done for outgoing packets and incoming packets are translated back and redirected to original destinationInternal port numbers are associated with random port numbers (address and port translation)Single public interface supports multiple simultaneous conversationsThree IP address type TRUSTED, BORDER, UNTRUSTEDCan be configured over leased, LAN, or PPP link
10.1.1.0
LAN or PPP link
Internet
© 2011 IBM Corporation
IBM Power Systems
15
Masquerading is used to allow the private network to hide behind and be represented by the address bound to the public interface of the NAT machine. In most situations, this will be the address that has been assigned by an ISP which may be dynamic in the case of a PPP connection. This type of translation can only be used for connections originating within the private network destined for the outside public network. Each connection out, is maintained by using a different source (client) IP port number.The main characteristics of hide NAT are:
A private IP address or a range of IP addresses are hidden behind a single public IP address on the AS/400 gateway performing NAT.Only clients in the internal network can initiate the connection which improves security.Translation is done for outgoing packets and incoming packets are translated back and redirected to original destination.Internal port numbers are associated with random port numbers. This means that both, the address and the port number are hidden
form the public network.The registered address on the NAT machine is a usable interface outside of NAT.Single public interface supports multiple simultaneous conversations.
Address typesWhen using NAT there are three address types that you must configure in the Defined Addresses rules.
Trusted, used for internal or private addresses. These addresses are hidden from the public network.Untrusted, used for external or public addresses.Border, used for addresses that are public and that form a boundary between trusted and untrusted networks. This is the public address on the AS/400 gateway to
which the internal address or addresses are translated. Figure 10 on page 29 illustrates these concepts.
© 2011 IBM Corporation
IBM Power Systems
16
Virtual IP Address
Powerful tool for load balancing, fault tolerance, unnumbered interface anchor, etc.Can be viewed as "primary" or "external" IP address -- "IP address of the system"Externally accessible local IP address unbound to a single physical interfaceIf TCP/IP is UP then the Virtual IP address should be available to the applications using it VirtualIP interfaces : Not directly routable:
Reachable only via indirect route through "physical IP address" (IP address of physical interface)AS/400 will never answer ARP request to *VIRTUALIP address ****** option in V5R2 ******Allows same *VirtualIP address to exist on multiple hosts
VirtualIP is also supported by other IBM server platforms (AIX, MVS)VirtualIP interfaces advertised by RIPv2May be called Circuitless or Loopback interfaces
CorporateNetwork
10.1.1.x
NetworkDispatcher or Router
10.1.1.11
10.1.1.12
*VirtualIP = 10.250.1.1*VirtualIP = 10.1.1.1 CAN ARP in V5R2
10.1.1.13
Internet
© 2011 IBM Corporation
IBM Power Systems
17
Virtual IP Address EvolutionV5R2 Added:
Proxy ARP automatic agent selection (based on first interface activated)
V5R3 Added:Agent selection based on highest speed available interfaceIf multiple VIPAs being proxied, spread across interfaces
V5R4 Added:Preferred Interface Selection
10.1.1.13
10.1.1.12
10.1.1.11
10.1.1.x
R1
R2
*VirtualIP10.1.1.1 Internet
DNS Entry10.1.1.1
© 2011 IBM Corporation
IBM Power Systems
18
Virtual IP Address
© 2011 IBM Corporation
IBM Power Systems
19
Virtual IP Address
© 2011 IBM Corporation
IBM Power Systems
20
Virtual IP Address
© 2011 IBM Corporation
IBM Power Systems
21
Virtual IP Address (V5R4)
© 2011 IBM Corporation
IBM Power Systems
22
Preferred Interface List for Virtual IP and Virtual Ethernet
Virtual IP Address (VIPA) enhancements introduced in V5R4 to have better control over VIPA proxy ARP agent selection
New preferred interface list available for virtual IP addresses
– is an ordered list of the interface addresses that will take over for the failed adapter
– allows you to manually select which adapters and IP addresses are to be the preferred interface for VIPA proxy ARP agent selection
Chart created by Thomas Barlen
i5/OS
VIPA10.1.1.15 / 32
TB TB
Intranet10.1.1.0 / 255.255.255.0
.11 .10
ProxyARPAgent
.12
© 2011 IBM Corporation
IBM Power Systems
23
i5/OS provides proxy ARP support for VIPAs that are in a non-local subnet
– physical interfaces answer ARP requests for IP addresses that are not in the same subnet as the physical interface address
Provides IP mobility support for local area networks (LANs)
– allows IP address to be moved from a home network to a different network (migration)Cisco’s IOS Local Area Mobility (LAM) feature can exploit this V5R4 enhancement
Proxy ARP on behalf of VIPAs in a different subnet
OriginalRouter(home)
LAMRouter
10.10.10.20 /24
10.10.10.30 /24VIPA
10.10.10.75 / 32
Proxy ARP enabledIGP used
Routing table10.10.10.0 /2410.10.10.75 /32
10.10.10.1 /24
Proxy ARP + LAM enabledIGP used
Routing table10.10.10.0 /2410.10.10.75 /32
10.20.20.1 /24
10.20.20.30 /24
IBM i
Network A Network B
© 2011 IBM Corporation
IBM Power Systems
24
Notes:
In OS/400 V5R2 and i5/OS V5R3, proxy ARP is supported for virtual IP addresses (VIPAs) that are in the same subnet as the IP interface addresses on the physical LAN adapters. For instance, when configured for proxy ARP, an Ethernet IP interface 10.10.10.10 / 24 would answer ARP requests for a VIPA 10.10.10.20 /24, but it would not answer ARP requests for 10.20.20.5 /24.
With i5/OS V5R4, support was added to provide proxy ARP support for VIPAs that are not in the same subnet as local interfaces. This allows, for example, an Ethernet IP interface of 10.10.10.10 / 24 to answerARP requests for a VIPA 10.20.20.5 /24.
This enhancement can be leveraged by a technology that Cisco introduced for mobility in local area networks. The feature that isonly supported by Cisco routers is called Local Area Mobility (LAM) and is part of Cisco’s router operating system IOS.
© 2011 IBM Corporation
IBM Power Systems
25
Notes:
What does LAM do? Let’s explore LAM based on the network diagram that is shown on the previous page. Assume that the iSeries system with its VIPA 10.10.10.75 /24 used to be installed in network A. When a client in network A (10.10.10.0 /24) wanted to communicate with the iSeries system, which was on the same subnet, the client would issue an address resolution protocol (ARP) request to obtain the MAC address for the iSeries interface. At a certain point in time it was necessary to move the iSeries system to a different building. This building had an IP subnet of 10.20.20.0 /24 (network B). However, many applications and clients had hardcoded the iSeries IP address 10.10.10.75, so a quick change of the iSeries IP address was not possible. The answer to this problem is LAM. LAM allows you to implement a migration scenario like the one described above. When LAM is used, the router (LAM router) where the iSeries system is now connected to will be enabled for proxy ARP and LAM (mobility) on the LAN interface. In addition, route redistribution has to be enabled on this router using one of the interior gateway protocols (IGPs), such as Open Shortest Path First (OSPF) or Enhanced Interior Gateway Routing Protocol (EIGRP). The host route for 10.10.10.75 is then distributed to other routers in the network. On the original router in network A, proxy ARP has also to be enabled. The original router’s routing table contains now an entry for network 10.10.10.0 /24 for its LAN interface and a more specific host route to 10.10.10.75 on the external interface. When a client now wants to communicate with 10.10.10.75 (VIPA), the original router answers the ARP request, thus pretending to be the interface the client wants to talk to and routes the packets to the LAM router, which in turn sends the packet to the network B. In order for the LAM router to send a packet to the iSeries, the router first sends an ARP request for 10.10.10.75 to the local network. This is where the V5R4 enhancement comes into the picture. The LAN interface on the iSeries will now answer the ARP request for the VIPAeven though the VIPA is in a different subnet than the LAN IP interface (10.20.20.30).For more information on Cisco’s LAM support, go to the Cisco home page at http://www.cisco.com and search for “Local Area Mobility”.
© 2011 IBM Corporation
IBM Power Systems
26
Schowler Routes
Extends Duplicate Route, Round Robin load balancing to local networksƒ DRRR - Based on "Duplicate Route Priority" and "Preferred Binding Interface" parmsƒ Problem: Neither parameter is available on *DIRECT routesƒ Solution: "Schowler" Routes
–Special indirect route that replaces a *DIRECT route–Same Route Destination, Subnet Mask & TOS as equivalent *DIRECT route–Next Hop and Preferred Binding Interface are set to the IP address of the equivalent local interface–Same local network connectivity as *DIRECT route but allows user to set Duplicate Route Priority and Preferred Binding Interface options for local network load balancing
–Requires PTFs (1Q00) for V4R3 or V4R4, is integrated into V4R5
ƒ Side Benefit: Host routes may be prioritized over *DIRECT routes
10.6.7.1
10.6.7.2
10.6.7.3
10.6.7.x
Private DNS Records:
MyServer 10.6.7.3
Schowler Route Replacements for *DIRECTs:Rte Dest. Subnet Mask Next Hop Preferred IFC Dup Rtr Pri10.6.7.0 255.255.255.0 10.6.7.1 10.6.7.1 610.6.7.0 255.255.255.0 10.6.7.2 10.6.7.2 610.6.7.0 255.255.255.0 10.6.7.3 10.6.7.3 6
Standard *DIRECT Routes:Rte Dest. Subnet Mask Next Hop10.6.7.0 255.255.255.0 *DIRECT10.6.7.0 255.255.255.0 *DIRECT
10.6.7.0 255.255.255.0 *DIRECT
© 2011 IBM Corporation
IBM Power Systems
27
The Duplicate Route, Round Robin method of load balancing that was introduced in V4R2 was oriented towards remotely connected clients. This method of load balancing is based on two indirect route parameters:
- Duplicate Route Priority - Preferred Binding Interface.Configuring multiple duplicate routes with the same priority caused the routes to be selected in a round robin fashion.The problem was that these two parameters were not available for the *DIRECT routes that are automatically added when an interface is added. Thus, this form of load balancing did not work with locally connected hosts."Schowler" routes extend this load balancing capability to locally connected hosts. A Schowler route is functionally equivalent to the *DIRECT route that it replaces, but since it is added just like any other indirect route, the above two load balancing parameters can now be configured by the user. Schowler routes have two special characteristics:
–The same route destination, subnet mask and TOS setting as the equivalent *DIRECT route–The Next Hop and Preferred Binding Interface IP addresses are both set to the IP address of the associated local interface.When the Duplicate Route Priority is set greater than the default of 5, the equivalent Schowler routes are selected in a round robin fashion, identical to what can be done with other indirect routes.In the previous chart, we have 3 interfaces configured, connecting the AS/400 to the 10.6.7.x network, 10.6.7.1, 10.6.7.2 and 10.6.7.3. The first box in the lower left shows the standard *DIRECT routes that are automatically added with the interfaces. However, by adding 3 equivalent Schowler routes, shown in the lower box, the three *DIRECT routes disappear and are replaced by the Schowlers.One final use of Schowler routes is to reverse the default AS/400 TCP/IP routing logic that always prioritizes *DIRECT routes over any indirect routes, even *HOST routes. By replacing the *DIRECT routes with Schowler routes, no "highest priority" *DIRECT routes will be found during route lookup. All candidate routes are now indirect, and prioritized by subnet mask. Thus, a *HOST route, with a subnet mask of 255.255.255.255 will be considered the highest priority route.
© 2011 IBM Corporation
IBM Power Systems
28
Virtual Ethernet Adapter Support
Added in V5R1
Provides 1Gb "Ethernet" LAN connections across the system bus
Used for communications between logical partitions
Included in OS/400 as "no charge" item
One system will support up to 16 virtual LANs
Created when logical partitions are defined
May be configured and maintained using System Service Tools or Operations Navigator
Appear as type 268C adapter in WRKHDWRSC *CMN and are assigned a regular resource name (CMNxx)
Port number of resource matches to LAN id (0 - 15) 4096 in V5R3 with Power5 hypervsor
Linux references the resource as ethnn where nn is 0-15
A single virtual LAN can be used to connect 2 or more partitions
Can also be used to connect to Windows Servers on IXS, IXA and iSCSI
© 2011 IBM Corporation
IBM Power Systems
29
Multiple Proxy ARP Agent Support with Virtual EthernetProxy ARP support on multiple interfaces in an LPARed system with virtual EthernetImproves availability when using transparent subnettingPreferred interface list defines priorities for proxy ARP agent selection V5R4 only – Additional PTFS also needed
– MF41339 5722999– SI27233 5722SS1 (requires endtcp and strtcp to activate)
Chart created by Thomas Barlen
Linuxi5/OS Linux i5/OS
•Web Appl
•Productionserver
TB TB
•File Server DevelopmentPartition
Virtual I/O
Intranet10.1.1.0 / 255.255.255.0
.10 .11 .12
10.1.1.32 / 255.255.255.248
.33 .34 .35 .36
ProxyARPAgent
Preferred Interface List
10.1.1.33
10.1.1.1010.1.1.1210.1.1.11
© 2011 IBM Corporation
IBM Power Systems
30
Virtual LAN 1 addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240
Provides fault tolerant proxy for virtual ethernetpartitionsReplaces use of associated local interface
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
ProxyARP
Proxy ARP will reply to ARP request on the real network for addresses on the virtual segment of the network
Preferred Interface List for Virtual Ethernet (transparent subnetting)
10.1.1.2
© 2011 IBM Corporation
IBM Power Systems
31
Notes:
Proxy ARP replies with the MAC address of the physical adapter
All traffic is routed through the proxying partition
DNS entries point to Virtual addresses
Proxy can be i5/OS, Linux, or AIX (i5/OS in this example to provide fault tolerance)
Virtual LAN address range must be a subset of the physical LAN addresses
VLAN MTU must be =< Physical LAN
© 2011 IBM Corporation
IBM Power Systems
32
What’s in a Name – Integrated Virtual Ethernet (aka HEA)Integrated Virtual Ethernet (IVE) – External name in documentation
Host Ethernet Adapter (HEA) – Name used on user interfaces
New Hardware capability - Built into GX+ bus (P5IOC2) on most p6 systems
– Provides accelerated Ethernet connectivityEssentially, a system with a HEA has several integrated Ethernet adapters, called logical
ports. IVE can be used by multiple partitions.
– Integrated on most POWER6 systems
– Several variations of physical, external ports Dual 1 Gbit copper: supporting 10BASE-T, 100BASE-T, 1000BASE-TQuad 1 Gbit copper: supporting 10BASE-T, 100BASE-T, 1000BASE-T
Dual 10 Gbit fiber: supporting 10GBASE-SR or 10GBASE-LR
– Logical PortsUp to 32 logical ports, but can also be configured as 1, 2, 4, 8, 16 logical ports
Number of logical ports controlled by parameter called “Multi-Core Scaling Value”
– Several other configuration parametersAll based on tuning performance to match customer configuration and environment
e.g. Speed, frame size, duplex
© 2011 IBM Corporation
IBM Power Systems
33
IVEAdvantages:
No POWER Hypervisor hits
Does not require a VIO server or hosting LPAR to be running
No configuration required on any VIO servers or hosting LPARs
Removes SW packet forwarding overhead from VIO server or hosting LPAR
Provides equivalent performance as a dedicated Ethernet adapter
Each LPAR owns an ethernet adapter and MAC address
Consideration:
Consider total amount of data and total bandwidth available
PH
YP
Linux i5/OS AIX
EthernetDriver
EthernetDriver
EthernetDriver
IVE
© 2011 IBM Corporation
IBM Power Systems
34
IVE - Integrated Virtual Ethernet
VPD card
2 x 1Gb Eth
Seria l 2
Seria l 1
VPD card
Serial 2
10Gb Eth
10Gb EthBase Offering: #5636
2 Serial, 2 1Gb Eth
10G b Upgrade O ffering: #5637 1 Serial, 2 10Gb Eth
VPD card
4 x 1Gb Eth
Serial 2
4 x 1G b Upgrade O ffering: #56391 Seria l, 4 1Gb Eth
The feature code number is dependant on the Machine type. These are for a 9117-MMA
© 2011 IBM Corporation
IBM Power Systems
35
A config view of HEA (quad 1-Gb)
Resources
Port Groups
Physical Ports
Logical Ports (LPorts)
4 - Physical ports
2 - Port groups
32 - logical ports (max)
with a MCS value of 1Recommended Value
© 2011 IBM Corporation
IBM Power Systems
36
Dual 10-Gb
2 - Physical ports
2 - Port groups
32 - logical ports (max)
with a MCS value of 1
Resources
Logical Ports (LPorts)
Port Groups
Physical PortPhysical Port
© 2011 IBM Corporation
IBM Power Systems
37
Dual 1-Gb
2 - Physical ports
1 - Port groups
16 - logical ports (max)
with a MCS value of 1Recommended Value
Resources
Physical Ports
Logical Ports (LPorts)
Port Group
© 2011 IBM Corporation
IBM Power Systems
38
Virtualization: HEA Logical Port Concept
Logical L2 switch
Physical Port
Logical Ports
Partition PartitionPartition
HEA
To a LPAR, a HEA logical port appears as a generic Ethernet interface
– With its own resources and MAC address
– Sharing bandwidth w/ other logical ports defined on same physical port
– OS sees the HEA Logical port as just another ethernet adapter and may be used exactly like any other ethernet adapter
© 2011 IBM Corporation
IBM Power Systems
39
Logical Port to Physical Port Mapping
Logical Ports (LPorts)LPAR1 LPAR1 LPAR2 LPAR3
LPAR4 LPAR4 LPAR5 LPAR6
LPAR7 LPAR8 LPAR9 LPAR9
Physical Port Physical Port
•Logical ports are allocated to partitions•Each Logical Port can be owned by a separate LPAR•A Partition can own multiple Logical Ports•Only one Logical Port per Physical Port per partition•When a Logical port is assigned to an LPAR, it is also associated with a physical port in the port group
•One Logical Layer 2 switch per Physical port•Physical port looks like an “uplink” to the rest of the network from the port group
Port Group
© 2011 IBM Corporation
IBM Power Systems
40
Host Ethernet Adapter (HEA) - considerations
Up to one logical port per physical HEA port on each LPARPartition mobility of a partition with a directly configured HEA logical port is not supportedHEA devices consume more system memory than other Ethernet devices LHEA (l-hea) is the parent device of an LHEA Port (lp-hea) (AIX)An LHEA can contain 1-4 LHEA Ports Dependent on the type of daughter card used
V5R4 – Default - Logical ports report as available to the LPARS regardless of physical port status– As a result, Virtual IP fault tolerance using Proxy ARP does not work.
IBM i 6.1 - Default - Logical ports report physical port status, behavior can be changed
APAR - MA36089PTF List:
– Release 545 : MF44862 available 08/06/03 (8183)– Release 610 : MF44073 available 08/05/19 (8127)
Licensed internal code has been changed to allow the physical link state to be used for Host Ethernet Adapter ports. Contact support for assistance in changing this behavior.
© 2011 IBM Corporation
IBM Power Systems
41
Added in V6R1
New routing protocol supported
– The i5/OS® operating system has been extended to support the Open Shortest Path First (OSPF) routing protocol. Open Shortest Path First (OSPF) is a link-state routing protocol in which routers or systems within the same area maintain an identical link-state database that describes the topology of the area.
Virtual IP enhancements that affect the TCP/IP routing and workload balancing topic collection are as follows:
– * Virtual IP address support has been extended to include IPv6 addresses.
– * A Point-to-Point Protocol (PPP) interface or a Layer Two Tunneling Protocol (L2TP) interface can use a virtual IP address as the local IP address to provide fault tolerance for remote connections.
– * You can configure virtual IP Proxy ARP while the virtual IP interface is active.
http://publib.boulder.ibm.com/infocenter/systems/scope/i5os/topic/rzajw/rzajwwhatnew.htm?tocNode=int_217851
© 2011 IBM Corporation
IBM Power Systems
42
TCP/IP Configuration
New alias names for IP interfaces
– CL commands (i.e. STRTCPIFC) can use name instead of IP addressADDTCPIFC ALIASNAME(LABNET1)STRTCPIFC ALIASNAME(LABNET1)ENDTCPIFC ALIASNAME(LABNET1)CHGTCPIFC ALIASNAME(LABNET1)RMVTCPIFC ALIASNAME(LABNET1)
Work with TCP/IP Interfaces System: RCHASM27
Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 9=Start 10=End
Internet Subnet Interface Alias Opt Address Mask Status Name
172.5.92.48 255.255.255.128 Active PROD 10.1.1.1 255.255.255.0 Active LABNET1 10.1.1.2 255.255.255.0 Active LABNET2 10.1.1.3 255.255.255.0 Active *NONE 10.1.1.50 255.255.255.255 Inactive VIPA1
127.0.0.1 255.0.0.0 Active LOCALHOST
© 2011 IBM Corporation
IBM Power Systems
43
Notes:
A new parameter was added to TCP/IP interfaces in V5R4. The parameter Alias Name provides administrators with an option to define a name for an IP address. i5/OS commands and interfaces also support the alias name. For example, interfaces can be started or stopped via a name rather than an IP address. This is especially useful when dealing with IPv6 interface addresses.
The Convert Interface ID (QtocCvtIfcID) API can be used to retrieve the IP address of an interface when given the name or the name of an interface when given the IP address.
© 2011 IBM Corporation
IBM Power Systems
44
Typical Solutions
© 2011 IBM Corporation
IBM Power Systems
45
Fault Tolerance using *VirtualIP
Interface 10.1.1.1 fails:
Any connections to 10.1.1.1 are lost, connections to 10.1.1.2, 10.1.1.3 remain active.
But connections to 10.2.1.1, the *Virtual IP address, remain active , system stays available
Use *VirtualIP to provide continuous availability even through an interface failure
ƒ What if, instead of an external router, an interface adapter fails?
ƒ Unbound routes automatically switched to active interface. (Routes explicitly bound to interface not moved)
ƒ But IP address of failed interface is still unavailable -- "system still appears down"
ƒ Solution: Use a "Virtual IP" address as the primary system address to which external users connect
Primary IP address of system remains active as long as system is active
System stays accessible so long as at least one physical interface remains active
10.1.1.3
10.1.1.2
10.1.1.1
10.1.1.x
R1
R2
*VirtualIP10.2.1.1
DNS Entry10.2.1.1
Internet
© 2011 IBM Corporation
IBM Power Systems
46
This chart demonstrates a powerful use of *VirtualIP addresses. Here, we define a *Virtual IP address as the primary address for the system. In the DNS, only the *VirtualIP address is defined. All external users access the system via the 10.2.1.1 *VirtualIP address.
If any of the local interfaces fail, the system remains accessible so long as at least one interface remains active. Connections can be transparently re-routed through any of the available interfaces as needed. The advantage of this is that because a *VirtualIP address is not tied to a hardware adapter, it remains active so long as TCP/IP is active.
© 2011 IBM Corporation
IBM Power Systems
47
Fault Tolerance using *VirtualIP - V5R2
Interface 10.1.1.11 fails:
Any connections to 10.1.1.11 are lost, connections to 10.1.1.12, 10.1.1.13 remain active.
But connections to 10.1.1.1, the *Virtual IP address, remain active , system stays available
System TCP/IP moves the APR response to a new working adapter
Use *VirtualIP to provide continuous availability even through an interface failure
ƒ What if, instead of an external router, an interface adapter fails?
ƒ Unbound routes automatically switched to active interface. (Routes explicitly bound to interface not moved)
ƒ But IP address of failed interface is still unavailable -- "system still appears down"
ƒ Solution: Use a "Virtual IP" address as the primary system address to which external users connect
Primary IP address of system remains active as long as system is active
System stays accessible so long as at least one physical interface remains active
10.1.1.13
10.1.1.12
10.1.1.11
10.1.1.x
R1
R2
*VirtualIP10.1.1.1
DNS Entry10.1.1.1
Internet
© 2011 IBM Corporation
IBM Power Systems
48
IP Address Takeover using *VirtualIP
AS1 is taken down:
IP Address Takeover inactivates 10.2.1.1 *VirtualIP interface on AS1 and activates equivalent interface on AS3
RouteD on AS3 advertises that it can now reach 10.2.1.1
After route change is propagated, all traffic to 10.2.1.1 should be directed to AS3
ƒ What if entire system is taken down?
ƒ V4R4: IP Address Takeover -> Switch primary server address to physically different machine
If backup machine is on the same network, route switchover is automatic (via ARP)
But backup machine can even be on a totally different network:
Define Primary server address as a *VirtualIP interfaceWith RIPV2, movement of *VirtualIP address is advertised throughout the network
Note: Also requires V4R4 Clustering product be installed
10.1.1.xR1
*VirtualIP10.2.1.1
10.1.1.1
10.1.2.xR2
10.1.2.4
*VirtualIP10.2.1.1
AS1
AS3
10.1.1.3
AS2
© 2011 IBM Corporation
IBM Power Systems
49
Finally, *VirtualIP addresses can improve system availability when used in conjunction with the V4R4 Clustering product. The Clustering application controls on which system is the *VirtualIP address active at any point in time. When that system is taken down, the same *VirtualIP address is activated on a backup system.
If the backup system is connected to the same network as the primary system, no special routing procedures are required. Consider AS1 as the primary system and AS2 the backup. When the takeover IP address comes active on AS2, it will broadcast an ARP packet to the rest of the local network, informing all other hosts that the IP address has moved to a new system,
But IP address takeover is not limited to both machines being on the same network. All we need is to define the takeover address as an address that is not directly accessible from either of the local networks -- in other words, a *VirtualIP address.
For example, consider the backup system being AS3, rather than AS2. In this case, we need to define the takeover address as a *VirtualIP address that is not part of either of the local networks to which the AS/400's are attached. That is why. on the previous page, the *VirtualIP address is defined as 10.2.1.1. This address is not part of either the 10.1.1.x or the 10.1.2.x networks.
When the 10.2.1.1 takeover address is moved from AS1 to AS3, RIPv2 will advertise to the rest of the network that 10.2.1.1 is now reachable by AS3. Assuming the intermediate routers are also running RIPv2, within a few minutes, the route tables throughout the rest of the network will be updated.
© 2011 IBM Corporation
IBM Power Systems
50
RouterX
Firewall10.1.1.1
10.1.1.2 10.1.2.1
Internet
Rest of the 10.0.0.0 Corporate Network
A 10.1.1.11*VIRTUALIP
10.250.250.1 SYSNAME10.250.250.2 HTTPSVR110.250.250.3 HTTPSVR210.250.250.11 DOM110.250.250.12 DOM210.250.250.13 DOM3
.
.
.10.250.250.20 DOM20
Router XRoute Directives
Next HopDestination Subnet Mask Gateway10.250.250.0 255.255.255.0 10.1.1.11
Multiple TCP/IP Servers using Virtual IP (separate network range)
10.1.1.101 Local PC
In this scenario we are hosting many servers on a single iSeries. Some of these are HTTP servers and some are Domino servers. We need multiple addresses so that each server can bind to a unique address and be accessed by the well known ports for the service it is providing (80, 443, and 1352 for Domino; 80 and 443 for HTTP). The traditional way to define multiple TCP/IP on the iSeries is to create multiple TCP/IP interfaces on the same line description. The applications will bind to the Virtual IP address and therefore will not notice if the physical adapter fails. This may prevent the restart of some servers.The benefits of using Virtual IP are not fully realized in this configuration because there is only one physical LAN adapter in this configuration.
© 2011 IBM Corporation
IBM Power Systems
51
Building the configuration1Get an address for the physical LAN adapter (if it is not already created)
In our example we use 10.1.1.112Select a subnet to use for the Virtual IP addresses
This range of addresses should not be in use anywhere else in the network.In our example we use 10.250.250.x
3Create the Line Description for the LAN adapterCRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)
4Define the IP interface for the LAN adapterADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')
5Add the corporate and Internet route entries to the TCP/IP route tableADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2')ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1')
6Define the Virtual IP addresses (repeat as needed)ADDTCPIFC INTNETADR('10.250.250.1') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388) ADDTCPIFC INTNETADR('10.250.250.20') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388)
7Add route entries to the routers, firewall, and systems that need to point to the subnet 10.250.250.0Tell the network administrator that the iSeries looks like a router for that subnetIf the routers etc. support RIP2 you can start ROUTED server and let the system broadcast the net route8Add entries to your DNS to point to the virtual IP addresses9Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for
each serverNOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition
• Start the servers and test.
© 2011 IBM Corporation
IBM Power Systems
52
RouterX
Firewall10.1.1.1
10.1.1.2 10.1.2.1
Internet
Rest of the 10.0.0.0 Corporate Network
A 10.1.1.11*VIRTUALIP
10.1.1.21 SYSNAME10.1.1.22 HTTPSVR110.1.1.23 HTTPSVR210.1.1.24 DOM110.1.1.25 DOM210.1.1.26 DOM3
.
.
.10.1.1.43 DOM20
Multiple TCP/IP Servers using Virtual IP (with proxy ARP same network address)
10.1.1.101 Local PC
In this scenario we are hosting many servers on a single iSeries. Some of these are HTTP servers and some are Domino servers. We need multiple addresses so that each server can bind to a unique address and be accessed by the well known ports for the service it is providing (80, 443, and 1352 for Domino; 80 and 443 for HTTP). The traditional way to define multiple TCP/IP on the iSeries is to create multiple TCP/IP interfaces on the same line description. The applications will bind to the Virtual IP address and therefore will not notice if the physical adapter fails. This may prevent the restart of some servers.The benefits of using Virtual IP are not fully realized in this configuration because there is only one physical LAN adapter in this configuration. In this case we have used IP addresses in the same network as the real network and checked the proxy APR check box so that all the addresses will reply to APR request using the MAC address of the 10.1.1.11 adapter
No additional routes needed in the router configuration
© 2011 IBM Corporation
IBM Power Systems
53
Building the configuration1Get an address for the physical LAN adapter (if it is not already created)
In our example we use 10.1.1.112Get addresses to use as virtual IP addresses. The addresses do NOT need to be consecutive. In our
example however they are.2Check the network equipment to make sure that the switches etc. support multiple IP addresses on a
single port.3Create the Line Description for the LAN adapter
CRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)4Define the IP interface for the LAN adapter
ADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')5Add the corporate and Internet route entries to the TCP/IP route table
ADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2')ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1')
6Define the Virtual IP addresses. For proxy APR you must use iSeries Navigator (repeat as needed)Refer back to page 19 for an example.
7No additional routes are needed in the routers or firewalls. 8Add entries to your DNS to point to the virtual IP addresses9Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for
each serverNOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition
• Start the servers and test.
© 2011 IBM Corporation
IBM Power Systems
54
BIND to an IP address - HTTP Servers
http://hostname:2001HTTP Config and AdminOriginal HTTP Server
ConfigurationBasic Settings
Select Bind server to host address
Apache ServerConfigurationGeneral Settings
Click Add and type IP addressSelect All and click Remove
© 2011 IBM Corporation
IBM Power Systems
55
BIND to an IP address - Domino Servers
notes.ini
Server Document
© 2011 IBM Corporation
IBM Power Systems
56
Router
X
Firewall10.1.1.1
10.1.1.2 10.1.2.1
Internet
Rest of the 10.0.0.0 Corporate
Network
A
10.1.1.12
10.1.1.13
10.1.1.11
10.1.1.14
10.1.1.15
*VIRTUALIP10.250.250.1 SYSNAME10.250.250.2 HTTPSVR110.250.250.2 HTTPSVR210.250.250.11 DOM110.250.250.12 DOM210.250.250.13 DOM3
.
.
.10.250.250.20 DOM20
Router XRoute Directives
Next HopDestination Subnet Mask Gateway
10.250.250.0 255.255.255.0 10.1.1.1110.250.250.0 255.255.255.0 10.1.1.12
B
C
D
E
Interface A and B are setup as primary input interfaces. Interface C, D, and E are setup for output connection balancing. As connections are made TCP/IP will round-robin between C, D, and E. If all three of these become unavailable the TCP/IP will move to the next lower Priority (6) and use A and B for output as well as input.The router directives are set up to round-robin between interface A and B. This is a function of the router. Most routers will provide this type of support. In this example the load on each interface is NOT considered. The assumption is that the traffic load is similar for all connections.
TCP/IP inbound and outbound balancing with Virtual IP
OS/400 TCP/IP Route Entries
Preferred Dup.Route Subnet Binding Route
Dest. Mask Next Hop Interface Pri.10.1.1.0 255.255.255.0 10.1.1.11 10.1.1.11 6
10.1.1.0 255.255.255.0 10.1.1.12 10.1.1.12 610.1.1.0 255.255.255.0 10.1.1.13 10.1.1.13 710.1.1.0 255.255.255.0 10.1.1.14 10.1.1.14 7
10.1.1.0 255.255.255.0 10.1.1.15 10.1.1.15 7
10.0.0.0 255.0.0.0 10.1.1.2 10.1.1.11 610.0.0.0 255.0.0.0 10.1.1.2 10.1.1.12 6
10.0.0.0 255.0.0.0 10.1.1.2 10.1.1.13 710.0.0.0 255.0.0.0 10.1.1.2 10.1.1.14 710.0.0.0 255.0.0.0 10.1.1.2 10.1.1.15 7
*dftroute *none 10.1.1.1 10.1.1.11 6
*dftroute *none 10.1.1.1 10.1.1.12 6*dftroute *none 10.1.1.1 10.1.1.13 7*dftroute *none 10.1.1.1 10.1.1.14 7
*dftroute *none 10.1.1.1 10.1.1.15 7
X
Y
Z
10.1.1.101 Local PC
© 2011 IBM Corporation
IBM Power Systems
57
Balancing example explained
This example takes full advantage of using virtual IP addresses. In addition to providing each application a unique address to bind to, it also provides support for :
–Inbound connection balancing–Outbound connection balancing–Some level of fault tolerance
Inbound connection balancing is provided by:–Virtual IP addresses defined on the system–External router, firewall, and/or switch with L3 routing built in
Outbound connection balancing is provided by the preferred binding interface and duplicate route priority parameters on the OS/400 TCP/IP route entries. The connection balance will round-robin between all the interfaces at the same duplicate route priority when this value is set greater than 5. If all the interfaces at one value become unavailable (7 in our example) the system will switch to a set at the next lower value (6 in our example). The entries are split into three groups in the example.
–Group X - provides connection balancing to the local segment of the LAN–Group Y - provides connection balancing to the rest of the corporate network using the router–Group Z - provides connection balancing to the Internet using the firewall
Fault tolerance:–When a connection such as TELNET is established it will occur between the remote host and the virtual IP address. The session on the iSeries will be bound to the virtual IP address. If the physical interface drops the session will stay active. The system will reroute the traffic over another existing outbound interface. The router will also reroute the traffic to a different interface. So long as the iSeries and the router can move the traffic the session will remain active.
© 2011 IBM Corporation
IBM Power Systems
58
1 Get an addressesfor the physical LAN adapters (if it is not already created)In our example we use 10.1.1.11 - 10.1.1.15
2 Select a subnet to use for the Virtual IP addressesThis range of addresses should not be in use anywhere else in the network.In our example we use 10.250.250.x
3 Create the Line Description for the LAN adapter for each adapterCRTLINETH LIND(ETHLAN1) RSRCNAME(CMN05) LINESPEED(*auto) DUPLEX(*auto)
4 Define an IP interface for each LAN adapterADDTCPIFC INTNETADR('10.1.1.11') LIND(ETHLAN) SUBNETMASK('255.255.255.0')
5 Add the local, corporate and Internet route entries to the TCP/IP route table for each interface (see table in example)ADDTCPRTE RTEDEST('10.1.1.0') SUBNETMASK('255.255.255.0') NEXTHOP('10.1.1.11') BINDIFC('10.1.1.11') DUPRTEPTY(6ADDTCPRTE RTEDEST('10.0.0.0') SUBNETMASK('255.0.0.0') NEXTHOP('10.1.1.2') BINDIFC('10.1.1.11') DUPRTEPTY(6)ADDTCPRTE RTEDEST(*DFTROUTE) SUBNETMASK(*NONE) NEXTHOP('10.1.1.1') BINDIFC('10.1.1.11') DUPRTEPTY(6)
6 Define the Virtual IP addresses (repeat as needed)ADDTCPIFC INTNETADR('10.250.250.1') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388) ADDTCPIFC INTNETADR('10.250.250.20') LIND(*VIRTUALIP) SUBNETMASK(*HOST) MTU(16388)
7 Add route entries to the routers, firewall, and systems that need to point to the subnet 10.250.250.0Tell the network administrator that the iSeries looks like a router for that subnetIf the routers etc. support RIP2 you can start ROUTED server and let the system broadcast the net route8 Add entries to your DNS to point to the virtual IP addresses9 Start TCP/IP or the Interfaces that you added and test the connectivity• Change all the HTTP servers and Domino servers to BIND specific to the Virtual IP address you set up for each server
NOTES.INIWRKHTTPCFGor use the GUI toolsBe sure to change the default HTTP server definition
• Start the servers and test.
Creating the example
© 2011 IBM Corporation
IBM Power Systems
59
Inbound load balancing revisited
•Can now limit proxy agent to the desired set of physical interfaces
Physical IP Address
10.1.1.11
10.1.1.12
Virtual IP Address
10.250.250.1
VIPA Preferred Interface List10.250.250.1 1) 10.1.1.11
2) 10.1.1.12
10.1.1.13
10.1.1.14
10.1.1.15
© 2011 IBM Corporation
IBM Power Systems
60
TCP/IP and LPAR
© 2011 IBM Corporation
IBM Power Systems
61
The advent of LPAR provided yet another environment to apply the same routing concepts as previously discussed.
With LPAR, a single AS/400 is logically partitioned in multiple virtual machines. Each partition has its own address space its own instance of TCP/IP, and may have its own dedicated I/O adapters. To TCP/IP, each partition appears like a distinct AS/400
Moreover, TCP/IP communication between the different partitions is done via a virtual opticonnect bus or a virtual ethernet LAN (V5R1). The TCP/IP routing code sees the path to another LPAR partition no differently than the path to another system connected via a physical opticonnect bus. All of the concepts and configurations that were previously described for "TCP/IP over Opticonnect" environments apply equally well to "TCP/IP with LPAR"
© 2011 IBM Corporation
IBM Power Systems
62
LPAR Scenario - Using Virtual Ethernet LAN
10.1.1.1
10.1.1.10 10.1.1.11
10.1.1.12
i5/OSPar ID 1
LinuxPar ID 2
Linux2Par ID 3
Virtual LAN 1
Virtual LAN 5
Requirments:One physical LAN adapterThree partitions
–i5/OS–Linux–Linux
All Partitions connected on LAN 1Partition 1 and 3 connected on LAN 5
subnet mask 255.255.255.0
addrs - 10.1.1.241 - 10.1.1.254subnet mask 255.255.255.240
addrs - 10.1.1.233 - 10.1.1.238subnet mask 255.255.255.248
© 2011 IBM Corporation
IBM Power Systems
63
Steps to Implement - After Partitioning is complete
1 Connect virtual LANs to the correct partitions2 Define physical LAN adapter (CRTLINETH)3 Define TCP/IP Interface over real LAN adapter4 Set IP Forwarding to *YES
5 Determine TCP/IP addresses to useSelect a contiguous range that is a natural subnet
6 Define TCP/IP Interfaces over virtual Ethernet adapters7 Vary on lines and start TCP/IP interfaces8 Build LAN configuration in other partitions• Test connectivity
© 2011 IBM Corporation
IBM Power Systems
64
Setting up Virtual Ethernet LAN (HMC LVL 6)
HMC is used to create, modify, and monitor logical partitions and their resources.
Select the virtual adapter type
Click Create
© 2011 IBM Corporation
IBM Power Systems
65
Setting up Virtual Ethernet LAN (HMC LVL 6)
HMC is used to create, modify, and monitor logical partitions and their resources.
Slot in the virtual partition
VLAN id – all partitions on the VLAN are on the same segment of the network and can communicate
© 2011 IBM Corporation
IBM Power Systems
66
Display Resource Detail System: ATSI5P1
Resource name . . . . . . . : CMN09 Text . . . . . . . . . . . . : Ethernet Port Type-model . . . . . . . . . : 268C-001 Serial number . . . . . . . : 00-00000 Part number . . . . . . . . :
Location: U9406.520.10A965C-V1-C2-T1
Logical address: SPD bus: System bus 255 System board 0 System card 0 Communications:
More... Press Enter to continue.
F3=Exit F5=Refresh F6=Print F12=Cancel Display Resource Detail System: ATSI5P1
Resource name . . . . . . . : CMN28 Text . . . . . . . . . . . . : Ethernet Port Type-model . . . . . . . . . : 268C-001 Serial number . . . . . . . : 00-00000 Part number . . . . . . . . :
Location: U9406.520.10A965C-V1-C5-T1
Logical address: SPD bus: System bus 255 System board 0 System card 0 Communications: More...
Press Enter to continue.
F3=Exit F5=Refresh F6=Print F12=Cancel
Find the Resource - i5
Use Slot ID to find Virtual LAN ID
WRKHDWRSC *CMN
© 2011 IBM Corporation
IBM Power Systems
67
ADDTCPIFC INTNETADR('10.1.1.1') LIND(ETHLAN) SUBNETMASK('255.255.255.0')
CRTLINETH LIND(ETHLAN) RSRCNAME(CMN05) LINESPEED(*AUTO) DUPLEX(*AUTO)
Add the TCP/IP Interface to the real LAN
Set IP forwarding to *YES
© 2011 IBM Corporation
IBM Power Systems
68
Determine TCP/IP addresses to use
Once the TCP/IP interface has been created the interface should be varied on.
Proxy ARP basically consists of the establishment of a sub-network within the larger network that the i5/OS physical interface is connected to.
The IP address of the virtual network connection along with the subnet mask determines the network range that i5/OS will proxy for
A subnet-calculator can be useful in helping to determine the address range.
The TCP/IP attribute Datagram Forwarding has to be set to *Yes to allow network packets to flow between the two network interfaces.
© 2011 IBM Corporation
IBM Power Systems
69
ADDTCPIFC INTNETADR('10.1.1.241') LIND(VIRTLAN) SUBNETMASK('255.255.255.240') LCLIFC('10.1.1.1')
CRTLINETH LIND(VIRTLAN) RSRCNAME(CMN11) LINESPEED(1G) DUPLEX(*FULL) TEXT('Virtual Ethernet to Other Partitions')
Add a TCP/IP Interface to the Virtual LAN
© 2011 IBM Corporation
IBM Power Systems
70
References
• http://www.redbooks.ibm.com/
• SG24-5190: " V4 TCP/IP for AS/400: More Cool Things Than Ever"
• SG24-6718 - iSeries IP Networks: Dynamic
• GC24-3376: "TCP/IP Tutorial Technical Overview",
•Chapter 11-- Availability, Scalability and Load Balancing
• SG24-5147: " AS/400 Autoconfiguration: DNS and DHCP Support
•Section 15.2: Transparent Subnet Masking
• SG24-6232: "Linux on iSeries: An Implementation Guide"
• http://www.as400.ibm.com/infocenter/
• Networking--> TCP/IP -->TCP/IP routing and workload balancing.
• IBM Network Dispatcher:
•http://www.software.ibm.com/network/dispatcher/
© 2011 IBM Corporation
IBM Power Systems
71
8 IBM Corporation 1994-2008. All rights reserved.References in this document to IBM products or services do not imply that IBM intends to make them available in every country.
Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at http://www.ibm.com/legal/copytrade.shtml.
Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registeredtrademarks of Intel Corporation or its subsidiaries in the United States and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce.ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.UNIX is a registered trademark of The Open Group in the United States and other countries.Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.
Information is provided "AS IS" without warranty of any kind.
The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.
Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products.
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.
Prices are suggested U.S. list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.
Photographs shown may be engineering prototypes. Changes may be incorporated in production models.
Trademarks and Disclaimers