z/OS OMPROUTE Hints and Tips - IBM WWW PageFILE/3930_OSPF_hints_and_tips.pdf · z/OS OMPROUTE Hints...

z/OS OMPROUTE Hints and Tips

Session 3930Monday, August 22, 2005, 4:30 pm

Mike FoxGwen Dente

[email protected]@us.ibm.com

IBM Corporation 2001, 2005

®AbstractPrerequisites:

Knowledge of Basic Routing Concepts OSPF IP RoutingExperience with implementing MVS or OS/3900-based TCP/IPBasic knowledge on starting, controlling, and configuring OMPROUTE

Abstract: OSPF as part of OMPROUTE is the strategic direction for implementing a dynamic IP routing protocol in CS for z/OS. The link-state algorithms of OSPF bring great advantages to an IP network over those offered by the distance vector algorithms of RIP. This presentation provides some basic design suggestions, diagnostic information, common problems, and other hints and tips for users of OSPF and OMPROUTE.


®

1. General Network Design Guidelines2. Summary of Coding Hints3. Diagnostics with OMPROUTE

1. OMPROUTE in the TCP/IP Stack2. OMPROUTE and multipath

4. Integration with non-OSPF Networks1. EIGRP2. Interoperability scenarios

5. Common Problems1. Performance problems2. Neighbor states and problems3. Configuration Problems4. Multicast Snooping5. Can't route out of the totally stubby

area6. Common Configuration Complaints

1. Subnet Mask2. Too Many Host Routes

7. Appendix1. Integration with zSeries Linux

Agenda

Warning: Fast-moving presentation!

1. This presentation moves at a brisk pace. Do not get discouraged -- we know you may not be able to absorb everything during the time allocated to this topic, but that is why we have included thorough notes!! The notes make it easy for you to review this presentation at your leisure at a later point in time.

© IBM Corporation 2001, 2005

1. General Network Design Guidelines


®

OS/390 OSPFStub area

XCFNetwork

Backbone -OSPF area0.0.0.0

z/OS

Area Border Routers: Channel-Attached

z/OS in an OSPF Stub Area

If your z/OS area is at the edge of the network (and not between a RIP AS and another OSPF Area), consider defining the z/OS Area as a Stub Area (STUB=YES).No OSPF virtual links through the Stub AreaNo AS external routes in the Stub Area

Or better yet, make it "Totally Stubby"No NSSA support in OMPROUTE

z/OS

z/OS

z/OS

z/OS

z/OS

1. OSPF design options allow an installation to limit the amount of network topology that OS/390 or z/OS needs to be aware of and to process through the OSPF area concept.

2. This allows us to limit the amount of CPU required to process required OSPF information updates. 3. If your OS/390 cluster is at the edge of the network, you can reduce the amount of routing information in the OS/390 environment

by defining the OS/390 sysplex as an OSPF stub area where the border routers attach the stub area to the backbone area.1. No OSPF virtual links through the stub area2. No AS external routes

4. You might even consider making CS a "Totally Stubby Area" that does not even receive Summary LSAs (except for the Default Route in a Summary LSA).

5. OMPROUTE does NOT support the "not so stubby area (NSSA)" concept described in RFC 15876. The following is a summary of what you may already know about different types of areas and router roles.7. Area Border Router

1. An area border router summarizes all OSPF links into other areas. 2. External Link State Advertisements flow freely across all OSPF areas, except for Stub Areas..

1. (External Links are RIP, Static, and Direct.) 3. An Area Border Router can interface with a Stub Area.

1. All OSPF links and summaries can flow into a Stub Area. 2. The Area Border Router sends summary of OSPF links into the Stub Area. 3. External Link State Advertisements cannot flow into a Stub Area. 4. The Area Border Router says, "I'm not telling you the external links, so here's a default route for you if you need to reach an

external route." 8. Stub Areas

1. Stub Areas can have multiple default routes from different attached Area Borders. (One for each.) 2. A Stub Area is defined with STUB=YES in the Area definition in each host that attaches to the area.3. A Stub Area can be defined as a "Totally Stubby Area" in CS for OS/390 with "Import_Summaries=NO." Only Default Routes are

accepted. 4. A stub area can be adjacent to the backbone or not adjacent to the backbone, but adjacent to a border area.

9. Connectivity to Backbone OSPF Areas and Virtual Links 1. Every area must be adjacent to the backbone. If non-adjacent physically, there must be a virtual link to the backbone. 2. Stub Areas can be the end-points of a virtual link, but they cannot be on the intermediate path of a virtual link.


®z/OS in an OSPF Stub Area with OSA-E

z/OS OSPFStub area

Or, to minimize linkstate advertisement traffic, attach an z/OS Stub Area to the backbone via OSA-Express QDIO Connections!

XCFNetwork


Area Border Routers: LAN-Attached

z/OS

z/OS

z/OS

z/OS

1. In the previous network, we saw point-to-point connections to channel-attached routers. Separate IP networks are used for each channel attachment, thus increasing the number of LSAs that must be advertised throughout the area.

2. In a broadcast network such as what you see on this page, fewer LSAs need to be advertised to establish adjacencies and to set up routing trees.


®Actual Customer Experience with Totally Stubby Area

iT-Austria changed from an "OSPF-isolated" to a Totally Stubby Area network design.

"OSPF-isolated" had z/OS in an area 0.0.0.0, isolated from the rest of the network using Cisco's OSPF to OSPF feature

By changing to Totally Stubby Area, they saw a significant reduction in OMPROUTE/OSPF workload

Routing table size dropped from 3876 routes down to 37 routesOMPROUTE CPU usage dropped by more than half, to 40% of its previous value

1. Cisco's OSPF to OSPF feature is similar to using BGP between two OSPF autonmous systems with access lists. It allows connection of two OSPF services, with filters.


®z/OS in a Non-Backbone Area

RIP

RIPRIPRouter13

Router11

AS1

AS2

Area 0.0.0.0Router12 ASBR

RIP <-> OSPF

If your z/OS area is not at the edge of the network You may define Area 1.1.1.1 as a non-backbone area so that it may receive RIP routes (i.e., "external routes") into the OSPF environment. You must design for Multiprotocol Route Cost Comparisons!With redundant ASBRs, configure for route tags to prevent routing loops.

Different Metrics & Perhaps Route Tags!

RouterXX ASBR

RIP <-> OSPF

XCFNetwork


Area Border Routers: LAN-Attached

z/OS

z/OS

z/OS

z/OS

Area 1.1.1.1

1. Here you see that RIP AS1 is connected to the OSPF AS2 by means of an Autonomous System Boundary Router and the OS/390 area lies between the RIP AS and the Backbone Area.

2. The ASBR (Router 12) may be a CS/390 IP stack.3. In this multiprotocol scenario, RIP routes with their metrics are imported into OSPF; OSPF routes with their Costs are imported

into RIP.4. Since metrics and costs mean different things, you must establish a route precedence methodology to determine which type of

route should be preferred in the case of multiple equal-cost routes of different flavors.5. It goes beyond the scope of this tutorial to examine these issues. For more information about coding for route precedence with

the OSPF statement "Comparison," please consult the IP Configuration Guide and the Information APARs on RETAIN regarding "route precedence" and "Type 1" and Type 2" external routes. Also the IP Configuration Guide for V2R10 and later releases has discussions of route precedence in a multiprotocol environment.

6. If you have redundant ASBRs, as you see here with Router12 and RouterXX, configure them to include route tags so that the receiving AS does not attempt to re-distribute the same routes back into the AS from which they came.


2. Summary of Coding Hints


®

Specify all interfaces in the OMPROUTE configuration file

Use link metrics (weights) that make sense

Always specify MTU size and Subnet Mask

Always specify the RouterID to avoid the accidental assignment to a Dynamic VIPA (Recommended: Static VIPA)

If possible on multiaccess networks, let nodes other than CS be the Designated Router (Set OMPROUTE Router_Priority=0); EXCEPTION on HiperSockets LAN

OSPF timer defaults are normally OK, but at the very least match the HELLO and Dead Router Interval with adjacent routers

Run OMPRoute in a good performance group/WLM policy (otherwise risk of lost adjacencies); make non-swappable only if necessary

OSPF/OMPRoute Hints and Tips (1)

1. Specify all interfaces in the OMPROUTE configuration file1. Otherwise OMPROUTE builds "Interface" definitions - not "OSPF_Interface" or "RIP_Interface."2. Also, OMPROUTE will use default values for MTU and subnet mask, which may not be desirable.

2. Use link metrics (weights) that make sense1. Plan ahead. With a metric = 1, difficult to lower metric as link speeds get higher

3. Always specify MTU size; otherwise a default of 576 is taken. Note that OMPROUTE MTU values override BSDRoutingparms MTU values, so if you are running OMPROUTE make sure OMPROUTE has the MTU defined with the values that you want.

4. Always specify subnet value; otherwise the native class network mask is used 5. Do not allow the RouterID to default; you need to ensure that the RouterID is not assigned to a Dynamic VIPA. Although we recommend a Static

VIPA as the RouterID, in fact any physical interface value is also acceptable.6. To eliminate extraneous CPU processing/cycles, allow nodes other than the CS node to be eligible to become the Designated Router (DR).

However, remember that a HiperSockets network requires a Designated Router and at least one of the LPARs participating in a HiperSockets LAN should be eligible to be a DR (Router_Priority=1).

7. OSPF timer defaults are normally OK1. Default time intervals:2. Hello Interval - 10 seconds (MUST match what is coded in all routers on same medium)3. Dead router interval - 40 seconds (Recommended to be at least 4x Hello interval value)4. Re-transmit interval - 10 seconds

8. Run OMPRoute in a good performance group/WLM policy1. If OMPRoute is starved for CPU

1. It can't respond quickly to network changes2. Neighbors may be lost or Neighbors may not become fully adjacent3. At times, if you are unsuccessful at managing performance with WLM you may consider making OMPROUTE non-swappable, although this

should be unnecessary if your WLM performance groups have been properly established.1. To make OMPROUTE non-swappable, start it as a started task rather than via BPXBATCH; this allows you to add OMPROUTE to the PPT

table as non-swappable. 9. OMPROUTE JCL may need to be revised to use an LE Environment variable to free storage in certain cases. The symptoms are loss of neighbors

for DYNAMICXCF connections of all flavors: Message BPXF024I (TCPIPUX) Feb 25 22:42:51 omproute 67108971 : EZZ7921I OSPF 194 Adjacency failure, neighbor 192.168.5.171, old state 32, new state 8, event 7 1. Change the EXEC statement in the OMPROUTE JCL to use the "HEAP(,,,FREE)" setting: 1.//OMPROUTE EXEC PGM=OMPROUTE,REGION=0K,TIME=NOLIMIT, 2.// PARM=('POSIX(ON)', 3.// 'ENVAR("_CEE_ENVFILE=DD:STDENV",', 4.// '"_CEE_RUNOPTS=HEAP(,,,FREE)")', 5.// '"TZ=EST"/ -t1')


®

Enable tracing only when necessary!!Use tracing to CTRACE, rather than HFS files, whenever possible

Be careful if mixing OSPF AS with AS of different protocol - metrics may not be consistentRecommended reading: Step 9: Defining Route Precedence in a MultiProtocol Environment (If OSPF Protocol is Used) in OMPROUTE section of IP Configuration Guide Some routers on a Token-Ring network do not understand the MAC multicast address used by OSPF and RIP V2. Use Translate Statement in the PROFILE: TRANSLATE 224.0.0.0 IBMTR FFFFFFFFFFFF linkname

Consult the Informational APAR on OMPROUTE Problems: II12026.

OSPF/OMPRoute Hints and Tips (2)

1. Be careful if mixing OSPF AS with AS of different protocol - metrics may not be consistent. In V2R10 we significantly enhanced the routing information in our pubs, including a discussion of this consideration.

2. Some routers on a Token-Ring network do not understand the MAC multicast address used by OSPF and RIP V2. Please consult the TCP/IP Hints and Tips FLASH (N3196) at www.ibm.com/support/techdocs or the IP Configuration Guide for a special use of the TRANSLATE statement in the PROFILE if token-ring-attached routers are not listening for the TR multicast mac address of 0xC000.0004.000. 1. TRANSLATE 224.0.0.0 IBMTR FFFFFFFFFFFF linkname

3. If you are counting on VIPA Host Routes on any release prior to V2R8, ensure that you have consulted APARs PQ25823 and PQ29859. Without these APARS only subnet destinations will be sent.

4. Periodically consult the Informational APAR II12026 for information on avoiding common OMPROUTE problems.


®Neighbor Coding for OSPFLink State Advertisements originated by both sides of a connection must agree on the media type and connection type.Some idiosyncrasies ...

Cisco codes CLAW as "point-to-multipoint" and "broadcast" in older releases of Cisco software. Higher releases allow coding of "point-to-point."

Ensure that IBM side of CLAW connection is compatible with what is coded in the Cisco. Consult RFC 2328, APAR PQ35265 and APAR PQ48766, and Flash N3196 for V2R8 IP Migration Hints and Tips.

MPC+ can be perceived as either P2P or P2MP by the TCP/IP stack in Communications Server; the coding on the Cisco side of an MPC+ connection is the same in either case.

1. See the CISCO Examples in the of the tutorial presentation (N03).2. CS/390 OSPF perceives a CLAW interface as point-to-point and a CISCO router codes it as point-to-multipoint. Note that Cisco

did not traditionally have a configuration parameter for P2P for CLAW; it had to be configured using the P2MP *AND* BROADCAST parameters to treat CLAW like P2P for compatibility with the IBM CLAW. 1. For example, a CISCO router considers a broadcast network to be point-to-multipoint; a non-broadcast network is

point-to-point. Accordingly the CISCO router will send in Link State Advertisements according to RFC 2328, section 12.4. If you have problems connecting properly or receiving appropriate route updates, you may have a mismatch in the configurations between CS/390 and the router.

2. APARs PQ35265 and PQ48766 describe how to correct these scenarios. See examples of Cisco CLAW coding in Part 3 of this OSPF series of presentations.

3. The latest levels of CISCO IOS now allow the definiton of CLAW as point-to-point as another way to avoid the problem.


3. OMPROUTE Basic Diagnostics


®

UNIX sockets

OMPRoute address space

TCP/IP stack

routingtable

routingtable

A B

C D

Eth1

E

1 1

1

4 6

1

Configuration file

OMPROUTE Routing Structure

13.25.43 d tcpip,tcp2,netstat,route 13.25.43 EZZ2500I NETSTAT CS V2R7 TCPCS2 263 DESTINATION GATEWAY FLAGS REFCNT INTERFACE DEFAULT 9.67.113.1 UG 000000 TR1 9.67.113.0 0.0.0.0 U 000000 TR1 192.168.2.0 0.0.0.0 U 000000 LINKSAME 192.168.2.4 0.0.0.0 UH 000000 LINKSAME 192.168.3.0 192.168.2.3 UG 000000 LINKSAME 192.168.4.0 192.168.2.4 UG 000000 LINKSAME 6 OF 6 RECORDS DISPLAYED

Link / Route Info

from network

13.24.24 D TCPIP,TCP2,OMP,RTTABLE 13.24.24 EZZ7847I ROUTING TABLE 260 TYPE DEST NET MASK COST AGE NEXT HOP(S) DFLT 0.0.0.0 0 0 2424 9.67.113.1 SBNT 9.0.0.0 FF000000 1 2394 NONE DIR* 9.67.113.0 FFFFFF80 1 2424 9.67.113.11 SPF* 192.168.1.0 FFFFFF00 1 2423 LINKVIPA1 DIR* 192.168.1.2 FFFFFFFF 1 2424 LINKVIPA1 SPF* 192.168.2.0 FFFFFF00 100 2324 LINKSAME DIR* 192.168.2.3 FFFFFFFF 1 2376 192.168.2.2 DIR* 192.168.2.4 FFFFFFFF 1 2367 192.168.2.2 SPF 192.168.3.0 FFFFFF00 101 2173 192.168.2.3 SPF 192.168.4.0 FFFFFF00 101 2173 192.168.2.4 DEFAULT GATEWAY IN USE. TYPE COST AGE NEXT HOP DFLT 0 2424 9.67.113.1 0 NETS DELETED, 0 NETS INACTIVE

routingtable

StaticRoutes

TCP Informational Socket

IOCTL Call

1. OMPROUTE maintains a routing table in its address space (RTTABLE); it receives information for this table from OSPF LSAs and from the TCP/IP stack via a TCP Informational Socket. Via an IOCTL call, Omproute sends its routing information to the TCP/IP stack, which makes the actual routing decisions.

2. OSPF support on CS for OS/390 has same code base as IBM 2216/22103. VM/TCPIP also has ported this code, their daemon is named MPROUTE. It was ported from V2R6 OMPROUTE.

1. Routing Daemon: OMPRoute, standing for OpenMVS MultiProtocol Router 1. Can run RIP on some interfaces, OSPF on others2. Receives information from the IP stack via a TCP Informational Socket. (This socket is visible with D TCPIP,,N,CONN.)3. Maintains its own routing table and feeds information to stack table via an IOCTL program call. Stack makes routing decision.4. Presents stack with multiple equal-cost routes so that stack can distribute Traffic Among Equal-Cost Routes

1. IPCONFIG MULTIPATH2. Both RFC 2328 and RFC 1583 define OSPF. However, CS for OS/390 OMPROUTE supports 1583+ and not RFC2328.3. OSPF uses raw sockets; unlike UDP and TCP, the concept of Port number is unknown to raw sockets. Therefore, no port number

is coded for OSPF in the IP Profile.4. Has Link State Database of network; computes from it a routing table. Updates the IP stack routing table with this one.5. Can discover/React to Failures More Quickly6. Cannot run ORouteD and OMPRoute on same stack7. Extends Addressing8. Varying-Sized Subnets9. Implements SNMP Subagent 10. The TYPE field of RTTABLE may have many values, some of which we will see later in a more complete discussion of this table. :

1. SBNT = placehoder to indicate net has been subnetted2. DIR = direct3. RIP = Learned through RIP4. SPF = Intra-Area destination5. SPIA = Inter-Area destination6. SPE1 = External Type 1 Route7. SPE2 = External Type 2 Route8. RNGE = if ranges have been set9. * = router has directly connected backup10. % = RIP updates unconditionally accepted for this net/subnet11. Next Hop = IP Address or link name


®

OSPF Multipath support : OSPF enables the stack to perform ...

Load balancing over multiple equal weight routes Multipath routes chosen in a round-robin scheme

Inactive multipath routes are skippedOutbound bind-specific connections and directed IP traffic over specified interfaces not affected

Including new versions of OPING and OTRACERT

Per Connection Multipath support Load balancing for multiple routes between same

source and destination address pairOne route per TCP/IP connection: PERCONNECTION

Per Packet Multipath support Load balancing of IP traffic at the packet level

New route chosen per IP packet: PERPACKET

OMPROUTE supports up to 16 multipath routes to a destination in z/OS V1R5 and later releases. Previous releases only support up to 4 multipath routes to a destination.

client 1 client 2

Router C Router D

Intranet/Internet

Router A Router B

1

1

1

2

2

1

33

IBM

IBM

IBM IBM

S1

OSPF & Multipath

IPCONFIG MULTIPATHPERCONNECTION;PERPACKETDATAGRAMFWDFWDMULTIPATH

S2IBM

Router E Router F

IBM

1. OSPF enables the stack to perform Multipath forwarding by presenting the stack with multiple routes to the same destination. However, the stack must have been coded to support MULTIPATH to be able to take advantage of the multiple routes with which it has been presented.

2. Multipath types:1. Perconnection:

1. The TELNET connection to Client1 passes over RouterA. The FTP connection to Client1 passes over RouterB.2. The TELNET connection to Client2 passes over RouterB. The FTP connection to Client2 passes over RouterA.3. PERCONNECTION is the DEFAULT.

2. Perpacket:1. The packets using the same destination and source IP addresses will use different outbound paths. IP will selct a route on

a round-robin basis from a multipath routing list to that destination host.3. DATAGRAMFWD allows IP to be an intermediate router and transfer data between networks

1. If you add FWDMULTIPATH, then multipath logic will be applied to the transferred data.2. NOFWDMULTIPATH is the DEFAULT.

4. If you have multiple OSPF interfaces on the same subnet, all are used for traffic, but only one is used for OSPF packets. (Per RFC 2178)1. You may control which interface becomes primary with the use of the PARALLEL_OSPF parameter on the OSPF_Interface

statement. Otherwise, OSPF chooses the primary, which sends all OSPF packets.2. One interface selected as "primary"3. If primary fails, backup interface becomes new primary

5. OMPROUTE supports up to 4 multipaths to the same destination. If you require more, you'll have to use static routing, which supports unlimited multipath. See slide 79 for more information on this limitiation.


4. Integration with Non-OSPF Networks


®

EIGRP is a proprietary routing protocol supported by Cisco routers. BGP is a standards-based protocol supported by many different vendor routers.

CS for z/OS does not support EIGRP or BGP.However, OSPF and EIGRP|BGP interoperate effectively using the appropriate routers as the ASBRs.

Though EIGRP and BGP are used in the following examples, in fact the backbone autonomous system could be any non-OSPF protocol.

How to configure the routers to interoperate EIGRP, BGP, and OSPF is beyond the scope of this presentation. See Redbooks SG24-6297 & SG24-6516.

OSPF/EIGRP/BGP Interoperability

1. This information is useful for customers who have an existing Cisco EIGRP backbone and want to attach z/OS sysplexes or mainframe complexes to them, and use dynamic routing.

2. This information is also useful to customer wanting to implement "filtering" between OSPF areas by designing them as separate Autonomous Systems interconnected by ASBRs running EIGRP or BGP.

3. Note that OMPROUTE does not support EIGRP or BGP; however it may interoperate with routers acting as ASBRs that do support these protocols.1. Cisco supports the proprietary routing protocol named EIGRP. 2. Many other router vendors, including Cisco, provide routers that support BGP.


®

Cisco ASBR:

RTR-SW

Cisco ASBR:

RTR-NE

Cisco ASBR:

RTR-SE

Area 1.1.1.1 Area 2.2.2.2

Area 3.3.3.3

EIGRP Network

Cisco ASBR:

RTR-NW

OSPF areas may be attached as non-backbone to an EIGRP Autonomous System.All routing between OSPF areas occurs through ASBRs and the EIGRP AS.

Cannot connect a second area to any of the OSPF areas unless second area is a backbone area (Area 0).Permits strategy to insert non z/OS routers as Area 0 at a later date between ASBR and the z/OS non-backbone area or to convert EIGRP to Area 0.Permits strategy to connect another AS to any of the OSPF areas.

A Direct Route between two OSPF areas (e.g., Area 1 to Area 2) does not exist because there is no required backbone area (Area 0) between them.

Potential Area 0.0.0.0

Area 4.4.4.4

Potential RIP or STATIC

AS

EIGRP + OSPF as Non-Backbone

1. ASBRs will be configured to ship only default routes into the OSPF areas. 2. Since there will be relatively small routing trees that need to be maintained within the OSPF areas, a Stub

Area or Totally Stubby configuration in the z/OS area is not necessary. This allows the attachment of RIP or Static Autonomous Systems to any of the OSPF areas because any router in them may be an ASBR.1. Remember: ASBRs cannot import external routes into an OSPF Stub Area.


®

Cisco ASBR:

RTR-SW

Cisco ASBR:

RTR-NE

Cisco ASBR:

RTR-SE

Area 0.0.0.0 Area 0.0.0.0

Area 0.0.0.0 Area 0.0.0.0

EIGRP Network

Cisco ASBR:

RTR-NW

OSPF areas may be attached as backbone areas to an EIGRP Autonomous System.All routing between OSPF areas occurs through an ASBR and the EIGRP AS.The Area 0s are isolated from each other via ASBR connections and are unaware of the existence of another "Area 0." (Ergo: No conflict with multiple Area 0s.) Permits use of EIGRP "redistribute" statements for filtering!A Direct Route between two OSPF areas (e.g., Area 1 to Area 2) does not exist; a virtual link cannot be built between OSPF areas across a separate AS (EIGRP).

EIGRP + OSPF as Backbone

1. OSPF by its very architecture provides for no filtering algorithms as does RIP. This is because OSPF requires that every router in an OSPF area must have the same image of the area topology as every other router. Filtering would prevent the building of identical link-state databases within an area. However, there are means within OSPF to "filter" out advertisements across areas and between autonomous systems. You heard about these methods in the presentation about OSPF architecture:1. Creating Stubby Areas2. Creating Totally Stubby Areas3. Creating an ASBR that does not "import" (or "redistribute") certain types of advertisements into OSPF4. Creating an ABR that suppresses the advertisement of a network through the RANGE statement.

2. By placing EIGRP between two OSPF Autonomous Systems and allowing a Cisco Router to be the ASBR between OSPF and EIGRP, you can take advantage of the Cisco "redistribute" statement to filter OSPF advertisements between the two OSPF AS's.


®

ASBR: RTR-NW

ASBR: RTR-NE

Area 0.0.0.0 Area 0.0.0.0

EIGRP Network

Area 1.1.1.1 Area 2.2.2.2 Area 3.3.3.3 Area 4.4.4.4

An OSPF AS with multiple areas may be attached to an EIGRP Autonomous System.All routing between OSPF areas in an AS occurs through ABRs.

No need for the OSPF areas within the AS to communicate with each other via the EIGRP AS.

The Area 0s are still isolated from each other via ASBR connections and are unaware of the existence of another "Area 0." (Ergo: No conflict with multiple Area 0s.)Area 0's may be physically interconnected to bypass the EIGRP network.

ABRABR ABRABR

EIGRP + OSPF: Multiple Areas

1. This is a variation of the theme introduced on the previous page.


5. Common Problems


®Performance Problems & TimingOSPF is a timer-intensive protocol

If dead router intervals are allowed to lapse without OMPROUTE sending and processing packets from neighbors, network routing problems can ensue

adjacencies droppedsnowballing bad performance as OMPROUTE and neighbors attempt to restore lost adjacencies

routes lost

APAR PQ51196 (z/OS V1R2) added a warning to impending problems:

EZZ7968I Hello interval missed on interface OSAQDIO

APAR II12026, "INFORMATIONAL APAR FOR OMPROUTE PROBLEMS."

1. Many performance and other problems are detailed in the RETAIN Informational APAR II12026, "INFORMATIONAL APAR FOR OMPROUTE PROBLEMS.") We include some of these hints and tips in this presentation, but you will want to consult this APAR frequently for information we have not covered and for any new information.

2. Hello and dead router intervals are a tradeoff. Higher values mean slower convergence of router outages. Lower values mean that adjacencies are quicker to be dropped if there is a temporary holdup in packets

3. It takes a lot more processing to restore lost adjacencies that it does to keep them up in the first place. So if you are losing adjacencies due to performance problems, the problem will only get worse as attempts are made to restore them

4. When adjacencies are dropped, a router deletes from its routing table all routes learned from the dropped neighbor and must relearn them either from another source or by re-establishing the lost adjacency. So network connectivity can be severely affected.

5. Message EZZ7968 is issued if OMPROUTE notices that it has not been able to send a required hello packet within two hello intervals. This indicates that the adjacency is in trouble, probably because OMPROUTE is not getting dispatched enough. If you see this message, immediate attention is needed to ensure OMPROUTE gets the necessary processing. Or perhaps tuning is needed to increase the hello and dead router intervals.

6. ADDITIONAL SYMPTOMS you might notice if this scenario occurs: 1. Storage growth in 4K ECSA - this is queued up protocol traffic which is not being read by OMPROUTE

because it was swapped out or not dispatched. 2. OMPROUTE neighbor state will loop in states 8-16 or 8-16-32 3. OMPROUTE may determine that DEAD_ROUTER_INTERVAL has elapsed, or that the RIP timeout of

3 minutes has elapsed, and will therefore remove dynamically learned routes from the routing table


®DYNAMICXCF: Suppressing Advertisements for XCF Links

PROFILE.TCPIP for Dynamic XCFIPConfig DynamicXCF 192.168.2.2 255.255.255.0 1

Can build: IUTSAMEH device/linkHiperSockets device/link

XCF Connection

But you might want to treat the XCF links differently from the others:InterfaceIP_address = 192.168.2.2Name = EZAXCFSAMTU=4096Subnet_mask = 255.255.255.0;;;;;;;;;;;;; ETC ;;;;;;;;;;;;;;;

Don't import XCF "Interface" into OSPF if you don't want ...

OSPF protocols (Hello, Exchange, etc.) to consume cycles and bandwidth; to have XCF links advertised!!

OSPF_InterfaceIP_address = 192.168.2.2Name = EZASAMEMVSMTU=4096Subnet_mask = 255.255.255.0Cost0=100Router_Priority=1; (default)

OSPF_InterfaceIP_address = 192.168.2.2Name = IQDIOLNKC0A80202MTU=4096Subnet_mask = 255.255.255.0Cost0=100Router_Priority=1; (default)

1. You already know that ... DYNAMIC XCF links are coded in the PROFILE.TCPIP with a statement similar to the following:1. IPConfig DynamicXCF 192.168.2.2 255.255.255.0 12. This definition can build up to three different device/link pairs at the same address.3. This very definition can also generate a HiperSockets DEVICE/LINK if HiperSockets have been enabled in the IOCPgen of a

zSeries platform and no competing HiperSockets DEVICE/LINK has been defined on the same HiperSockets CHPID. (Note that no XCF link is built for the address if a HiperSockets connection to the same target is available.)

4. This definition also generates an IUTSAMEH device/EZASAMEMVS link for the case in which multiple TCP/IP stacks reside in the same MVS image.

2. Although HiperSockets represent a broadcast network, XCF is a point-to-multipoint network. There is no designated router on a point-to-point network, so -- depending on the size of the Sysplex -- you could have fully meshed exchanges of the OSPF protocol packets, leading to an increase in CPU activity. To cut down on this, you could make the XCF connections into an INTERFACE instead of an OSPF_INTERFACE. How could you distinguish among three interfaces that all have the same IP Address?

3. Here's where you could use the NAME parameter on the various Statements in the Omproute Configuration file. 1. If all the interface types are "OSPF_Interface," you may use wildcard addresses, but distinguish one or more individual entries

by their names. If an exact IP address match is not found, then the NAME parameter is used first when searching wildcard addresses. Until you get to IPv6, the NAME parameter cannot be wildcarded.

2. If, however, the XCF address is to be an "Interface" and the HiperSockets and EZASAMEVS links are to be "OSPF_Interface," the OSPF_Interfaces will take precedence and the Interface entry will not be used at all. In such a case, you must build individual, full-octet address entries for each type of interface, with the NAME parameter specified.

3. So, in the example above, you see that we have split out the XCF definitions by specifying an exact NAME match. (All XCF connections built with DYNAMICXCF are given the prefix of EZAXCF with a two-character suffix that matches the SYSID of the partner, as in "SA." If there are 10 members of a SYPLEX with XCF connections, then each OMPROUTE Configuration File would contain 9 definitions (one for each possible partner); to make the coding simpler, you would simply code all 10 individual OSPF_Interface or Interface statements, including yourself, so that the same file could be shared among all members.

4. If all you care about is applying a different COST to the HiperSockets connection, you could simply split out the HiperSockets NAME: IQDIOLNK<Ipaddr in hex> as in "IQDIOLNKC0A80202."


®OSPF Timers;Sample OSPF_INTERFACE (Broadcast: ethernet, token-ring, fddi, etc.): OSPF_Interface IP_Address=9.59.101.5 Name=TR1 Subnet_Mask=255.255.255.0 Attaches_To_Area=2.2.2.2 MTU=1500 Hello_Interval = 10 Dead_Router_Interval = 40 DB_Exchange_Interval=40 Cost0=2 Router_Priority=0;

PQ51195 or z/OS V1R4 and later:Specifying OMPROUTE_OPTIONS=hello_hi coded in the STDENV file or in the 'ENVAR" parameter of the OMPROUTE procedure will change the way OMPROUTE processes the OSPF Hello packets. These packets will be given a higher priority than other updates and processed by the first available OMPROUTE task ahead of other received packets. Prior to specifying this parameter customers must be cognizant of the impact to their network of processing hello packets out of the received order sequence.

//OMPROUTE PROC //OMPROUTE EXEC PGM=OMPROUTE,REGION=4096K,TIME=NOLIMIT, // PARM=('POSIX(ON)', // 'ENVAR("TZ=EST5EDT","OMPROUTE_OPTIONS=hello_hi",', // '"_CEE_ENVFILE=DD:STDENV")/') ...

1. Hello_Interval 1. This parameter defines the number of seconds between OSPF Hello packets being sent out this

interface. This value must be the same for all routers attached to a common network. Valid values are 1 to 255 seconds.

2. Dead_Router_Interval 1. The interval in seconds, after not having received an OSPF Hello, that the neighbor is declared to be

down. This value must be larger than the Hello_interval. Setting this value too close to the Hello_Interval can result in the collapse of adjacencies. A value of 4*Hello_Interval is recommended. This value must be the same for all routers attached to a common network. Valid values are 2 to 65535.

3. DB_Exchange_Interval 1. The interval in seconds that the database exchange process cannot exceed. If the interval elapses, the

procedure will be restarted. This value must be larger than the Hello_Interval. If no value is specified, the DB_Exchange_Interval will be set to the Dead router interval. Valid values are 2 through 65535.

4. STDENV Omproute Environment File coding1. OMPROUTE_OPTIONS=hello_hi

1. Specifying OMPROUTE_OPTIONS=hello_hi changes the way OMPROUTE processes the OSPF Hello packets. These packets are then given a higher priority than other updates and processed by the first available OMPROUTE task ahead of other received packets. Prior to specifying this parameter, customers must be cognizant of the impact to their network of processing hello packets out of the received order sequence.

2. Note: Specifying OMPROUTE_OPTIONS=hello_hi only helps to keep adjacencies up when OMPROUTE is running and getting flooded with protocol packets. It does not provide any help for the case when adjacencies are not staying up because OMPROUTE is not getting enough cycles (that is, swapped out or running in too low a priority).


®The Missing DR! Unreachable VIPAs via iQDIO

D TCPIP,TCP1,N,ROUTE (at TCP1)DESTINATION GATEWAY FLAGS REFCNT INTERFACE192.168.40.51 0.0.0.0 UH 000000 VLINK192.168.47.0 192.168.63.129 UGO 000000 OSAG31L192.168.47.51 192.168.63.133 UGHO 000000 OSAG32L192.168.47.51 192.168.63.129 UGHO 000000 OSAG31L

D TCPIP,TCP1,OMPR,OSPF,IF (at TCP1)IFC ADDRESS PHYS ASSOC. AREA TYPE STATE #NBRS #ADJS192.168.40.51 VLINK 192.168.32.0 VIPA N/A N/A N/A192.168.63.134 OSAG32L 192.168.32.0 BRDCST 32 1 1192.168.63.130 OSAG31L 192.168.32.0 BRDCST 32 1 1192.168.63.149 IQDIOLNK...1 192.168.32.0 BRDCST 32 1 0

TCP1LPAR1

TCP2LPAR2

OSAG41L

iqdiolnk...1

OSAG42L

IP NetworkIP Network

192.168.63.149/30

192.168.63.130/30 192.168.63.134/30

192.168.63.150/30

192.168.63.138/30192.168.63.142/30

VIPA 192.168.40.51/30 VIPA 192.168.47.51/30

iqdiolnk...0

192.168.63.129/30192.168.63.133/30

OSAG32L

OSAG31L

1. Problem: Unable to route to VIPAs over HiperSockets Connection. 1. Note: Due to space limitations, some of the lines have been omitted from the command output displays that you see in the visual and below in

these notes. 2. The VIPAs do not show up in the routing tables as being reachable via the HiperSockets connection.

1. Above in the visual you see the output from the NETSTAT GATE. Here in these notes you see the output from the OMPR,RTTALE display:D TCPIP,TCP1,OMPR,RTTABLEEZZ7847I ROUTING TABLE TYPE DEST NET MASK COST AGE NEXT HOP(S)SPIA 0.0.0.0 0 5 275 192.168.63.129 (2)DIR* 192.168.40.51 FFFFFFFF 1 279 VLINKSPF 192.168.47.51 FFFFFFFF 5 271 192.168.63.129

3. So ... Neither the NETSTAT ROUTE nor the OMPR,RTTABLE display shows HiperSockets as being a path to the VIPAs. Why? Because there is absolutely no Designated Router on the HiperSockets LAN. SOMEONE has to be DR in a Broadcast Network. These two HiperSockets interfaces are not connected to any other OSPF node that functions as DR; therefore one of them must assume the role. (Normally IBM recommends not allowing MVS to be a DR in a broadcast or NBMA network if there is a router out there that can assume the responsibility. In the case of HiperSockets, there is no other router, so one of the MVS's must be the DR.)

4. If you look at the IF display next -- D TCPIP,,OMPR,OSPF,IF -- you notice that the HiperSockets interface "iqdiolnk...1" has no adjacency whatsoever with any other node. (Pay particular attentioin to the final column for iqdiolnk...1.) This explains why the HiperSockets route to the VIPA has not been retained in the routing tables.

5. You need to ensure that the HiperSockets OSPF_Interfaces are DR-capable (Priority must be non-zero).1. Important Note: the default value of router_priority is 1. Therefore you would only see this problem if you had overridden the default

with a value of 0 on each Hipersockets interface. 6. You can set the HELLO and DEADROUTER intervals to be even higher on these two OSPF_Interfaces in order to minimize the keepalive flows if

that is a concern. 7. You may have been asking yourself about the results of the Neighbor display. Here they are:

D TCPIP,TCP1,OMPR,OSPF,NBRS (at TCP1) EZZ7851I NEIGHBOR SUMMARY NEIGHBOR ADDR NEIGHBOR ID STATE LSRXL DBSUM LSREQ HSUP IFC 192.168.63.133 192.168.1.2 128 0 0 0 OFF OSAG32L 192.168.63.129 192.168.63.253 128 0 0 0 OFF OSAG31L 192.168.63.150 192.168.47.51 8 0 0 0 OFF IQDIOLNK...11. Neighbor State of 8 means that the IP interface is "merely a neighbor," but that it has not established adjacencies with the neighbor indicated.

This does not indicate a problem if we are looking at the Neighbor display -- we saw an explanation of this earlier. As long as the interface in question has an adjacency with some neighbor -- it need not be with all neighbors -- then routing will be as expected in the network.


®Why Are Adjacencies Dropped?A hated message:

EZZ7921I OSPF adjacency failure, neighbor 9.5.1.2, old state 128, new state 1, event 12

What happened depends on the event code. The most common events are:

7 -- sequence number mismatchUsually means the neighbor is attempting to restart the adjacency

12 -- no hellos seen recentlyWe haven't heard from the neighbor for a full dead router interval, we assume he's down

15 -- failure to thriveAdjacency was trying to come up, but failed to complete within the DB_EXCHANGE_INTERVAL

1. The message indicating lost adjacencies might have been missed in some installations in which the message was being sent only to SYSLOGD and not to the MVS error log. This has been corrected with the following APAR. . 1. PQ50918 and PQ46720: V1R2 and V2R10

1. Message EZZ7921I being sent only to SYSLOGD and not to MVS error log when designated router adjacency is lost. 1. EZZ7921I OSPF Adjacency Failure, neighbor neighbor, old state state, new state state, event event

2. For reason code 7, usually it means the neighbor has not heard from OMPROUTE for a full dead router interval. So the neighbor marks its adjacency with OMPROUTE down, and then attempts to restart the adjacency. Since OMPROUTE still thinks the adjacency is up, it's not expecting an initial hello so considers it to be a sequence number mismatch. This could indicate that, while OMPROUTE is receiving HELLO packets from the neighbor, for some reason the neighbor is not getting ours.

3. For reason code 12, OMPROUTE is not receiving HELLO packets from the neighbor. This may indicate that the neighbor has actually gone down. If that is not the case, then perhaps the dead router interval is too short for this network design and load. Or perhaps OMPROUTE is not getting dispatched in a timely manner so is not processing hte neighbor's HELLO packets quickly enough

4. Reason code 15 can indicate a death spiral. Recall that it takes more work to restart an adjacency than it does to maintain one. If OMPROUTE is barely able to keep up with hello timers, because of processor or network load, and a bunch of adjacencies drop, then the work of trying to restore them all will not help, and they may not be able to come back. This reason code is governed by the DB_EXCHANGE_INTERVAL. If an adjacency establishment does not fully complete inthat time, it is torn back down and started over. 1. If the DB_Exchange_Interval is too short, Neighbors will cycle between states 8-16-32...8-16-32...8-16-32

and may never actually reach state 128 (full adjacency) . The cycling is shown in EZZ7919I and EZZ7921I messages indicating event 12. You may need to raise this interval.


®Displaying Interfaces and NeighborsD TCPIP,TCPIP,OMPR,OSPF,IF EZZ7849I INTERFACES 153 IFC ADDRESS PHYS ASSOC. AREA TYPE STATE #NBRS #ADJS 1.1.2.201 LNKVIPA1 0.0.0.0 VIPA N/A N/A N/A 192.168.20.2 MPC49LNK 0.0.0.0 P-2-MP 16 1 1 192.168.10.2 MPC28LNK 0.0.0.0 P-2-MP 16 1 1 192.168.110.2 LNKOSAF1 0.0.0.0 BRDCST 32 3 2 192.168.100.2 GIGFE 0.0.0.0 BRDCST 32 3 2

D TCPIP,TCPIP,OMPR,OSPF,NBRS EZZ7851I NEIGHBOR SUMMARY 842 NEIGHBOR ADDR NEIGHBOR ID STATE LSRXL DBSUM LSREQ HSUP IFC 192.168.10.3 206.10.20.1 128 0 0 0 OFF MPC28LNK 192.168.110.4 206.10.20.2 128 0 0 0 OFF LNKOSAF1 192.168.110.3 206.10.20.1 128 0 0 0 OFF LNKOSAF1 192.168.100.3 206.10.20.1 128 0 0 0 OFF GIGFE 192.168.100.4 206.10.20.2 128 0 0 0 OFF GIGFE

A

Interface State of 32 = DR Other (Another node is the DR.)

Neighbor State of 128 = This node has "full adjacency" with the neighbor node.

A

B

B

1. You have seen this visual previously. It shows the DR role that the Broadcast and NBMA interfaces play. The "NBR" display at the bottom of the visual lets you know with which nodes/neighbors you have adjacencies.

2. When you first bring up OSPF with OMPROUTE, issue the Interface and the Neighbor displays that you see here. An interface state shows you the status of an interface and also the status of DR selection. A state of 128 on the Neighbor display indicates that the connection is healthy and that you are fully adjacent.

3. Message EZZ7849I State Values1. 1 = down2. 2 = Backup3. 4 = looped back4. 8 = waiting for DR determination5. 16 = pt-pt6. 32 = DR Other7. 64 = Backup DR8. 128 = DR

4. # NBRs = # of routers whose hellos have been received plus those configured5. # ADJS = # of neighbors in state Exchange or greater; they have synchronized the DB or are in the process of synchronizing6. Message EZZ7851I : This reflects the neighbor states that you heard about in part 1 of this 2-part presentation on OSPF (An

OSPF Tutorial - The Architecture).1. Neighbor ID = Router ID2. 1 = down3. 2 = attempt4. 4 = initializing5. 8 = two-way6. 16 = Exchange Start7. 32 = Exchange8. 64 = Loading9. 128 = Full Adjacency

7. LSRXL = Link-State ReXMIT List8. DBSUM = Size waiting to be sent9. LSREQ = Size requested by neighbors10. HSUP = Hello Suppression


®

D TCPIP,NM1ATCP,OMPR,OSPF,IFS EZZ7849I INTERFACES 520 IFC ADDRESS PHYS ASSOC. AREA TYPE STATE #NBRS #ADJS 192.168.5.168 EZASAMEMVS 1.1.1.1 P-2-MP 16 1 1 192.168.5.168 EZAXCFM2 1.1.1.1 P-2-MP 16 1 1 10.82.5.122 VIPL0A52057A 1.1.1.1 VIPA N/A N/A N/A 192.168.51.168 TRLSM94A 0.0.0.0 P-2-MP 16 1 1 192.168.31.168 TRLSM93A 0.0.0.0 P-2-MP 16 1 1 192.168.11.168 TRLSM92A 0.0.0.0 P-2-MP 16 1 1 172.18.2.168 CTCC128 1.1.1.1 P-P 1 0 0 10.82.5.121 VLINK2 1.1.1.1 VIPA N/A N/A N/A 10.82.5.120 VLINK1 1.1.1.1 VIPA N/A N/A N/A REPLY WITH VALID NCCF SYSTEM OPERATOR COMMAND

How to Track Adjacency Problems (1)

State is "Interface State" -- not "Neighbor State"State "1" = Interface is downState "16" = Interface is Point-to-Point

1. Interface STATE can be one of the following. (Do not confuse these states with Neighbor States.)1. 1 (down)2. 2 (backup)3. 4 (looped back)4. 8 (waiting)5. 16 (point-to-point)6. 32 (DR other)7. 64 (backup DR)8. 128 (designated router)9. N/A (interface is a VIPA and does not actually come up)

2. #NBRS Number of neighbors. 1. This is the number of routers whose hellos have been received, plus those that have been configured.

3. #ADJS 1. Number of adjacencies. This is the number of neighbors in state Exchange or greater. These are the

neighbors with whom the router has synchronized or is in the process of synchronization.


®

D TCPIP,NM1ATCP,OMPR,OSPF,IFS,NAME=EZAXCFM2 EZZ7850I INTERFACE DETAILS 524 INTERFACE ADDRESS: 192.168.5.168 ... INTERFACE TYPE: P-2-MP STATE: 16 DESIGNATED ROUTER: 0.0.0.0 BACKUP DR: 0.0.0.0 DR PRIORITY: 1 HELLO INTERVAL: 20 RXMT INTERVAL: 5 DEAD INTERVAL: 80 TX DELAY: 1 POLL INTERVAL: 0 DEMAND CIRCUIT: OFF HELLO SUPPRESS: OFF SUPPRESS REQ: OFF MAX PKT SIZE: 16384 TOS 0 COST: 1 DB_EX INTERVAL: 80 AUTH TYPE: NONE # NEIGHBORS: 1 # ADJACENCIES: 1 # FULL ADJS.: 1 # MCAST FLOODS: 32 # MCAST ACKS: 20 DL UNICAST: OFF MC FORWARDING: OFF NETWORK CAPABILITIES: POINT-TO-POINT EMULATED-BROADCAST DEMAND-CIRCUITS

How to Track Adjacency Problems (2)

1. Interface STATE can be one of the following. (Do not confuse these states with Neighbor States.)1. 1 (down)2. 2 (backup)3. 4 (looped back)4. 8 (waiting)5. 16 (point-to-point)6. 32 (DR other)7. 64 (backup DR)8. 128 (designated router)

2. Had this been a broadcast connection, the DR and BDR fields would have been filled in with the address of these two nodes. In z/OS V1R5 this display will show N/A for irrelevant fields1. DESIGNATED ROUTER IP address of the designated router. BACKUP DR IP address of the

backup designated router. 3. This display shows you exactly with how many nodes the interface has established FULL Adjacencies.

This is in contrast with the previous display that contained information only on "neighbors" and on "adjacencies."

4. # NEIGHBORS Number of neighbors. This is the number of routers whose hellos have been received, plus those that have been configured.

5. # ADJACENCIES Number of adjacencies. This is the number of neighbors in state Exchange or greater.

6. # FULL ADJS Number of full adjacencies. This is the number of neighbors whose state is Full (and therefore with which the router has synchronized databases).

7. # MCAST FLOODS Number of link state updates that flooded the interface (not counting retransmissions).


®How to Track Adjacency Problems (3)D TCPIP,,OMPR,OSPF,NBRS EZZ7851I NEIGHBOR SUMMARY 015 NEIGHBOR ADDR NEIGHBOR ID STATE LSRXL DBSUM LSREQ HSUP IFC 192.168.5.170 192.168.5.170 128 0 0 0 OFF EZAXCFM2 192.168.11.92 172.16.1.92 128 0 0 0 OFF TRLSM92A

D TCPIP,,OMPR,OSPF,NBRS,IPADDR=192.168.11.92 EZZ7852I NEIGHBOR DETAILS 019 NEIGHBOR IP ADDRESS: 192.168.11.92 OSPF ROUTER ID: 172.16.1.92 NEIGHBOR STATE: 128 PHYSICAL INTERFACE: TRLSM92A DR CHOICE: 0.0.0.0 BACKUP CHOICE: 0.0.0.0 DR PRIORITY: 1 NBR OPTIONS: E DB SUMM QLEN: 0 LS RXMT QLEN: 0 LS REQ QLEN: 0 LAST HELLO: 0 NO HELLO: OFF # LS RXMITS: 1 # DIRECT ACKS: 0 # DUP LS RCVD: 10 # OLD LS RCVD: 0 # DUP ACKS RCVD: 1 # NBR LOSSES: 0 # ADJ. RESETS: 0

1. This example shows some basic neighbor displays. 2. In the second display, note that the neighbor IP address and the OSPF router ID may be different. Recall

that the OSPF Router ID identifies the router as a whole, while the Neighbor IP address identifies the IP address of the interface that attaches the router to the common medium.

3. Since the neighbor is fully adjacent yet there are no Designated Routers nor Backup Designated routers assumed, this is not a multiaccess medium. Beginning in z/OS V1R5 this will be made clearer by using "N/A" for irrelevant fields.


®How Does OMPROUTE Drop Adjacencies

Other load on the z/OS machine keeps OMPROUTE from dispatching enough

taking dumps -- all address spaces marked non-dispatchable during dump processing

if the dump takes longer than a dead router interval, the adjacencies will fail

too many other address spaces running higher priority than OMPROUTE

OMPROUTE should be one less than TCP/IP's dispatching priority

Not enough dispatching priority for OMPROUTE

either increase OMPROUTE's priority or increase the dead router intervals don't run OMPROUTE as BPXBATCH program

1. If you are unwilling or unable to give OMPROUTE the priority and cycles it needs on the machine, than increase the dead router interval so that adjacencies can survive prolonged OMPROUTE starvation

2. When OMPROUTE runs as a BPXBATCH program, it cannot be made non-swappable; i.e., it cannot be placed in the PPT. Furthermore, running it as BPXBATCH makes the job of setting an appropriate WLM Service Class difficult.


®OMPROUTE: High Priority or Service Class

JOBNAME ... DP ... U% Workload SrvClass BPXOINIT FF 4 SYSTEM SYSTEM ... CRON6 FF 4 SYSTEM SYSOTHER CSSVTAM FE 4 SYSTEM SYSSTC ...JESXCF FF 4 SYSTEM SYSTEM JES2 FE 4 SYSTEM SYSSTC ... NM1AOMPR FE 4 SYSTEM SYSSTC NM1ASYSL FF 4 SYSTEM SYSOTHER NM1ATCP FE 4 SYSTEM SYSSTC ...NM1RSLVR FE 4 SYSTEM SYSSTC

To change Service Class: Key over SDSF SrvClass column (requires proper WLM Authority), or RESET NM1AOMPR,SRVCLASS=xxxx, or Make change permanent through WLM displays


®

The following things can cause OMPROUTE performance to be unacceptable even when it gets enough cycles

Too much tracingOMPROUTE debug tracing using file I/O and really hurts performancez/OS V1R5 introduces CTRACE tracing to improve this performance

routed all the way back to OS/390 V2R10 with APAR PQ76172

Too big a route tableOnce OMPROUTE has more than 1000-2000 routes, start expecting troubleTry to keep z/OS out of backbone areas. Stub areas are ideal.

Too many adjacenciesThis may especially be a problem in an XCF environment

do XCF interfaces really have to be OSPF?

Self-inflicted OMPROUTE Performance Problems

1. In z/OS V1R5 OMPROUTE will be able to write its debug trace to CTRACE, which is much more efficient than the current file I/O method


®

Use recommended network design practicesTry to put z/OS and the sysplex into a stub or totally stubby area or isolate areas with BGP or EIGRP

to minimize routing table size and advertisements that have to be processedTry to avoid OMPROUTE becoming designated router on LANs

to mimimize OSPF adjacenciesmay be unavoidable on Hipersockets LAN since no routers attached

OMPROUTE has full OSPF function but it's unlikely you bought a z/OS box to be a network router

let the routers do the routing work unlike on dedicated routers, OMPROUTE has to share machine resources with your business-critical applications!

Only use debug tracing when necessaryuse CTRACE tracing whenever possible

Consider taking XCF links out of the OSPF domainif not used for data trafficcuts down significantly on adjacencies

Get the Most out of OMPROUTE Performance

1. The sysplex function maintains its own routing table. Therefore it is not necessary for OMPROUTE to advertise XCF links for basic sysplex functions like DVIPA and DDVIPA to work properly. If XCF links will not be used for application data traffic, define them to OMPROUTE using INTERFACE statements to avoid adjacencies being set up over them.

2. Because XCF is a point to multipoint medium, if it is in the OSPF domain it will have fully meshed adjacencies instead of the more efficient designated router model of LANS. Fully meshed adjacencies over XCF can really be a load on your system if there are a lot of LPARs in the sysplex.


®Spurious Network Route

EZD0101I NETSTAT CS V1R6 TCPCS3 IPV4 DESTINATIONS DESTINATION GATEWAY FLAGS REFCNT INTERFACE9.67.101.0/29 0.0.0.0 UC 000000 CTC3TO4

If CTC3TO4 is coded in OMPROUTE.CONF: Routing Table Display "NETSTAT ROUTE"

EZD0101I NETSTAT CS V1R7 TCPCS3 IPV4 DESTINATIONS DESTINATION GATEWAY FLAGS REFCNT INTERFACE9.0.0.0/8 0.0.0.0 UC 000000 CTC3TO4

Most likely cause: stack interface(s) not defined to OMPROUTE

z/OS with OMPROUTE

9.67.101.3/29CTC3TO4

10.4.4.4/24INT2

MTU 1492

MTU 1492

Router(Static

Routing) Network with Dynamic Routing

If CTC3TO4 is not coded in OMPROUTE.CONF: Routing Table Display "NETSTAT ROUTE"

Wrong!!

1. Stack interfaces may not be defined to OMPROUTE because they simply were not defined, or because of a typographical error in the definition

2. For IPv4, OMPROUTE sets the BSDROUTINGPARMS MTU value and subnet mask on every interface1. If an interface exists on the stack but is not defined to OMPROUTE, OMPROUTE sets default values for

the MTU and subnet mask, overriding any other BSDROUTINGPARMS definitions1. Default MTU size is 5762. Default subnet mask is the class mask of the interface's home address

1. The default subnet mask could cause undesirable results. For example, an undefined interface with home address 9.67.101.3 would cause OMPROUTE to assign a subnet mask of 255.0.0.0. This in turn would cause OMPROUTE to add a route to the stack's routing table to 9.0.0.0/8 over that interface -- which is probably not was was intended.

2. If OMPROUTE is an Autonomous System Boundary Router and is importing direct routes, it would also advertise that 9.0.0.0/8 route to the rest of the OSPF routers, causing all traffic to unknown destinations in the 9 network to be sent to the TCP/IP stack and then forwarded over that interface!

2. Solution: make sure all stack interfaces are defined to OMPROUTE, even the ones that are not running RIP or OSPF.

3. Because of these problems, IBM has long recommended that every stack interface be defined to OMPROUTE, regardless of whether or not dynamic routing will be used over it.

3. INTERFACE configuration statement used for interfaces over which no dynamic routing will be done.


®Wrong MTU and Routing Doesn't Work!

EZZ7883I Processing interface from stack, address 10.211.63.142, name OSAGIG1L, index 2, flags 441EZZ7871I No matching interface statements for 10.211.63.142 (OSAGIG1L) ...EZZ7865I Class mask 255.0.0.0 being used for interface OSAGIG1L ...EZZ7938I SPF Interface 10.211.63.142 (OSAGIG1D) is not an IP interface, interface not installed

D TCPIP,,N,DEVEZZ2500I NETSTAT CS V1R4 TCP9IP DEVNAME: OSAGIG1D DEVTYPE: MPCIPADEVSTATUS: READY CFGROUTER: NON ACTROUTER: NONLNKNAME: OSAGIG1L LNKTYPE: IPAQENET LNKSTATUS: READY .............. ACTMTU: 8992 BSD ROUTING PARAMETERS: MTU SIZE: 00576 METRIC: 01 DESTADDR: 0.0.0.0 SUBNETMASK: 255.0.0.0 ...

OSPF_Interface IP_Address=10.211.63.142 Name=OSAGIG1D Subnet_Mask=255.255.255.252 MTU=1500 ...

; TCP/IP PROFILEDEVICE OSAGIG1D MPCIPA NONROUTERLINK OSAGIG1L IPAQGNET OSAGIG1D;

WRONG

RIGHT

What??

MTU of 576??Mask of 255.0.0.0??

1. Here is a situation in which the definitions in the OMPROUTE Configuration file are wrong. The clue was that the MTU size for OSAGIG1L is wrong as is the subnet mask, as depicted on the NETSTAT DEV display.

2. The NETSTAT DEV output shows that the GIGOSA has an "ACTMTU" of 8992 but a BSDROUTINGPARMS MTUsize of 576. So either the OSPF_Interface statements are not being processed properly ... or ... something is wrong with the configuration file.

3. The next clue is in the OMPROUTE startup, which shows a series of messages that are very revealing:1. EZZ7871I shows that there is no OSPF_Interface statement for a link that was found named

"OSAGIG1L" and, of course, there isn't. (Because the wrong name is on the OSPF_Interface statement!!)

2. EZZ7883I says that OSPF had to resort to "brute force" to build an Interface named OSAGIG1L and in message EZZ7865I it shows you that it therefore had to ASSUME a native class mask because it wasn't given the proper class mask!!!

3. And finally: message EZZ7938I shows that the OSPF_Interface could not be installed!! (There was no such OSPF interface!) The Name OSAGIG1D is the device name and not the Link name, as is required for an IPv4 OSPF_Interface statement.


®Debugging Undefined Stack Interfaces

If you suspect an undefined stack interface, look for the following message in OMPROUTE trace:

-t1 trace level must be activated

EZZ7966I No matching interface statements for 9.67.101.3 (CTC3TO7)

Starting with z/OS V1R5, you will be able to display undefined interfaces in OMPROUTE:

d tcpip,,omproute,generic,ifsEZZ8060I IPV4 GENERIC INTERFACES IFC NAME IFC ADDRESS SUBNET MASK MTU CFG IGN CTC3TO7 9.67.101.3 N/A N/A NO NO


®Watch out for Multicast Snooping!Some switches offer an optimization called Multicast Snooping, which is not architected in RFCs.

Normally switches should forward all multicast packets to all attached hosts to let them decide if they are interested. But with multicast snooping, the switch listens for multicast joins and drops and does not forward multicast packets if it thinks a host is not in a multicast group.

IBM

Host

Switch

LAN

IGMP message: "Join 224.0.0.5"

IGMP message: "Join 224.0.0.5"

IBM

Host

Switch

LAN

IBM

Host

Switch

LAN

(remembers thathost dropped 224.0.0.5)

IGMP message: "Drop 224.0.0.5"


(remembers thathost joined 224.0.0.5)

packet dest: 224.0.0.5

(does not forwardpacket because "knows" host notinterested)


®The Problem with Multicast SnoopingSwitches identify hosts by MAC addresses.

multiple LPARs on a z/OS box which are sharing an OSA port have the same MAC address, and the switch doesn't know there are multiple "hosts" if the switch sees a drop of an OSPF multicast address from a MAC address, it stops forwarding of all OSPF multicast packets to that MAC address.

Therefore, if one LPAR drops the OSPF multicast group, any other LPARs on that box that are running OSPF will be crippled!

LAN

IBMSwitch

2 IGMP messages: "Join 224.0.0.5"

(only remembers that the shared MAC address has joined 224.0.0.5)

Join


(only remembers that the shared MAC address has dropped 224.0.0.5, doesn't realizethere is still a host up there that still belongs to the group)

Join

LPAR1 LPAR2

IBMSwitch

LPAR1 LPAR2

IBMSwitch

LPAR1 LPAR2

224.0.0.5 packetfor LPAR1

Drop

1. This problem could be overcome if switches that implemented multicast snooping kept a count and only stopped forwarding packets for a multicast group when there have been as many drops as joins

2. However, the designers of this optimization did not do that, most likely because they did not think that multiple hosts could be using the same MAC address.


®Multicast Snooping Conclusion

Don't enable multicast snooping on a switch when: it is attached to a z/OS box running multiple LPARs with shared OSAs, andmore than one of those LPARS will be running OMPROUTE (or any application or service that uses multicast, for that matter)

If you don't know what you're looking for, problems caused by multicast snooping can be very hard to debug

because the multicast packets that the switch drops don't show up at all in OMPROUTE or TCP/IP traces.

Symptoms can include:Adjacencies dropping for no apparent reasondifferent routers on the same LAN having different neighbor listsmore than one designated router on a LAN network


®

Totally Stubby Area 1.1.1.1

Area 0.0.0.0

I can't route out of my totally stubby area even though I have redundant routers!

HOST11 HOST14

ABRs interconnected only via Link in Totally Stubby Area; two Default Gateways for each HOSTnn.LSA about loss of connectivity to subnet 9.67.103.0 via ABR #2 is NOT sent into Totally Stubby Area 1.1.1.1 by either ABR #1 or ABR #2, since only defaut routes are sent into Totally Stubby Areas.As far as hosts in the Totally Stubby Area know, all destinations outside the area can be reached via either ABR. They will continue sending packets for 9.67.103.0 to both ABR #1 and/or ABR#2 (depending on multipath settings). The packets sent via ABR #2 wil not be deliveredThis is an undesirable way to configure a Totally Stubby Area. It violates the rule that it should never matter which ABR is used to reach any destination outside the area.

Def

ault

via

#1

Def

ault

via

#2

9.67.112.0

ABR #1 ABR #2HOST01 HOST02

9.67.102.0 9.67.103.0OSPF Router

1. This page and the next one are also in the appendix of the OSPF Tutorial presenation (N03)2. One ABR in a stub area does not accept the summary default route from another ABR in the same stub area. In other words,

ABR #2 will not accept the stub area summary default route from ABR #1, so packets sent to the backbone via ABR #2 cannot be delivered by going to ABR #1 through the totally stubby area

3. Dead gateway processing may kick in when packets fail to be delivered via ABR #2, but only in some cases, and its recovery is not robust enough to give the redundancy that should be expected when two ABRs attach a totally stubby area to the backbone.


®


Area 0.0.0.0

How to Interconnect ABRs for Totally Stubby Area

Def

ault

via

#1

Def

ault

via

#2

9.67.102.0 9.67.103.0OSPF Router

ABRs interconnected via Link in Backbone Area 0 AND Link in Area 1; two Default Gateways for each HOST1n.

As before, LSA about loss of connectivity to subnet 9.67.103.0 via ABR #2 is NOT sent into Totally Stubby Area 1.1.1.1 by either ABR #1 or ABR #2 since only default routes are sent into Totally Stubby AreasPackets sent to ABR#2 for 9.67.103.0 will still reach the destination, via ABR#1 and the link in Area 0HOST11 and HOST14 know of two default gateways; regardless of which gateway they choose (they have coded IPCONFIG MULTIPATH), the packets can arrive at 9.67.103.0

9.67.112.0

HOST14HOST11

ABR #2HOST02

ABR #1HOST01


6. Common Configuration Complaints


®My Router Folks Want a "255" Mask

Subnet_Mask The subnet mask of the subnet to which this interface attaches. This value must

be the same for all routers attached to a common network. For VIPA use new SUBNET values to work around this limitation (described in later visuals)

OMPROUTE.Conf Coding for Dynamic VIPAs

OSPF_InterfaceIP_address = 192.168.1.*Subnet_mask = 255.255.255.248Attaches_To_Area=1.1.1.1Subnet=NoMTU=65535;

OMPROUTE

TCPIP Stack#1192.168.1.1192.168.1.2192.168.1.3192.168.1.4192.168.1.5192.168.1.6

Subnet 192.168.1.0

Host 192.168.1.1

Host 192.168.1.n .....

Host 192.168.1.6

OSPF Network

1. Many router administrators are used to being able to code a host route mask for individual IP addresses when they code static routes. The host route mask is 255.255.255.255 or 32 bits long in IP Version 4.

2. z/OS System Programmers are often pressed to code this host route mask in the OMPROUTE configuration file, when, in fact, this is a violation of the RFCs. 1. It is also unnecessary because many types of interfaces, including point-to-point and VIPA interfaces,

will automatically have both their host routes and their subnet routes broadcast into the network. 3. New SUBNET parameters (described later) also provide a workaround for this limitation


®My Router Folks Don't Like Host Routes PROFILE.TCPIP for Dynamic VIPA Block

VIPADYNAMIC VIPADEFINE 255.255.255.248 192.168.1.001 ........................................................................ VIPADEFINE 255.255.255.248 192.168.1.006 ENDVIPADYNAMIC


OSPF_InterfaceIP_address = 192.168.1.*Subnet_mask = 255.255.255.248Attaches_To_Area=1.1.1.1Subnet=NoMTU=65535;

But don't suppress host routes unless you have

to!!

Subnet = No (default) Send Host Routes plus Subnet RoutesSubnet = Yes Send Subnet Routes ONLY

1. Many router networking technical people are overwhelmed if they have to manage routing tables for a plethora of host routes. In some cases it is possible to suppress host route advertisements. In OMPROUTE the parameter SUBNET may be used to suppress host routes on certain types of interfaces if necessary and to broadcast only subnet routes for certain interfaces. However, heed the warning note in the description of this parameter below.

2. Subnet = YES | NO on point-to-point lines1. For an interface to a point-to-point serial line, the option "SUBNET=YES" enables the advertisement of a

stub route to the subnet that represents the serial line rather than the host route for the other router's address. In effect, this parameter controls whether, for this interface, OMPROUTE implements option 1 (SUBNET=NO) or option 2 (SUBNET=YES) described in RFC 2328 (OSPF version 2) section 12.4.1.1.

3. Subnet = YES | NO | NOVIPAHOST | NOVIPASUBNET for VIPA Interfaces1. By default, OMPROUTE advertises both host and subnet routes for a VIPA into the OSPF autonomous

system. In some environments it may be undesirable to advertise both host and subnet routes.2. For a VIPA interface, the option "SUBNET=YES" suppresses advertisement of the VIPA host route.

Normally CS for z/OS advertises both a host route and a subnet route for owned VIPA interfaces. With this option set to YES, only the subnet route will be advertised. 1. Note: Do not use this option of SUBNET=YES for dynamic VIPAs or for any VIPA whose subnet

might exist on multiple hosts. If you do, problems can occur routing to all VIPAs that share the subnet.3. For a VIPA interface, the option "SUBNET=NO" is the default. "SUBNET=NO" means that both host

routes and subnet routes will be advertised.4. CONTINUED on NEXT VISUAL.


®Subnet= no | yes | novipahost | novipasubnet OMPROUTE.Conf Coding for Dynamic VIPAs

OSPF_InterfaceIP_address = 192.168.1.*Subnet_mask = 255.255.255.248Attaches_To_Area=1.1.1.1Subnet=NOVIPAHOSTMTU=65535;

Subnet = No (default) Send Host Routes plus Subnet RoutesSubnet = Yes Send Subnet Routes ONLY

Subnet=NOVIPAHOST Send Subnet Routes ONLYSubnet=NOVIPASUBNET Send Host Routes ONLY -- provides

"hostmask-like" function

APAR PQ82792:V1R4 and V1R5

1. This page is a continuation of the previous page. It deals with enhancements to the SUBNET= parameter in the OMPROUTE Configuration file.

2. Subnet = YES | NO | NOVIPAHOST | NOVIPASUBNET for VIPA Interfaces1. By default, OMPROUTE advertises both host and subnet routes for a VIPA into the OSPF autonomous

system. In some environments it may be undesirable to advertise both host and subnet routes.2. For a VIPA interface, the option "SUBNET=YES" suppresses advertisement of the VIPA host route. Normally

CS for z/OS advertises both a host route and a subnet route for owned VIPA interfaces. With this option set to YES, only the subnet route will be advertised. 1. Note: Do not use this option of SUBNET=YES for dynamic VIPAs or for any VIPA whose subnet might

exist on multiple hosts. If you do, problems can occur routing to all VIPAs that share the subnet.3. For a VIPA interface, the option "SUBNET=NO" is the default. "SUBNET=NO" means that both host routes

and subnet routes will be advertised.4. With "SUBNET=NOVIPAHOST" -- introduced via APAR PQ82792 for z/OS V1R4 and V1R5 -- the VIPA host

route will be suppressed and only the VIPA subnet route will be advertised. With this option set to NOVIPASUBNET, the VIPA subnet route will be suppressed and only the VIPA host route will be advertised.1. Specifying SUBNET=YES on a VIPA interface has the same effect as specifying SUBNET=NOVIPAHOST. 2. Note: Do not use this option of SUBNET=NOVIPAHOST for dynamic VIPAs or for any VIPA whose subnet

might exist on multiple hosts. If you do, problems can occur routing to all VIPAs that share the subnet.5. With "SUBNET=NOVIPASUBNET" -- introduced via APAR PQ82792 for z/OS V1R4 and V1R5 -- the VIPA

subnet route will be suppressed and only the VIPA host route will be advertised. This causes OMPROUTE to behave as if a subnet mask of 255.255.255.255 were coded, which works around the limitation that does not permit the use of that mask in the configuration file. 1. In order to fully suppress the VIPA subnet route, SUBNET=NOVIPASUBNET must be specified on every

VIPA OSPF_INTERFACE statement that defines a VIPA in a common subnet.


®

VIPA Advertisements V1R6: Advertise_VIPA_Routes

Advertise_VIPA_Routes=Host_and_Subnet (default)

Send Host Routes plus Subnet Routes

Advertise_VIPA_Routes=Subnet_Only

Send Subnet Routes ONLY

Advertise_VIPA_Routes=Host_Only

Send Host Routes ONLY -- provides "hostmask-like" function


OSPF_InterfaceIP_address = 192.168.1.*Subnet_mask = 255.255.255.248Attaches_To_Area=1.1.1.1Advertise_VIPA_Routes=SUBNET_ONLYMTU=65535;

Advertise_Vipa_Routes= ...(a more user-friendly alternative to the SUBNET parms described on the previous page)

Be careful with "Subnet_Only" - See Notes

1. This page is a continuation of the previous page. It deals with the new Advertise_VIPA_ROUTES = parameter on the OSPF_INTERFACE statement in the OMPROUTE Configuration file introduced in z/OS V1R6.

2. Advertise_VIPA_Routes = HOST_ONLY | SUBNET_ONLY | HOST_AND_SUBNET for VIPA Interfaces1. This is exactly equivalent to using SUBNET=... It was added to be more user friendly and less cryptic than the SUBNET=...

parameters described on the previous page. 3. Advertise_VIPA_Routes = HOST_ONLY

1. This is equivalent to SUBNET=NOVIPASUBNET4. Advertise_VIPA_Routes = SUBNET_ONLY

1. This is equivalent to SUBNET=NOVIPAHOST or SUBNET=YES5. Advertise_VIPA_Routes = HOST_AND_SUBNET

1. This is equivalent to SUBNET=NO6. You can use either parameter to accomplish this, but "Advertise_VIPA_Routes" is the preferred method. This new parameter is

available on CS for z/OS V1R6 and beyond. 7. NOTE: The value specified on the ADVERTISE_VIPA_ROUTES option will override any value specified on the SUBNET option. 8. RULE: Do not specify SUBNET_ONLY for dynamic VIPAs or for any VIPA whose subnet might exist on multiple hosts.

Problems can occur routing to all VIPAs that share the subnet when the subnet exists on multiple hosts. 9. TIP: The HOST_ONLY option must be specified for every VIPA in a common subnet. If the HOST_ONLY option is not specified

for every VIPA in a common subnet, OMPROUTE will still advertise the VIPA subnet route for the interfaces not specifying HOST_ONLY.


Information Sources


®

URL Content

http://www.ibm.com/servers/eserver/zseries

http://www.ibm.com/servers/eserver/zseries/networking

http://www.ibm.com/servers/eserver/zseries/networking/technology.html

http://www.ibm.com/software/network

http://www.ibm.com/software/network/commserver

http://www.ibm.com/software/network/commserver/library

http://www.redbooks.ibm.com

http://www.rfc-editor.org/rfcsearch.html

http://www.ibm.com/support/techdocs/

IBM Enterprise Servers (z900 & S/390)

z900 Networking

Networking White Papers and Information

Networking & Communications Software

Communications Server

CS White Papers, Product Doc, etc.

ITSO Redbooks

RFCs

Advanced Technical Support (Flashes, Presentations, White Papers, etc.)

For More Information ...

Whitepaper on OSPF with IBM & CISCO:"OSPF Design and Interoperability Recommendations for Catalyst 6500 and OSA-Express Environments" http://www-1.ibm.com/servers/eserver/zseries/networking/pdf/ospf_design.pdf

Redbook on "Networking with z/OS and Cisco Routers: An Interoperability Guide (SG24-6297)


®Bibliography

RFCs 1583 & 2328 OSPF V2 by John Moy: URL http://www.ietf.cnri.reston.va GG24-3376 TCP/IP Tutorial and Technical ReferenceSC31-8513 OS/390 Communications Server: IP ConfigurationHuitema, Christian Routing in the Internet, Prentice HallBlack, Uyless IP Routing Protocols: RIP, OSPF, BGP, PNNI & CISCO

Routing Protocols (SR23-9498)Parkhurst, William R. Cisco Router OSPF Design and Implementation Guide,

Parkhurst McGraw-Hill (SR23-8683)Cisco Systems CCIE Fundamentals: Network Design and Case Studies,

2nd Edition (SR23-9437)Doyle, Jeff (Cisco Systems)

CCIE Professional Development: Routing TCP/IP, Volume 1 (SR23-9241)

SG24-5631 SecureWay Communications Server for OS/390 V2R8 TCP/IP Guide to Enhancements

SG24-5227 V2Rn TCP/IP Implementation Guide, Vol. 1 (Configuration and Routing)

SG24-5228 V2Rn TCP/IP Implementation Guide, Vol. 2 (UNIX Apps)SG24-5229 V2Rn TCP/IP Implementation Guide, Vol. 3 (MVS Apps)FLASH N3196 TCP/IP Migration Tips & Hints (V2R8) at

www.ibm.com/support/techdocsFLASH N3190 OMPROUTE and VIPA at www.ibm.com/support/techdocs


®Bibliography V1R2 Redbooks

SG24-5227-03 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 1: Base and TN3270 Configuration

SG24-5228-03 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 2: UNIX Applications

SG24-6516-00 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 4: Connectivity and Routing

SG24-6517-00 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 5: Availability, Scalability, and Performance

SG24-6839-00 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 6: Policy and Network Management

SG24-6840-00 Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 7: Security

SG24-5229-01 OS/390 eNetwork Communications Server V2R7 TCP/IP Implementation Guide Volume 3: MVS Applications

SG24-6297-00 Networking with z/OS and Cisco Routers: An Interoperability Guide

1. These redbooks are all now available from www.redbooks.ibm.com. 2. The new Volume 4 contains information not only on implementing OSPF in z/OS, but also

information on integrating with Cisco Routers, including the integration with EIGRP.3. Note also that the MVS Applications volume of the redbook series has not been updated

since OS/390 V2R7, although there have been enhancements to several of the socket applications discussed there. Notably, the CICS sockets application has received important enhancements in the z/OS V1R2 release. These CICS enhancements are included in one of the presentations in this course.

4. The final redbook mentioned also contains information on using EIGRP and BGP with Cisco.


Appendix: Integrating LINUX on zSeries into a

Network with z/OS

"It depends. And with LINUX, it depends even more ...."

1. This section on integrating LINUX into an OSPF network approaches the integration from the perspective of OSPF best practices.

2. This part of the presentation therefore makes little attempt to say that one way to integrate "performs better" or "is better" from another. This is because factors like "Total Cost of Ownership," adapter speeds, path lengths through various types of operating system code, and so on can influence an implementer's or user's perspective of what is "better."

3. For more information on Linux and VM routing and bridging technologies discussed herein, be sure to attend relevant sessions at this and other conferences.


®Physical Network Design for zSeries Linux

LINUX for zSeries

LPAR1

Interface 1 Interface 2

LINUX1 for

zSeries

LPAR1

Interface 2

LINUX2 for

zSeries

Interface 1

z/VM

Direct Attachment to Physical Interface (LCS, Pt-to-Pt, QDIO, HiperSockets, etc.)1. Native LPAR2. Under z/VM

1 2

Direct Attachment to z/VM1. Virtual CTC (VCTC)2. IUCV3. Guest LAN

LPAR1

LINUX1 for

zSeries

LINUX2 for

zSeries

4

z/VMGuest LAN

Interface 2Interface 1

LINUX1 for

zSeries

LPAR1

LINUX2 for

zSeriesz/VM

Virtual CTCIUCV

Interface 2Interface 1

3z/VM

Virtual Switch Later!

1. zSeries LINUX may be attached to a multitude of physical interfaces when it resides standalone in an LPAR or a physical zSeries footprint. (See Visual 1.) In this case, the LINUX operating system and TCP/IP stack "own" and control the physical interfaces.

2. zSeries LINUX may have dedicated physical interfaces of many types when it is operating as a Guest under z/VM. (See Visual 2.) In this case, the z/VM operating system has dedicated the physical interfaces to the LINUX operating system with LINUX controlling/activating the physical interfaces to the network.

3. z/VM can support LINUX guests by connecting to them in any of several ways.1. If using VCTC or IUCV connections (i.e., point-to-point connections), the z/VM system can use PROXYARP to respond to

requests for addresses that share the same IP subnet as the physical LAN attachment of the z/VM system itself. This sometimes facilitates routing in the network, particularly when there is a dearth of addresses that can be handed out to the new LINUX guests. Alternatively, PROXYARP can be eliminated if a different IP subnet has been assigned to each of the IUCV and VCTC interfaces. (See Visual 3.) In this case, the physical interfaces are owned and controlled by z/VM; the "virtual" CTC and IUCV interfaces are owned on one end by the LINUX guests and on the other end by z/VM.

2. Guest LAN support is the recommended way to attach Guest LINUX systems, as future support will be invested here versus with VCTC or IUCV connections. (See Visual 4.) In this case, the physical interfaces are owned and controlled by z/VM; the connections to the Guest LAN are owned on one end by the LINUX guests and on the other end by z/VM.

3. z/VM V4R2 supports Guest LANs that emulate Hipersockets connectivity. Because this is an emulation, the actual machine on which z/VM is running need not be a zSeries platform. HOWEVER, z/VM V4.2 Guest LANs do not support broadcast or multicast mode. Therefore, at z/VM V4.2, MPROUTE cannot communicate with z/VM Guest LINUXes over the Guest LAN using OSPF or RIPv2/v1 protocols. (Recall that OSPF generally requires multicast; RIPv2 requires multicast; RIPv1 requires broadcast mode.) 1. NOTE: DRNEIGHBOR Coding could be employed to use OSPF over non-multicast interfaces, but this solution is not optimal

in light of other, more elegant solutions for LINUX integration. Therefore, DRNEIGHBOR coding will not be addressed in this section.

2. NOTE: As a result of the points just made regarding multicast and broadcast, static routing is required over the Guest LANs at the z/VM V4.2 level.

4. Starting with z/VM V4.3 Guest LANS may also be defined to emulate QDIO mode LANs. QDIO-mode Guest LANs at V4.3 support both multicast and broadcast; HiperSockets Guest LANs at V4.3 support multicast only. So, at VM V4.3, MPROUTE with Multicast can be used over the Guest LAN interfaces.

5. At z/VM V4.4 more capability has been added to Guest LAN support. One new feature in z/VM V4.4 is the use of the Virtual Switch. You will see this in a later visual. Another enhancement is the support of broadcast for HiperSockets Guest LANs. Yet another enhancement is the support of VLANs (802.1q) for Guest LANs.


®z/VM as ASBR for LINUX Guests

z/OS11 z/OS12

Area 0.0.0.0ABR11

Range 10.1.0.0/16ABR21

Range 10.1.0.0/16

AS1Static

Routing

AS2 OSPF RoutingReal HiperSockets 10.1.4.0 / 24

2 Autonomous SystemsStatic Routing for LINUXNon-Backbone Area ConfigurationHiperSockets Between VM and z/OSLANs and HiperSockets Between VM and z/OS

Reach LINUX via External Routes to 10.1.1.0 over OSA-E or HiperSockets

PRIROUTERPRIROUTER

Routing Burden on z/VM

Non-Backbone Area 1.1.1.1

Linux for

zSeries

.2 .3

10.1.2.1 / 24 10.1.3.1 / 24

> Default is z/VM

Linux for

zSeries> Default is z/VM

ASBR (MPROUTE)> Static Route to 10.1.1.0 / 24

HiperSockets or QDIO10.1.1.0 / 24

z/VM

.1

OSA-E (QDIO)OSA-E (QDIO)

10.1.2.0 / 24

10.1.3.0 / 24

.1

1. Note in our diagram how the z/OS images and z/VM itself are attached to two different OSA-Es, which are presumably attached to the rest of the network in such a way as to provide redundancy. z/VM and z/OS are also attached to each other via HiperSockets.1. The z990 has introduced "adapter interrupt handling" in the Guest Environment to optimize the performance of Guests accessing

HiperSockets and QDIO adapters.2. Please imagine that the ABRs in the network provide redundancy to reach any of the depicted LANs. (There were too many lines to

draw, so we appeal to your imagination for this.) 3. This diagram depicts zSeries LINUX guests under z/VM. z/VM is acting as a router to the guests. It communicates with the LINUX

guests over a Guest LAN using address 10.1.1.1. 1. If using Guest Virtual LAN support, the z/VM system cannot use Proxy Arp. Remember that External Routes (LSA Type 5) are not

sent into Stub Areas; this is why our configuration has z/VM and z/OS in a non-backbone area that is allowed to receive LSA Type 5s.

2. Since z/VM itself can support the OSPF dynamic routing protocol via the MPROUTE code (ported from z/OS Communications Server's OMPROUTE), we can configure z/VM as a node in the OSPF AS. (z/VM supports MPROUTE starting with z/VM V3R1. ) And yet, note how z/VM is communicating with the LINUX guests using STATIC routing protocols. Since z/VM must be able to advertise the static routes to the rest of the Non-Backbone Area 1.1.1.1, including to the z/OS images, as well as to the ABRs that interconnect Area 1.1.1.1 with Area 0.0.0.0, z/VM is configured as an Autonomous System Boundary Router (ASBR).

3. Note how the z/VM LPAR has still been designated the PRIROUTER for the shared OSAs; otherwise the packets destined for the LINUX guests in subnet 10.1.1.0/24 would be rejected. (OSAs must either recognize the full host address of received packets or be allowed to recognize unknown host addresses in order to pass the packets up to the IP stack for further routing.)

4. This solution is all right, but ... depending on the size of the Autonomous System and the number of other types of AS's, the routing tables could be very large. It might be better to come up with another network design that minimizes the routing table at each node and the amount of computation required to build a routing table.

4. So, in summary, our configuration shows:1. Two Autonomous Systems

1. OSPF + Static Routes2. Area 1.1.1.1 (a non-backbone area able to receive LSAs for external routes)

1. z/VM = ASBR2. z/OS = Intra-Area Routers

3. Areas 1.1.1.1 and 0.0.0.01. ABRnn = Area Border Routers


®

OEM ASBR:

RTR-NE

Area 1.1.1.1 Area 2.2.2.2 EIGRP or BGP

Network OEM ASBR:

RTR-NW

OSPF areas may be attached as non-backbone to an Autonomous System of another protocol (e.g., EIGRP or BGP)

All routing between OSPF areas occurs through ASBRs and the intermediate AS.

Permits strategy to connect another AS to any of the OSPF areas.

For example, LINUX Static ASMinimizes Routing Table size at z/OS, z/VM, and LINUX.

LINUX STATIC AS

Separating z/VM + LINUX from z/OSz/OS Images z/VM ASBR + LINUX Images

1. In this configuration we have z/VM with LINUX in one Area and z/OS in another. We have also placed each area in a separate OSPF Autonomous System (AS).

2. The non-zSeries ASBRs will be configured to ship only default routes into the OSPF areas. 3. Since there will be relatively small routing trees that need to be maintained within the OSPF areas, a Stub

Area or Totally Stubby configuration in the z/OS area is not necessary. This allows the attachment of a Static Autonomous System for LINUX Guests to the z/VM OSPF areas because any router in them may be an ASBR. It also allows the z/VM system to be an ASBR in its own right (as you saw in the previous visual) and to interface with the non-zSeries ASBR, which can perform extra filtering to reduce the size of the routing tables at z/OS, z/VM, and LINUX.1. Remember: ASBRs cannot import routing destinations into OSPF Stub Areas.


®

z/OS11 z/OS12

Area 0.0.0.0ABR11

Range 10.1.0.0/16ABR21

Range 10.1.0.0/16

AS1Static

Routing


PRIROUTERPRIROUTER

z/VM is a minimal Router


10.1.2.0 / 24

2 Autonomous Systems, but Static Routing is invisible to OSPF!Static Routing at LINUX but not at z/VM!Totally Stubby Area ConfigurationHiperSockets and QDIO LANs between VM and z/OSReach LINUX via OSPF Intra-Area Routes to 10.1.1.0 over OSA-E or HiperSockets

LINUX Guests: z/VM & z/OS in Stub Area

Linux for

zSeries

.2 .3

10.1.2.1 / 24 10.1.3.1 / 24

> Default is z/VM

Linux for

zSeries> Default is z/VM

HiperSockets or QDIO10.1.1.0 / 24

MPROUTE: > OSPF_Interface for 10.1.1.1 / 24

z/VM

.1

OSA-E (QDIO)OSA-E (QDIO) 10.1.3.0 / 24

.1

1. Note in our diagram how all of the z/OS images and z/VM itself are attached to two different OSA-Es, which are presumably attached to the rest of the network in such a way as to provide redundancy. z/VM and z/OS are also attached to each other via HiperSockets. 1. The z990 has introduced "adapter interrupt handling" in the Guest Environment to optimize the performance of Guests accessing

HiperSockets and QDIO adapters.2. Again, please imagine that the ABRs in the network provide redundancy to reach any of the depicted LANs. (There were too many

lines to draw, so we appeal to your imagination for this.) 3. This diagram depicts the zSeries LINUX guests under z/VM. Again, z/VM is acting as a router to the guests - however its routing

duties are not heavy. It communicates with the guests over a Guest LAN. The advantage to this configuration is that z/VM and the z/OS images reside in a Totally Stubby Area, communicating with each other via Intra-Area Routes and with Default Routes to reach the backbone area and any other areas or autonomous systems beyond the two Area Border Routers (ABRs).

4. The LINUX guests are using static routing that points to the z/VM router as the default router over the HiperSockets or QDIO Guest LAN. However, the z/VM attachment to the Guest LAN is defined as an OSPF_Interface so that the z/VM node can advertise the Guest LAN IP subnet address throughout the Totally Stubby Area 1.1.1.1 and to the ABRs (ABR11 and ABR21). Although the z/VM interface to the Guest LAN sends out periodic HELLOs, the LINUX Guests will not respond since they have not joined the appropriate OSPF multicast group. In fact, even if z/VM is at a level that does not support multicast over the Guest LAN, thus preventing the OSPF_Interface from even receiving HELLO responses (were they possible), the MPROUTE code will still send out LSAs about the subnetwork 10.1.1.0 over the HiperSockets and the QDIO interfaces.

5. Note how the z/VM LPAR has still been designated the PRIROUTER for the shared OSAs; otherwise the packets destined for the LINUX guests in subnet 10.1.1.0/24 would be rejected. (OSAs must either recognize the full host address of received packets or be allowed to recognize unknown host addresses in order to pass the packets up to the IP stack for further routing.)

6. Although our Real HiperSockets connections connect only as far as z/VM, we could have extended the HiperSockets all the way to the LINUX guests. You'll see a variation on this next.

7. So, in summary, our configuration shows:1. Two Autonomous Systems

1. OSPF + Static Routes (an AS hidden from OSPF)2. Area 1.1.1.1 (a Totally Stubby Area)

1. z/VM and z/OS communicate with each other via intra-area routes2. z/VM and z/OS communicate with the rest of the network by using the Default routes sent to them by the ABRs (ABR11 and

ABR21), which have connections to the Totally Stubby Area and to Area 0.3. Area 0.0.0.0 (backbone area)

1. Used to reach other OSPF Areas in the same Autonomous System or to reach other ASs that are not depicted.


®

z/OS11 z/OS12

Area 0.0.0.0ABR11

Range 10.1.0.0/16ABR21

Range 10.1.0.0/16

AS1Static

Routing


PRIROUTERPRIROUTER


LINUX Using z/OS as a Router in Stub

Linux for

zSeries

.2

10.1.2.1 / 24 10.1.3.1 / 24

> Default is z/OS

Linux for

zSeries> Default is z/OS

OMPROUTE: > OSPF_Interface for 10.1.1.1n / 24 & 10.1.4.1n

z/VM

.1

OSA-E (QDIO)OSA-E (QDIO) 10.1.3.0 / 24

2 Autonomous Systems, but Static Routing is invisible to OSPF!Static Routing at LINUX but not at z/OS!Totally Stubby Area ConfigurationHiperSockets & QDIO LANs between VM & z/OSHiperSockets LAN between z/OS, z/VM, & LINUXUsing HiperSockets Accelerator at z/OS.

HSA HSA

.3

z/VM TCP/IP with Static Routing or Dynamic Routing

HSA is very efficient for

HiperSockets

10.1.2.0 / 24

1. This diagram is quite similar to the previous one -- with a major exception: z/OS is acting as the "minimal" router for the z/LINUX images instead of z/VM, as was the case in the previous diagram. Of course, there are a few other differences as well.

2. This diagram depicts the zSeries LINUX guests under z/VM. However, in this scenario, z/VM is NOT A ROUTER. z/OS takes over the ROUTING role and uses HiperSockets Accelerator function to reduce the CPU requirements for z/OS. z/OS communicates with the guests and with z/VM over a HiperSockets connection -- we are not using a Guest LAN in this configuration, since each LINUX owns a connection to the HiperSockets LAN as does z/VM. We still have the Stubby Area we had before, but now z/OS can be designated the PRIROUTER for the shared OSAs.1. Note: The use of "HSA" at the z/OS nodes to indicate "HiperSockets Accelerator" can cause confusion because HSA"

also stands for the "Hardware System Area." We use "HSA" here only because of space limitations. 3. Note in our diagram how all of the z/OS images and z/VM itself are attached to two different OSA-Es, which are

presumably attached to the rest of the network in such a way as to provide redundancy. z/VM and z/OS are also attached to each other via HiperSockets. 1. The z990 has introduced "adapter interrupt handling" in the Guest Environment to optimize the performance of Guests

accessing HiperSockets and QDIO adapters.4. Again, please imagine that the ABRs in the network provide redundancy to reach any of the depicted LANs. (There were

too many lines to draw, so we appeal to your imagination for this.) 5. If a client in the network beyond the ABRs need to reach z/VM, he can do so by means of the OSAs, since z/VM's TCP/IP

will download its own addresses for the HiperSockets connection and the QDIO attachments into the OSAs. 6. You might still configure z/VM with a TCP/IP stack that is using MPROUTE, as you saw in the previous diagram. In such a

case, z/VM s part of the Totally Stubby Area, but it can advertise connections to the HiperSockets and QDIO LANs. Alternatively, you can choose to use static routing at z/VM so as to essentially eliminate dynamic routing overhead completely.

7. Of course, if static routing is being used at z/VM you have the option of making the ABRs into ASBRs and allowing them to import the static host routes at z/VM and advertise them into the backbone. However, this is probably not necessary since the subnet routes are already being advertised ito the backbone because of the OSPF protocols at z/OS. Once the network packets reach the OSA LANs, the ARP process will ensure that z/VM is reached over the OSA attachments. The z/LINUX guests are reached over the z/OS images that are using the HiperSockets Accelerator function and acting as minimal routers themselves.


®Physical Network Design for VSWITCH

Virtual Switch ("VSWITCH") in z/VM V4.4VM Control Program bridges to the AdapterGuest Machines are on same subnet as QDIO Adapter of Guest LAN.Direct Attachment to Physical QDIO Interface via Bridge

Traditional Layer 3 routing can be combined with VSWITCH to provide increased flexibility and availability.

Router can be: z/VM Guest, z/OS Guest, LINUX Guest

LPAR1

1

LAN 2LAN 1

z/VMGuest LAN

LINUX2 for

zSeries

LINUX3 for

zSeriesz/VM

LPAR1

2

LAN 2LAN 1

LINUX2 for

zSeries

LINUX3 for

zSeries

TCP/IPwith

Routing: z/VM, z/OS, LINUX

LAN 0

z/VM

z/VMGuest LAN

qdio qdio qdio qdio qdio

1. The "Virtual Switch" (or "VSwitch"), a layer-2 routing technique, was announced together with other new functions for z/VM V4.4 in Announcement Letter 203-128. The Announcement Letter, dated May 13, 2003 with availability August 15, 2003, is entitled: 1. "IBM z/VM V4.4 Improves Virtualization Capabilities for Linux on zSeries"

2. What is a "Virtual Switch"?1. In a nutshell, the Virtual Switch allows you to connect guests over a Guest LAN that resides in the same

IP subnet as the real QDIO adapter. (HiperSockets Guest LANs cannot take advantage of the Virtual Switch configuration.) VM's Control Program (CP) bridges the Guest LAN to the real QDIO LAN. The z/VM Control Program and the associated Virtual Switch controller (i.e., a copy of the z/VM TCP/IP stack itself) push the Guest LAN addresses into the QDIO adapter. Which addresses are pushed down is a function of the operating system that the VM Guest is using. z/VM and z/OS Guests are similar to each other in this respect; zSeries LINUX is different.

2. The z/VM Virtual Switch employs transparent bridging to enable the switch to dynamically determine and maintain node connectivity so that the LAN administrator has less network maintenance to perform. The z/VM Virtual Switch also adds 802.1q VLAN support, although a discussion of this particular feature goes beyond the scope of this presentation.

3. The Virtual Switch supports both broadcast and multicast mode. Many configurations will use just simple static routing; but other configurations may warrant the implementation of OSPF over the Virtual Switch.


®

Virtual Switch (10.1.2.0)

Linux3Linux2

10.1.2.0 / 24

10.1.2.210.1.2.3OSA1

.2 .3

z/VM V4.4 and the VSWITCH Concept

Default Router

z/VM V4.4

I am ready for some easy

pictures now!

1. We will get back to our look at LINUX in an OSPF network later. But for now, we need to look at some simple pictures to understand the Virtual Switch.

2. Here you see two LINUX Guests using the z/VM V4.4 feature called "Virtual Switch." 3. The virtual switch allows transparent bridging to the OSA adapter (and also introduces VLAN 802.1q capability to the VSWITCH

connections). 4. Note how the addresses on the VSWITCH belong to the same IP Subnet as the LAN Segment to which the OSA is attached. 5. Note how the all the addresses known to the VSWITCH are downloaded into the OSA adapter. Thus, any packets from the

network that are destined for addresses known to the OSA Adapter will be forwarded to those addresses; the OSA will not discard packets to known IP destination addresses on the zSeries platform.

6. Since the Guest Machine images appear attached to the OSA itself, these images can bypass the z/VM routing function and connect directly to an external network over a QDIO OSA-E. Any other node besides z/VM -- a downstream router, for example -- can function as the default router for the Guest Machine, thus reducing the CPU load on z/VM. The Guest Machine (e.g., LINUX, z/OS, z/VM) configuration with VSWITCH looks like a flat network, since there is no z/VM router in between the Guest and the external network. Every node serviced by the virtual switch has an adapter coupled directly to the virtual switch’s LAN.

7. For more information about Virtual Switch, see www.vm.ibm.com. Here are some excerpts from the z/VM V4.4 Announcement Letter 203-128, "IBM z/VM V4.4 Improves Virtualization Capabilities for Linux on zSeries". It is here for your reference only. :

8. "Network Consolidation using the Virtual Switch: z/VM V4.4 further enhances virtualization technology by introducing a virtual IP switch that is capable of bridging a z/VM Guest LAN to an associated real LAN connected by an OSA-Express adapter. The z/VM virtual switch is designed to help eliminate the need for virtual machines acting as routers to provide IPv4 connectivity to a physical LAN through an OSA-Express adapter. Further, it helps eliminate the need to define a separate routable subnet for the exclusive use of the members of a Guest LAN. Using the virtual switch, the convenience of a Guest LAN is maintained while allowing the guests to be assigned IP addresses in the real LAN subnet."

9. "Virtual routers consume valuable processor cycles to process incoming and outgoing packets requiring additional copying of the data being transported. The virtual switch helps alleviate this problem by moving the data directly between the real network adapter and the target or originating guest data buffers."

10. "Centralized network configuration and control of the virtual switch within CP helps allow the z/VM Guest LAN administrator to more easily grant and revoke access to the real network and to manage the configuration of Guest LAN VLAN segments. While the z/VM system can be a member of multiple VLANs, the z/VM Guest LAN administrator can control which guests belong to which real VLAN, without requiring additional network adapters or switch port configuration. If a guest does not support IEEE 802.1q, z/VM will transparently join the virtual network interface into the desired VLAN."


®

Virtual Switch (10.1.2.0)

Linux3Linux2

10.1.2.1 10.1.3.1110.1.2.210.1.2.3OSA0

10.1.3.1110.1.2.1 OSA

PRIROUTER

VM TCP/IPMPROUTE

(Static?) .2 .3.1.11

z/VM V4.4

VSWITCH with a Router on zSeries

10.1.2.0 / 24

10.1.3.0 / 24

1. Recall that a Guest Machine with TCP/IP Routing Function may be added to the VSwitch configuration to provide increased flexibility and availability. . That Guest machine could be running z/VM, z/OS, or zSeries LINUX.

2. Here we have added a z/VM TCP/IP machine attached both to an OSA-E in its own right and also to the VSWITCH at 10.1.2.0. The TCP/IP stack could be using static routing or dynamic routing. The advantage to using any kind of routing stack in front of network adapters is that you may use multipath outbound. The VSWITCH direct bridging cannot perform multipath over multiple OSA adapters; it can perform multipath only over the Guest LAN if the guest has two interfaces to the guest LAN. Note which addresses have been downloaded into OSA0:1. z/VM's TCP/IP sends all addresses known to itself down to the attached OSA: 10.1.3.11 and 10.1.2.1.2. No LINUX machine has direct connectivity to the OSA0, so none of the LINUX addresses are downloaded onto that QDIO

adapter. 3. Note which addresses have been downloaded into OSA1:

1. z/VM's TCP/IP sends all addresses known to itself down to the attached VSWITCH: 10.1.3.11 and 10.1.2.1. 2. LINUX2 and LINUX3 send their addresses on LAN Segment 10.1.2.0 to OSA1: 10.1.2.2 and 10.1.2.3.

4. Packets coming in over LAN Segment 10.1.2.0 will reach the LINUX Guests directly, as the OSA knows and accepts these destination addresses, since they have been downloaded as previously described. The packets could even reach the z/VM TCP/IP stack at address 10.1.2.1; again, this address is known to the OSA card.

5. Packets coming in over LAN Segment 10.1.3.0 will reach the z/VM TCP/IP stack at address 10.1.3.11. Such packets could be routed out of z/VM TCP/IP over the VSWITCH into the LINUX Guests if z/VM were made PRIROUTER on OSA0.

6. In case of a complete VSWITCH failure, Linux can still get out through VM TCP/IP because the Guest LAN portion of the Virtual Switch continues to operate. Likewise and in the reverse direction, packets needing to reach the LINUX guests can use z/VM's TCP/IP as a router. Of course, z/VM must be the PRIROUTER for the OSA so that packets to destinations unknown to OSA0 (i.e., LINUX2 and LINUX3) may be accepted by the OSA0 and forwarded through z/VM TCP/IP using z/VM's routing protocol.1. Note: Although our diagram depicts z/VM as the router for this configuration, in fact, z/OS could serve this purpose.

7. Just in case you were wondering if you could attach both the z/VM TCP/IP stack to the same OSA as the one to which the VSWITCH is attached, the answer is "yes." 1. You might even add a second OSA and a second VSWITCH to which VM TCP/IP and a different set of Guest LANs are

attached. 2. For more information about these types of configurations, please consult the literature on designing with VSWITCHes under

z/VM V4.4.8. IBM has announced Layer 2 Switching function for zSeries LINUX. Its availability is dependent on its incorporation into the

appropriate LINUX distribution. Please attend other presentations on LINUX to learn the details.


®

z/OS11 z/OS12

Area 0.0.0.0ABR11 & ASBR

Range 10.1.0.0/16ABR21 & ASBR

Range 10.1.0.0/16

AS1Static

Routing


PRIROUTERPRIROUTER


2 Autonomous Systems, but Downstream Routers must be ASBRs to import Static Routes from LINUX!Totally Stubby Area Configuration at z/OSHiperSockets between LINUX and z/OS not advertised to rest of networkRest of network can reach LINUX or z/OS via ASBR/ABRs

.20 .3010.1.3.0 / 24

z/VM

QDIO VSWITCHes

Bridge,Use FLAT

Network for z/VM; very efficient.

Linux for

zSeries> Default is OEM Router

Linux for

zSeries> Default is OEM Router

10.1.2.1 / 24 10.1.3.1 / 24

10.1.2.0 / 24

LINUX & VSwitch: 2 Subnets, Shared OSAs

OSA-E (QDIO)OSA-E (QDIO)

10.1.2.0 / 24

10.1.3.0 / 24

HSA HSA

1. Here you see a configuration with z/VM and LINUX residing in a STATIC Autonomous System. The Virtual Switch creates a "Flat Network" look with no router between the VM Guests and the OSA QDIO Adapters themselves. Note how z/VM does not own any IP address on the Guest LAN because it will not be functioning as a router.

2. Again, please imagine that the ABRs/ASBRs in the network provide redundancy to reach any of the depicted LANs. (There were too many lines to draw, so we appeal once again to your imagination for this.)

3. Note that our visual now shows the Virtual Switch bridging directly to the adapter. Finally, note how HiperSockets connections are defined via static routing in the LINUX images and in z/OS. Since these connections will be used ONLY for z/OS-LINUX communications, there is no need for these static routes to be imported into OSPF for broadcasting through the OSPF AS; as a result, our z/OS images may remain in a Totally Stubby Area.

4. The LINUX guests point not to z/VM as their default router, but to some external OEM router. In our visual the router might be ASBR11 or ASBR21, depending on the physical connectivity. 1. The z990 has introduced "adapter interrupt handling" in the Guest Environment to optimize the performance of

Guests accessing HiperSockets and QDIO adapters. 5. LINUX and z/OS can communicate with each other over the HiperSockets path or by using the shared OSA path.6. With a FLAT Network design you may lose the traffic splitting afforded by dynamic routing over the OSAs to the

LINUX guests unless you manage traffic distribution inbound by some means other than dynamic routing with OSPF. Of course, the static routing protocols at the downstream router might allow you to entertain two equivalent paths. Outbound the LINUX Guests may use multipath only on the Guest LAN itself if they have multiple interfaces to it. Although CP does not support Multipath on the physical adapters -- even if they reside on the same subnet -- CP does provide for automatic failover. Automatic failover with CP takes place no matter which IP subnet the multiple QDIO adapters are attached to; however routing is not successful unless the failover QDIO adapter is attached to the same subnet as the failing QDIO adapter. (CP does not support ARP Takeover as does z/OS.) 1. Although our network diagram is therefore not optimally configured for CP automatic failover since the OSAs are

on different LAN Segments, the network does have redundant paths with HiperSockets connections and z/OS configured as PRIROUTER. In fact, we also have z/OS configured for HiperSockets Accelerator should it becomenecessary to use z/OS as a router.


End

z/OS OMPROUTE Hints and Tips - IBM WWW PageFILE/3930_OSPF_hints_and_tips.pdf · z/OS OMPROUTE Hints...

Documents

Transcript of z/OS OMPROUTE Hints and Tips - IBM WWW PageFILE/3930_OSPF_hints_and_tips.pdf · z/OS OMPROUTE Hints...