Parallel Sysplex Implementation

Parallel Sysplex CenterPoughkeepsie, NY USA

Trademarks

The following terms used in this presentation are trademarks of the IBM Corporation

ƒ ACF/VTAMƒ CICS/ESAƒ DB2ƒ DFSMS/z/OSƒ ESA/390ƒ ESCONƒ ES/9000ƒ IBMƒ z/OS/ESA SPƒ PR/SMƒ PS/2ƒ RACFƒ S/390ƒ SYSPLEX TIMERƒ VM/ESAƒ VTAM

Overview– Pre-Sysplex Multisystem Applications

z/OS Parallel Sysplex–Components/Terminolgy–Multisystem Sysplex Requirements–IBM 9037 Sysplex Timer–XCF Signalling Requirements–XCF Sysplex Couple Data Sets–GRS in a Sysplex–Parallel Sysplex Resource Managers–Policies

Agenda

Parallel Sysplex Implementation StepsSummaryParallel Sysplex Extensions

–z/OS Logger–Automatic Restart Manager (ARM)–Sysplex Failure Manager (SFM)

Agenda (cont.)

Pre-Sysplex Multisystem Applications

CICS IMS JES2

VTAM

GRS

JES3

CONSOLES

JES2 IMS CICS

VTAM

GRS

JES3

CONSOLES

CTC

CTC

CTC

MCS MCSJES3MVS MCS

MVS

z/OS Parallel Sysplex

Multiple systems coupled together by hardware and software services that can be viewed as a single entity.

Improves: AvailabilitySystems ManagementGrowth

121

2

3

4

56

7

8

9

10

11

CouplingTechnology

Shared data

Sysplex Timer

ESCON/FICON

9672zSeries

S/390 Sysplex Components/Terminology

Sysplex A set of 1-32 z/OS systems coupled togetherusing hardware and software elements.

XCF Cross-system Coupling Facility-provides the z/OS communications services within a Sysplex.Multisystem

ApplicationAn authorized application that has functions distributed across more than one z/OS system.CTC Channel to Channel connection-direct connection between two z/OS systems (ESCON CTC, 3088).Sysplex

Couple Data Set

A data set shared by all XCF components in a sysplex. Contains XCF-related data for the sysplex.

Sysplex Timer

IBM 9037 Sysplex Timer-external clock used to synchronize the TOD clocks of the systems.

Member A specific function of a multisystem application that is defined to XCF and assigned to a group.Group A collection of related members.

Serialization The function provided by GRS that is used to serialize resources used by XCF and other users.

S/390 Parallel Sysplex Components/TerminologyParallel Sysplex

2 or more z/OS images in a Sysplex using the coupling facility to share data (up to 32)XES Cross-system Extended Services-provides the z/OS communication services to the coupling facility

Coupling Facility

Hardware element that provides high-speed caching, list processing, and locking functionsCoupling

Facility Link/Channel

High bandwidth fiber optic link that provides high speed connectivity required for data sharing between the CF and attached CPCsStructures Dynamically partitioned areas of storage within a coupling facility manipulated by z/OS/ESA system services

Data Sharing

The ability of concurrent subsystem programs to directly access and change data with integrity.Data

Sharing Group

Collection of two or more subsystems that access and share data in the coupling facility

Role of the CF

LOCKS DATA BUFFERS

DATABASE MANAGER

DATA BUFFERS

DATABASE MANAGER

z/OS Sysplex Services

Coupling Technology

REQUESTS REQUESTS

z/OS

S/390

z/OS

S/390

LocksListsCaches

LOCKS

Multi-SystemSerializationChanged Data

Multisystem Parallel Sysplex Requirements

Requirements for all systems in an z/OS Sysplex:

Be connected to a common time sourceUse the same Couple Data SetsBe in the same GRS configurationHave full connectivity to every other system in the SysplexHave unique system names within the SysplexSpecify the same Sysplex nameHave connectivity to all CFs

IBM 9037 Sysplex Timer

S0000 Main MenuTime HelpPort

sProfile

LogMaint

01/09/9009:17:23

UTCStatus changed. View log for details.F1=Help

9037 SysplexTimer Unit

(TOD) Clock Management using ETR Synchronization Mode

TOD values kept in synchronization with the Sysplex TimerNo operator intervention required to set TOD clocksExpanded availability configuration is strongly recommended

Sysplex Modes

XCF

APPLICATION

XCF

APPLICATION

XCFLocal

XCFMonoplex

XCF

CTCs

Couple Data sets

Timer

GRS

APPLICATION

XCF

GRS

APPLICATIONXCF Multisystem

Otherz/OSWork

Otherz/OSWork

Otherz/OSWork

Otherz/OSWork

Couple Data sets

XCF-Multisystem Sysplex ConfigurationsXCF

XCF

GRS

GRS

APPL

APPL

Multiple Physical Processors

XCF

XCF

GRS

GRS

APPL

APPL

Multiple LPARs on a Single Processor

XCF

XCF

GRS

GRS

APPL

APPL

Multiple VM Guests

*

**Note: Sysplex Timer and CTC considerations

Processor 1

Processor 2

z/OS 1

z/OS 1

z/OS 2

z/OS 2

Full Connectivity to all systems is required– Redundant configurations are recommended

Potential configurations include:–ESCON Channels operating in CTC mode (Point-to-Point)

ƒ Channels supporting CTC communications must be paired; one operating in CTC (Type=CTC) mode and the other operating in native mode (Type=CNC)

–ESCON Channels with ESCON Directors (ESCD)ƒ Reduces the number of CTCs that must be usedƒ Allows the native ESCON channels to be used for attaching other

ESCON devices–PR/SM LPARs with ESCON Multiple Image Facility

ƒ Using shared channels further reduces channel requirementsƒ Signalling structures

z/OS Sysplex Connectivity

XCF CTC Signalling Requirements

Outbound Inbound

Inbound Outbound

System A

System BConnecting Link

At least one connecting link must be defined between each system in the Sysplex - A connecting link consists of a pair of inbound-outbound signalling paths between systems - Signalling paths are uni-directional - Recommended minimum is 4 signalling paths between each system (2 inbound and 2 outbound)

XCF Logical connections:

XCF CTC Signalling Requirements . .

Outbound Inbound

Inbound Outbound

Connecting Link

XCF Logical connections:

ESCON CTC Physical connections

600 100

601 101

SYS1 SYS2

XCF

XCF

PATHOUT (600)PATHIN (601)

PATHIN (100)PATHOUT (101)

SYS1 SYS2 ctc cnc

System 1 System 2 CHPID 03 CHPID 22

System 1 I/O Definition

System 1 System 2

CHPID PATH=((03)),TYPE=CTCCNTLUNIT CUNUMBR=600,PATH=((03)), UNITADD=(00,2)),UNIT=SCTCIODEVICE ADDRESS=(600,2),CUNUMBR=(600), UNIT=SCTC

System 2 I/O Definition

CHPID PATH=(22)),TYPE=CNCCNTLUNIT CUNUMBR=100,PATH=((22)), UNITADD=(00,2)),UNIT=SCTCIODEVICE ADDRESS=(100,2),CUNUMBR=(100), UNIT=SCTC

XES signalling provides the ability to establish XCF signalling connectivity through the coupling facility

– In addition to, or instead of, the existing signalling capability using CTC links

XES automatically configures paths to all systems using a common CF list structure

–Defining a coupling facility list structure as an outbound definition for XCF will cause an XCF signalling path to be established with every other system that has the structure defined as an inbound path

–A coupling facility list structure can be defined and be used for both outbound and inbound XCF signalling paths

XES Signalling Structures

InstallationUnidirectional, point-to-point; Minimum CTC paths to

provide full connectivity for 8 system Sysplex is 56 (8*(8-1)); 112 for redundancy

Bidirectional, one-to-many; one coupling facility list structure can support more than an 8 system Sysplex; 2 for redundancy

AdministrationPARMLIB, I/O Definition, Shared COUPLExx member not

practical because of CTC addressesPARMLIB, Administrative policy, shared COUPLExx

member possible if all systems share the same list structure(s) and all other XCF parameters are the same

RecoveryManualXES and Coupling Facility initiated

XCF CTC

XES Signalling

XES Signalling

XES Signalling

XCF CTC

XCF CTC

XCF CTC Signalling versus XES Signalling

COUPLE SYSPLEX (PLXM) PCOUPLE (SYS1.XCF#PLXM.CDS01) ACOUPLE (SYS1.XCF#PLXM.CDS02)

PATHIN STRNAME(IXCSTR1) PATHOUT STRNAME(IXCSTR2)

Signalling Definitions

Coupling Facility

CTC COUPLE SYSPLEX (PLXM) PCOUPLE (SYS1.XCF#PLXM.CDS01) ACOUPLE (SYS1.XCF#PLXM.CDS02)

PATHIN DEVICE(E40,E41) PATHIN DEVICE(890,891)

PATHOUT DEVICE(100,101) PATHOUT DEVICE(200,201)

XCF Sysplex Couple Data Set

Required for XCF-Monoplex or XCF-Multisystem Sysplex modesAn alternate couple data set may be specifiedCouple data set(s) must be shared by all systems in the SysplexSome information kept in the couple data set includes:

–Names of the systems in the Sysplex–Status of the systems in the Sysplex–A system status field that is periodically updated by each system to indicate its "health"

–Names of groups and members–Member status for members using permanent status recording

All systems in the Sysplex will go in wait state, if couple data set(s) are not usable

Defined in PARMLIB; May be added or removed using SETXCF

Shared data repository for the systems in a Sysplex

XCF Sysplex Couple Data Set . . .

Avoid placing Couple Data set on volume with high use data setsAvoid a volume that has the RESERVE macro issued against itNormal I/O activity - Status Updates - 2 I/Os every 3 seconds per system in Sysplex

Additional I/O activity–I/O to the Couple Data Set for every action message in Sysplex (for example, WTOR)

–Modifications to the Sysplex (for example, activation of a new group)–During recovery

GRS Multisystem (Global) serialization is required for XCF-Multisystem Sysplex mode

In a multisystem Sysplex GRS can use lock structure to manage GRS configuration. Benefits:–Improved Performance

All systems in the Sysplex must be in the same GRS configuration

Global Resource Serialization in a Sysplex

z/OS z/OS

RSA

RNLs RNLs

CTC

GRS

GRS

XCF

XCF

z/OS Parallel Sysplex Resource Manager Policies

Sysplex-wide administrative policies–Automated, system-managed goals–Changes communicated to all systems in the sysplex–Dynamically changed using commands–IPL not required

z/OS Coupling Facility Resource Manager (CFRM) Policyz/OS Sysplex Failure Mangement (SFM) Policyz/OS Workload Manager (WLM) Policyz/OS Automatic Restart Manager (ARM) Policyz/OS Logger Inventory

Parallel Sysplex Implementation Steps

Planning for a z/OS Parallel Sysplex

Review:

–Sysplex Implementation Requirements

ƒ z/OS Sysplex and S/390 Parallel Sysplex

ƒ Software/Hardware

–Network Considerations

–Potential Applications

–Operations Changes

–Testing Requirements

Customization Tasks for Parallel Sysplex

ƒ SYS1.PARMLIBIEASYMxx

SYSNAME, SYMDEF definitions

IEASYSxxSYSNAME, GRS, GRSCNF, CLOCK, PLEXCFG, CON, COUPLE

CLOCKxxGRSCNFxxGRSRNLxxCONSOLxxCOUPLExx

SYSPLEX name, COUPLE data sets, Signalling paths

ƒ Format Couple data setsƒ IPL with new configurationƒ Bring other systems into the SYSPLEX

IEASYMxx Member of PARMLIB

SYSDEFSYSCLONE(&SYSNAME(3:2))SYSPARM(PS)SYMDEF(&SYSR1='BMRS11')SYMDEF(&SYSR2='&SYSR1(1:5).2')SYMDEF(&GRSPARM=‘STAR’)SYMDEF(&COUPLE=‘PS’)

SYSDEF HWNAME(AB02) LPARNAME(Z021) SYSNAME(ZMB0) SYMDEF(&APPN='8B') SYMDEF(&SSCPID='811') SYSDEF HWNAME(AB01) LPARNAME(Z011) SYSNAME(ZMB1) SYMDEF(&APPN='8C') SYMDEF(&SSCPID='812')

IEASYSxx Member of PARMLIB

PLEXCFG=ANY,GRS=&GRSPARM,GRSRNL=PS,CON=PS,CLOCK=PS,COUPLE=&COUPLE,

CLOCKxx Member of PARMLIB

ETRMODE YESETRZONE YESETRDELTA 10TIMEZONE E.08.00.00

GRSRNLxx Member of PARMLIB

/* System EXCLusion List */ RNLDEF RNL(EXCL) TYPE(SPECIFIC) QNAME(SYSDSN) RNAME(SYS1.LOGREC)

/* System INCLusion List */ RNLDEF RNL(INCL) TYPE(GENERIC) QNAME(SYSDSN)

/* System CONversion List */ RNLDEF RNL(CON) TYPE(PATTERN) QNAME(*)

SYSPLEX provides the capability to dynamically change the GRS Resource Name List

Implementing Parallel Sysplex

ƒ Format SYSPLEX couple data setsUpdate COUPLExx for signalling paths

ƒ Format other couple data setsCoupling Facility Resource Management (CFRM)Sysplex Failure Management (SFM)Workload Management (WLM)Automatic Restart Management (ARM)

ƒ Define CF structuresXCF/XES signallingVTAM JES2 RACFIRLM LOGGERothers......

Sysplex Couple Data Set

Required for:XCF-Multisystem and XCF-Monoplex

Must be shared by ALL systems in sysplexAvoid placing on busy volumesDefine primary and alternate

Alternate is optional but recommendedDefined by IXCL1DSU batch utility

IXCSYSPF in SYS1.SAMPLIB

Control Statement:

DEFINEDS DSN(SYS1.XCF#PLXM.CDS01) VOLSER(BMCD01) SYSPLEX(PLXM) MAXSYSTEM(6) CATALOG DATA TYPE(SYSPLEX) ITEM NAME(GROUP) NUMBER(20) ITEM NAME(MEMBER) NUMBER(50)

COUPLE SYSPLEX (PLXM) PCOUPLE (SYS1.XCF#PLXM.CDS01) ACOUPLE (SYS1.XCF#PLXM.CDS02)

PATHIN STRNAME(IXCSTR1) PATHIN DEVICE(E40,E41) PATHIN DEVICE(890,891) ... ...

PATHOUT STRNAME(IXCSTR1) PATHOUT DEVICE(100,101) PATHOUT DEVICE(200,201) ... ...

Signalling Definitions - COUPLExx

Coupling Facility

CTCs

Coupling Facility Resource Manager Policy

Coupling Facility Resource Manager (CFRM) policy

Defines Coupling Facilities (CFs) to be usedƒ Type of Coupling Facilityƒ CPCIDƒ Sequence Numberƒ etc.....

Defines structures and their placement in the CFsƒ Preference List, Exclusion List

Batch Administrative Utility

Define CFRM Couple Data Set

ƒ IXCCFRMF member in SYS1.SAMPLIBExecute IXCL1DSU batch utility Alternate CFRM couple data set is HIGHLY recommended

Control Statements:

DEFINEDS SYSPLEX(PLXM) DSN(SYS1.CFR#PLXM.CDS01) VOLSER(BMCD02) CATALOG DATA TYPE (CFRM) ITEM NAME(POLICY) NUMBER(6) ITEM NAME(CF) NUMBER(5) ITEM NAME(STR) NUMBER(20) ITEM NAME(CONNECT) NUMBER(32)

Define CFRM Policy

ƒ IXCCFRMP member in SYS1.SAMPLIB

Execute IXCMAIPU administrative data utility CFRM couple data set must be pre-formatted

IXCMAIPU administrative data utility is used to addupdate, or delete policy data Structure sizes must be estimated

CFSizer has recommendations

RACF protect IXCMAIPUResource name ‘MVSADMIN.XCF.CFRM' in FACILITY class

DATA TYPE(CFRM) REPORT(YES) DEFINE POLICY NAME(PLXM) REPLACE(YES) CF NAME(C033) TYPE(002064) MFG(IBM) PARTITION(3) SEQUENCE(000000019B1A) CPCID(00)

CF NAME(C021) TYPE(002064) ... ... STRUCTURE NAME(IXCPATH1) SIZE(16316) PREFLIST(C033,C021) REBUILDPERCENT(1) STRUCTURE NAME(ISTGENERIC) SIZE(10000) PREFLIST(C033,C021) STRUCTURE NAME(ISTMNPS) SIZE(12288) PREFLIST(C021,C033) STRUCTURE NAME(IXCPATH2) SIZE(16316) PREFLIST(C021,C033) EXCLLIST(IXCSTR1)

CFRM Policy Control Statements

Example:

C033

C021

IIXCPATH1

ISTGENERIC

ISTMNPS

IXCPATH2

Activating CFRM Policy

Make CFRM Couple Data Set available to z/OS:

SETXCF COUPLE,PCOUPLE=SYS1.CFR#PLXM.CDS01,TYPE=CFRMSETXCF COUPLE,ACOUPLE=SYS1.CFR#PLXM.CDS02,TYPE=CFRM

Activate CFRM Policy:

SETXCF START,POLICY,POLNAME=PLXM,TYPE=CFRM

Only one member of the sysplex needs to activate the policy

Update COUPLExx Member of PARMLIB

COUPLE SYSPLEX(&SYSPLEX) PCOUPLE(SYS1.XCF#&SYSPLEX..CDS01) ACOUPLE(SYS1.XCF#&SYSPLEX..CDS02)

DATA TYPE(CFRM) PCOUPLE(SYS1.CFR#&SYSPLEX..CSD01) ACOUPLE(SYS1.CFR#&SYSPLEX..CDS02)

PATHOUT STRNAME(IXCSTR1) PATHIN STRNAME(IXCSTR2)

z/OS Workload Manager Policy

z/OS Workload Manager (WLM) Policy

Performance goalsNot parameter driven

ƒ Replaces SRM IPS & ICSƒ Business importance

ISPF Dialogs to define

Define WLM Couple Data Sets

ƒ IWMFTCDS member in SYS1.SAMPLIBExecute IXCL1DSU batch utility Alternate WLM couple data set is strongly recommendedDefine resource name MVSADMIN.WLM.POLICY in the FACILITY class of RACFUse PERMIT to allow access

Control Statements:

DEFINEDS SYSPLEX(PLXM) DSN(SYS1.WLM#PLXM.CDS01) VOLSER(BMCD01) CATALOG DATA TYPE (WLM) ITEM NAME(POLICY) NUMBER(6) ITEM NAME(WORKLOAD) NUMBER(35) ITEM NAME(SRVCLASS) NUMBER(100)

Activating WLM Policy

Make WLM Couple Data Set available to z/OS:

SETXCF COUPLE,PCOUPLE=SYS1.WLM#PLXM.CDS01,TYPE=WLMSETXCF COUPLE,ACOUPLE=SYS1.WLM#PLXM.CDS02,TYPE=WLM

Activate WLM Service Policy:

VARY WLM,POLICY=wlm_policy_name

Don't forget to update COUPLExx member

Sysplex Failure Management Policy

Sysplex Failure Mangement (SFM) policy

Automates Sysplex recovery actions:ƒ Loss of connectivityƒ System status update missingƒ Storage Reconfiguration

Batch Administrative Utility

Sysplex Failure Management

z/OS A

VTAM NNJES2CMC

962

Example

z/OS BVTAM ENJES2Test onlines

390 micro

VTAM ENJES2 Prod Onlines

390 micro

z/OS C

Sysplex Failure Management

z/OS A

VTAM NNJES2CMC

962

Example

z/OS BVTAM ENJES2Test onlines

390 micro

VTAM ENJES2 Prod Onlines

390 micro

z/OS C

Sysplex Failure Management

z/OS A

VTAM NNJES2CMC

962

Example

z/OS BVTAM ENJES2Test onlines

390 micro

VTAM ENJES2 Prod Onlines

390 micro

z/OS C

WT 100 WT 30

WT 80

Define SFM Couple Data Set

ƒ IXCSFMF member in SYS1.SAMPLIBExecute IXCL1DSU batch utility Alternate SFM couple data set is recommended

Control Statements:

DEFINEDS SYSPLEX(PLXM) DSN(SYS1.SFM#PLXM.CDS01) VOLSER(BMCD01) CATALOG DATA TYPE (SFM) ITEM NAME(POLICY) NUMBER(6) ITEM NAME(SYSTEM) NUMBER(8) ITEM NAME(RECONFIG) NUMBER(4)

Define Sysplex Failure Management Policy

ƒ IXCSFMP member in SYS1.SAMPLIB

Execute IXCMAIPU administrative data utility SFM couple data set must be pre-formatted

IXCMAIPU administrative data utility is used to addupdate, or delete policy data

RACF protect IXCMAIPUResource name ‘MVSADMIN.XCF.SFM' in FACILITY class

DATA TYPE(SFM) REPORT(YES) DEFINE POLICY NAME(NORMAL) CONNFAIL(YES) REPLACE(YES)

SYSTEM NAME(*) WEIGHT(1)

ISOLATETIME(0) SYSTEM NAME(ZMB0) WEIGHT(40) SYSTEM NAME(ZMB1) WEIGHT(40) SYSTEM NAME(ZMB3) WEIGHT(10)

RECONFIG FAILSYS(SYSC) ACTSYS(SYSD) TARGETSYS(ALL) ESTORE(YES) STORE(YES)

SFM Policy Control StatementsExample:

Activating SFM Policy

Make SFM Couple Data Set available to z/OS:

SETXCF COUPLE,PCOUPLE=SYS1.SFM#PLXM.CDS01,TYPE=SFMSETXCF COUPLE,ACOUPLE=SYS1.SFM#PLXM.CDS02,TYPE=SFM

Activate SFM Policy:

SETXCF START,POLICY,POLNAME=NORMAL,TYPE=SFM

Only one member of the sysplex needs to activate the policy

Update COUPLExx member

CDS Types: Sysplex Couple Data Set CFRM SFM Separate/Combined WLMAllocation: Space requirements Primary/AlternateFormat: IXCL1DSU (All)ADMIN: IXCMIAPU (CFRM,SFM) ISPF Interface (WLM)

Sysplex Implementation Summary

SysplexImplementation Activities

IEASYSxx: PLEXCFG GRS GRSCNF GRSCNFxx GRSRNL GRSRNLxx CLOCK CLOCKxx CONSOLE CONSOLxx CNGRPxx COUPLE COUPLExx

CSS ConfigurationOS ConfigurationIOCDSSecurity Profiles CBD.CPC.IOCDS CBD.CPC.IPLPARMIPLPARMHMC Setup Profiles

SYSPLEX NameNetwork Name (S/390 T/R)Processor Id names (HCD)CPC NamesCF NamesStructuresConsolesJES2 CheckpointSignalling Pathsz/OS System NamesHW/SW Requirements

HCD

PARMLIB

Planning

COUPLE

Define checkpoint structure in CFRM

CKPTDEF Parametersƒ CKPTn=(STRNAME=str_name)ƒ MODE=DUPLEX

Recommendationsƒ CKPT1 on CF, CKPT2 on DASDƒ VOLATILE=(ONECKPT=IGNORE,ALLCKPT=DIALOG)ƒ Activate via Dialog

JES2 Checkpoint on CF

z/OS System Logger

APPL2

LOGGER

APPL1

z/OSA

Log Data Set Log Data Set

APPL2

LOGGER

APPL1

z/OSB

List structurelogstream 1

List structurelogstream 2

Coupling Facility

Sysplex-Wide

Offload Process

Offload Process

CICS Tran. Server for z/OSIMS/ESA 6.1 Shared QueuesOperlogLogrecAPPC/MVSz/OS Resource Recovery ServicesWebsphereTVS...

Users of System Logger:LOGWTR A

z/OS_A

IXGLOGR

LOGWTR B

CF_1

LOGWTR A

z/OS_B

LOGWTR B

IXGLOGR

System Logger Users

System Logger Recommendations

Logstream/Structure Recommendations:

Aim for 10-20 logstreams per structure. During recovery, each z/OS will maximize parallelism ACROSS structures, but serially within a single structure. Remember (current) overall limit of 255 structures.

Try to have at least 2 active logstreams per CF structure, connected to more than one system, to allow peer recovery in case of failure.

Try to put logstreams with similar storage requirements in same structure - storage is split evenly between all logstreams, so CF storage will be wasted if logstreams have vastly different usage patterns.

When another logstream is connected, the existing logstreams will be re-sized. This could potentially cause short term logstream-full conditions and impact exploiters.

Logstream/Structure Recommendations:

AVGBUFSIZE & MAXBUFSIZE are specified at structure level, so try to ensure all logstreams in a structure have similar block size characteristics.

Use a realistic number for LOGSNUM - every POTENTIAL logstream takes up some storage in the structure.

Do not oversize the logstream structures - every logstream is potentially duplexed in a data space - so very large logstreams can lead to processor storage constraint. Also, very large logstreams mean long offloads which can impact logger performance. More frequent small offloads are better than a few large ones. One offload every 5-10 minutes would be an ideal balance.

System Logger Recommendations cont...

Offload Dataset Recommendations:

Use a CISZ of 24576 for offload datasets. Staging datasets MUST have a CISZ of 4096 or writes to them will fail.

Make the offload datasets as large as reasonable, to reduce overhead of allocating additional datasets in the middle of an offload. Bear in mind retpd and age of data in each offload dataset.

RETPD and AUTODELETE to control deletion of data from logstream and deletion of log data sets.

Ensure data sets are defined with SHROPTIONS(3,3)

System Logger Recommendations cont...

Recommendations:

For OLTP workloads, use HIGHOFFLOAD of 80% and LOWOFFLOAD of 60% as a starting point. Check the documentation for the connector (CICS or whatever) for specific recommendations, and monitor SMF Type 88 records using IXGRPT1 provided in SAMPLIB.

Do not set HIGHOFFLOAD higher than 80%, in case spurts of activity would fill the log stream before OFFLOAD can provide additional space.

Do not set HIGHOFFLOAD so high that you rarely get offloads - log stream and offload data sets only get cleaned up when offload is invoked.

Make sure you have sufficient auxiliary storage to back the logger data spaces.

IXGRPT1 provides counts of number of offloads, number of structure full and entry full, and avg buffer size, amongst other things. See SMF manual, macro IXGSMF88 in SYS1.MACLIB, and member IXGRPT1 in SYS1.SAMPLIB for a description of all the fields.

System Logger Recommendations cont...

Offload devices should be fast enough to offload difference between HIGHOFFLOAD and LOWOFFLOAD faster than space above HIGHOFFLOAD fills up.

System Logger Review

Format Logger Inventory

Format the z/OS Logger Inventory Couple Data Set with:LSR – number of structuresLSTRR – number of log streams

//STEP1 EXEC PGM=IXCL1DSU//SYSPRINT DD *//SYSIN DD *DEFINEDS SYSPLEX(PLXM)DSN(SYS1.LGR#PLXM.CDS01) VOLSER(BMCD03)CATALOG

DATA TYPE(LOGR)ITEM NAME(LSR) NUMBER(5)ITEM NAME(LSTRR) NUMBER(10)

Define Log Streams

Associate data sets, log stream names, structure names and attributes.–Structures may be defined or deleted –Streams may be defined, deleted or updated

//STEP1 EXEC PGM=IXCMIAPU//SYSIN DD *DATA TYPE(LOGR) REPORT(YES)DEFINE STRUCTURE NAME(LOGGER_OPERLOG)AVGBUFSIZE(512)MAXBUFSIZE(4096) LOGSNUM(5)

DEFINE LOGSTREAM NAME(SYSPLEX_OPERLOG)STRUCTNAME(LOGGER_OPERLOG)...

Update the CFRM Policy

Update CFRM policy with names of the structures to be used by z/OS System Logger.

//STEP1 EXEC PGM=IXCMIAPU//SYSIN DD *DATA TYPE(CFRM) REPORT(YES)

. . .DEFINE STR(LOGGER_OPERLOG) SIZE(13568) PREFLIST(C033,C021)

Automatic Restart Manager

The Automatic Restart Manager (ARM) is an z/OS recovery function– Improves the availability of specified subsystems and applications–Automatically restarting them as requested–Function of the Cross-System Coupling Facility (XCF) component ƒ Runs in XCF address space ƒ Uses data spaces

–Augments the functions of production controllers, automation products, and job schedulers–Subsystems that use ARM are expected to perform their own transaction level and database recovery–Requires:ƒ ARM Couple data sets which have connectivity to all the systems on which

ARM is active

ARM Environment

XCF XCF

ARM

ARM

ARM

Couple Data Set

StartedTasks/

Batch

StartedTasks/

Batch

CouplingFacility

SYSA SYSB

z/OS z/OS

ARM Restart Inplace

CF

SYSA SYSB

RTM

Error

Application

AutomaticRestart Manager

EOM

Resource Manager

- Invoke Exits

- Determine ifElement isrestartable

- ExecuteappropriateRestartMetho

d

Restarton SYSA

Executingon SYSA

ARM Restart on other Systems

ARM

SYSA

SYSB SYSC

GRP01

Appl1Appl2Appl3

GRP02

Appl4Appl5 ARM

Policy

Couple Data SetCF

GRP01

Appl1Appl2Appl3

GRP02

Appl4Appl5

SYSB leaves Sysplex

XCF Address Space

Applicationsdiscontinueon SYSB

Restart on SYSA

Restart on SYSC

XCF Statusupdate missing

ARM Couple Data Sets

PrimaryARM

Couple Data Set

AlternateARM

Couple Data Set

Couple Data Set Format Utility

.

.

DEFINEDS SYSPLEX(PLXM) DSN(SYS1.ARM#PLXM.CDS01) VOLSER(BMCD01) DATA TYPE(ARM) ITEM NAME(POLICY) NUMBER(3) ITEM NAME(MAXELEM) NUMBER(10) ITEM NAME(TOTELEM) NUMBER(500)

ARM Policy - IXCMIAPU

//IXCARMP0 JOB//STEP1 EXEC PGM=IXCMIAPU//SYSPRINT DD SYSOUT=A//SYSIN DD * DATA TYPE(ARM) DEFINE POLICY NAME(ARMPLXM1) RESTART_GROUP(DEFAULT) TARGET_SYSTEM(*) ELEMENT(*) RESTART_ATTEMPTS(0) RESTART_GROUP(VTAMGRP) TARGER_SYSTEM(*) ELEMENT(NET@V*) RESTART_ATTEMPTS(3,300) READY_TIMEOUT(900) TERMTYPE(ELEMTERM)/*

IXCMIAPU Utility example