IBM System z Technical University – Berlin, Germany– May 21-25

© 2012 IBM Corporation

Session code zZS25***** PRELIMINARY VERSION V *****Software Deployment Customer ExperiencesTim Alpaerts, Euroclear

IBM System z Technical University – Berlin, Germany– May 21-25

2

Terminology

The old situation

The new design

The Product Catalog in TDSL

Non-uniform software stacks

TDSL Product Selection & Generate (build) processes

The roll-out (restore) process

VSAM challenges

zFS on the installation system

Agenda

3

Installation system: one particular MVS system dedicated to installing software and installing fixes.

End-User system: all MVS systems other than the installation system.

Dead libraries: product libraries created on the installation system by software installation, (maybe SMP/E installed or non-SMP/E installed) not actively in use by an instance of the product.

Retrofit: after making a local correction on an end-user system, it needs to be incorporated into the dead libraries for future builds.

Version-less dsnames: product library dsnames that do not imply a particular version or release of a software, they do not contain a version release qualifier.

Non-Uniform software stacks: selectively activating only the needed parts of the complete enterprise software stack on every system, mainly for the purpose of reducing the license charge.

Terminology

4

Terminology

The old situation

The new design





VSAM challenges


Agenda

5

Development systems Production systems

K-systems each group of systems: Test systems Paris Active MVS target & dlibs Inactive MVS target & dlibs

MVS SMPE, within each group 2 target zones, non-IBM DASD was shared with CA-MIM 2 dlib zones products

The old situation, overview

IBM IBM

SMPE

Test systems Brussels

Pre-production Paris

Pre-production Brussels

7

Version-release qualifiers in dsnames of product libraries make system upgrades risky.

Enqueue and reserve contention interference between lpars, instability from dead-locks.

No inventory nor overview of what system is using what software releases.

No obvious distinction between software product libraries and databases and work datasets.

Locally maintained PROGxx parmlib members accumulate errors over time

Low confidence when doing IPL.

Some issues with the old set-up

9

Terminology

The old situation

The new design





VSAM challenges


Agenda

10

Software installation needs to create dsnames (dead libraries) different from those the driving system is using.

All software installation work concentrated on one single system dedicated for this purpose.

All product libraries with version-less dsnames, to avoid widespread impact from system upgrades.

Distinction between R/O product libraries and R/W databases immediately apparent from the dsname.

Complete isolation of DASD between lpars in the IODF, no more cross-plex sharing of dasd.

Objectives for the new design (1)

11

All software, OS and all software products are upgraded in one go.– Upfront agreement on scope and planning (IPL slots).– Integrated testing.– All the “hold actions” are performed in parallel and are repeatable.

Provide a ‘perfect’ APF-list and link-list, complete but without any needless entries.

Ensure at all times we have an SMP perfectly in-sync for every software rolled-out on every system.

Upgrade per system if required by a freeze of the environment.

Complete inventory of software installation and deployment states.– Visibility of what is used where (licencing, cleanup of old S/W)

Installation guidelines and documentation is available the same way for all softwares.

Objectives for the new design (2)

12

K-systems Production systems

Development systems Test systems

each system:&SYSR1 Active IPL-set Inactive IPL-set &SYSR2 6 model 9 vols 6 model 9 vols&SYSR3&OEMR1&OEMR2&OEMR3

End-user systems

The new design, overview (1)

13

IPL-set is restored from a (virtual) tape that was created on the installation system.

Each sysplex has two IPL-sets, the active set and the inactive set.

First volume of an IPL-set is the load device for the end-user system.

First volume restored from full volume dump to include the IPL text in the dump, the other volumes are dumped logically by dataset.

Second volume contains an ICF user catalog for the VSAM datasets on the IPL-set.

IPL-set is protected against updates by DFP OPEN exit IFG0EX0B, datasets can only be opened for INPUT.

The new design, the IPL-set

15

Datasets on the IPL-set are cataloged with symbolic volsers defined in IEASYMxx parmlib member with substrings from &SYSR1:

SYSDEF SYMDEF(&SYSR2.='&SYSR1(1:5)2')SYMDEF(&SYSR3.='&SYSR1(1:5)3')SYMDEF(&OEMR1.='&SYSR1(1:5)5')SYMDEF(&OEMR2.='&SYSR1(1:5)6')SYMDEF(&OEMR3.='&SYSR1(1:5)7')

When MVS IPL processing sets &SYSR1, the volser of the IPL device to "IPCAA1", then IEASYMxx sets

&SYSR2. = "IPCAA2"&SYSR3. = "IPCAA3"&OEMR1. = "IPCAA5"&OEMR2. = "IPCAA6"&OEMR3. = "IPCAA7"

The new design, the IPL-set, continued

16

K-systems Production systems

Development systems Test systems

Level-set, 6 model 9 volumes each system:&SYSR1 Active IPL-set Inactive IPL-set &SYSR2 6 model 9 vols 6 model 9 vols&SYSR3&OEMR1&OEMR2&OEMR3

Installation system End-user systems


17

An exact image of the software for the end-user systems.

Generated by the TDSL application on the software installation system.

A ‘contents’ dataset is added with the list of all products and libraries on the LS.

The non-VSAM datasets on the LS are not cataloged, VSAM datasets get a special temporary HLQ.

Dsnames on the LS have no version-release qualifier (limited exceptions are allowed),

Exists only on the software installation system.

The new design, the Level-set

19

K-systems REF-set, SMS storage pool Production systems

REFx datasets Development systems Test systems TDSL




20

A large SMS-pool with dead-libraries

All the REF-datasets created by software installation go in this pool

ACS routines direct all allocations of REFx datasets to this pool

All REF-dsnames contain a version-release qualifier

Mostly vendor product libraries, but also Euroclear ‘config’ libraries

Exists only on the central software installation system

The new design, the REF-set

21

K-systems REF-set, SMS storage pool Production systems

REFx datasets Development systems Test systems TDSL



The new design, RENAME

22

Dsnames in the REF-set need to be renamed when they are copied to the Level Set.

Standard processing: rename removes the version release qualifier, i.e.

REFN.T119A.OPS.LOAD REFN / SYSN dataset:

SYSN.OPS.LOAD one qualifier dropped

REF1.T113A.ZOS.LINKLIBREF1 / SYS1 dataset:

SYS1.LINKLIB two qualifiers dropped

The new design, rename rules (1)

23

The standard rename does not support two or more releases side by side.

Keep-Qualifier processing: rename keeps the version release qualifier, i.e.

REFN.T119A.OPS.LOAD

SYSN.T119A.OPS.LOAD

Keep-Qualifier processing is never done for SYS1 datasets: REF1.T113A.ZOS.LINKLIB

SYS1.LINKLIB

The new design, rename rules (2)

24

Terminology

The old situation

The new design





VSAM challenges


Agenda

25

TDSL application handles software deployment on the installation system.

TDSL Product Catalog describes all products and the datasets belonging to each product in SQL tables.

Each product (and version plus maintenance level) is identified by an 8-char acronym.

Product Catalog has attributes at the level of the product and at the level of the dataset.

Products can be duplicated in the Product Catalog to apply maintenance

The TDSL Product Catalog

26

The TDSL Product Catalog, screenshot

27

The TDSL Product Catalog, screenshot

Each product is identified by an 8-char acronym

Filter line provides a fast navigation facility

More attributes on the right not in this screenshot

LR indicator

28

The TDSL Dataset List screenshot

29

Dataset Type:

INSTALL product installation jcl SMP the CSI clusters, SMPPTS, SMPMTS, SMPLOG, etc.DLIB SMP dlibs, 100% inert for TDSLTARGET SMP/E target library or other distributable datasetVTARGET VSAM target datasetUTARGET z/fs linear, always keeps version-release qualifier for automount NODIST not distributed, but it contributes to the PROGxx member

APF dataset should be APF on end-user system

LNK dataset should be link-listed on end-user system.

The TDSL Product Catalog, dataset attributes

31

The TDSL Product Catalog, dataset attributes, continued DSTVOL:

a symbolic volser like &SYSR1, &SYSR2 ...

In the dataset list of the z/OS 1.13 A product: &SYSR1

…

REF1.T113A.ZOS.NUCLEUS &SYSR1

REF1.T113A.ZOS.LINKLIB &SYSR1 &SYSR2

REF1.T113A.ZOS.IMW.SIMWSDCK &SYSR2

REF1.T113A.ZOS.OMEG.TKANMOD &SYSR3 … &SYSR3

Why fix the location of target datasets on the Level-set?

- to ensure SYS1.NUCLEUS etc. on &SYSR1 volume &OEMR1- fewer catalog updates on the end-user system

32

The TDSL product catalog, screenshot of z/OS 1.13 dataset list

33

Terminology

The old situation

The new design





VSAM challenges


Agenda

34

swap validation production systems

test systems

development production K-systems systems

Systems using the same software stack:

EB systems EF systems

SSE systems K systems Installation system

Non-uniform software stacks (1)

35

Why not deploy all software on all systems?

Procurement dept. Licensing all software on all systems needlessly expensive.

Audit dept. libraries in APF list for software products that will never be used.

Why not deploy software per-system?

Hard to maintain the multitude of parmlib members (i.e. PROGxx members)Many different combinations to validate before a system upgrade.

A solution in-between is needed.

All software will be present on all systems, but we make the libraries APF & linklisted only on those systems where it is licensed.

To do this we add a where-used attribute to each product in the product catalog.

Non-uniform software stacks (2)

36

Non-uniform software stacks (3)7 flags attributed to each product, each flag is on (used in this region) or off (not used in region)

SSE PROGEGEF PROGEF the libraries of product CORT660A will be included in EP PROGEP PROGEG, PROGEF, PROGEP, PROGEN and PROGEBENL PROGEN but not in PROGEK or PROGES K PROGEKBSOF PROGEBSY90 PROGES

37

Why create PROG members with a program?

Software installers know what datasets need to be APF or Link-listed, it’s in the product installation instructions -> can be entered in TDSL.

We keep lists of datasets we don’t distribute, but that need to be in the PROG member.

TDSL has al the info it needs to create APF lists and Link lists for the End-User systems.

Creating PROG members

38

Creating PROG members (2)

39

Creating PROG members (3)

40

Terminology

The old situation

The new design





VSAM challenges


Agenda

41

Choose software products to go onto a Level-Set.

An SQL query finds all unique DSTVOLs

Engineer specifies the ‘real’ volsers and device numbers for each symbolic DSTVOLs, i.e. &SYSR1 will be build on real volser L13A11

The dialog produces jcl to build the LS with the selected software products. Only the TARGET datasets will be copied to the LS

TDSL Generate steps

43

The TDSL Generate, screenshot

44

The problem:

ALL REF-Set libraries have secondary space so SMP APPLY won’t fail with abend x37.

On the Level-Set, Link list libraries should have zero secondary allocation, to prevent an extent being added when someone adds/updates a member.

The solution:

DFDSS dataset copy will copy every library into a single extent. Copy with ALLDATA & ALLEXCP will preserve the allocated space. A homegrown utility clears the secondary allocation value in the VTOC of the

Level-Set.

The treatment of link list datasets, space allocation

45

Terminology

The old situation

The new design





VSAM challenges


Agenda

46

Each sysplex has an active and inactive IPL-set with associated master catalogs:

ACTIVE ACTIVE INACTIVE INACTIVEMASTER CATALOG IPL-SET IPL-SET MASTER CATALOG

IPCAA1 IPCAB1

CATALOG.PLXP.IPCAA1.MASTER CATALOG.PLXP.IPCAB1.MASTER

Inactive IPL-set is formatted, ICKDSF INIT New master catalog is delete/defined A dump is restored onto the inactive IPL-set REPRO NOMREGECAT of the active master catalog to the new one Inactive master catalog is fixed for new- or changed datasets IPL from the inactive IPL-set.

Roll-out on the end-user system

47

This slide just for the notes

48

Terminology

The old situation

The new design





VSAM challenges


Agenda

49

VSAM datasets need to be cataloged for all processing.Interferece with the driving system when copying VSAM datasets w same name.

All IDCAMS processing requires a correct VSAM ‘back-pointer’

Catalog volume Volume with VSAM dataset

forward pointer

ICF catalog VVDS

SYSV.OMVS.ROOT.ZFS VVR back pointer

VSAM dataset SYSV.OMVS.ROOT.ZFS

The Challenges of VSAM datasets

50

On the installation system: Define a new ICF usercatalog on one of the LS volumes

Define an alias for a new temporary HLQ with RELATE to the new usercat, i.e. SYSDEF

Copy the VSAM datasets to the LS with RENAMEUNCONDITIONAL, REFx SYSDEF

Level-set is dumped to tape.

On the end-user system: Define a new ICF usercatalog on one of the IPLset volumes

Delete/define the SYSDEF alias

Restore VSAM clusters with the SYSDEF HLQ from tape (no rename)

IDCAMS ALTER all SYSDEF dsnames in the user catalog to the normal HLQ for VSAM

The distribution of VSAM datasets (1)

51

An alias with symbolic relate accomplishes the same ‘volume switching function’ as the symbolic volumes in the catalog do for non-vsam datasets:

DEFINE USERCATALOG – (NAME(CATALOG.SYSV.IPCAA1) – IPL-set A ICFCATALOG – VOLUME(IPCAA2) – CYLINDERS(1 1) DEFINE USERCATALOG - (NAME(CATALOG.SYSV.IPCAB1) – IPL-set B ICFCATALOG – VOLUME(IPCAB2) – CYLINDERS(1 1)

DEFINE ALIAS(NAME(SYSV) – SYMBOLICRELATE(CATALOG.SYSV.&SYSR1))


52

LISTCAT ENTRY(SYSV) ALIAS ALL ALIAS --------- SYSV IN-CAT --- CATALOG.PLXP.IPCAA1.MASTER HISTORY RELEASE----------------2 ASSOCIATIONS SYMBOLIC-CATALOG.SYSV.&SYSR1 RESOLVED-CATALOG.SYSV.IPCAA1


53

Terminology

The old situation

The new design





VSAM challenges


Agenda

54

Risks of APPLY on statically mounted file systems:

If the wrong file system is mounted, an APPLY job could update USS components belonging to a different SMP/E zone than the OS libraries.

Potentially there could be data not hardened in the file system when a R/W mounted file system is copied with dfDSS.

How automounted file systems can help:

USS path in dddef’s can contain a zFS qualifier that makes automount mount the file system corresponding to that zone.

Automount will unmount file systems that have not been accessed for some amount of time. This reduces the time window during which uncommitted data can exist.

z/FS file systems on the installation system (1)

56

/etc/auto.master:

/SERVICE/DB2/DB2BASE /etc/service_db2_db2base.map

/etc/service_db2_db2base.map

name * type ZFS filesystem REFN.<uc_name>.DB2.DB2BASE.ZFSmode rdwr duration 60 delay 10 security Yes setuid no

DDDEF SDSNABIN in target zone TD1010A contains USS path:

'/SERVICE/DB2/DB2BASE/T10101A/bin/IBM/'


57

The SMP/E APPLY job: //SMPCNTL DD * SET BDY(TD1010A) . APPLY SELECT(…) .

When SMP/E opens a file in the SDSNABIN directory it will cause the automount daemon to mount REFN.T1010A.DB2.DB2BASE.ZFS onto mount point /SERVICE/DB2/DB2BASE/T10101A

Automount replaces <uc_name> in the map file with the path-qualifier from the SMP/E dddef path to obtain the zFS dsname:

i.e. Filesystem REFN.<uc_name>.DB2.DB2BASE.ZFS

becomes REFN.T1010A.DB2.DB2BASE.ZFS


58


A Loose End

How to distribute large zFS file-systems that need SMS extended format data class attribute?An SMS managed volume in the IPL-set is most unpractical.

59

The End

Question Time.

64

Reporting on changes in the field

At levelset generation time a hash is created for all datasets.

Reporting is available to identify all changes between the levelset and a resident set deployed in a user system.

IBM System z Technical University – Berlin, Germany– May 21-25

Documents

Transcript of IBM System z Technical University – Berlin, Germany– May 21-25