VSAM’s Alive! But we only have 60 minutes to save your performance…

Helmut Zwicklbauer

November 2020

Session 3BC

VSAM’s Alive

• Things to know about VSAM

• What is IAM?

• What makes the difference?

• How to enable IAM

• What is the benefit?

• Other IAM features and advanced functions

• How to check how much VSAM is still active in your shop

A word about VSAM VSAM has a long history…

What is VSAM?

• The term VSAM is used for the data set organization and for the access method as well

• VSAM means Virtual Storage Access Method

• Developed by IBM in the 1970s

• A way to see a data set just like Virtual Storage • Addressable by RBA (Relative Byte Address)

• API allows to access data by records

VSAM Data Set Types

• KSDS - Key Sequenced Data SetData that uses an INDEX (a KEY)

• ESDS - Entry Sequenced Data SetSequential Data

• RRDS - Relative Record Data Setaccessed by record number

• LDS - Linear Data Setaccessed as byte-addressable strings in virtual storage.

VSAM Usage today

• Over the years many applications that used VSAM data sets in the past had been converted to DB2

• This required a change of the application code

• Not every shop changed to DB2

• And not every application was converted

• So there is some VSAM left in almost every shop

• A lot of Vendor products still use VSAM

Vendor Products that use VSAM

• Accero (CYBORG) Payroll

• American Software DRP

• ASG (Mobius) ViewDirect

• Atos Origin CARDLINK

• CGI CGI Advantage

• CSC Hogan. Cyberlife, Capsil

• CSI Bank Trade

• Fidelity National Systematics Banking, BASE2000, CSF

• First Data VisionPLUS

• Fis (SunGuard) OMNI DC, SIS+, FRS

• Fiserv Pep Plus, Mortgage-Serv

• Infor (GEAC) GL, AR, MSA, Millennium, Walker, Infopoint Deposits

• JDA Compass Contract

• LRS PageCenter

• LSI Popims

• Mackinney Software JES Queue Print

• McKesson (HBO) HealthQuest

• Pitney Bowes (Group 1) Finalist, Code 1, MailstreamPlus

• Retalix (NCR) Biceps, ABS

• SEA TRMS, SAVRS

• Serena Changeman ZMF

• Shaw Systems Collections

• Siemens Medical Systems Invision, Signature

• Sigma SAM (Student Aid Mgmt)

• Empower Software (Tesseract) Payroll, HR, Benefits

• TriZetto ClaimFacts, GroupFacts

• VIPS Medicare Part B, SuperOP

• … and probably a lot more

VSAM Usage today

For today’s discussion we won’t be talking about…

• VSAM data sets allocated by DB2 or IMS

• Catalogs or other z/OS system datasets (SMF recording, Checkpoints etc.) that use VSAM for special purposes

Today’s focus is KSDS

Key Sequenced Data Set means• Data accessed by Key value in the index component

(Can be Sequential, Random or Dynamic)

• Keys can be any character strings like• Names

• Numbers

• Booking References

• Timestamps

• Etc.

VSAM KSDS works fine …

… but there is room for improvement

• In particular with large data sets

• And this is the point where IAM comes in

• What is IAM ?

What is IAM ?

• IAM means Innovation Access Method

• High performance index access Method

• Development started in 1972 by IDP (now owned by Compuware)

• Well established for over 45 years

• Replacement of VSAM for selected data sets

What is IAM ?

IAM is used to achieve the following benefits:

• Reduction of computing costs

• Reduction of CPU time

• Reduction of EXCPs

• Reduction of DASD space utilization

• Reduce nightly batch processing window

• Delay processor and DASD upgrades

How do we achieve those benefits?Let’s see the major issues with large VSAM KSDS

The Basic Issue with large VSAM KSDS

• The KSDS Index is organized in levels

• A large dataset will have multiple index levels

• Each level requires an additional I/O

• Finally you may end up with four or five I/Os (EXCPs) in order to read or update a single record

VSAM KSDS INDEX Levels

• The INDEX describes which key is found in what block

• Each pointer means an additional I/O

Prime Data Area

Rec1…Rec2…Rec3…

Control Information

Index

Extended Overflow

Extended PE

The IAM solution

IAM is “self describing”: VSAM attributes are stored

here

Entire compressed Index is loaded into 64-Bit storage eliminating all Index I/O

• Non-VSAM data set containing data and index

• Random accessto data recordswith a single I/O

The IAM solution

• Load the full index into virtual storage during OPEN

• No index I/Os needed to read a data record

• In the past IAM used z/OS dataspaces to retain the index

• Now we can use z/OS 64-bit virtual storage

The VSAM SPLIT issue

• VSAM keeps all records in sequence within the smallest unit of data➢ A block or CI (Control Interval in terms of VSAM)

• If there is no room to insert additional records the CI has to do a SPLIT➢ Half of the block is moved to a new CI within the same CA (Control Area)

• Consequently the CA (usually one 3390 cylinder of data) may split as well➢ in this case half of the cylinder is moved. These are a lot of extra I/Os

Data Records ordered by KEY

CI-Split if there is no space left in the block

VSAM Data Block = Control Interval (CI)

CI-Splits create new data blocks within the same Control-Area(CA, usually one Cylinder).

As long there is free space.

Otherwise a CA-Split occurs. This will move a half cylinder of data.

CI-Splits vs. CA-Splits

Prime Data Area

Rec1…Rec2…Rec3…

Control Information

Index

Extended Overflow

Extended PE

The IAM solution

Inserted records go into the extended overflow without

any CI/CA split overhead

• No SPLITs are done. If some record doesn’t fit in the home block it will be written to an overflow area at the end of the file.

• No extra I/Os are done to re-order any data blocks.

• In order to avoid CI- and CA-SPLITs many KSDS Clusters are allocated with a lot of extra space.

• But this doesn’t prevent all Splits while other Freespace-Areas are left empty.

The VSAM SPACE issue

The VSAM SPACE issue

• Usually a lot of extra free space is allocated in each CI and/or CA because of the SPLIT issue

• Since Inserts by applications are not evenly spread across the data set, over time the result is an overallocated file with many splits which need to be reorganized.

• VSAM has to use fixed CI (Block) Sizes of a multiple of 2K (up to 32K)

• This doesn’t match 3390 track capacity

The IAM solution

• IAM simulates VSAM CI Sizes when required but it doesn’t use it.

• IAM used optimal 3390 block size (half track or 3 or 4 blocks per track).

• No need for additional free space to be allocated for future inserts.

• Inserted records are written to the end of the file that may lead to additional extents.

• The result is a smaller dataset using less I/O to read the same amount of data.

How to enable IAM?That’s pretty easy…

Enable IAM

• Run IAMSTART program (usually done during IPL)

• This will load the IAM VSAM InterFace (VIF) into Common Storage

• VIF will detect at OPEN timewhether some data set isVSAM or IAM... and give control to theappropriate method

How to change a VSAM data set to IAM

//DEFINE EXEC PGM=IDCAMS

//SYSPRINT DD SYSOUT=*

//SYSIN DD *

DEFINE CLUSTER -

(NAME(EXAMPLEl.DATASET) -

VOLUMES(VOL00l) -

CYL(500 50) -

OWNER($IAM) - ADD

SPEED REUSE ) -

DATA( -

NAME(EXAMPLE.DATASET.DATA)-

RECORDSIZE(200 256) -

KEYS(16 0) -

CISZ(4096) -

FREESPACE(10 10) ) -

…

/*

• No changes to Programs

• No changes to JCL

• Conversion by DEFINE CLUSTER

• Just add OWNER($IAM) or assign a DATACLAS name like DC$IAM

What is the impact on your VSAM jobsLet’s see some samples from real customer jobs …

Run existing jobs with VSAM and IAM

Comparison using existing customer’s job:

• 77% less I/O (EXCPs)

• 35% less Service Units

• 67% less Elapse Time

Elapse Time Service

JOBNAME STEP mm:ss EXCPs (I/O) Units

S160VSAM DELDS 00:00 564 3,298

JOB09588 PS05 00:01 476 1,511

PS10 00:02 7,125 18,230

PS20 01:28 192,000 797,000

PS30 00:00 158 1,326

PS40 00:43 55,827 448,000

PS50 00:00 134 935

Totals: 02:14 256,284 1,270,300

S160IAM DELDS 00:00 36 316

JOB09594 PS05 00:00 477 1,445

PS10 00:02 7,120 17,762

PS20 00:35 36,119 559,000

PS30 00:00 255 1,814

PS40 00:06 15,877 238,000

PS50 00:01 277 1,507

Totals: 00:44 -67% 60,161 -77% 819,844 -35%

Results will vary depending on application

This is some unusual result.

It turned out the application opened the VSAM file for UPDATE while most I/O was sequential reading.

IAM real time tuning will handle this.

Elapse Time Service

JOBNAME STEP mm:ss EXCPs (I/O) Units

BRE4VSAM DELDS 00:00 262 1,127

JOB09560 PS010 13:19 3,451,000 4,209,000

PS030 00:01 165 717

Totals: 13:20 3,451,427 4,210,844

BRE4IAM DELDS 00:00 44 299

JOB09590 PS010 00:09 2,216 330,000

PS030 00:01 224 922

Totals: 00:10 -99% 2,484 -99.9% 331,221 -92%

Results will vary depending on application

In this case most I/O is TAPE I/O not related to VSAM.

IAM will just speed up the VSAM part.

Elapse Time Service

JOBNAME STEP mm:ss EXCPs (I/O) Units

S405VSAM DELDS 00:00 34 267

JOB17918 PS06 00:01 247 2,718

PS10 05:54 1,455,000 14,687,000

PS20 00:39 118,000 727,000

PS30 00:01 4,885 30,624

PS45 00:00 158 1,305

PS50 00:39 91,449 426,000

PS60 00:00 136 945

Totals: 07:14 1,669,909 15,875,859

S405IAM DELDS 00:00 36 236

JOB12551 PS06 00:01 954 6,866

PS10 03:04 1,271,000 13,892,000

PS20 00:40 116,000 712,000

PS30 00:01 4,888 29,767

PS45 00:00 260 1,612

PS50 00:14 51,485 227,000

PS60 00:00 679 2,535

Totals: 04:00 -45% 1,445,302 -13% 14,872,016 -6%

Other featuresUp to now we discussed Index in Virtual Storage, Inserts without SPLITs and smaller Data Set Size, but there is more …

LARGE BUFFERS

• With 64-bit virtual storage IAM can assign significant more I/O buffers

• 64-bit virtual buffers can be used even when the application program is not running in 64-bit AMODE.

• Even in CICS environment when many file are used in parallel

• Number of buffers is dynamically adjusted within a minimum to maximum number specified by you

Real Time Tuning

• LRU (Least recently used) Buffer Management

• Sequential read ahead

• Deferred writes for batch updates

• In mixed random/sequential environments, dynamically balances buffer usage based on application I/O demands

Data Compression

• Increases amount of data transferred per I/O

• Reduces physical I/O (EXCP counts)

• Reduces data set size

• IAM Software compression• high performance proprietary software algorithm

• IAM may use System z Hardware Compression• Dictionary is dynamically built during file load

• User may provide customized dictionaries

• Customized dictionaries may achieve higher compression rates

IAMINFO Run Time Report

• Display data set attributes

• Sizes and timestamps

• Internal storage and buffer usage

• I/O macros used by application

• Is written to IAMINFO DD if available

• Is written to SMF if recording is enabled

IAMINFO Run Time Report

IAM FILE FORMAT-----------= LARGE-FRMT

DATASET TYPE--------------= KSDS

LOGICAL RECORD LENGTH-----= 512

KEY SIZE------------------= 4

BLOCKING FACTOR-----------= 4

INTEGRATED OVERFLOW (CI%)-= 0

IAM FILE CHARACTERISTICS -

FILE DEFINED DATE---------= 2020.301

FILE LOADED DATE----------= 2020.301

FILE UPDATE DATE----------= 2020.301

TOTAL RECORDS-------------= 100,000

TOTAL RECORDS UPDATED-----= 200

TRACKS IN USE-------------= 555

NUMBER OF EXTENTS---------= 1

NUMBER OF IAM DATA BLOCKS-= 2,217

IAM FILE STATUS-----------= LOADED

SHARE OPTIONS-------------= 2

CI SIZE-------------------= 8192

KEY OFFSET----------------= 8

BLOCK SIZE----------------= 13682

DASD RESERVE (CA%)--------= 10

FILE DEFINED TIME---------= 06:37:26

FILE LOADED TIME----------= 06:37:27

FILE UPDATE TIME----------= 06:43:30

TOTAL RECORDS DELETED-----= 0

TOTAL RECORDS INSERTED----= 0

VARIABLE LENGTH OVERFLOW--= YES

NUMBER OF VOLUMES---------= 1

IAMINFO Run Time Report – Part 2TOTAL STORAGE REQUIRED----= 1,519,616

STORAGE ABOVE THE LINE----= 1,507,328

64-BIT INDEX STORAGE (K)--= 128

REQUESTS PROCESSED--------= 100,204

DISK BLOCKS READ----------= 41

DYNAMIC BUFFER RETRIEVALS-= 2,177

MINIMUM BUFFERS USED------= 64

IAM COMMAND EXECUTION SUMMARY -

GET RANDOM----------------= 0

GET SEQUENTIAL------------= 100,001

GET PREVIOUS--------------= 0

GET KGE/GENERIC-----------= 0

GET (SKIP SEQUENTIAL)-----= 0

CLOSE---------------------= 1

CLOSE TYPE=T--------------= 0

INVALID REQUESTS----------= 0

SEQ CHAINED BLOCKS READ---= 2,177

PRIME INDEX(COMPRESSED)---= 5,700

COMPRESSED DATA STRUCTURE-= YES

TOTAL JOB 64-BIT INDEX (K)= 768

REQUESTS FAILED-----------= 1

DISK BLOCKS WRITTEN-------= 39

MAXIMUM BUFFERS USED------= 64

MAXIMUM BUFFERS AVAILABLE-= 4,904

PUT UPDATE----------------= 200

PUT ADD-------------------= 0

POINT (START BROWSE)------= 1

POINT KGE/GENERIC---------= 0

ERASE---------------------= 0

OPEN----------------------= 1

VERIFY--------------------= 0

RECORD LENGTH CHANGES-----= 0

SEQ CHAINED BLOCKS WRITTEN= 162

Tuning and Reorg Recommendations

• IAM will issue tuning and/or reorg recommendations during OPEN and CLOSE if appropriate.

• IAM368 SPECIFYING A BUFNO VALUE GREATER THAN nnnn MAY IMPROVE PERFORMANCE

• issued when dynamic tuning finds more buffers would be useful

• IAM373 REORGANIZATION OF THIS FILE IS RECOMMENDED

• issued when the Overflow area becomes too large

Advanced functionsFeatures that are not used by every shop …

Dynamic Reorg for CICS

• IAM Dynamic Reorg increases data availability by reducing the time the data is not available due to reorganizations.

• The bulk of the reorg is done to a copy of the file while the original data set is used by CICS.

• Any updates done by CICS will be applied to the reorganized copy as well.

• Finally the original data set will be closed and renamed to some backup DSN and the reorganized copy will be opened instead.

• CICS GLUE exits provided by IAM will cause any CICS transaction to tolerate the file Close, Rename, and Open.

Dynamic Reorg for CICS

• Benchmark Environment for IAM Dynamic Reorg Processing

• Z13, CICS TS 5.2, z/OS 2.2, Reorg of 3 files

• 50,000 track VSAM KSDS loaded with 9,000,000 records followed by a batch update of 227,000 records and the insertion of 15,000 records. The file has one alternate index.

• 25,000 track VSAM KSDS loaded with 3,000,000 records followed by a batch update of 111,000 records and the insertion of 7,200 records.

• 17,000 track VSAM KSDS loaded with 3,000,000 records followed by a batch update of 105,000 records and the insertion of 6,000 records.

VSAM IDCAMS Reorg vs. IAM Dynamic ReorgElapsed Time File Is Not Available to CICS

0

200

400

600

800

1000

Elapsed Time in Seconds

869

772

VSAM IDCAMS

IAM Dynamic ReorgWhile Closed to CICS

IAM Dynamic ReorgWhile Open to CICSAccess Delayed by

14 Minutes & 29 Seconds

1 Minute & 17 Seconds

2 Seconds

43

* IAM Dynamic Reorg also used 93% less CPU Time than the VSAM IDCAMS Reorg

IAM/RLS Record Level Sharing

• Enables shared access to IAM files with data integrity

• Allows sharing with multiple address spaces on same LPAR

• Supports CICS, Batch, TSO and other address spaces

• Requires IAMRLS address space to handle all physical I/O

• User address spaces exchange data with IAMRLS by cross memory service

• Supports CICS with no application program changes required

• IAM files are defined to CICS as non-RLS VSAM data sets

IAM LimitationsThings you cannot do with IAM …

Seldom used VSAM functions currentlynot supported

• with KSDS – No CI or RBA processing

• with ESDS – No updating by CI

• with RRDS – CI access limited to files with 1 record per CI

• In development: Sysplex RLS - Record Level Sharing on multiple LPARs

Check VSAM activity in your shopMany users think they have no VSAM data sets but in most cases there are some …

Run IAM VSAM Usage Report or send SMF type 30 & 64 records

Summary

Please remember:

• IAM can have significant benefits for selected data sets

• Very easy to implement

• No change to application programs or JCL to achieve this

• Conversion is done by DEFINE CLUSTER using a simple keyword

• Application are not aware of IAM. Everything will look like VSAM

• z/OS, DFSMS and backup tools see a sequential DSORG=PS file instead of VSAM

Thanks for your timeDon’t hesitate to ask if you have any questions …

Driving Digital Success with the Mainframe: Adapt, Automate, and Secure

bmc.com/forms/mainframe-survey-results.html

Please submit your session feedback!

• Do it online at http://conferences.gse.org.uk/2020/feedback/3BC

• This session is 3BC

GSE UK Conference 2020 Charity

• The GSE UK Region team hope that you find this presentation and others that follow useful and help to expand your knowledge of z Systems.

• Please consider showing your appreciation by kindly donating a small sum to our charity this year, NHS Charities Together. Follow the link below or scan the QR Code:

http://uk.virginmoneygiving.com/GuideShareEuropeUKRegion