Dynamic Full Function

Database Buffer Pools

Nancy G. Stein - ngstein@us.ibm.com

IBM Washington System Center / IMS Team

September 16, 2015

Philadelphia IMS User Group

Acknowledgements and Disclaimers

Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

© Copyright IBM Corporation 2015. All rights reserved.

— U.S. Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM, the IBM logo, ibm.com, z/OS, IMS, ODM, DB2, CICS, WMQ, WAS, WOLA, RAA, RAD, RDz, RTC, are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or TM), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at

•“Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

•Other company, product, or service names may be trademarks or service marks of others.

2

3

Dynamic Full Function Database Buffer Pools

4 4

Dynamic Full Function Database Buffer Pools

• Dynamic Full Function DB Buffer Pool support was part of IMS 12

Users can dynamically manage full function buffer pools

Buffer pool definitions can change without taking IMS down

Application activity is internally quiesced

• Allows new buffer pools to be created

• Allows existing buffer pools to be changed or deleted

• Benefits

Improved buffer pool management

• Provides type-2 commands for better usability

• Eliminates system down time to make modifications to buffer pool definitions

• Helps application performance with improved buffer pool specifications

5

Overview of Dynamic Full Function Dynamic Buffer Pools

IMS Proclib

DFSVSMxx

POOLID=VCCC 2048,10000 4096,10000

System Initialization

2048,10000 4096,10000

IMS CONTROL REGION

UPDATE POOL TYPE(DBAS) SECTION(VSAM111)

IMS Proclib

DFSDFxxx

<SECTION=VSAM111> POOLID=VCCC

2048,30000 4096,30000

Define OSAM and VSAM Changes in DFSDFxxx

2048,30000 4096,30000

IMS CONTROL REGION

1

2

3

6 6

Full Function Database Buffer Pools (Pre-IMS 12)

• VSAM and OSAM definitions stored only in DFSVSMxx

proclib member

IMS processes DFSVSMxx once during system initialization

No facility to change buffer pool definitions with online

commands

Buffer pool modifications required an IMS system restart

7

VSAM Buffer Pools (as Defined in DFSVSMxx)

• VSAM Buffer Pool Specification

POOLID= (One for each shared resource pool)

id, (shared resource pool user id)

FIXDATA=NO|YES, (long-term page-fixing -- Data)

FIXINDEX=NO|YES, (long-term page-fixing -- Index)

FIXBLOCK=NO|YES, (long-term page-fixing – I/O Control Blocks)

STRINGNM=n (max VSAM I/O requests concurrently active)

VSRBF= (One/more to define subpools within shared pool)

buffersize, (buffer size for subpool)

number of buffers, (Number of buffers in this subpool)

type, (Index (I) subpool vs. Data (D) subpool)

HSO|HSR, (Specifies action if Hiperspace unavailable)

HSn (Number of hiperspace buffers for subpool)

7

8

VSAM Buffer Pools (as Defined in DFSVSMxx)

• VSAM Buffer Pool Specification (Continued)

DBD= (Optionally assigned to POOLID shared pool id)

DBDname( (DBD from NAME= keyword on DBD macro)

dataset number, (Specific data set of data set group)

id, (Shared resource pool identifier)

ERASE=YES|NO, (Treatment of deleted logical records)

FREESPACE=NO|YES) (Treatment of defined free space % in KSDS)

• Example VSAM Specification in DFSVSMxx:

POOLID=VCCC

VSRBF=2048,8000,I

VSRBF=2048,26000,D

VSRBF=4096,32000,D

VSRBF=8192,104000,I

DBD=PVHDJ5B(B,VCCC,ERASE=YES,FREESPACE=YES)

8

9

OSAM Buffer Pools (as Defined in DFSVSMxx)

• OSAM Buffer Pool Specification

IOBF= (One for each OSAM subpool definition)

(length, (length of buffers in subpool)

number, (number of buffers in subpool)

fix1, (long-term page-fixing – buffers + prefixes)

fix2, (long-term page-fixing – prefixes + headers)

id, (subpool identifier)

co) (caching option)

DBD= (Optionally assigned to POOLID shared pool id)

DBDname( (DBD from NAME= keyword on DBD macro)

dataset number, (Specific data set of data set group)

id) (Subpool identifier)

• Example OSAM Specification in DFSVSMxx

IOBF=(8192,8000,N,N,OCCC)

DBD=POHIDKA(B,OCCC)

9

10

Dynamic Full Function Database Buffer Pools

• Dynamic VSAM and OSAM Buffer Pool Specification

Initial definitions exist in DFSVSMxx proclib member

New and changed definitions are in the DFSDFxxx proclib member

• OSAM format is similar to DFSVSMxx

• VSAM format is similar to DFSVSMxx

– POOLID in DFSDFxxx includes the VSRBF statement

Example:

POOLID=

(VSM1,

FIXDATA=N,

FIXINDEX=Y,

FIXBLOCK=N,

STRINGNM=255,

VSRBF=(1024,30000,D),

VSRBF=(1024,10000,I))

10

11

Full Function Database Buffer Pools (DFSDFxxx)

• Buffer Pool Specifications use <SECTION=section_name>

<SECTION=OSAMxxx>

• IOBF=(bufsize,bufnum,fix1,fix2,id,co)

• DBD=(DBDname,dsid,id) or

DBD(DBDname,dsid,id)

<SECTION=VSAMxxx>

• POOLID=(id,FIXDATA=,FIXINDEX=,FIXBLOCK=,STRINGNM=,

VSRBF=(buffersize,buffer number,type,HSO|HSR,HSn))

• DBD=(DBDname,dataset number,id,ERASE=,FREESPACE=) or

DBD(DBDname,dataset number,id,ERASE=,FREESPACE=)

11

12

VSAM Buffer Pools (Change Specification)

• VSAM Buffer Pool Specification Example

Example VSAM Specification in DFSVSMxx:

POOLID=VCCC

VSRBF=2048,8000,I

VSRBF=2048,26000,D

VSRBF=4096,32767,I,HSO,HS40000

VSRBF=8192,32767,I,HSR,HS40000

DBD=PVHDJ5B(B,VCCC,ERASE=YES,FREESPACE=YES)

Example of changed VSAM Specifications in DFSDFxxx

<SECTION=VSAMEXM>

POOLID=(VCCC,

VSRBF=(2048,12000,I),

VSRBF=(2048,12000,D),

VSRBF=(4096,32767,I,HSO,HS80000),

VSRBF=(8192,32767,I,HSR,HS80000))

DBD=(PVHDJ5B,B,VCCC,ERASE=YES,FREESPACE=YES)

12

13

OSAM Buffer Pool (Change Specification)

• OSAM Buffer Pool Specification Example

Example OSAM Specification in DFSVSMxx:

IOBF=(8192,8000,N,N,OCCC)

DBD=POHIDKA(B,OCCC)

Example of changed OSAM Specification in DFSDFxxx

<SECTION=OSAMEXM>

IOBF=(8192,12000,N,N,OCCC)

DBD=(POHIDKA,B,OCCC)

13

14

DFSDFxxx Considerations

• Multiple DFSDFxxx proclib members may be used

DFSDFMON

DFSDFTUE

• DFSDFxxx may have multiple sections defined

<SECTION=OSAMMON>

IOBF=(8192,8000,N,N,OAAA)

DBD=(POHIDKA,B,OAAA)

<SECTION=VSAMMON>

POOLID=(VAAA,

VSRBF=(4096,32000,I,HSO,HS10))

DBD=(PVHDJ5B,B,VAAA,ERASE=YES,FREESPACE=YES)

<SECTION=OSAMTUE>

IOBF=(8192,104000,N,N,OBBB)

DBD=(POHIDKA,B,OBBB)

<SECTION=VSAMTUE>

POOLID=(VBBB,

VSRBF=(4096,10320,I,HSO,HS10))

DBD=(PVHDJ5B,B,VBBB,ERASE=YES,FREESPACE=YES)

1

4

15

UPDATE POOL Command

• UPDATE POOL Command used to add & change buffer pools

Add and Change commands: • For VSAM or OSAM buffer pool definitions:

– UPDATE POOL TYPE(DBAS) SECTION(OSAMxxx)

– UPDATE POOL TYPE(DBAS) SECTION(VSAMxxx)

• Or in one command:

– UPDATE POOL TYPE(DBAS) SECTION(OSAMxxx,VSAMxxx)

• Add or Change definitions in an alternate DFSDFyyy proclib member:

– UPDATE POOL TYPE(DBAS) SECTION(OSAMxxx) MEMBER(yyy)

Example:

Add (8) 8K OSAM buffers, (24) 8K VSAM buffers

– DFSDFxxx

<SECTION=OSAMMON>

IOBF=(8192,8000,N,N,OAAA)

<SECTION=VSAMMON>

POOLID=(VAAA,VSRBF=(8192,24000,I))

– Issue: UPDATE POOL TYPE(DBAS) SECTION(OSAMMON,VSAMMON)

15

16

Deleting VSAM Buffer Pools

• UPDATE POOL Commands required to Delete subpools

Deleting VSAM Subpool:

• Specify a POOLID=() statement with a bufnum parm of 0

• UPDATE POOL TYPE(DBAS) SECTION() activates subpool deletion

Example:

Delete a VSAM 4K Subpool

– DFSDFxxx

<SECTION=VSAMTUE>

POOLID=(VBBB,

VSRBF=(4096,0)

VSRBF=(8192,20000))

– Issue: UPDATE POOL TYPE(DBAS) SECTION(VSAMTUE)

16

17

Deleting OSAM Buffer Pools

• UPDATE POOL Command required to Delete buffer pools

Deleting OSAM Subpool:

• Specify a IOBF=() statement with a bufnum parm of 0

• UPDATE POOL TYPE(DBAS) SECTION() activates subpool deletion

Example:

Delete an OSAM 4K Subpool

– DFSDFxxx

<SECTION=OSAMTUE>

IOBF=(4096,0,N,N,OCCC)

– Issue: UPDATE POOL TYPE(DBAS) SECTION(OSAMTUE)

17

18

Querying VSAM and OSAM Buffer Pools

• QUERY Command required to Query buffer pools

QUERY

POOL

TYPE(DBAS)

SUBTYPE(OSAM,VSAM) SIZE() POOLID()

SHOW(ALL/STATISTICS/MEMBER)

18

ALL

Show both STATISTICS and MEMBER information

STATISTICS

Show statistical information similar to /DIS POOL DBAS

MEMBER

Show active PROCLIB members used for buffer pool definitions

19

Querying VSAM and OSAM Buffer Pools

• QUERY POOL Command Example 1:

QRY POOL TYPE(DBAS) SUBTYPE(OSAM,VSAM)

Response:

Subpool MbrName CC BufSize PoolId NBuf ProcMbr Section FixOpt

-------------------------------------------------------------------------------

OSAM IMS1 0 512 10000 DFSDFGS1 OSAM001 N/N

OSAM IMS1 0 1024 OSM1 16000 DFSDFGS1 OSAM001 N/N

VSAM-D IMS1 0 1024 VSM1 20000 DFSVSMGS N/Y/N

VSAM-I IMS1 0 512 VSM1 30000 DFSDFGS1 VSAM001 N/N/N

Note: All columns not shown

19

20

Querying VSAM and OSAM Buffer Pools

• QUERY POOL Command Example 2:

QRY POOL TYPE(DBAS) POOLID(VSM1) SIZE(1024)

Response:

Subpool MbrName CC BufSize PoolId NBuf ProcMbr Section FixOpt

-------------------------------------------------------------------------------

VSAM-D IMS1 0 1024 VSM1 8000 DFSVSMGS N/N/N

Note: All columns not shown

20

21

• UPD POOL TYPE(DBAS) change requests are processed serially

UPD command can have partial success and partial failures

Failed changes can be located anywhere in the update sections

Determining partial success and failure of command:

• Two possibilities:

– Failure occurred prior to SPOC Timeout

– Failure occurred after SPOC Timeout

21

UPD POOL TYPE(DBAS) Command Execution

22

UPD POOL TYPE(DBAS) Command Execution

• UPD POOL TYPE(DBAS) cannot be cancelled or aborted

Command may produce unintended or partial results

Two execution possibilities:

• UPD command completes before SPOC Timeout

– Reason codes displayed in SPOC show results of command changes

or

• UPD command completes after SPOC Timeout

– Use additional QRY POOL TYPE(DBAS) commands to determine the

changes made

or

– Use OM Audit Trail to determine the changes made

» Token “rqsttkn1” ties issued commands to command responses

22

23

• Failures occurring prior to SPOC Timeout

Reason codes are displayed in SPOC

23

CC CCText Code

x’EA’ Dynamic buffer pool failure N/A

x’EB’ Resource unable to be quiesced N/A

x’EC’ Reduced buffer allocation ICC_REDBUF

x’ED’ Minimum buffer allocation ICC_MINBUF

x’1C0’ Poolid error in DFSDFxxx member ICC_POOLID

x’1C1’ DBD error in DFSDFxxx member ICC_INVDBD

x’1C2’ Lock request failed ICC_LOCKFAIL

UPD POOL TYPE(DBAS) Command Execution

24

• Failures occurring after SPOC Timeout

SPOC no longer available to show reason codes

Use QRY POOL TYPE(DBAS) commands

• Determine which changes were made partially or completely

• Determine which changes were not made at all

– Use the OM Audit Trail

• Connect issued commands and responses

24

UPD POOL TYPE(DBAS) Command Execution

25

UPD POOL TYPE(DBAS) If Storage Is Unavailable

• When UPD POOL TYPE(DBAS) cannot satisfy the storage request

IMS tries to complete command with “less storage” than requested

• OSAM

– Attempts to allocate a smaller amount of storage

» First attempt is 90% of requested amount

» Second attempt is another smaller percentage

» Threshold is reached at 4 buffers

• VSAM

– Attempts to allocate a smaller amount of storage

» First attempt is 50% of requested amount

» Second attempt is another smaller percentage

» Threshold is reached at 3 buffers

Reason codes indicate requested storage was reduced

• x’EC’ = ICC_REDBUF -> Reduced buffer allocation

25

26

Effect of Long-Running BMPS on UPD POOL Command

• Long-running BMP can prevent UPD POOL TYPE(DBAS) completion

Two execution possibilities:

• Subpool(s) is eventually “freed up” for change:

– OSAM waits for target subpool to be “Not Owned”

» See OSAM Buffer Pool Quiesce Processing

– VSAM waits for DL/I activity to reach commit points

» All PSTs with affected subpools held until UPD completes

» Recommend spreading databases across multiple subpools

» See VSAM Buffer Pool Quiesce Processing

or

• Subpool(s) is not “freed up” for change

– UPD command waits indefinitely

26

27

OSAM Buffer Pool Quiesce Processing

• When subpool ownership goes to zero, subpool is destroyed

Applications can own one OSAM (or VSAM) buffer at a time

UPDATE POOL command quiesces buffers

• Only after application ownership of buffer goes to zero

After buffer is quiesced

• Applications must wait for buffer to be reconfigured

Buffer reconfiguration causes:

• Subpool to be purged and altered buffers to be committed

• Subpool and buffer prefixes are released

• Fixed pages and CF resources are released

• Local cache is released

• Sequential Buffers are invalidated

After reconfiguration of buffers

• Applications requesting buffer resume processing

27

28

VSAM Buffer Pool Quiesce Processing

• Activity against affected subpools is quiesced, subpool is destroyed

IMS looks at PSTs to find PSBs with intent (read or update) on

databases

• If intent found, PST is quiesced at commit point

When all PSTs are quiesced

• Buffer pools are purged

• Open database data sets are closed and reopened

• New applications with sensitivity to database

– Held until UPDATE POOL command completes

– Applies to both READ and UPDATE access

• Old subpool is deleted and new subpool is created

Assigning a database data set to a new pool

• Does not trigger quiesce processing

No TIMEOUT parameter for UPDATE POOL command

• No ability to interrupt the command once issued 28

29

Buffer Pool Statistics and Database Data Set Reassignment

• Buffer Pool statistics handled differently for VSAM vs. OSAM

VSAM statistics are reset

• Old statistics are not carried over

OSAM statistics are accumulated

• Old statistics are carried over

• Database data set reassignment

OSAM

• Reassignment occurs after database data set is closed and reopened

• Close of data set is explicit (not part of command)

VSAM

• Reassignment occurs after database data set is closed and reopened

• Close of data set is implicit (if target subpool is also changed)

– If no change to target subpool, close of data set is explicit

29

30

Initialization and IMS Restart for Buffer Pools

• Buffer Pool Initialization

During IMS initialization, buffer pools are created using DFSVSMxx

Buffer pools are dynamically modified using DFSDFxx and UPDATE POOL

command

• IMS Restart

Committed buffer pool changes are written to Restart Data Set (RDS)

• Emergency Restart will restore buffer pools using RDS

• Normal Restart (COLD or WARM) will initialize buffer pools from DFSVSMxx

XRF Takeover loads committed buffer pool changes from RDS

RSR and FDBR do not track committed buffer pool changes

• Log Records

UPDATE command changes are logged with x’22’ Log record

• Log record is for information only

• Non-recoverable command

30

31 31

Dynamic Full Function Database Buffer Pools Summary

• IMS 12 added dynamic buffer pool support for full function databases

• Users can dynamically manage full function buffer pools

• Buffer pool definitions can change without taking IMS down

• For VSAM, application activity is internally quiesced at commit

• For OSAM, buffer activity is quiesced when application usage is zero

• Benefits

Improved database buffer pool management

• Provides ability to change pool specifications dynamically for

better usability

• Eliminates system down time for modifications to pool definitions

32

IMS Buffer Pool Analyzer

• Provides a method to tune IMS database buffer pools without any guesswork

• Projects the impact of potential changes to buffer pool definitions

• Projects the impact of potential changes to database definitions (like changing from OSAM to VSAM or changing block size)

• OSAM coupling facility caching information to allow analysis of the benefit of caching OSAM buffers in a coupling facility

• Support for IBM IMS Tools Knowledge Base to store and retrieve Buffer Pool Analyzer reports

What it does:

33

IMS Buffer Pool Analyzer

34

IMS Buffer Pool Analyzer

• BPA will tell you exactly what each subpool's performance would be with 25 different numbers for buffer allocations

• The user can judge where they are on the I/O rate curve, and how many buffers would have been required to achieve a particular hit ratio or buffer life

• You don’t have to guess whether adding buffers will help performance! Or how many buffers would help performance!

What it shows:

35

IMS Buffer Pool Analyzer

36

IMS Buffer Pool Analyzer

37

IMS Buffer Pool Analyzer

38

IMS Buffer Pool Analyzer

39

IMS Buffer Pool Analyzer

40

IMS Buffer Pool Analyzer

• Allow user to specify an updated subpool configuration and show

resulting statistics

– Add or remove subpools

– Change which DBDS are assigned to subpools

• BPA reports all the same information as the “actual configuration”

reports, including subpool details and recommendations for number

of buffers for subpools.

Additional Features:

41

IMS Buffer Pool Analyzer

42

IMS Buffer Pool Analyzer

• Allow the modelling of applications being added to a environment to

judge the impact on an entire system

• Allow the user to request a report showing the impact of changing a

DBDS access method (OSAM/VSAM) or block size

– Just specify a change DBD statement in report JCL

– Produces all the same reports and recommendations

• Can be used with updates to subpool definitions to project the impact

with subpool changes

Additional Features:

//BPL EXEC PGM=BPLMAIN,REGION=64M //STEPLIB DD DSN=&SBPLLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //TRACE DD DSN=&TRACE,DISP=SHR //SYSIN DD * CHANGE DBD=(PDB2001,A),AM=VSAM,DATACISIZE=8192

43

IMS Buffer Pool Analyzer

44

IMS Buffer Pool Analyzer

• Improved system performance compared with manual (guesswork) tuning … This impacts IMS performance AND overall system I/O rates

– May also impact DASD tuning requirements

– Provides reports that show buffer pools that can be REDUCED in size – something dangerous to try to do in a busy system

– Consequences of reducing the number of buffers in a pool can be highly disruptive!

– Reduced real memory utilization

– If DB buffers are fixed, this can be a very large savings. When a subpool is over allocated, no one wants to try reducing the number of buffers when there could be a huge negative impact on performance. BPA tells you how much a subpool can be reduced, and what the performance impact will be

Benefits:

45

IMS BPA - ROI Savings

• Save CPU cycles – When buffers are appropriately allocated the data is immediately available

to the end user. Data that is already in the buffer reduces both CPU and I/O needed to present data. SLAs are more easily met and kept.

• Improve online response times and batch elapsed time – Properly allocated buffers provide consistent and reliable access to data

without transaction volume spikes impacting access. User can expect the same responses at any time of day.

• Simulate buffer pool behavior – Prevent buffer pool changes made that will adversely impact system

performance by modelling new application function that allows confidence in buffer allocation before changes are made that may impact current functions and SLAs.

• Improve staff efficiency – Reduce the time to properly tune IMS buffers by having BPA providing the

information for the correct buffer allocations, eliminates trial and error tuning.

46

IMS Buffer Pool Analyzer with IMS V12

IMS V12 provided capability for Dynamic FF Database Buffer Pool

management

• Improved buffer pool management

• Provides ability to change specifications dynamically for better

usability

• Eliminate system down time for modifications to buffer pool definitions

Using IMS Buffer Pool Analyzer:

• Measure actual buffer pool performance

• Model changes without IMS outage

• Ability to evaluate multiple changes vs. implementing single changes

• Perform accurate IMS Buffer Tuning with confidence and no

guesswork

47