IMS Cloning Tool - IMS UG May 2012 Philadelphia

1 © 2012 IBM Corporation

IMS Cloning Tool

IMS Regional User GroupPhiladelphia, PA5/2/2012

Rick EngelIMS Tools – Technical Sales [email protected]


Topics for Today’s Meeting

• Fast Replication Storage– Host-based data copy options

– What is Storage-based Fast Replication?

– Why is Storage-aware Fast Replication Needed?

• IMS Cloning Tool– Cloning an IMS System

– Refreshing Databases


Host-Based Data Copy Options

• Volume copy options

– DFSMSdss (IBM)

– FDR (Innovation Data Processing)

– TDMF (IBM)

– FDRPAS (Innovation Data Processing)

• Data set copy options

– DFSMSdss (IBM)

– FDR (Innovation Data Processing)

“Traditional” data copy processes use host based CPU and I/O facilitiesMore costly and slower than storage-based fast replication

Host-basedCopy Process

An “expensive” use of processing power


Storage Processor = Storage + Processor

• Storage

– Non-volatile

– Computer components and recording media that retain digital data

• Processor

– Computing system that executes software programs

• Storage Processors

– Enable data movement through host-based API

– Allows CPU reduction from host-based systems

Host based APIs


What is Storage-based Fast Replication?

• An instant copy of a volume/data set at a specific point in time

• Data movement (CPU and I/O) offloaded to storage processor

– Frees up resources on host processor

– No host CPU or I/O costs

• For volume replication a relationship is established between a source and a target (Pair)

– Geometrically similar devices

• Consistency Groups

– Group of volumes copied at exactly the same point in time while maintaining the order of dependent writes

Storage Processor-based

Copy Process


Local Storage-based Fast Replication• First product availability - late 90s

– Used to streamline batch processing

– Speed for backup processing

• Data copied using storage processor fast-replication facilities

– Volume based

– Data set based

• No application or database knowledge

• Typically used by storage administrators

• Examples

– IBM FlashCopy

– EMC TimeFinder

– HDS Shadow Image

Storage Processor APIs

Local Storage Replication

Interface

Data set BasedFast Replication

Volume BasedFast

Replication


Remote Storage-based Fast Replication

Remote Replication

Interface

Primary Production Site Secondary Production Site

Remote ReplicationPPRC, SRDF, HUR

Fast Rep DASD at another level Volume based remote

storage replication Used for business

continuity and/or high availability

Synchronous or asynchronous

Consistency technology required

No application or database knowledge

Typically used by storage administrators

Examples EMC SRDF IBM PPRC, XRC HDS HUR

‘Mirroring’


Fast Rep Data Copy Options – Many Choices

• Volume-based Fast Replication options– FlashCopy (IBM,EMC,HDS)

– SnapShot (IBM,STK)

– TimeFinder / Clone Volume Snap (EMC)

– TimeFinder / Snap (EMC)

– Mirror processes

PPRC (IBM,EMC,HDS) TimeFinder / Mirror, SRDF (EMC) ShadowImage HUR (HDS)

• Dataset-based Fast Replication– Data set FlashCopy (IBM,EMC,HDS)

– Data set SnapShot (IBM,STK)

– TimeFinder/Clone data set Snap (EMC)

Common denominator = eliminate usage of host-based resources

Fast copy processes offloaded to the storage processorNo host CPU or I/O resources = $Savings

Fast ReplicationCommands from

z/OS


Application and Database Storage Integration

MainframeApplication and

Database Systems

Storage-AwareData

Management Tools

Application and Database Management

Domain

Storage Administration and

Business ContinuityDomain

“Having storage-aware tools” provide for: • Organizational Integration• New Backup Methods• New Recovery Strategies• Business Recovery Monitoring• Disaster Restart Solutions• Cloning Automation

SourceData

Backup,Clone,

DR


Storage-based Fast Replication Usage - Today

• Storage-based fast replication is a huge success but used ineffectively by many (most) mainframe installations

– Requires application and database knowledge

– Operationally complex to use

– Historically used solely by storage administrator role

• Storage-aware application and database products provide storage awareness and simplify operational usage


IMS Cloning Tool

Capitalizing on your Storage Subsystem


Cloning – Some Terminology

• A clone is an exact replica

– Clone an IMS system by volume

– Clone an IMS database by data set

• IMS system cloning and database refresh

– The act of replicating the data, making the replica accessible, and then using the replica in lieu of the original data

• IMS system cloning automation

– Clones a complete IMS system including all its databases.

– Creates a new, or cloned, IMS system from it without having to repeat the entire installation and system generation processes

– Less than 30 minutes (average when using Fast Replication)

• IMS database refresh automation

– Refreshes specific databases


IMS ‘System’ Cloning - Today

Production LPAR

User Catalogs

IMS001

IMS System Data Sets

IMS002

Application Libraries

IMS003

RECON Data Sets

IMS004

Data Base Data Sets

IMS005

Test LPAR

User Catalogs

IMS001


IMS002


IMS003

RECON Data Sets

IMS004

Data Base Data Sets

IMS005

Two reads of the data


IMS ‘Database’ Cloning – How you do it ?

Production IMS


RECON Data Sets

Data Base Data Sets


Test IMS


RECON Data Sets

Data Base Data Sets


Unload Load

Image Copy

Recovery

Index Rebuild

???

z/OS


Problems with Traditional “Cloning” Methods

• Manual step by step methods can be error-prone, inefficient, time consuming, and complicated

– Can require the data to be stopped for a long time while the copy operation is performed

– Downtime impacts high availability, business waits

• Expensive

– Resource intensive using large amounts of host CPU and I/O

• For IMS system cloning

– Requires a separate LPAR to house the copy

Same volsers, same data set names

– Requires Sysgen


Production Database

IMSCloneIMS

Cloning ToolCloning

Automation

SourceDatabaseVolumes

CloneDatabaseVolumes

• Performs IMS system cloning automation– Takes an existing IMS system (complete installation and

system generation process completed) and creates a new(“cloned”) IMS system from it without having to repeat the entire installation and system generation processes

– You benefit by: Simplfying IMS cloning processes Reducing cloning time and administration costs

• Performs rapid volume reconditioning and dataset renaming on cloned volumes to solve the data access challenges

– Target volumes have different volser label than the source volume

– Renames the VTOC, VTOCIX, and VVDS to match the target volume

– Renames and catalogs all data sets to a new HLQ

• Adjusts target IMS to accommodate and accept the cloned data

– IMS RECONs, PROCLIB, JOBS, JCL, MDA members

Create IMS Systems Using IMS Cloning Tool Automation


Production Database

IMSCloneIMS

Cloning ToolCloning

Automation



• Updates done in IMS cloned system– RECONs data sets

Data set names, IMS subsystem IDs, and VOLSERs are updated in the following RECON records:

→ header record, database data set records, online log records, and back-out records

Optionally, the following RECON records are updated if they were on volumes that were cloned:

→ image copy records, change accumulation records→ system log data set (SLDS) records→ recovery log data set (RLDS) records

– IMS PROCLIB, JOBS and user JCL libraries New values for IMSID, VOLSERS, and data set names in the JCL members

within these libraries

– MDA (MVS Dynamic Allocation) members for databases or system data sets, are updated to reflect the new data set names. Updates include:

The RECON data sets Online Log data sets (OLDS) Write-ahead data sets (WADS)

– If IMS data sharing is involved, each additional IMS data sharing member is updated

• Leverages fast replication facilities to clone data

– Data can be cloned while online or offline

Create IMS Systems Using IMS Cloning Tool Automation


What makes IMS Cloning Tool so “fast”?

• Host-based Copy Options

– Data copy processes use host based CPU and I/O facilities

– Slower than storage-based fast replication

– Volume copy options for IMS system cloning

TDMF (IBM),

FDRPAS (Innovation Data Processing)

DFSMSdss (IBM) FDR (Innovation Data

Processing)

– Data set copy options for IMS database refresh

Any traditional data set copy processes

• Fast Replication Data Copy Options

– Fast copy processes offloaded to the storage processor

– No host CPU or I/O resources

• Data set based fast replication options for IMS database refresh

– Data Set FlashCopy (IBM,EMC,HDS)

• Volume based fast replication options for IMS system cloning

– FlashCopy (IBM,EMC,HDS)

– SnapShot (IBM,STK)

– TimeFinder/Clone Volume Snap or Snap (EMC)

– Mirror processes PPRC (IBM,EMC,HDS) TimeFinder/Mirror, SRDF (EMC) ShadowImage HUR (HDS)

– Data set SnapShot (IBM,STK)

– TimeFinder/Clone Data set Snap (EMC)

Host-based

Copy Process

Fast ReplicationCommands from z/OS


IMS Cloning - Methods

• IMS System Skeleton– Clones an IMS system without cloning any data by volume

Recommended for IMS new version migration activities

– The replicated system is accessible and is usable in lieu of the original system without requiring a system generation

• IMS Full System Clone– Copies an IMS system WITH the databases by volume

– The replicated system is accessible and is usable in lieu of the original system without requiring a system generation

– Less than 30 minutes average when using “Fast Replication”

• IMS Data Refresh– Copies individual or groups of databases by data set


Production Database

IMSCloneIMS

IMS Cloning ToolCloning

Automation

SourceSystem

Volumes

CloneSystem

Volumes

• Performs IMS system “skeleton” cloning automation– System, no databases

– Takes an existing IMS system and creates a new, or cloned, IMS system from it

– Do not need to repeat the entire installation and system generation processes

– Simplifies IMS cloning processes

– Reduces cloning time and administration costs

• Leverages fast replication facilities to clone data– Can also leverage traditional Host Based copy options

• Performs rapid volume reconditioning and data set renaming on cloned system volumes to solve the data access challenges

– Target volumes retain their target volume label



• Adjusts target IMS to accommodate and accept the cloned data

– IMS RECONs, PROCLIB, JOBS, JCL, MDA members

Using IMS Cloning Tool - IMS System Skeleton


IMS System Skeleton - What can it deliver to you

• Quickly creates a “Sandbox Environment” with an exact copy of the IMS configuration of the source system (system only, no data).

– Takes up very little space

• A safe place to test RECON fixes

• A safe place to test the apply process

– Install prerequisite software and maintenance

– Install IMS new version

– System Definition steps including DBRC migration process

• Can you still bring your IMS up?

Are you missing any modules?

Did your RECON upgrade correctly?

Did you skip a step in the apply process?

Run the System Skeleton clone for each configuration and repeat apply testing

Repeatable – automation cleans up between cloning cycles


Production Database

IMSClone

IMS



• Performs IMS system cloning automation– Takes an existing IMS system (complete installation and system

generation process completed) and creates a new, or cloned, IMS system from it without having to repeat the entire installation and system generation processes

– Simplifies IMS system copy processes

– Reduces cloning time and administration costs

• Leverages fast replication facilities to clone data– Data can be cloned while online or offline

– Can also leverage traditional Host Based copy options

• Performs rapid volume reconditioning and data set renaming on cloned volumes to solve the data access challenges

– Target volumes retain their target volume label



• Adjusts target IMS to accommodate and accept the cloned data– IMS RECONs, PROCLIB, JOBS, JCL, MDA members

Using the IMS Cloning Tool - IMS Full System Clone

IMS Cloning ToolCloning

Automation


IMS Full System Clone - What it can deliver to you

Quickly Copies an entire IMS system without having to repeat the entire installation and system generation processes

Fast with no interruption to the source system

Copies all the data and is faster than database refreshes

• Offload business reporting to an IMS clone– Improve production performance

• Create or refresh test, development, or quality assurance environments – Development refresh

– To stage data-warehouse loads

• Aid in problem determination– Troubleshoot a production problem

– Test new releases of IMS

– Apply maintenance and verify integrity before applying to production


IMSSystem Volumes

IMSTarget

IMS Clone

Volumes

IMSSource

IMS Cloning Tool Database Refresh

Automation

• Performs automated IMS database refresh operations– Fast refresh of IMS databases

– IMS DB support (FF, HALDB, DEDB)

• Verifies source and target database compatibility

• IMS data copied using storage-based data set fast-replication

– Target takes up the same amount of space as the source

• Performs target system meta-data management– Updates DBRC information for target databases

• Performs HALDB cloning with integrity– Updates the target IMS RECON– Primary/Secondary indexes and ILDS can be copied to eliminate rebuild time– Manages partition number and gap sequencing

Using IMS Cloning Tool - IMS Database Refresh

Production Database


IMS Database Refresh - What it can deliver to you

• Quickly copies a specific database or set of databases for targeted testing. – Flexibility to select exactly what database you need thus saving space and time

• Application Testing Before a major upgrade to the application To debug an error that only happens in production Create consistent test environments between phases of the testing cycle Test an application in conjunction with system

parameter tweaking

• Database Testing HALDB conversions can be complex and affect database

availability Run performance and integrity utilities


IMS Cloning Tool – Benefits and Advantages• Creates an “exact” Copy

– Avoid surprises

– Avoid errors

– Enable the “lesser” skilled

• Fast– No interruptions to your production

– Less than 30 minutes average for a full system clone when using Fast Replication

• Easily Repeatable– One batch job

– Automation includes clean up features between cloning cycles

• Flexible– Easily controlled, parameter and command driven

• Disposable– If anything fails or you want to try something different, just re-clone the environment


Using IMS Cloning Tool to Migrate to New IMS ReleasesTeleconference – Webcast on May 8, 2012 11:30 AM EST

- Automating the cloning process of an entire IMS System with all its data, just an IMS environment, or just IMS

- Reducing production downtime when creating exact clones for migration testing

- Applying resources based on which vendor and home-grown products need upgrades

Speakers: Margaret M Wilson, IMS Tools Marketing Manager, Rosemary Shay, Quality Assurance and Product Specialist for IMS Tools

Broadcast date: May 8, 2012, 11 a.m., Eastern Daylight Time

http://www-01.ibm.com/software/sw-events/teleconference/J210596Q30517L24.html

http://www-01.ibm.com/software/sw-events/teleconference/J210596Q30517L24.html


Rick EngelIBM IMS Tools Technical Sales [email protected]

Thank You!

IMS Cloning Tool - IMS UG May 2012 Philadelphia

Technology

Transcript of IMS Cloning Tool - IMS UG May 2012 Philadelphia