Cloud Object Storage | Store & Retrieve Data Anywhere ...€¦ · Created Date: 20170804173710Z
Accelerate with IBM Storage: TS7700 Object Store and ... · Object Storage 1. Read Data from Object...
Transcript of Accelerate with IBM Storage: TS7700 Object Store and ... · Object Storage 1. Read Data from Object...
© Copyright IBM Corporation 2020
Accelerate with IBM Storage: TS7700
Object Store and DS8000 Transparent
Cloud Tiering
Eddie Lin, DS8000 Architect and Development
Lourie Goodall, TS7700 Architect and Development
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Accelerate with IBM Storage Webinars
The Free IBM Storage Technical Webinar Series Continues in 2020...
Washington Systems Center – Storage experts cover a variety of technical topics.
Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome.
To automatically receive announcements of upcoming Accelerate with IBM Storage webinars, Clients, Business Partners and IBMers are welcome to send an email request to [email protected].
Located on the Accelerate with IBM Storage Site: https://www.ibm.com/support/pages/node/1125513
Also, check out the WSC YouTube Channel here: https://www.youtube.com/channel/UCNuks0go01_ZrVVF1jgOD6Q
2020 Upcoming Webinars:
April 21 – CSM 6.2.8 - What's New!Register Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=ef2d68c2c198e337c1e222998b32528e0
April 23 - TS7700 Security - Keeping Your Castle SafeRegister Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e4f9c3ec70aa0856171e3255ab0136993
May 12 - Artificial Intelligence (AI) Scalability for a DecadeRegister Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e6a684ae106025eeebb60ad03ec0de15b
May 21 - Storage Insights, Storage Insights Pro or Spectrum Control, which one is right for me?Register Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e32d1989e32fe468ef72c082519920fc1
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Session objectives
• Introduction
• DS8000 Transparent Cloud Tiering
• TS7700 as an Object Store
• Q & A
3
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Evolution of Mainframe Space Management
FlashCopy
Class
Transitions
Transparent
Cloud Tiering
TCT to TS7700
as an Object
Store
DFSMShsm wants OUT of the
business of physical data
movement
Easy Tier
Tape Tiering
Efficiency
5
The Transparent Cloud Tiering Advantage Cost SavingsSecurity
DFSMS
Primary Disk
Virtual or Physical Tape
1. Read Data from Tape into z/OS2. Write Data from z/OS to Disk3. Read Data from Disk to z/OS4. Data Available to User
Data Recall Steps
Efficiency
6
The Transparent Cloud Tiering Advantage IntegrationSecurity
Primary Disk
Object Storage
1. Read Data from Object Store into Primary Storage2. Data Available to Customer
Data Recall Steps
Direct data transfer from object storage to primary storage
• Eliminates data movement through IBM Z• Eliminates Tape Recycle Processing• Eliminates 16k Block Size restructuring• Eliminates Serial Access to Tape
• SSL/TLS Authentication
• TCT Encryption with Dataset Level Encryption Support
• Seamless integration into DFSMShsm and DSS
• Supports Off-Prem and On-Prem Object Storage and now TS7700 Virtual Tape
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Transparent
Cloud Tiering
7
z/OS Transparent Cloud Tiering with DS8880 / DS8900 Current Key Use Case: DFSMShsm Dataset Migrate/Recall
• MIPS reduction via direct data transfer from DS8000 to Object Storage• Recall to volumes participating in Simplex, Flashcopy, Metro Mirror, Global Mirror, MGM Environments
• Includes support for Hyperswap• TCT Client Side Encryption• 1Gbit (Included) and 10Gbit (Additional) Ethernet Connectivity• Command Line and Automated HSM Migration – Dataset movement to Cloud by policy, size based override• Supports IBM Cloud Object Storage, Amazon S3 API, Openstack Swift, TS7700
IBM Cloud
Object Storage
IBM Z IBM DS8880
Transparent Cloud Tiering
IBM
TS7700
IBM Cloud
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage9
TCT CPU Savings Estimator
9
Based on approximations from preliminary internal IBM data measurements. Results will vary by
customer based on particular workloads, configurations and software levels applied.
Estimation for Daily PSM
CPU reduction
Total reported CPU compared
against an estimated CPU:
(.05 seconds * # data sets)
Remember, reduction is based
on the size of the data being
moved. Clients with mostly
small data sets will see modest
reductions.
ftp://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/HSM/zTCT/
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage10
Cloud Objects within the Object Store
10
• TCT stores datasets as individual objects, not as logical tape volumes.
• DFSMShsm considers objects a separate tier versus ML1, ML2 (Migration Level CLOUD)
• VOLSERs are modified to MIGRATC
• Datasets are a collection of objects : metadata and customer data are split
• Allows for future work with analytics, data sharing, and further tiering
DCSDMC01.GDG.V12S.G0001V00/APPMETA
DCSDMC01.GDG.V12S.G0001V00/DTPDSHDR
DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/META
DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/EXTENTS
VOLSER Z20022
VS.
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage11
Use Case : Targeted HSM Automatic Migration
11
• Management Class Enhancements
• Use new “Level 2 Days Non-Usage”
parameter with value of “0” to define moving
directly to CLOUD based on primary days
non-usage
• Use new SIZE based parameters to further
differentiate datasets to CLOUD
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
AB C D
L0
MLC
AutoMigration Window 2
• Runs 12 hours• Process large sized
IF DS Size < 1 GBNo Action
ELSECloud
ML2
AutoMigration Window 1
• Runs 4 hour• Skip small / Process medium sized
IF DS Size <= 1 MBNo Action
ELSE IF DS Size >= 1 GBNo Action
ELSEML2
Combine HSM Auto Migration and TCT’s minimized CPU footprint
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Use Case : DSS Backups Migrated to TS7700
13
DFSMSdss
Source Storage Group
DSS Backup Copies
DFSMShsm TCT
▪ Step 1: Create DSS Disk backup with zEDC
▪ If source data set is already zEDC
compressed, backup will be same
size as source
▪ Otherwise, backup copy will be zEDC
compressed (VSAM, etc)
▪ Write backup copies to one or more
storage groups managed by HSM
▪ Step 2: Management Class and Storage
Group policies used by HSM to automatically
expire/migrate older backup copies to TS7700
▪ Backup copies with a short life can
expire on disk (HSM expiration)
▪ Backup copies with a long life can be
automatically migrated to TS7700 after
a policy based time period
X
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Other TCT use cases
Flashcopy to a Storage Group, HSM Migrate to Cloud
14
Move very old ML2 data to MLC to eliminate recycle
• Create Point in Time Copy via DSS
FlashCopy
• eg. DB2 Image Copy
• Use HSM: MIGRATE
STORAGEGROUP(DB2IMGC)
CLOUD(MYCLOUD)
• Note: For now, manual recall
required before DSS restore
• ML2 data must be recalled and
then remigrated from ML2 to MLC
• Target very old datasets to avoid
additional recycles
Source
VolumesStorage Group
FlashCopy HSM
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
TCT in DS8000 Replicated Environments
15
Topology Minimum Code Level Support
Metro Mirror DS8880 R8.3• DFSMS APARs OA52386
• PTFs on z/OS V2R2 and z/OS V2R3
• GDPS APAR PI98596
DS8900 R9.0
Global Mirror DS8880 R8.5
DS8900 R9.0
MGM DS8880 R8.5
DS8900 R9.0
Multi-Target MM or MM/GM DS8880 R8.5
DS8900 R9.0
zGlobal Mirror (XRC) Not Supported
FlashCopy DS8880 R8.4• DFSMSdss APAR OA55043
• PTFs on z/OS V2R2 and z/OS V2R3
DS8900 R9.0
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
TCT Encryption for Object Storage
• Client Side Encryption• 256-bit AES Encryption
• Encryption happens in the DS8000, data remains encrypted
across the link and in the cloud
• Hardware accelerated encryption using POWER8 and 9
• Only customer data is encrypted
• Requires an external key manager (KMIP)• ISKLM 3.0.0.2+ (Distributed)
• Interlock with DFSMS dataset level encryption• If the dataset is already encrypted, DS8000 will not attempt
to encrypt again
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
TS7700 AS AN OBJECT STORE
17
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Transparent Cloud Tiering with TS7700 as Object Store
18
• TS7700 as a target object store
• Supports TS7760/TS7770 Virtual Tape
and DS8880/DS8900 Disk systems
• Intermix existing FICON tape workloads
and TCT Object workloads to the same
TS7700
• Data flows from DS8000 to TS7700 via
GRID network links
• Utilizes logical partitions dedicated to
TCT objects
• Supports data redundancy via writing to
two TS7700 clusters within the same
GRID
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage19
Grid Cloud
Clo
ud
Clo
ud
TS7760C TS7760CTS7760T TS7760T
DS8880 DS8880 DS8880
Ethernet
Z System
FIC
ON
FIC
ON
• Co-existence with up to 8-way GRIDs
• TS7700 can be tapeless, tape attached,
or cloud attached.
• Objects cannot be migrated to tape or
cloud.
• Support multiple DS8Ks connect to a
TS7700
• Uses existing cache. MES of additional
cache not required
Transparent Cloud Tiering with TS7700 as Object Store
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Logical Cache Partition for Objects
DS8000
FICON
Workload
Policy
Managed
TS7770T or TS7770C
Object
Workload
Cloud
• Utilizes existing TS7700 Logical Cache Partitioning
• Separates FICON logical volume data from object data
• Coexists with Tape or Cloud logical volume partitions
• Allocates from Cache Partition 0 (Resident only partition) and returns to CP0
Logical volume data
Object data
Resident Only
Partition
20
zHost
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Parallel Access Objects
21
DCSDMC01.GDG.V12S.G0001V00/APPMETA
DCSDMC01.GDG.V12S.G0001V00/DTPDSHDR
DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/META
DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/EXTENTS
• Allows for parallelism of object retrieval and store• Eliminates serial access to tape
DS8000
TS7700
Z
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Eliminates HSM Reclaim
22
Z
DS8000
TS7700
dksksksk
21%
dksksksk
20%
dksksksk
23%
dksksksk
21%
dksksksk
20%
dksksksk
23%
dksksnnnksk
Recall
Reclaim
Rewrite
HDELETE
dksksnnnks
k
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Requirements for DS8000 TCT with TS7700 as an Object Store
• DS8880 and DS8900 supported (all models)
• Part of the zSynergy License Bundle
• DS8880 R8.5SP4.1 minimum level
• DS8900 R9.0SP1 minimum level
• Ethernet connections
• Note: DS8900 10Gb Ethernet CANNOT direct connect to TS7700 GRID adapters (SR vs. LR connectivity). DS8880 10Gb Adapters can direct connect.
• TS7760 (VEC) and TS7770 (VED) supported
• Minimum code level is R5.0
• Feature Code #5282
• TS7760T tape attached, TS7760C cloud attached, TS7760 tapeless supported
• Uses existing GRID adapters and will share TCPIP traffic with the GRID
• z/OS V2R3 and V2R4 supported
• APARs required to enable ”Tape Object” type in SMS Cloud Construct/Network Connection panel23
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Easy Setup Steps
24
TS7700Activate the FC 5282 and enable DS8000 Object Store01
TS7700Enter DS8000 serial number and IP addresses02
TS7700Resize Object cache partition03
z/OSCreate Cloud Network Connection construct in the z/OS ISMF Cloud
panel using provider “TAPE-OBJECT”.04
z/OSCreate z/OS Management Class construct “Level 2 Days Non-
usage”=0 05
DS8000Enable Ethernet ports and enter TS7700 Grid IP address and
Network Connection name in DS8000 mkcloudserver command06
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Object Cache Partition
• Object Cache Partition now visible on the TS7700 MI (tapeless)
• New object partition will now exist - created during Enable process
• Resize partition
• Object Count
25
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Library Request CACHE2
Displays the partition type and cache utilization statistical information
26
> SHOWING RESULTS FOR COMMANDS: CACHE2
"TAPE VOLUME CACHE STATE V5 .0 "
"TOTAL SPACE INSTALLED/ENABLED: 191TB/ 191TB "
"TOTAL ADJUSTED CACHE SPACE USED: 1044GB "
"CACHE ENCRYPTION STATUS: CAPABLE "
"OVERCOMMITTED CACHE PARTITIONS: NONE "
"CACHE RESIDENT ONLY PARTITION "
" PRIVATE CACHE SPACE USED: 1042GB "
" SCRATCH CACHE SPACE USED: 0.0GB "
" CP ALLOC USED PIN PKP PRM COPY CPYT "
" 0 70.8TB 1042GB 0.0GB 1042GB 0.0GB 0.0GB 0 "
" FLASH COPY INFORMATION "
" INDEX ENABLED SIZE "
" 1 NO 0.0GB "
" 2 NO 0.0GB "
" 3 NO 0.0GB "
" 4 NO 0.0GB "
" 5 NO 0.0GB "
" 6 NO 0.0GB "
" 7 NO 0.0GB "
" 8 NO 0.0GB "
"TIERED CACHE PARTITIONS "
" CP TCO ALLOC USED PG0 PG1 PMIGR D_PMIGR COPY PMT CPYT "
" 1 NNY 121TB 2189GB 0.0GB 2189GB 0.0GB 0.0GB 0.0GB 0 0 "
" 2 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
" 3 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
" 4 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
" 5 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
" 6 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
" 7 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "
> EXECUTING COMMANDS: CACHE2
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Library Request SETTING2, RSDOLOW/RSDOHIGH
• New settings RSDOLOW and RSDOHIGH
• LI REQ SETTING2 - Displays current settings
• LI REQ SETTING2,RSDOHIGH,<value GB> - High warning limit of resident data in the object partition
• LI REQ SETTING2,RSDOLOW,<value GB> - Low warning limit of resident data in the object partition
27
> SHOWING RESULTS FOR COMMANDS: SETTING2
"SETTING2 V4 .0 "
" SCRATCH CONTROLS "
" PFRLOCO = ENABLED "
"----------------------------------------------------------------------"
" ALERTS "
" PDRVSLOW = 0 PDRVSCRT = 0 "
" LMTDTHR = 0 "
" CAGALOW = 0 CAGAHIGH = 0 "
" CAGLOW = 0 CAGHIGH = 0 "
" RSDOLOW = 1000 RSDOHIGH = 10000 ”
"----------------------------------------------------------------------"
" CACHE CONTROLS "
" MAXLGMC = 128 "
" OVRSPLVL = 2000 "
"----------------------------------------------------------------------"
…..
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Library Request STATUS,GRLNKACT
28
• "GRLNKACT STATUS V2 .0 "
• "CLUSTER INDEX: 1 LINK COUNT: 4 Time: Wed Feb 27 00:04:15 CUT 2019 "
• "GRID LINK ESTABLISHED SOCKET CONNECTIONS------------------------------"
• "LN INTF IP C0 C1 C2 C3 C4 C5 C6 C7 MQ RFA CLD "
• "L0 en8 9.11.219.164 0 0 55 0 0 0 0 56 111 0 0 "
• "L1 en10 9.11.219.165 0 0 0 0 0 0 0 0 0 0 0 "
• "L2 en9 9.11.219.169 0 0 0 0 0 0 0 0 0 0 0 "
• "L3 en11 9.11.219.170 0 0 0 0 0 0 0 0 0 0 0 "
• "LN INTF IP GGM OBJ "
• "L0 en8 9.11.219.164 0 2 "
• "L1 en10 9.11.219.165 0 0 "
• "L2 en9 9.11.219.169 0 0 "
• "L3 en11 9.11.219.170 0 0 "
• "NET ACTIVITY -----TCP RECV/SEND ADAPTER BUFFER ACTIVITY BYTES---------"
• "LN TxMBs RxMBs MQ_REC MQ_SND GFA_REC GFA_SND CLD_REC CLD_SND "
• "L0 100 100 0 0 80 80 0 0 "
• "L1 0 0 0 0 0 0 0 0 "
• "L2 0 0 0 0 0 0 0 0 "
• "L3 0 0 0 0 0 0 0 0 "
• "TOT 100 100 0 0 80 80 0 0 "
• "LN GGM_REC GGM_SND OBJ_REC OBJ_SND "
• "L0 0 0 20 20 "
• "L1 0 0 0 0 "
• "L2 0 0 0 0 "
• "L3 0 0 0 0 "
• "TOT 0 0 20 20 "
• "GRID LINK THROUGHPUT ESTIMATES-MB/s-----------------------------------"
• "DIR C1 C2 C3 C4 C5 C6 C7 GRD_TOT CLD GGM OBJ TOT "
• "Tx 0 50 0 0 0 0 30 0 0 0 20 100 "
• "Rx 0 30 0 0 0 0 50 0 0 0 20 100 "
<--C0 doesn’t show since it is the local
New command added in R4.2. Grid link point-in-time activity.
Add new columns for data for Objects
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
TS7700 Management Interface Changes
29
• Cache Utilization
• Object Count
• Historical Statistics
• Grid link throughput
Flash Copy Data
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
DS8000 Object Store - Object Data Resync/Recovery
30
In the event a TS7700 becomes
unavailable, DS8K will track
transactions not written to the
unavailable cluster and
automatically synchronize the
data once the cluster becomes
available.
XTODO
Object1Object2Object3Object4
Object1Object2Object3Object4
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
TCT, TS7700, GRID and DR Distance
31
• Planning to use synchronous writes for data redundancy?
• Planning on one of those TS7700 Clusters to be at the remote
site?
• Use existing IP Extension options to eliminate latency, just like
GRID to remote locations today
DS8000
Local TS7700
GRID Network (WAN)
Remote TS7700
IP Extension
IBM SAN 42B-R
Brocade 7840
IBM SAN 42B-R
Brocade 7840
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM Storage
Preview Announcement – TS7700 R5.0.1
TCT Compression to TS7700
• Compression will be done within the DS8900
• Uses a hardware compression engine within the POWER9
• Data compressed before transmission as well as when it lands in object store / TS7700
• Bandwidth optimization and storage savings
• Interlock with DFSMS if the dataset is already encrypted or compressed
TCT with Secure Data Transfer to TS7700
• TCT to TS7700 as an object store currently does not support encryption
• Uses the same protocol as Secure Data Transfer for the GRID
• Via embedded TS7700 data movement engine
• Encryption is only for data in flight – decrypted when landing in TS7700
• Uses hardware engine for encryption
• Does not require an external key server
32IBM Systems Technical University © Copyright IBM Corporation 2019
Washington Systems Center - Storage
© Copyright IBM Corporation 2020 Accelerate with IBM StorageIBM Systems Technical University © Copyright IBM Corporation 2019 33
http://publibz.boulder.ibm.com/zoslib/pdf/OA51622.pdf
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.idak100/cloud23.htm
New - Available in DRAFT format for TCT and TCT to TS7700 as an Object Store
Other Links to publications:www.redbooks.ibm.com › Cloud
https://www.ibm.com/support/knowledgecenter/ST5GLJ_8.5.0/com.ibm.storage.ssic.help.doc/f2c_configuring_trans_
cloud_tiering.html
IBM TS7700 Series DS8000 Object Store User’s Guide Version 1.0,REDP-5583
http://www.redbooks.ibm.com/abstracts/redp5583.html
IBM DS8000 and Transparent Cloud Tiering,SG24-8381
http://www.redbooks.ibm.com/Redbooks.nsf/RedpieceAbstracts/sg248381.html
Publications
Unleash the potential of your data.
Put smart to work today.
IBM Storage