Enterprise Removable Media Manager - University of...
Transcript of Enterprise Removable Media Manager - University of...
IBM TotalStorage
© 2005 IBM Corporation
Enterprise Removable Media Manager
A Service Offering forOpen System Tape Management
Ulf Troppens, eRMM Development, IBM [email protected]
2 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Disclaimer
3 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion
4 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion
5 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Customer Problems and Requirements today
Libraries with SCSI Media Changer(e.g. IBM 3584)
Removable Media Hardware
Media Changer Devices
Streaming Devices (Drives)
SCSI/FC SCSI/FC
SAN
Removable Media Hardware
Media Changer Devices
Streaming Devices (Drives)
SCSI/FC prop. InterfaceLibrary Mgmt.
Libraries with integratedManagement (e.g. IBM 3494)
Problems:• No centralized access control, administration and reporting
• Media management has to be done by every application
• Difficult (or nearly impossible) to share libraries, drives and scratch pools, especially in heterogeneous environments
• Multitude of Media Changer andLibrary Management Interfaces(SCSI, IBM 3494, STK ACSLS/LS, …)
• Adding storage or upgrading storage technology requires changes to every application
Requirements:• Improved manageability to cope with rapidly growing storage• Reduced TCO• Efficient usage of tape storage capacity• Increased utilization of tape drives• Integrated support for vaulting
6 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Customer Scenarios
…
Statically partitioned IBM 3584
SAN
Legato Networker and HiBack Server
TSM Server
…EDT
EDT
EDT
STK SN 6000
STK Silos
STK ACSLSServer
…
IBM Tape Hardware
SAN
TSM on z/OSTSM LAN-free clients for open systems
Customer wants:• consolidated media
management for Open Systems first, later with z/OS too
• centralized administration and control
• to get rid of Gresham EDT
Customer wants:• enhanced utilization of drives• to create drive groups and
cartridge pools that span multiple libraries
• centralized media management administration and control for Networker and HiBackapplications
• enhanced monitoring and reporting
Customer wants:• consolidated media
management for heterogenous environment
• Integrated support for vaulting• policy-based cartridge
lifecycle management
Scenario IScenario II
Scenario III
7 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Case Study: Backup Consolidation on zLinux (1/2)
Business needThe customer wants to consolidate its backup infrastructure with TSM on zLinux.
Business valueThe customer builds a backup infrastructure which is very flexible and which scales very well. The advanced management functions of zLinux/VM allow to provision new TSM Servers on demand.
Use case (planned new service)At the customer a business unit can request to establish a new security zone including a dedicated VPN to protect certain applications and its confidential data. The infrastructure team automatically creates a new zLinux Instance with a new TSM server which is only visible in the new VPN. The concept of having a dedicated TSM server for each security zone simplifies the mechanisms to secure confidential backup data.
8 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Case Study: Backup Consolidation on zLinux (2/2)
Similar setup available in collocated data centerBoth data centers connected via DWDM (Gigabit Ethernet + Fibre Channel)Fail-over of zLinux instances between both data centers planned for laterNew TSM servers added on demand
eRMM TSM1
TSM2
… TSMn
TSMn+1
TSMn+2
TSMn+3
… TSMn+m
z/VM 1 z/VM 2
z900
40 zFCP Ports for Tape, 8 zFCP Ports for disk
Fibre Channel SAN
IBM 3584 Tape Library65 x IBM 3592 Tape Drive2320 x 3592 JA Cartridge
IBM DS 830014,7 TB RAID 5 netDisk cache for TSM
eRMM’s advanced tape management capabilitieshelp to master the complexity of this frequently changing setup.
9 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value of the Enterprise Removable Media Manager
Improve the flexibility of the removable media infrastructureResults
Improved Application Availability– Create cross library drive and scratch pools
Optimized Storage Resource Utilization– Share drives between heterogeneous applications– Provide a cross application scratch pool– Define policies that priories concurrent mount requests– Define policies to move cartridges to vaults
Enhanced Storage Personnel Productivity– Create a single point of control, administration, reporting, security and auditing
for removable media resources – Move, add or change removable media resources (drives, cartridges, libraries)
without requiring modifications to applications like TSM
10 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Tape Configuration without eRMM
SAN
IBM 3584 IBM 3494 StorageTek
11 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Enterprise Removable Media Manager
IBM 3584 IBM 3494 StorageTek
Enterprise Removable Media Manager
SAN
12 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Enterprise Removable Media Manager
SAN
IBM 3584
Enterprise Removable Media Manager
IBM 3494 StorageTek
TSMCartridges
Networker Cartridges
STK DrivesLTO Drives
Common Scratch Pool
Drive Pool• Pools of capacity• Segmented basedon business need
Scratch Pool• For all applications• Segmented basedon business needs
Cartridge Pool• Access for owningapplication only
13 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Enterprise Removable Media Manager
SAN
IBM 3584
Enterprise Removable Media Manager
IBM 3494 StorageTek
TSMCartridges
Networker Cartridges
STK DrivesLTO Drives
Common Scratch Pool
Policies• Drive and cartridgeallocation based onpriorities
• Cartridge movement• Cartridge life-cyclemanagement
New Efficiencies• Cross applicationdrive sharing
• Cross applicationscratch pool
• Mount request queuing• Centralized reporting• Audit trails• Integrated vaulting
14 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Dynamic grouping and pooling of drives and cartridges
DriveGroup 1 DriveGroup 2
CartridgeGroup 1
Cartridge Group 2
Library 1 Library 2
DriveGroups:• Used to aggregate drives• Drives can be added and removed dynamically
• Used to implement access permission model and preferential usage policy
• May span multiple libraries
CartridgesGroups:• Used to aggregate cartridges• Cartridges can be added and removed dynamically
• Used to implement access permission model and preferential usage policy
• May span multiple libraries
15 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Access Roles and Preferential Usage Policy for DriveGroups and CartridgeGroups
DriveGroup 1 DriveGroup 2
CartridgeGroup 1
CartridgeGroup 2
Library
App 1 App 2
100
10
10
100
100
100
100
100
Preferential Usage Policy:• DriveGroup-Application relations have priority attribute
• Drives have priority attribute too• Priority is used by eRMM to find drive for a mount operation
Access Roles:• Applications are subject to access control• Two – level access permission model• Administrative applications may access every Drive• Standard applications may only access Drives within DriveGroups which have been assigned to them by an administrative application
16 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Features
Dynamic library and drive sharing across heterogeneous application boundaries and heterogeneous operating systemsDynamic drive and cartridge poolingMount request queuingPolicy-based drive and cartridge allocationPolicy-based media life cycle management (*)Integrated off-site media management and tracking (Vaulting) (*)Centralized access control, administration and reportingAdvanced reporting and auditing (*)
(*) Planned for future releases
17 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
SAN
App 3App 2App 1
SAN
App 3App 2App 1
Media Management and Sharing
Eliminate changes when storage is added and minimize impact of technology upgrades.
Adding storage or upgrading storage technology requires changes to every application
Provide integrated support for manually operated libraries and vaults together with policy-based cartridge life cycle management (planned for future release).
No integrated support for Vaulting
Provide centralized access control, administration and reporting accessible through open standards based API (IEEE1244) and SSG Console WUI
No centralized access control, administration and reporting
Provide centralized mainframe-class media management accessible through open standards based API (IEEE1244)
Media management has to be done by every application
Provide dynamic sharing of tape drives including mount request queuing + additional utilization enhancements through dynamic grouping and pooling of drives
Inefficient drive utilization because its difficult to share drives (physical mapping App->Lib. Partition)
Provide access to and sharing of a common scratch pool and dynamically resizable cartridge groups with access control.
Inefficient capacity usage because tape storage is statically partitioned by application
ERMM SolutionTop Customer Problems/Requirements
18 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion
19 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Architecture
IBM 3584 IBM 3494 STK ACSLS
Media Manager
LibraryManager
SAN
eRMM
Admin Console
DM DM DM DM
20 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Architecture
IBM 3584 IBM 3584 StorageTek
Media Manager
LibraryManager
SAN
eRMM
Admin Console
•Coordinates accessto drives and cartridges
• Central repository• Log of all activities
• Controls libraries onbehalf of the mediamanager
• Encapsulateslibrary hardware
• Command lineinterface and GUI
DM DM DM DM
• Controls driveson behalf of themedia manager
21 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Control Flow
IBM 3584 IBM 3584 StorageTek
Media Manager
LibraryManager
SAN
eRMM
Admin Console
(1.) TSM Server wants to mount a scratch tape
(2.) TSM ELM sends request to Media Manager via TCP
DM DM DM DM
22 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Control Flow
IBM 3584 IBM 3584 StorageTek
Media Manager
LibraryManager
SAN
eRMM
Admin Console
(3.) Media Manager selects scratch cartridge and idle drive
(4.) Media Manager forwards request via TCP
(6.) Drive Manager loads cartridge into drive
(5.) Library Manager moves cartridge to drive
DM DM DM DM
23 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Control Flow
IBM 3584 IBM 3584 StorageTek
Media Manager
LibraryManager
SAN
eRMM
Admin Console
(8.) Response to TSM Server via TCP and ELM
(7.) Response to Media Manager via TCP
DM DM DM DM
24 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Control Flow
IBM 3584 IBM 3584 StorageTek
Media Manager
LibraryManager
SAN
eRMM
Admin Console
(9.) Direct access to drive
DM DM DM DM
25 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSM eRMM RoadmapConclusion
26 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Offline Demo (Screenshots)
I. SetupII. Drive and cartridge sharing
(between EMC/Legato Networker and TSM)III. ReportsIV. Advanced Reporting
27 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (1) – managed libraries
28 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (2) – managed drives
29 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (3) – managed cartridges
30 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (4) – authorized applications
31 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (5) – drive groups
32 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (6) – drive groups (authorized applications)
33 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (7) – cartridge group
34 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Setup (8) – cartridge group (authorized application)
35 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Sharing (1) – common scratch pool
36 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Sharing (2) – Networker begins using one drive …
1. Allocate scratch volume
3. Start backup
2. Label volume
37 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Sharing (2) – TSM wants two drives …
4. TSM is waiting for second drive
3. MOVE DATA needs two drives
38 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Sharing (3) – eRMM synchronizes drive access …
5. TSM is blocked
39 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Sharing (4) – Networker is done and TSM continues …
6. Networker is ready and unmounts cartridge
7. TSM gets second drive
40 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Reports (1) – drive “LTO263” was shared
Drive Sharingbetween TSMand Networker
41 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Reports (2) – cartridge groups have changed
42 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Reports (3) – drive usage statistics
43 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting
The following screen shotsshow prototyped features,
which may become part of eRMMin a future release.
44 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (1a) – Audit trailsStep 1: Select filter in GUI
Which Cartridge was mounted in
“DRIVE1” in January 2004?
45 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (1b) – Audit trailsStep 2: Download report as CSV file
46 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (1c) – Audit trailsStep 3: Import CSV file into Excel
47 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (2a) – Error logsCentral error log file for all drive errors
48 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (2b) – Error logsDetails for a specific error
49 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (2c) – Error logsError statistics by day
50 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (3a) – Drive UtilizationTotal number of drives in use
51 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (3b) – Drive UtilizationUtilization of a single drive
52 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Advanced Reporting (4) – PerformanceTotal throughput by day
53 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSM eRMM RoadmapConclusion
54 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (1/6)
TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications
Why do I need eRMM in a homogeneous TSM environment?
55 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (2/6)
Flexible provisioning of drives and cartridges– sharing of TSM drives with other applications like EMC Legato NetWorker
and OS tools like mksysb and tar– simplified setup of TSM LAN-free environments
no need to define tape paths inside TSMall tape paths are automatically generated inside eRMMcentral check whether drive is used by TSM Server or Storage Agent
– flexible adding and removing of drives to TSM storage poolsall changes are handled inside eRMMupdates of the TSM servers or other applications are not required
– enabling of cross library scratch pools– enabling of cross library drive pools– automatic labeling of scratch cartridges (can be disabled in config file)
56 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (3/6)
Advanced reporting and auditing– eRMM provides statistics on how many drives were used when by
which application (future).– eRMM provides statistics on how many data was written during a
mount. This data can be aggregated to provide throughput reportsfor drives, drives groups, applications and a whole library.
– eRMM provides statistics on how often a cartridge was mounted and how many data was written/read for the whole cartridge life cycle (future). TSM looses its data, when a cartridge is returned to the TSM scratch pool.
– The TSM server stores tape alerts in the TSM activity log. In addition to that, eRMM stores all log sense pages in the eRMM database. This additional data can be very useful for error analysis.
57 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (4/6)
Enhanced access control to library resources– In an outsourcing environment eRMM can share the drives across several
TSM servers whilst it strictly separates the cartridges of each TSM server.– eRMM can also provide a separate scratch pool for each TSM server thus
cartridges are not exchanged between the TSM servers.
Additional operational benefits– dismount of cartridge when TSM server goes down– physical erase of cartridges before return to scratch pool (future)– improved cartridge life-cycle management
(e.g. threshold for wornout tapes) (future)
58 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (5/6)Customer EnvironmentThe customer backs-up a large application server using TDP for Oracle. This backup job starts four sessions and thus uses four drives.
Customer IssueWhen TSM starts a backup of the TSM database and all drives are in use, then TSM dismounts one volume and uses it for the TSM database backup. As a result the backup of the large application server fails, even when it was running for several hours and even when this backup job was nearly complete.
eRMM SolutionConfigure TSM with two external libraries which use eRMM managedresources. Use one library for the backup of the application server and the other for the backup of the TSM database. Inside eRMM map these two libraries to the same resources (DriveGroup, CartridgeGroup).
As a result eRMM will queue the backup of the TSM database.Now the TSM administrator can decide which job to cancel.
59 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Value add in a homogeneous TSM environment (6/6)
TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications
eRMM’s advanced tape management capabilitiesprovides value add even in a homogeneous TSM environment!
60 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion
61 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Supported platforms (eRMM 1.2)eRMM Media Manager
– SLES 8 for zSeries GA– AIX 5.2/5.3 GA– SLES 8 for Intel RPQ
Applications– Tivoli Storage Manager GA– Native OS commands (e.g. tar, mksysb) GA– EMC Legato Networker RPQ– Veritas NetBackup future– iSeries BRMS future– zSeries DFSMSrmm future– Other backup applications future
Tape Libraries– IBM 3584 Open Systems GA– IBM 3494 Open Systems GA– StorageTek ACSLS GA– StorageTek HSC-LS RPQ– Other tape libraries future
62 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Implemented features (eRMM 1.2)
Dynamic/heterogeneous drive sharing GADynamic/heterogeneous drive and cartridge pooling GAMount request queuing GATape library virtualization GAPolicy-based drive and cartridge allocation GACentralized access control, administration and reporting GAPolicy-based cartridge lifecycle management futureOffsite-media management and tracking (vaulting) futureAdvanced reporting and auditing future
63 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Agenda
eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSM eRMM RoadmapConclusion
64 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
Summary
eRMM is an enterprise tape library virtualization solution whichscales very well and provides very high availability and reliability.eRMM provides mainframe class, centralized management functions like tape cartridge pooling, tape drive pooling, tape drive sharing, mount request queuing, policy based cartridge lifecycle management, offsite media management, centralized access controland auditing, and advanced reporting functions.eRMM’s intelligent sharing capabilities increase tape drive, tape cartridge and therefore tape library utilization whilst it reduces the time spent for tape library administration.
The goal of eRMM is to dramatically reduce the Total Cost of Ownership for data stored on tape by improving its access and its management across heterogeneous application, server, and tape library boundaries.
65 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation
IBM TotalStorage
eRMM Resources
Internethttp://www.ibm.com/Search/?q=eRMM&v=14&lang=en&cc=us
eRMM Business DevelopmentDetlef KrauseMail: [email protected]: +49 171 3 33 87 41