August 4th

Objective : This project is a TECHNICAL INSIGHT of the Improvement Opportunities in the

BackUp and Restore Area for Century Link .

Accomplishments. New Findings – Help Needed.

Initiatives

1) Developed Health Check Script & implemented health checks of TSM SERVERS as a process every six hours.

2) TSM STORAGEPOOL Script implemented

3) Implemented Journal Based Backup to resolve the issue on server "QTCMHPNTIAP01".

Benefits

1) Pro-active monitoring of TSM Servers Every Six Hours – Has Reduced the Downtime Risk of TSM Servers and improved the Back up Success Rate from 91% to 97% .

2) This script has helped in Evaluating the storage Pool space to accommodate fresh back ups from oracle with out having them to Fail – This has resulted in Improving the Oracle Backup Success Rate.

3) This technic can be implemented on servers which carry more numbers files.This helps in reducing the Backup Window from For Eg 2 Million Files which take 5 working days can be backed up in approximately 2 hours.

New Findings - Streamline of TSM TAPE Storage Utilization New Findings - Streamline of TSM TAPE Storage Utilization ◦ .

NODENAME110330ELXDNP49EL110330ELXDNP50EL110330ELXDNT47EL110330ELXDNT48EL110413ELXDENVMTC057L110413EQTDENVMDT143L110417EQTDENVD188N110417EQTDENVD189N110509ERASCALSS

We have around 573 Expired Nodes to be removed from TSM SERVERS

Removing expired nodes data from TSM SERVERS will release 1000 Tapes approximately .

Can benefit the account from XX $$ Value.

Tape Data associated to Decommissioned servers is not managed (Deleted)– Resulting into More Tapes Consumption.

Note : Not aware of any Hidden Associated Risk to delete the nodes data

New – Findings Scope (Intel) : Configuration Issues resulting in to Data Duplication

NTOMT78CS NTOMT78A NCLUNTOMT78AN NCLUNTOMT78SQL1N No 'S' Drive AIOMB52JQTOMASQLDB4CS QTOMASQLDB4A BCLUQTOMASQLDB4AN BCLUQTOMASQLDB4SQL1N No 'S' Drive AIOMB53JQTOMASQLDB4CS QTOMASQLDB4B BCLUQTOMASQLDB4BN No 'R' Drive BCLUQTOMASQLDB4SQL2N AIOMB53Jqtomasql16acs QTOMASQL16A DCLUQTOMASQL16AN DCLUQTOMASQL16SQL1N DCLUQTOMASQL16SQL2N AIOMB54JQTOMASQL14CS QTOMASQL14A MCLUQTOMASQL14AN MCLUQTOMASQL14SQL1N No 'S' Drive AIOMB55JQTOMASQL11CS QTOMASQL11A DCLUQTOMASQL11AN DCLUQTOMASQL11SQL1N No 'S' Drive AIOMB54JQTDENSQL1CS QTDENSQL1B HQTDENSQL1BN No 'R' Drive HCLUQTDENSQL1SQL2N AIDNB866QTDENSQL2CS QTDENSQL2A HQTDENSQL2AN HCLUQTDENSQL2SQL1N No 'S' Drive AIDNB866QTDENSQL2CS QTDENSQL2B HQTDENSQL2BN No 'R' Drive HCLUQTDENSQL2SQL2N AIDNB866QTDENSQL3CS QTDENSQL3A ACLUQTDENSQL3AN ACLUQTDENSQL3SQL1N No 'S' Drive AIDNB864

Identified few Cluster nodes with TB’s of duplicated Data :Getting Rid of this data will save around 500 Tapes Approximately.

SCHEDULE_NAME COMMAND ACTIVE Issues in this TSM SERVERBACKUP_DEVCONF ba devconf YESBACKUP_STG_ARCHIVE_DISKba stg archive_disk offsite_tapeYESBACKUP_STG_ARCHIVE_TAPEba stg archive_tape offsite_tapeYESBACKUP_STG_DBA_DISK ba stg dba_disk dba_offsiteYESBACKUP_STG_DBA_FILE ba stg dba_file dba_offsiteYESBACKUP_STG_DBA_TAPE ba stg dba_tape_t10k dba_offsiteYESBACKUP_STG_GOLD_DISK ba stg gold_disk gold_offsiteYESBACKUP_STG_GOLD_FILE ba stg gold_file gold_offsiteYESBACKUP_STG_GOLD_TAPE ba stg gold_tape_t10k gold_offsiteYES

AIDNB866-2

Utilization 0%Utilization 0%

No Backup Stg for DBA_TAPENo Backup Stg for DBA_VTL

No Backup Stg for DBA_XIV_VTL

New – Findings : Configuration Issues resulting in to TSM TSM SERVER Storage pools having no offsite copy.SERVER Storage pools having no offsite copy.

Approximately 40% of the Data doesn’t have an offsite Copy.

What all new tools/technologies can be implemented to resolve few issues?

Tivoli Monitoring to be in place. Use Journal Based Backup wherever required. Implementing LANFREE for the servers which has 500GB and above data.

What all changes needed in DESIGN to have better environment. TSM SERVER Configuration needs to be thoroughly checked. TEST/DEV to be separated with different Storagepools. Password Expiration.

Help Needed Approval for removing the expired & Duplicate data . Once Approved, the plan of action will be submitted after working with the US Counter part.

S3 – Save Storage Space [YES CUBE]

Team Responsibilities Weekend Tasks Technical KT

Finding the nodes which has more backup data. [List Collected] Through check on that host’s each object and checking with customer or

AIP whether these files/objects really need for backup.

Ex:

AIOMB08R◦ NSUOMT102S 326.3 GB◦ NSUOMT800S 2.8 TB◦ NECOMT106L 379.6 GB

AIOMB09R◦ BHPOMP267H 372.5 GB

AIOMB54J◦ DSUOMP34QS 187.5 GB◦ DSUOMP34QS 206.2 GB

Taking individual responsibility on the remedy tickets and resolve them with in the SLA.

Ex:

More attention on monitoring the TSM Storage Pool migrations to reduce the production database backup failures count.

◦ Indentified File systems which has no backup from long time and working with Server Owner’s/AIP’s to get confirmation to remove the data from TSM Storage pool which has occupied TB’S of space.[Report will be generated by Friday and Weekend Shift Team is working on this ]

◦ Irrespective of daily missed/failed backups; we have initiated digging in to servers which has no backup in last 1-5 Days. [This is to ensure that we should not have any of missed/failed backups repeated on Monday.]

We have started Knowledge Share within the team members on the topics raised by team◦ Duration : 1 Hour◦ Frequency : 2/Week◦ Topics : Everything related to BACKUP/SAN/AIX

Topics Covered◦ LANFREE BACKUPS◦ NAS BACKUPS