Post on 14-Mar-2022
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Silo Busting
Removing the wall between system and storage administrators
Chuck Laing Senior Technical Staff Member (STSM) IBM GTS SO/IS Delivery
Link: http://www.ibmsystemsmag.com/power/infrastructure/storage/silo_busting/
1 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Objectives
• Considerations that trigger Crit Sits and Performance issues
• Share and Share Alike – Top Ten things to know
• Example 1 - Concepts for solutions and example 4 step migration process
• Example 2 - Physical/Logical Considerations Understand the underlying
disk structure of LUNs
• Understand the virtual nature of SVC, DS8K, V7000 and XIV Storage
• Making it Work – communicating key points
• My object is to teach:
• Examples that foster communication between towers
• Concepts that will make you better
• Concepts that will make your job easier
2 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Problem statement
• Storage and System Administrators often clash in the common goal to
achieve data performance and availability, leading to:
• Too many logical configuration related outages
• Performance related enhancements not working to specification
• Leading causes:
• Lack of understanding configurations
• No cohesiveness between the logical and physical implementations
• Lack of communication between System and Storage Administrators
• Resulting in:
• A lack of data reliability and IO throughput
• Customer dissatisfaction
2
3 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Addressing considerations that trigger
Crit Sits and Performance issues
• The 3 most common issues that cause performance
degradation and application outages are:
• Hardware component failures
• Configuration changes
• IO load shifts - caused by anything ranging from imbalance through data growth to
failure of components
• How do I address and avoid:
• Data Safety
• Outages
• Extended or unexpected downtime
• Data corruption
• Missing data or data loss
• Application performance issues
• Technical compatibility issues
• Risk associated with changes in the environment
4 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
• As System’s Administrators – we don’t always KNOW what we don’t know
about storage
• Ask for storage, leveraging what you know
• Avoid bottlenecks
• Use tools available
• Speed problem isolation
• Make more informed architectural decisions
• As Storage Administrators – we don’t always KNOW how the storage will
be utilized
• Make more informed architectural decisions
• Ask what is needed for best performance and IO separation
• What we are NOT going to do today:
• Try to turn Sys Admins into Storage Admins or vice versa
• Boil the ocean
Share and Share Alike - Knowledge is Power
5 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Systems Administrators should know about Storage
1. What should I be aware of/what should I avoid? (Tips & Pitfalls-Tuning)
2. Storage Overview - what's inside?
• What is the Physical makeup – drive types – RAID - Size?
• What is the Virtual makeup (good throughput design tips - IOPs)?
• What is a Storage Pool ?
• Where do I place data?
3. Connectivity- Picking the right drivers
4. Host Attachment Kits
5. How to Improve Performance using LVM
6. Documentation - why it matters
7. Topology Diagrams
8. Disk Mapping (view at a glance)
9. Easy Storage Inquiry Tools
10. How to avoid Bottlenecks
Storage Admins should know
about Hosts
1. What should I be aware of/what should I avoid? (Tips & Pitfalls-Tuning)
2. Host Operating System (OS) type
3. Hdisk Volume - LUN Layout - Purpose?
• DB type
• Access Patterns
• Number of spindles required
• Stripe
• Spread
• Mirror
4. What is a Volume Group (VG)?
5. What is a Logical Volume (LV)?
6. Disk Mapping (view at a glance)
7. Easy Storage Inquiry Tools
8. Cluster Properties
9. LPAR FC HBA port mappings
10. What causes Bottlenecks
The top ten things to know
5
6 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Plan
Execute
Validate
Evaluate
Example 1
Four Steps to communicate pattern for any process
Solutions Process
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Evaluate
• Data and Storage cross pollination for Power Systems
8 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Example: Evaluate the data migration process
• Migrating data is always a disruptive process. Whatever the migration
technique used, it always affects to some degree the normal
operations of the system.
• Selecting the appropriate technique depends on:
• The criticality of the data being moved
• The resources available
• Other business constraints and requirements.
• Note: Risks should be identified depending on the migration technique
used. We strongly recommend that you consider selecting the
technique that is the best compromise between efficiency and the least
impact to the system users.
9 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Disk consolidation can trigger data migration of storage when:
• You want to change computer systems
• You need to upgrade to new products to
stay competitive
• New functions of evolving technology are
introduced
• Database growth
• You need newer, faster, higher density
devices
• Taking advantage of the ever improving
price to performance ratio of new storage
devices
• You just require more flexibility
• You need to relocate you data center
• You need to reduce the foot print of your
storage subsystem within the data center
• Leveraging data migration to provide
Disaster Recovery solutions
Storage migration can trigger LVM
data movement when:
• You want to spread IO evenly across all the disks in the VG
• You need to align IO access patterns • Random access
• Sequential access
• You want to protect the data integrity
• Database growth
• Database refreshes
• You need to consolidate the space in a VG or multiple VGs
• You need to troubleshoot an ailing volume for • Performance
• Availability (failure boundary)
• You need to separate data into separate LVs
Evaluating migration triggers
10 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Evaluate the migration technique summary for risks
• Make a list of Pros and cons (each offering strengths and limitations)
Migration technique Pros Cons
– Host-based – LVM
– LDM
– Add-on software such
as VxVM
– Volume (block) level
– TDMF
– Generally lowest initial implementation
cost
– Leverages existing and IP network
– LVM or LDM tools available
– Storage device-agnostic
– Leverages existing Operating System
skills
– Migration can happen on-line during peak
hours
– Consumes host resources
– Operating system specific
– Management can become complex and
time consuming
– Each host is its own island – no central
management console
– May cause an initial outage to install the
utility or software if it is not already
existing on the host
– Network-based – Fabric
– TDMF-IP
– Supports heterogeneous environments –
servers and storage
– Single point of management for replication
services
– Higher initial cost due to hardware &
replication software
– Requires proprietary hardware and may
require implementation of Storage
– Application-based – SVC
– Migration can happen on-line during peak
hours
– Supports heterogeneous environments –
servers and storage
– Single point of management for migration
– Requires an initial outage to bring the
host volumes on-line to SVC
– Requires the host to reboot to load or
upgrade the multipathing drivers
– Tape based – TSM
– Etc
– Does not require additional special tools,
software or hardware
– Does not require additional skills or
training
– Requires the disruption of the
applications and down time
– Slow and cumbersome
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Plan
• Data and Storage cross pollination for Power Systems
12 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Planning phase – Data Protection & identify risks
Example migration methodology plan
Action Item Assigned Status Date
– Establish a migration management team
– Gather availability and production schedules
– Document Change Control procedures and
incorporate into the plan
– Document the time line for migration activities
– Announce the migration at least 30 days prior to the
intended target migration date
– Gather information about the storage server
environment and applications (lists, commands,
scripts and/or drawings)
– Schedule a pre-migration rehearsal that includes all
the members on the migration team and a data
sampling that will enable the application groups to
appropriately conduct the pre- and post migration
verification process
– Establish a “Migration Status” call-in process
– Utilize a “Migration Planning Checklist” to assure
that all of the pre migration planning steps have
been executed
13 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Planning phase
• A successful data migration always requires substantial evaluation and
planning
• Adequate planning is the critical success factor in a migration project
• Develop a high level migration plan
• Develop a detailed migration plan
14 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Planning phase – design requirements
Action
Item
Application Environment
Databases to be moved (DB2, Informix®, Oracle®, SQL, Sybase)
Database version
Database size
Availability requirements of databases (any existing SLA’s, downtime issues
to consider)
Cluster environment (MSCS, Veritas, Sun, HACMP™, MC/Service Guard,
etc.)
Understanding the requirements may help simplify migration process
Action
Item
Network Environment (if applicable)
Topology
Speed of network
15 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Planning phase – design requirements
Action
Item
Storage Environment
Storage Vendor and model (EMC, HDS, IBM, STK, Dell, HP)
Channel type (ESCON, FICON, Fibre, iSCSI, SAN)
SAN HBA & Model (Qlogic, Emulex, JNI)
Number of Channel Paths
Logical to Physical mapping (i.e. RAID-1 vs. RAID-5)
Number of Source volumes to be migrated
Volume sizes
Identify Target volumes to receive source data
Understanding the requirements may help simplify migration process
16 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Planning summary – avoiding data corruption
Example migration methodology approach
Action
Item Migration and validation methodology checklist
Based on the information gathered in the planning phase, structure the
migration architecture to match the production requirements
Use checklists to ensure any operating patches and software are at
the correct levels
Build detailed migration procedures following the chosen architecture
Put together a schedule of events with time lines to implement the
migration procedures
Establish an initial test plan to validate the initial installation of all
required components
Develop a cooperative deployment plan
Write and configure any automation scripts that will speed up the
process
Run a simple initial test plan that validates the migration process
Implement the migration procedures and time line built in the design
phase
Verify the migration completion by checking the successful completion
and status of the migration jobs
Utilize a “Migration Planning Checklist” to assure that all of the pre migration planning steps
have been executed.
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Execute
• Data and Storage cross pollination for Power Systems
18 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Execute
• During the migration phase, you will need to:
• Communicate your plans
• Obtain, install and configure any necessary:
• Hardware
• Software
• Automation scripts and tools (to perform the actual data migration)
19 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Establish good communication with towers and teams
• Team members may include but are not limited to:
– Project manager
– Client (account) manager
– DBA/Application owners
– System administrator
– Network administrator
– Security administrator
– Firewall administrator
– Disk storage administrator
– Backup/Recovery administrator
– SAN fabric administrator
– Hardware CE
– Floor planner
– Cable vendor
– Disaster/Recover administrator
– IT Architect
20 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Execute
An example migration may go as follows:
–This high level illustration is the execution migratepv –l
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Validate
• Data and Storage cross pollination for Power Systems
22 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Validate
• It is important to validate that you have the same data and functionality
of the application after the migration
• You should make sure that the application runs with the new LUNs,
that performance is still adequate, that operations and scripts work
with the new system
23 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
A sample validation list may include
but not be limited to the following items:
• Compile migration statistics
• Prepare a report to highlight: • What worked
• What didn’t work
• Lessons learned
• Share the report with all members of the migration team
• These types of reports are critical in building a repeatable and consistent process through
continuous process improvement, building on what worked and fixing or changing what
didn’t work. Further, documenting the migration process can help you train your staff, and
simplify or streamline the next migration you do, reducing both expense and risk
24 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Migration Methodology Summary
Validate Evaluate Plan Execute
– Analyze business
impact
– Risks
– Business
interviews
– Criticality of data
being moved
– Performance
– Migration types
– Key factors
– Multi-vendor
environment
requirements
– Application down
time
– Determine
migration
requirements
– Identify existing
environment
– Define future
environment
– Create migration
plan
– Develop design
requirements
– Migration types
– Create migration
architecture
– Develop test plan
– Obtain software
tools and licenses
– Communicate
deployment plan
– Validate HW & SW
requirements
– Customize
Migration
procedures
– Install & configure
– Run pre-validation
test
– Perform migration
– Verify migration
completion
– Run Post validation
test
– Perform knowledge
transfer
– Communicate
Project information
– Create report on
migration statistics
– Conduct migration
close out meeting
25 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Share and Share Alike
• If the storage expert and the OS expert share information and produce
vital documentation together, the environment is less vulnerable to
performance woes:
• Missing dependencies
• Outages
• Risks introduced through change
• Data corruption
• Performance degradation
• Etc…etc…etc
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Example 2
• Data and Storage cross pollination for Power Systems
27 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
27
Grid Building Block
-Data Module (1-15)
-CPU
-Memory (360GB/720GB)
-12 disk drives (1, 2, 3, 4, 6TB)
-Optional SSD 360/720 cache
External Connect
-Interface/Data Module (4-9)
-24 FC ports – 8Gb
-iSCSI ports – 22 GbE or 12 10GbE (Model 214)
Internal Interconnect
-2 Infiniband Switches
-3 UPSs Gen 3 Spectrum Accelerate
Example 2 - Physical/Logical Considerations
Build with a Strong Foundation
• Model 961 base and model 96E expansion
• 961 with up to 3 x 96E expansion frames
• 2.5” small-form-factor drives; 3.5” nearline; 1U High Performance Flash Enclosures
• 6 Gb/s SAS (SAS-2)
• Maximum of four frames
• Maximum of 1,536 drives plus 240 Flash cards
• Top of rack exit option for power and cabling
• Each frame has its own redundant set of power cords
• Two POWER7+ servers 4.228 GHz processors
• 2, 4, 8, and 16 core processor options per storage controller
• DS8870 has dual active/active controllers
• Up to 1TB of processor memory
• Host adapters
• Up to 64 16 Gb/sec or 128 8 Gb/sec ports or combination of 8/16 Gb/sec ports
• Each port supports FCP and FICON at the port level
• Base frame and first expansion frame allows 16 adapters. For 8 Gb/sec, both 4 and 8 port host adapter cards available. For 16 Gb/sec, 4 port host adapter card available
• All Flash configuration has all host adapters in the base frame
• Efficient front-to-back cooling (cold aisle/hot aisle)
V7000 V9000 • Great for
multiple mixed
workloads that
drive huge I/O
• Scale out for
more all flash
capacity, IOPS
and bandwidth
• Up to 2.5M
IOPS, 200µs
(.2ms)
• Up to 228TB
usable, 1.1PB
Effective
28 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Making it work
• Fully document LPAR to hdisks, databases to LPARs, HBA, paths and so forth.
• Document how virtualization is implemented and underlying physical structures to clarify the solution.
• Map out application-specific requirements or mappings introduced by mirroring or availability solutions.
• List parameters that differ from defaults (and why).
• Pipe historical output (for example network, path, virtualization results, data path queries Fibre channel statistics) to an alternative server to keep a history of changes. This output can be quite useful in diagnostic endeavors after a change.
• Diagram storage pools and how striping and mirroring are implemented. (Striping twice or incorrectly is a very negative performance factor.)
• Create switch and cable diagrams and network diagrams. It’s essential.
• Produce job-flow diagrams to pinpoint data-access patterns.
28
29 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Foundation - Build a SAN Environment
Seems simple enough right?
• Build the Pools, at the Storage Device • Choose your storage type by disk characteristics, speeds and feeds
• Create Volumes from those pools
• Use ET, Compression, technology to render best Performance
• Connect Hosts to the Storage though the SAN Fabric
• Zone for redundancy and resiliency
• Configure settings to Best Practices
• Configure hosts to take advantage of the Storage Foundation
• Configure VIOS, LPARs, VMs, etc.
• Distribute virtual aspects appropriately
• Map the volumes to the hosts
• Create the file systems , LPs, PPs, PVs, LPs, VGs, etc
• Place the applications on the configured hosts
30 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Foundation - Slow Performance or Outage Occurs
Now What?
• You followed the recipe • You took advantage of all the technology, features and functions by:
• Minimizing and automatically migrating volume IO hotspots – using ET in Pools
• Dual connecting all ports from the Storage to the Hosts
• Using good redundant performing storage foundation building blocks
• What happened?
• The cookies came out of the oven with clumps of: • Salt, baking soda and brown sugar in spots.
31 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
What automated storage inquiry tools can
help me understand my setup?
• Storage tools
• Gathers information such as, but not limited to:
• LUN layout
• LUN to Host mapping
• Storage Pool maps
• Fabric connectivity
• SAT - (SATHC) - (SATPD) - (HW/SW Currency) – (ECM) –Cisco – Brocade-SVC-
DS8K,-DS6K-DS5K-DS4k-XIV-NetApp-EMC VMAX-Quantastor-V7000b-V7000u-
• Go to the following Website to download the tool:
• http://bldgsa.ibm.com/projects/s/storage_automation/sat/index.html
• DS8QTOOL –DS8000 type
• Go to the following Website to download the tool:
• http://congsa.ibm.com/~dlutz/public/ds8qtool/index.htm
• SVCQTOOL – Spectrum Virtualize
• Go to the following Website to download the tool:
• http://congsa.ibm.com/~dlutz/public/svcqtool/index.htm
32 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
© Copyright IBM Corporation 2014
DEV#: 81 DEVICE NAME: hdisk81 TYPE: 2145 ALGORITHM: Load Balance
SERIAL: 60050768019002F4A8000000000005C7
======================================================================
Path# Adapter/Path Name State Mode Select Errors
0 fscsi0/path0 FAILED NORMAL 89154 2
1* fscsi0/path1 FAILED NORMAL 63 0
2 fscsi1/path2 OPEN NORMAL 34014 3
3* fscsi1/path3 OPEN NORMAL 77 0
Using the right tools -
Mapping Virtual LUNS to Physical Disks
• On the host server using SDD
LUN
to
Pool
to
Array
• Ask the StoAdmin to find disk/device UID or Raid-group in Storage Pool
• StorAdmin cross-references Storage Pool UID with Controller’s Arrays in Pools
33 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Throughput and Performance
Key Optimization Factors
• Throughput
• Spreading and balancing IO across
hardware resources
• Controllers
• Ports & zoning connections
• PCI’s Cards
• CPUs, RAM
• Disks spindles
• Compression
• Thin Provisioning
• Easy Tier - SSD
• Etc….
• IO Performance Tuning
• Using utilities, functions and features
to tweak (backend, frontend)
• Qdepths
• HBA transfer rates
• FC adapters
• LVM striping vs spreading
• Data Placement
• Random versus sequential
• Spreading versus Isolation
• Application characteristics
Configuring throughput optimally increases potential performance scalability
34 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Troubleshooting - Sys Admin -
How do I improve disk performance on the Host?
• Reduce the number of IOs
– Bigger caches
• Application, file system, disk subsystem
– Use caches more efficiently
– No file system logging
– No access time updates
• Improve average IO service times
– Better data layout
– Reduce locking for IOs
– Buffer/queue tuning
– Use SSDs or RAM disk
– Faster disks/interfaces, more disks
– Short stroke the disks and use the outer edge
– Smooth the IOs out over time
• Reduce the overhead to handle IOs
35 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Troubleshooting- StorAdmin –
How do I improve disk performance?
• Data layout affects IO performance more than any tunable IO parameter
• If a bottleneck is discovered, then some of the things you need to do are:
– Identify the hardware resources the heavy hitting volumes are on
• Identify which D/A pair the rank resides on
• Identify which I/O enclosure the D/A pair resides on
• Identify which host adapters the heavy hitting volumes are using
• Identify which host server the problem volumes reside on
• Identify empty non used volumes on other ranks – storage pools
– Move data off the saturated I/O enclosures to empty volumes residing on less used
ranks/storage pools
– Move data off the heavy hitting volumes to empty volumes residing on less used hardware
resources and perhaps to the another Storage Device
– Balance LUN mapping across
• Backend and host HBAs
• SVC IOgrps
• SVC preferred nodes
– Change Raid type.
36 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Summary
• Knowing - what's inside will help you make informed decisions?
• You should make a list of the things you don’t know
– Talk to the Storage Administrator or those who do know
• A better Admin understands • The backend physical makeup
• The backend virtual makeup
• What's in a Storage Pool for better data placement
• Avoids the Pitfalls associated with IO Tuning
• Knows where to go to get right device drivers
• Knows why documentation matters
• Keeps Topology Diagrams
• Keeps Disk Mapping documentation
• Is able to use Storage Inquiry Tools to find answers
Understands how to troubleshoot storage performance bottlenecks Silo Busting article in IBM System Magazine ->
http://www.ibmsystemsmag.com/power/infrastructure/storage/silo_busting/
37 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Questions and Answers
© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Definitions
• Data and Storage cross pollination for Power Systems
39 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
IOPS
• IOPS (Input/Output Operations Per Second) is a common performance
measurement used to benchmark computer storage devices like hard disk
drives (HDD), solid state drives (SSD), and storage area networks (SAN).
• IOPS is the standard unit of measurement for the maximum number of
reads and writes to non-contiguous storage locations. The RAID type
defines the number of IOPS
40 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Throughput
• Throughput is a measurement of the average number of megabytes transferred within a period of time for a specific file size. Back in the day this was performed using a single computer making a single request for a disk, but in today’s age with large storage arrays that are providing storage to a number of clients we need to measure based on a lot of small read/writes verses a single computer making a large request.
• IO size (random) x IOPS = Throughput
• IOPS = Throughput/IO (all value in kb) or IOPS=[MB/KB]*1024 or IOPS = (MBps Throughput / KB per IO) * 1024 [since 1mb=1024kb] So using the above, if I wanted to configure an IOPS limit to satisfy a 10 MBps throughput using a 8KB IO request size I would require to set the IOPS limit to 1280. First let us convert 10MBps to kbps 10*1024=10240 IOPS = (10240/8) = 1280
41 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
What is Latency?
• For storage subsystems, latency refers to how long it takes for a single
data request IO ( input/output) to be received and the right data found and
accessed from the storage media.
• In a disk drive, read latency is the time required for the controller to find
the proper data blocks and place the heads over those blocks (including
the time needed to spin the disk platters) to begin the transfer process.
• The reality is that three factors intertwine to impact overall storage
performance: IOPS, latency, and bandwidth.
• The effect of low latency storage I/O is more data flows through the same
hardware infrastructure in the same amount of time as legacy storage
systems
42 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
The Storage Performance Ecosystem
• Storage systems have four basic components that create an ecosystem.
• First is the media on which users store and access data.
• The second component is the storage software. It controls how data is
written to the media as well as providing advanced features like data
protection, snapshots, and replication. This software component should
also dispatch and schedule I/O traffic to back-end media.
• The third component is the CPU processing that drives the storage
software.
• Finally, there is the storage network. It transfers data back and forth to the
application tier.
43 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
LUNs, Volumes and SAN
• LUNs are logical drives
• Storage subsystems have their physical disk partitioned into logically addressed
portions that allow host servers to access them. This partition is called a LUN. For
example, most PC users will be familiar with the partition of a single disk into a C: drive
• Volumes are groups of LUNs
• LUN and volume are frequently used interchangeably. But it is worth noting that volume
is also often used to describe groups of several LUNs created with volume manager
software.
• SAN zoning and masking maintains security on the fabric
• Provisioning LUNs and volumes is only one part of storage provisioning. The SAN
fabric must also be configured so that drive arrays and LUNs are managed, and
security on the SAN is achieved by ensuring only those servers that have authorization
can access specific LUNs. For this we use SAN zoning and masking.
• For more details on zoning best practices please go the following link ;
http://www-01.ibm.com/support/docview.wss?uid=tss1wp102488&aid=1
44 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
1. StorAdmin -The stressing of the data layout for Oracle, SQL Databases and other application data especially for
customers using SVC based Storage with XIV, since Storage Administrators don't really have the ability to set where
the data is located, so the DB Admins need to organize the data and spread the data better...
2. StorAdmin - I think somehow we need to get equal cooperation from both sides when dealing with issues, as I said its
usually on us as a Storage group to prove that issue isn't with the SAN or Storage..
3. StorAdmin - Dynamic disks(Intel OS) can be created using one or more LUNS and in case of performance issues we
cannot move the data of one single LUN which is a part of the Dynamic Disk. So we always recommend the system
admins to use one single bigger LUN or the LUNs from the same class of disk from the back-end to get the similar
performance or else we get into the performance issues. -Bottlenecks
4. StorAdmin – Server Activation & Deactivation (SA&D) --This is not being followed by the system admins in several
accounts. When the server is getting deactivated, they should request the SAN team to un-map and release the
storage from the server during the deactivation process. Otherwise we run into SAN slow drain issues ( TSM servers
on the SAN and not being in use would cause this ). This would eliminate the Orphan LUNs as well from our side.
5. StorAdmin - With the deployment of ASP - Automated Storage Provisioning the System Administrators should have a
crash course for basic storage for their respective accounts.
6. StorAdmin - a simple deck - called it Storage fundamentals maybe 8 slides should be presented to the System Admins
Sample Content :
1. What's IOPS?
2. What's LUN?
3. When do we use bigger LUN size versus smaller?
4. What the sequence to follow to remove a LUN?
5. What are the different drive type ( speed) and when to use them?
6. What is the proper zoning best practice?
7. What makes a SAN storage environment ( just stick to the basics)?
What are Storage Administrators saying?
Feedback examples
45 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
1. GTS TI&A Architect -Slide 28 - With automated provisioning, the tool will be taking over a lot of what is currently being
done manually and documented (or not) by a storage admin. Would like your input on what the tool ought to
automatically create in terms of documentation. Also, where should the automatically created documentation be sent
and/or stored?
2. GTS TI&A Architect - Slide 44 item 5 - I agree it would be good to have a "crash course" in storage for server admins in
accounts where automated storage provisioning is being deployed. Do you have an outline or deck we could start
from?
What are the TI&A team saying?
Feedback examples
46 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
1. ??
2. ??
What are the Sys-Admins saying?
Feedback examples
47 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Session Evaluations
YOUR OPINION MATTERS!
Submit four or more session
evaluations by 5:30pm Wednesday
to be eligible for drawings!
*Winners will be notified Thursday morning. Prizes must be picked up at
registration desk, during operating hours, by the conclusion of the event.
1 2 3 4
48 IBM Systems Technical Events | ibm.com/training/events
© Copyright IBM Corporation 2016. Technical University/Symposia materials
may not be reproduced in whole or in part without the prior written permission of
IBM.
Continue growing your IBM skills
ibm.com/training
provides a comprehensive
portfolio of skills and career
accelerators that are designed
to meet all your training needs.
If you can’t find the training that is right for you with our
Global Training Providers, we can help.
Contact IBM Training at dpmc@us.ibm.com
Global Skills Initiative