Monthly Stroage Capacity Report_JAN2011_final

Compute & Storage Team

Monthly Storage Capacity

Report

120

FREE

USED

RAID

ARRAY

Prepared by:

Luca Viscomi

Date Prepared: 01/02/2011

Monthly Storage Capacity Report

Compute & Storage Reports of 17

GSTT-Technical Monthly Stroage Capacity Report_JAN2011_final.docx

Last changed: 21 April 2012 at 22:09

Document Information

Reporting Group: Storageerver &Storage Team

Reviewers: Storage TeamStakeholders

Document Date: 01/02/2011

Review Method: Team based

Prepared By: Luca Viscomi Preparation Date: 01/02/2011

Checked By: Review Date:

Distribution List

From Date

[email protected]

To Action Due Date

John Aldhouse Reviewer

Simon Creighton Reviewer

Chris Myers Reviewer

Distribution History

Distib. Month Date Revised By Description Filename

JAN 2011 01/02/2011 John Aldhouse First Draft Monthly Storage Capacity Report JAN-11





Table of Contents

1.1. Purpose ............................................................................................................................................. 4

1.2. Scope ................................................................................................................................................. 4

1.3. Out of Scope ...................................................................................................................................... 4

1.4. Intended Audience ............................................................................................................................ 4

1.5. References ......................................................................................................................................... 4

1.6. Definitions and Conventions ............................................................................................................. 4

THE HEADLINES

2.0 SAN STORAGE (GLOBAL VIEW) .......................................................................................................... 6

3.1. SAN Storage ‘NETAPP’ Growth and Projections (PROD-vSphere 4.1) .............................................. 7

3.2. SAN Storage Growth and Projections (PROD-3.5)............................................................................. 9

THE DETAILS

3.1. Storage Design (VMFS) .................................................................................................................... 10

3.2. Storage Design (RDM) ..................................................................................................................... 11

3.3. Storage Design (NETAPP VMFS TOPOLOGY) .................................................................................... 13

3.4. SAN Storage ‘THE RAID PROVISION’ (PROD-3.5)............................................................................. 14

3.5. SAN Storage ‘THE SAN VOLUME PROVISION’ (PROD-3.5) ............................................................... 15

3.6. SAN Storage ‘THE SAN VOLUME PROVISION’ (PROD-3.5) ............................................................... 15

3.7. SAN Storage ‘THE VMFS PROVISION’ (PROD-3.5) ........................................................................... 16

3.8. SAN Storage ‘THE SAN SYSTEM PROVISION’ (PROD-3.5) ................................................................ 17

4.1. SAN Storage ‘NETAPP’ (PROD-vSphere 4.1) ................................................................................... 19





Introduction

1.1. Purpose

The purpose of this document is to describe the present state of the ‘SAN’ storage environment with respect to presentation, usage, capacity and limited forecasting. From a global perspective the storage environment is reported as the ‘PROD-3.5’ incorporating the Falconstor/EMC/SGI/NEXSAN and ‘PROD-4.1’ for the vSphere NETAPP environment. The later half of the document dissects the environments into five separate layers, beginning at the base with the ‘RAID layer’ then the ‘SAN Volume’, ‘SAN LUN’, ‘VMFS Datastore’ and finally the Operating/system layer. Each zone is described and articulated in Terabytes (TB) and a consistent capacity schema will be used to define these parameters. The NETAPP Storage environment is reported in tandem with some slight alteration its design model and definitions.

1.2. Scope

Within the context of SANan Storage, the following items are deemed to be in scope for this report:

Core Virtualised Storage provision (Tier 1 and Tier 2)

1.3. Out of Scope

Within the context of SanAN Storage, the following items are deemed to be inout of scope for this report

Non-virtualised Storage provision

Utilisation Statistics

1.4. Intended Audience

This document is intended for those interested in Storage Capacity/planning and is of interest but not limited to:

GSTT Compute and Storage Team

GSTT Design Team

Relevant project managers

1.5. References

A list of the documents referred to in this document:

Ref no. ID Title Author Date

1.6. Definitions and Conventions

A list of terms, acronyms and abbreviations contained within this document.

Item Definition

VMDK VMware Virtual machine Disk

VMFS Vmware Virtual machine File System





Item Definition

RDM Raw Device mapping

SAN Storage Area Network

LUN Logical unit Number

VM Virtual Machine

Volume Logical Disk Container (Virtualised layer)

PROD-3.5 The production Vmware 3.5 (VI3) Environment, including Cluster A,B,D and E)

PRDD-4.1 The production Vmware 4.1 (vSphere) Environment, including Cluster F, Quarantine and Test)

Commissioned Physical or logical storage apparatus in operation

Presented Physical or logical storage element presented to an upper layer of the storage model

Disk used Physical or logical storage container anchored at that layer and in operation.

Disk Free Physical or logical Storage container available for anchoring at that layer on the storage model

Committed NETAPP ‘Thin Provisioning’ term referring to space allocated to the OS/System partition without physical reservation

Reserved NETAPP ‘Thin Provisioning’ term referring to physical blocks reserved to a Fixed LUN.

SAN Design Definition

‘SAN’ Tiered Storage Structure

By definition the genealogy of Storage Tiering has no set technological definition. We at GSTT have defined our own storage Tier’s and this can best be described in the following statements.

Type Definition

Tier 1 SCSI, Fibre Channel disk, operating at 2-4 Gbit/s

Tier 2 SATA, (1 or 2), operating at 1.5-3.0 Gbits/s

Tier 3 Backup tape, LTO-3 running at 800 Mbit/s (0.8 Gbit/s). Also LTO-4 1.7Gbit/s (approximate figures)





2. SAN STORAGE (GLOBAL VIEW)

0

20

40

60

80

100

120

140

160 153

68

14

85

16

70

TOTAL STORAGE COMMISSIONED (TB)

TIER 1 STORAGE COMMISSIONED (TB)

TOTAL RAIDED STORAGE FREE TIER 1(Below the OS/System Layer)(TB)


TOTAL RAIDED STORAGE FREE TIER 2(Below the OS/System Layer)(TB)

THIN PROVISIONED ESTIMATE ~(TOTAL 'FREE SPACE')

0

10

20

30

40

50

60

70

8078

18

10

61

42

55

TOTAL COMMISIONED (TB)

TOTAL 'TIER 1'COMMISSIONED (TB)

TOTAL 'TIER 1' FREE (TB)

TOTAL 'TIER 2'COMMISSIONED (TB)


TOTAL COMMITTED (TB)





3.1. SAN Storage ‘NETAPP’ Growth and Projections (PROD-vSphere 4.1)

Tier 1/2 Aggregates (1 Year Trend) Tier 1/2 Aggregates (3 Monthly Trend)

Tier 1/2 Aggr Used vs Committed( 3 Monthly Trend) Tier 1/2 Aggr Used vs Total (3 Monthly Trend)

------------------------------------------------------------------------------ Tier 1 Aggr used vs Total (3 Monthly Trend) Tier 2 Aggr used vs Total (3 Monthly Trend)





3.2. SAN Storage Growth and Projections (PROD-3.5)

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

29/0

8/2

007

29/1

1/2

007

29/0

2/2

008

31/0

5/2

008

31/0

8/2

008

30/1

1/2

008

28/0

2/2

009

31/0

5/2

009

31/0

8/2

009

30/1

1/2

009

28/0

2/2

010

31/0

5/2

010

31/0

8/2

010

30/1

1/2

010

28/0

2/2

011

31/0

5/2

011

PROVISIONED TIER 1STORAGE

PRESENTED TIER 1CAPACITY

Tier 1 Forcast

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

29/0

8/2

007

29/1

0/2

007

29/1

2/2

007

29/0

2/2

008

30/0

4/2

008

30/0

6/2

008

31/0

8/2

008

31/1

0/2

008

31/1

2/2

008

28/0

2/2

009

30/0

4/2

009

30/0

6/2

009

31/0

8/2

009

31/1

0/2

009

31/1

2/2

009

28/0

2/2

010

30/0

4/2

010

30/0

6/2

010

31/0

8/2

010

31/1

0/2

010

31/1

2/2

010

28/0

2/2

011

30/0

4/2

011

Size (GB)

PROVISIONED TIER 2STORAGE

PRESENTED TIER 2CAPACITY

Tier 2 Forcast





3.1. Storage Design (VMFS)

Concept diagrams to more accurately explain the storage layer and the manifestation of ‘Free Space’

Storage Design (VMFS) SAN layered Topology for Virtual Machines using a VMFS files system

Structural Analysis of Virtualised SAN Storage ~ VMFS Topology

OS~

VMDK Disk

Layer

RAID Disk

Layer

SAN Volume

Layer

LUN

Presentation

Layer

VMWARE

~VMFS

Layer

‘ Free Space’ exists in all layers of the system!

The example layers, names, Concepts are exclusive to this Document(excluding VMDK/VMFS). All rights reserved to the Author L.S.Viscomi

SAN Storage DesignInfrastucter Schematics

Multiple System

partitions in a Flat

VMDK Disk

Multiple VMDK Disks

in a VMFS Volume

VMFS VOLUME/S in

a SAN LUN

SAN LUN’S in a SAN

VOLUME

HP

ProLiant

DL320s

UID

1

2

SAN

= Used Space

120

FREE

USED

RAID

ARRAY

DatastoreDatastore

FreeVMDK-

DISK

VMDK-

DISKVMDK-

DISK

VMDK-

DISKVMDK-

DISK

VMDK-

DISK

= Free SpaceVMFS VOLUME

PARTITION PARTITION Free

VMDK DISK

VMVM

PARTI

TION

Free

SAN VOLUME

Free

Free Free

SAN LUN

Comments: Like the classical ‘Russian dolle’ design, each layer of this model sits inside the layer below. These logical containers each have their own capacity rationale in term of used and available space which provides the ability to provision a logical storage container to its upper layer.

Go to Section

3.5

Go to Section

3.6

Go to Section

3.4

Go to Section

3.7

Go to Section

3.8





3.2. Storage Design (RDM)

Storage Design (RDM) SAN layered Topology for Virtual Machines using RDM’s on the SAN

Structural Analysis of Virtualised SAN Storage ~ RDM Topology

RAID Disk

Layer

SAN Volume

Layer

LUN

Presentation

Layer

OS~

Vmware/

VMDK Disk

Layer

‘ Free Space’ exists in all layers of the system!



Multiple System


VMDK Disk

VMDK Disk in a SAN

LUN

SAN LUN’S in a SAN

VOLUME

HP

ProLiant

DL320s

UID

1

2

120

FREE

USED

= Used Space

= Free Space

SAN VOLUME

VMVM

SAN

Free

RAID

ARRAY

PARTI

TION

FreePARTITION PARTITION

VMDK DISK

FreePARTITION PARTITION Free FreePARTITION PARTITION Free

FreePARTITION PARTITION Free

SAN LUN

Comments: While almost identical to the ‘VMFS’ design model, the RDM topology does not have a VMFS (Virtualised) Tier. Therefore the Operating system partitions are presented and anchored directlyed from the SAN LUN via a VMDK disk.

Go to Section

3.4

Go to Section

3.5

Go to Section

3.6

Go to Section

3.8





3.3. Storage Design (NETAPP VMFS TOPOLOGY)

Storage Design (NETAPP VMFS TOPOLOGY) SAN lLayered Topology for Virtual Machines using VMFS on the NETAPP SAN

Structural Analysis of Virtualised SAN Storage ~ ‘NETAPP’ VMFS Topology

NETAPP’

RAID-DP Disk

Aggregate

Layer

VMWARE

~VMFS Layer

‘NETAPP‘

FLEX Volume

Layer

Free

‘NETAPP’ LUN

Presentation

Layer

OS~

VMDK Disk

Layer

‘ Free Space’ will only exist at the aggregate layer when ‘Thin Provisioning’

is enabled



Multiple System


VMDK Disk

Multiple VMDK Disks

in a VMFS Volume

VMFS VOLUME/S in

a SAN LUN

SAN LUN’S in a

’FLEX’ VOLUME

FreeVMDK-

DISK

VMDK-

DISKVMDK-

DISK

VMDK-

DISKVMDK-

DISK

VMDK-

DISK

Free

PARTI

TION

Free

NETAPP-

FAS2040

PARTITION PARTITION Free

VMDK DISK

120

FREE

USED

RAID-DP

AGGEGATE

NETAPP LUN

= Free SpaceVMFS VOLUME

VMVM

DatastoreDatastore

= Used Space

Free

‘FLEX’ VOLUME

Comments: While almost identical to the Standard ‘VMFS’ design model, the NETAPP topology uses Thin pProvisioning at the root of the system. In this design, blocks on the RAID-DP aggregate are only committed when the system/OS actually commit writes to the file system, other-wise the blocks remain unallocated. When ‘Thin Provisioning’ is enabled at the ‘FLEX Vols’ layer, all the perceived free space from that layer upwards is virtualised and therefore fabricated. This system allows for radical over allocation in all layers above the RAID-DP aggregate. The ‘FLEX Vols’ layer also support Deduplication enhancing block assignment and space utilisation.

Go to Section

4.1





3.4. SAN Storage ‘THE RAID PROVISION’ (PROD-3.5)

Comments/Analysis

TOTAL STORAGE COMMISSIONED (TB) This represents the total gross aggregate of RAID disk commissioned in PROD 3.5 for both Tier

1 and Tier2.

TIER 1 STORAGE COMMISSIONED (TB) This represents the total gross aggregate of RAID disk commissioned in PROD 3.5 for Tier 1.

TIER 2 STORAGE COMMISIONED (TB) This represents the total gross aggregate of RAID disk commissioned in PROD 3.5 for Tier 2

TOTAL RAIDED STORAGE FREE (TB) This represents the total gross aggregate of RAID disk commissioned in PROD 3.5 that is

available for commissioning.

0

20

40

60

80

100

120

140

160153

6885

0

TOTAL STORAGE COMMISSIONED (TB)



TOTAL RAIDED STORAGE FREE (TB)





3.5. SAN Storage ‘THE SAN VOLUME PROVISION’ (PROD-3.5)

Comments/Analysis

TOTAL STORAGE COMMISSIONED (TB) This represents the total gross aggregate of RAID disk commissioned in PROD 3.5 for both Tier

1 and Tier2.

TOTAL STORAGE PRESENTED TO THE VIRTUAL ESTATE (RAW Tier1/2) (TB) This represents the total gross aggregate of used space at the ‘SAN volume’ layer presented

to PROD 3.5.

TOTAL STORAGE AVAILABLE FOR PRESENTATION (RAW Tier1/2) (TB) This represents the total gross aggregate of unused/Free space at the ‘SAN volume’ layer

presented to PROD 3.5.

3.6. SAN Storage ‘THE SAN VOLUME PROVISION’ (PROD-3.5)

Comments/Analysis

TOTAL SPACE ALLOCATED TO THE VIRTUAL ESTATE (Tier1/2) (TB) Total aggregate of LUN space presented to the PROD3.5 estate

TOTAL SPACE ALLOCATED TO ‘NON-VIRTUAL CLIENTS’ (Tier1/2)(TB) Total aggregate of ‘LUN’ space presented to non PROD3.5 SAN clients

0

50

100

150

200

153

125

18TOTAL STORAGE COMMISSIONED (TB)

TOTAL STORAGE PRESENTED TO THE VIRTUAL ESTATE (RAW Tier1/2) (TB)

TOTAL STORAGE AVAILABLE FOR PRESENTATION (RAW Tier1/2) (TB)

0

50

100

150101

23

0

TOTAL SPACE ALLOCATED TO THE VIRTUAL ESTATE (Tier 1/2) (TB)

TOTAL SPACE ALLOCATED TO 'NON-VIRTUAL CLIENTS' (Tier 1/2) (TB)

TOTAL FREE SPACE (Tier 1/2) (TB)





3.7. SAN Storage ‘THE VMFS PROVISION’ (PROD-3.5)

Comments/Analysis

TOTAL VMFS DATASTORE CAPACITY ALLOCATED (TB) Total aggregate of VMFS (Datastore) space presented to the PROD3.5 estate

TOTAL VMFS DATASTORE CAPACITY USED (TB) Total aggregate of VMFS (Datastore) space used in the PROD3.5 estate

TOTAL VMFS DATASTORE CAPACITY FREE (TB) (Real-time fluctuation) Total aggregate of VMFS (Datastore) space available in the PROD3.5 estate

Comments/Analysis

TOTAL VMFS (Tier 1/2) DATASTORE CAPACITY ALLOCATED/USED/FREE(TB) Capacity Allocated ~ Presented and CutUT asAS VMFSvmfs Datastores Capacity Used ~ VMFS volumes consumed by VMDK disks and all other File Types Capacity Free ~ Capacity within the VMFS Datastore that is available (Real-time fluctuation)

0

20

40

6057

44

13

Total VMFS Datastore Capacity Allocated (TB)

Total VMFS Datastore Capacity Used (TB)

Total VMFS Datastore Capacity Free (TB)

0

10

20

3029

23

6

28

21

7

Total VMFS (TIER 1)Datastore Capacity Allocated (TB)Total VMFS (TIER 1)Datastore Capacity Used (TB)Total VMFS (TIER 1)Datastore Capacity Free(TB)Total VMFS (TIER 2)Datastore Capacity Allocated (TB)Total VMFS (TIER 2)Datastore Capacity Used (TB)Total VMFS (TIER 2)Datastore Capacity Free(TB)





3.8. SAN Storage ‘THE SAN SYSTEM PROVISION’ (PROD-3.5)

Comments/Analysis

TOTAL STORAGE ALLOCATED IN THE VIRTUAL ESTATE (TB) Capacity Allocated to the PROD3.5 environment for both RDM and VMSF formats

TOTAL USED SPACE AT THE SYSTEM LEVEL (TB) This describes what space is actually used in the OS partition/s

TOTAL FREE SPACE AT THE SYSTEM LEVEL (TB) The total amount of Free space at the OS level inside the partition/s

0

20

40

60

80

100 89

50

40

TOTAL STORAGE ALLOCATED IN THE VIRTUAL ESTATE (TB)

TOTAL USED SPACE AT SYSTEM LEVEL (TB)

TOTAL FREE SPACE AT SYSTEM LEVEL (TB)

56%

44%

TOTAL USED SPACE AT SYSTEM LEVEL (TB)

TOTAL FREE SPACE AT SYSTEM LEVEL (TB)





4.1. SAN Storage ‘NETAPP’ (PROD-vSphere 4.1)

78.3

25.9

52.4 55.3

TOTAL COMMISIONED (TB) TOTAL USED (TB) TOTAL FREE (TB) TOTAL COMMITED (TB)

0

10

20

30

40

50

60

70

18

7 10 14

61

19

4241

TOTAL 'TIER 1' COMMISSIONED (TB)

TOTAL 'TIER 1' USED (TB)


TOTAL 'TIER 1' COMMITTED (TB)

TOTAL 'TIER 2' COMMISSIONED (TB)

TOTAL 'TIER 2' USED (TB)


TOTAL 'TIER 2' COMMITTED (TB)





NETAPP CLUSTER : Disk Commissioned (Available), Disk Used and space committed(allocated/over allocated)

0.0

5.0

10.0

15.0

20.0

25.0

aggrsas aggrBsas aggrsata aggrMEDCON1

aggrsas2 aggrMEDCON2

Total TB 9.1 14.6 11.1 17.5 8.7 17.5

Used TB 6.5 7.3 9.0 2.3 0.9 0.0

Committed TB 13.7 20.1 11.6 9.1 0.9 0.0

FREE (TB) 2.7 7.2 2.1 15.2 7.8 17.5

9.1

14.6

11.1

17.5

8.7

17.5

6.5

7.3

9.0

2.30.9

0.0

13.7

20.1

11.6

9.1

0.90.0

2.7

7.2

2.1

15.2

7.8

17.5

Monthly Stroage Capacity Report_JAN2011_final

Documents

Transcript of Monthly Stroage Capacity Report_JAN2011_final