Post on 09-Mar-2018
1 Copyright © 2000,2003, Storage Networking Industry Association
Obligatory rubric
• Copyright © 2000,2003 Storage Networking Industry Association (SNIA).
• SNIA-TC members may use this material freely; all others require permission from the SNIA.
• See the speaker notes for this slide for important usage information.
• See SNIA-SSM-colors.ppt for printing and color/graphics conventions.
Copyright © 2000,2003, Storage Networking Industry Association
SNIA shared-storage modelA work in progress …
An architectural overviewThis revision:• 2001-06-05 last content update• 2003-04-13 last graphics update
3 Copyright © 2000,2003, Storage Networking Industry Association
Contents
•Purpose•The SNIA storage model
Layers, functions, and servicesNetworks and interfaces
•Applying the SNIA storage modelCommon storage architectures
•Conclusions
4 Copyright © 2000,2003, Storage Networking Industry Association
Purpose
•Present a simple model for shared storage architectures
•Use it to describe common examples graphically
•Expose, for each one:What services are provided, whereWhere interoperability is required[future] Pros and cons of the architecture
5 Copyright © 2000,2003, Storage Networking Industry Association
Benefits
•A common “architecture vocabulary”•Reference comparisons between common
solutions•Help to align the industry
Customers can better structure their choicesVendors can better explain their differences
6 Copyright © 2000,2003, Storage Networking Industry Association
What the model is and is not
• It is not:A specification, an architecture, a design, a product, a recommendation, or an installation
• It is:A framework that captures the functional layers and properties of a storage system
7 Copyright © 2000,2003, Storage Networking Industry Association
Classic storage model
Application
Storage domain:“anything goes!”
Network?Appliance?
Data mover?Array?
NAS?
JBOD?
8 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerFile/record layer
Block layerBlock layer
Storage devices (disks, …)Storage devices (disks, …)
Database(dbms)
File system(FS)
The SNIA shared storage modelS
tora
ge
do
ma
in
Ser
vice
sS
ervi
ces
Network
Host
DeviceBlock aggregation
Application
9 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerFile/record layer
The SNIA storage model:File/record layer
Database(dbms)
File system(FS)
10 Copyright © 2000,2003, Storage Networking Industry Association
The SNIA storage model:File/record layer — functions
•Aka “access methods”File system, database
•Primary responsibility: packing many smaller things into a few larger ones
Fine-grain naming, space allocation
•Secondary responsibilitiesCaching for performanceCoherency in distributed systems
11 Copyright © 2000,2003, Storage Networking Industry Association
The SNIA storage model:Block layer
Block layerBlock layer
Storage devices (disks, …)Storage devices (disks, …)
Block aggregationBlock aggregation
12 Copyright © 2000,2003, Storage Networking Industry Association
The SNIA storage model:Block layer — functions
• Storage devices – storing datadisk drives, tape drives, solid-state disks, …
• Block aggregation – address mappingin-SN aggregation, or “virtualization”slicing & concatenation, stripinglocal & remote mirroring, RAID-n
• Examplesvolume managersdisk array LUs
• Secondary responsibilitiescaching
13 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerFile/record layer
Database(dbms)
File system(FS)
Block layerBlock layer
The SNIA storage model:Access path examples
Storage devices (disks, …)Storage devices (disks, …)
Block aggregationBlock aggregation
Note: all 8 possible paths can be used!
Note: all 8 possible paths can be used!
Application
14 Copyright © 2000,2003, Storage Networking Industry Association
Block layer
•Block-mapping functions:what can be done
•Functional decomposition:where it can be done
•Sample architectures
Block layerBlock layer
Storage devices (disks, …)Storage devices (disks, …)
Block aggregation
Block aggregation
15 Copyright © 2000,2003, Storage Networking Industry Association
Block layerWhat can be done
• Space managementmaking a large store from many small onespacking many small stores into one large one
• Stripingfor performance (load balancing, throughput)
• Redundancyfull(local & remote mirroring, RAID-1, -10, …)partial (RAID-3, -4, -5, …)point-in-time copy
16 Copyright © 2000,2003, Storage Networking Industry Association
Block layerWhere it can be done
•Host-sidelogical volume managersdevice drivers, HBAs
•SN-basedHBAs, specialized SN appliances
•Device-basedarray controllers (e.g., RAID)disk controllers (e.g., sparing)
Storage devicesStorage devices
Block aggregationBlock aggregation
Network
Host
Device
17 Copyright © 2000,2003, Storage Networking Industry Association
Block layerHow it is done
•Building blocksinput: vector of blocksoutput: vector of blocks
•Result: building blocks can be stacked
enables the 3 layer model for theblock layerlayers can be nested on one anothercould be extended to more layers
……
…
Block aggregation
Block aggregation
18 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Block layerSample architectures
ApplicationHo
st. w
ith L
VM(+
softw
are R
AID?
)
1. Direct-attach
Disk array
SN
Host
with
LVM
2. SN-attach
File
/reco
rdla
yer
Blo
ckla
yer
3. SN aggregation
Aggregationappliance
Host
,no
LVM
Storage devices
19 Copyright © 2000,2003, Storage Networking Industry Association
File/record layer
•Byte-mapping functions:what can be done
•Functional decomposition:where it can be done
•Sample architectures
File/record layerFile/record layerDatabase
(dbms)File system
(fs)
20 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerWhat can be done
• Database management systemstuples → tablestables → table-spacestable-spaces → volume
• File systemsfiles → volume
• New typeshttp caches: a kind of distributed file system?
21 Copyright © 2000,2003, Storage Networking Industry Association
device
File/record layerWhere it can be done
•Host-sidefile systems and databasesNFS, CIFS, etc. are client-server splits inside the file system
•SN-basedNAS head
•Device-basedNAS functions in array box
NAS
Hos
t with
NFS
/CIF
Scl
ient
Hos
t with
loca
lFS
/dbm
s
22 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
File/record layer
Host
. with
LVM
and
softw
are R
AID
1. Direct-attach
File
/reco
rdla
yer
Blo
ckla
yer
NASserver
4. NAS server
Host
. with
LVM
Disk array
2. SN-attach
Host3. NAS head
NAS head
Host
LAN
Application
SN
23 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerFile/record layerDatabase
(dbms)File system
(fs)
The SNIA storage modelA layered view
IV. ApplicationIII. File/record layer
IIIa. DatabaseIIIb. File system
II. Block aggregationIIa. HostIIb. NetworkIIc. Device
I. Storage devices
Storage devicesStorage devices
Block aggregationBlock aggregation
Application
Network
Host
Device
IV
III
IIcIIbIIa
I
24 Copyright © 2000,2003, Storage Networking Industry Association
Sto
rag
e d
om
ain
File/record layerFile/record layer
Block layerBlock layer
The SNIA storage modelServices subsystem
Application
Storage devices (disks, …)Storage devices (disks, …)
Block aggregationBlock aggregation
Database(dbms)
File system(fs)
Ser
vice
sS
ervi
ces
Dis
cove
ry, m
onito
ring
Dis
cove
ry, m
onito
ring
Res
ourc
e m
gmt,
conf
igur
atio
nR
esou
rce
mgm
t, co
nfig
urat
ion
Sec
urity
, bill
ing
Sec
urity
, bill
ing
Red
unda
ncy
mgm
t (ba
ckup
, …)
Red
unda
ncy
mgm
t (ba
ckup
, …)
Hig
h av
aila
bilit
y (fa
il-ov
er, …
)H
igh
avai
labi
lity
(fail-
over
, …)
Cap
acity
pla
nnin
gC
apac
ity p
lann
ing
25 Copyright © 2000,2003, Storage Networking Industry Association
Services
• Operations off the critical pathnaming, discovery, monitoring, configuration, security, billing,redundancy management (backup, …), high availability management (fail-over, …), capacity planning, …strong ties into system-wide management services
• Vital for successful operationand a major opportunity for SNIA …
… but not discussed further in this presentation
26 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Caching… can be added to almost any layer
Host
. with
LVM
and
softw
are R
AID
File
/reco
rdla
yer
Blo
ckla
yer
Host
NAS head
Host
Host
. with
LVM
Disk array
SN
NAS head
LAN
Ideally, caching only affects performance. But: coherency implications do affect management (protocols, interfaces)
Ideally, caching only affects performance. But: coherency implications do affect management (protocols, interfaces)
Application
Cache
Cache
Cache
Cache
Cacheappliance
Cache
27 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
ClusteringInter-box aggregation
Host
. with
LVM
and
softw
are R
AID
File
/reco
rdla
yer
Blo
ckla
yer
Host
NAS head
Host
Host
. with
LVM
Disk array
SN
NAS head
LAN
Cluster FS
Multi-node LVM
Purposes:• load spreading across peers (scalability)
• alternate paths (high availability, scalability)
Purposes:• load spreading across peers (scalability)
• alternate paths (high availability, scalability)
Application
28 Copyright © 2000,2003, Storage Networking Industry Association
Q: Data versus storage?A: Putting information into containers
user: data (“learning my preferences”)application: container(“user keystroke history”)
application: data (“user keystroke history file”)file system: container (“byte vector”)
file system: data (“a named file”)volume system: container (“blocks in volume”)
volume system: data (“replicated, striped layout”)disk array: container (“blocks in LU”)
29 Copyright © 2000,2003, Storage Networking Industry Association
SharingContent sharing or resource sharing?
•Content sharing (“logical”, “data”)contents accessed and understood by multiple clients• e.g., file system, Oracle Parallel Server dbms
some of the hard issues:• coherency• heterogeneous data formats
•Resource sharing (“container”, “physical”)e.g., disk array where hosts access disjoint LUs
30 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
SharingContent sharing and resource sharing
Host
with
LVM
File
/reco
rdla
yer
Blo
ckla
yer
Host
with
LVM
SN
NAS system
HostHost
LAN
Application
Resource sharing
Array is shared,but LUs are disjoint
Resource sharing
Array is shared,but LUs are disjoint
Data sharing
NAS system is shared – and so are the files
Data sharing
NAS system is shared – and so are the files
31 Copyright © 2000,2003, Storage Networking Industry Association
Storage device
Block layer
File layer
Operating System
Application
Networks and interfacesare pervasive in the model
Network or interface
Network
Bus
API
Full benefits come only fromcommon, openinterfaces
Full benefits come only fromcommon, openinterfaces
32 Copyright © 2000,2003, Storage Networking Industry Association
Network or interface
Network or interface
Network or interface
Network or interface
Networks and interfacesComposition and scaling
Open interfacesallow:1. vertical
composition2. horizontal scaling3. supplier
independence
Open interfacesallow:1. vertical
composition2. horizontal scaling3. supplier
independence
Network
Bus
API
API
Network
Bus
API
33 Copyright © 2000,2003, Storage Networking Industry Association
Networks and interfacesOpen interfaces require …
•Well defined:functions (what they do)interface protocols (data formats)access protocols(system call, RPC, flow control, …)
•That are:publishedsupported by multiple products
=> standards (which is where SNIA comes in)
Network
Bus
API
34 Copyright © 2000,2003, Storage Networking Industry Association
? NAS
Q: “SAN” versus “NAS”?A: a poorly-formed question
• Q: hardware: FibreChannel vs Ethernet vs InfiniBand?
• Q: API: blocks vs files (aka “NAS”)vs objects (OSD)?
• Q: protocol: FCP vs TCP/IP vs … ?• A: (to all the above) it depends …
• Storage network (SN):any (mostly) dedicated network, installed (mostly) for storage trafficwhatever the hardware, API, or protocol
“SAN”
FC E’net
iSCSI
File
s B
lock
s
FC E’net
File
s B
lock
s FS/block combo
Copyright © 2000,2003, Storage Networking Industry Association
Some common storage architectures
Mapping the SNIA model onto some current implementations
36 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Direct-attach block storage
Disk array
Host
,no
LVM
File
/reco
rdla
yer
Blo
ckla
yer
Host
. with
LVM
and
softw
are R
AID
Host
wi
th L
VM
Application • Direct-attach
• Multi-attach boxE.g., SCSI
• Direct-attach
• Multi-attach boxE.g., SCSI
37 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
SN-attached block storage
Disk array
Host
,no
LVM
File
/reco
rdla
yer
Blo
ckla
yer
SN
“SN” = any networkused for storage access.E.g., Fibre Channel,Ethernet, …
“SN” = any networkused for storage access.E.g., Fibre Channel,Ethernet, …
Host
with
LVM
Application
38 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
SN-attached block storagewith metadata server
Disk array
Host
with
LVM
File
/reco
rdla
yer
Blo
ckla
yer
SNBlock-aggregationmetadata server
Host
,no
LVM
Aggregation map(metadata) issupplied by anexternal server
Aggregation map(metadata) issupplied by anexternal server
Application
39 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Block storage aggregationin a storage network appliance
Disk array
SN
File
/reco
rdla
yer
Blo
ckla
yer
Aggregationappliance
Host
with
LVM Ho
st,
no L
VM
Application aka “SNvirtualization”Functions: LVM, caching
aka “SNvirtualization”Functions: LVM, caching
40 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Multi-site block storage
Disk array
SNAggregation
appliance
Host
. with
LVM
Application
Disk array
Aggregationappliance
Host
. with
LVM
Application
SN applianceSN appliance
WAN
WAN
Host-to-hostHost-to-host
Functions: point-in-timecopy, caching, local & remote mirroring, …
Functions: point-in-timecopy, caching, local & remote mirroring, …
WAN
Device-to-deviceDevice-to-device
File
/reco
rdla
yer
Blo
ckla
yer
41 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
File server
NAS system
HostHost
File
/reco
rdla
yer
Blo
ckla
yer
LAN-attached“NAS” systemMay do SN/blockaggregation, etc. insidein the NAS system box
LAN-attached“NAS” systemMay do SN/blockaggregation, etc. insidein the NAS system box
Application
LAN
privateSN?
42 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
File server controller(“NAS head”)
NAS head
Disk array
File
/reco
rdla
yer
Blo
ckla
yer
No storage in thefile server controller box
No storage in thefile server controller boxHostHost
LAN
Application
SN
43 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
NAS/file server metadata manager (“asymmetric”)
Disk array
File
/reco
rdla
yer
Blo
ckla
yer
Hosts get file meta-data from FS/NAS controller, thenaccess the data directly
Hosts get file meta-data from FS/NAS controller, thenaccess the data directly
Host
. with
LVM
Filemetadata
Blockaccesses
FS controller canalso be NAS serverFS controller canalso be NAS server
HostLAN
SN
Application
File system metadata
44 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Object-based Storage Device (OSD), CMU NASD
OSDdevice
File
/reco
rdla
yer
Blo
ckla
yer
Host
Object metadata
Host
Application
LAN
Securitymetadata
Filemetadata
46 Copyright © 2000,2003, Storage Networking Industry Association
File/record layerFile/record layer
Database(dbms)
File system(FS)
The SNIA shared storage modelSt
orag
e do
mai
n
Block layerBlock layer
Storage devices (disks, …)Storage devices (disks, …)
Serv
ices
Serv
ices
Dis
cove
ry, m
onito
ring
Dis
cove
ry, m
onito
ring
Res
ourc
e m
gmt,
conf
igur
atio
nR
esou
rce
mgm
t, co
nfig
urat
ion
Sec
urity
, bill
ing
Sec
urity
, bill
ing
Red
unda
ncy
mgm
t (ba
ckup
, …)
Red
unda
ncy
mgm
t (ba
ckup
, …)
Hig
h av
aila
bilit
y (fa
il-ov
er, …
)H
igh
avai
labi
lity
(fail-
over
, …)
Cap
acity
pla
nnin
gC
apac
ity p
lann
ing
Application
Network
Host
DeviceBlock aggregation
47 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
Block layerSample architectures
Host
. with
LVM
and
softw
are R
AID
1. Direct-attach
SN
Host
. with
LVM
Disk array
2. SN-attach3. SN aggregation
Aggregationappliance
Host
,no
LVM
File
/reco
rdla
yer
Blo
ckla
yer
Application
48 Copyright © 2000,2003, Storage Networking Industry Association
Device block-aggregation
Network block-aggregation
Host block-aggregation
File/record layerSample architectures
Host
. with
LVM
and
softw
are R
AID
1. Direct-attach
File
/reco
rdla
yer
Blo
ckla
yer
NASserver
4. NAS serverHost3. NAS head
NAS head
Host
LANHo
st. w
ith L
VM
Disk array
SN
2. SN-attachApplication
49 Copyright © 2000,2003, Storage Networking Industry Association
Uses for the model
•Vendorsplace products in the space of architecturesclarify product differences
•Customersunderstand vendor offerings better
•The industrybasis for common definitions, communication, understanding, interoperability
50 Copyright © 2000,2003, Storage Networking Industry Association
Conclusions
• The SNIA shared storage model is both simple and useful
to highlight similarities and differencesas a basis for comparisons
• Still a work in progressdata movers, tape drives, …better comparisons …suggestions?
• The SNIA-TC welcomes input:<snia-tc@snia.org>
51 Copyright © 2000,2003, Storage Networking Industry Association
Authors
• John Wilkes, Hewlett-Packard (project lead)• Harald Skardal, NetApps• David Black, EMC• Wayne Rickard (SNIA TC chairperson), Gadzoox• Co-conspirators: the rest of the SNIA Technical
CouncilDave Anderson, Seagate TechnologyJim Carlson, IBMGarth Gibson, CMU/PanasasKevin Phaup, HighGround SystemsDavid Thiel, Compaq