Red Hat Storage Day Boston - Why Software-defined Storage Matters
-
Upload
redhatstorage -
Category
Technology
-
view
296 -
download
3
Transcript of Red Hat Storage Day Boston - Why Software-defined Storage Matters
WHY SOFTWARE-DEFINED STORAGEMATTERSWill McGrathRed HatNovember 2016
2
THE DATA STORAGE SHORTFALL
Data stores are growing exponentially, while IT budgets are not
HDDs are becoming more dense, but $/GB decline is slowing
Software and hardware advances are needed to close the gap
GROWTH OF DATA
IT STORAGE BUDGETS
20202010
3
APPLIANCES AREN’T ENOUGH
Complexity hidden from end users, along with flexibility
Vendor lock-in leads to pricing premium
Price premium over constituent components is difficult to sustain
THE TRADITIONAL APPROACH TO STORAGE
ADMINS
MANAGEMENT INTERFACE
DATA
NFS/SMB/ISCSI ENDPOINTS
STANDARD SERVERS AND MEDIA
PROPRIETARY SOFTWARE
4
PUBLIC CLOUD STORAGE ISN’T ENOUGH
Complexity still hidden from end users, pay-as-you-go pricing
Fastest-growing segment of IT storage budgets
Mostly built with proprietary software
CONVENIENT BUT LIMITEDADMINS
MANAGEMENT INTERFACE
DATA
CLOUD STORAGE APIS
STANDARD SERVERS AND MEDIA
LINUX + PROPRIETARY SOFTWARE
5
THE INDUSTRY IS RETHINKING STORAGE
38% of IT decision makers
report inadequate storage
capabilities as one of their top
three weekly pain points
70% of IT decision makers admit that their organization’s
current storage can’t cope with
emerging workloads
98% of IT decision makers believe a more agile storage solution could benefit their organization
Vanson Bourne Ltd: Storage limitations, frustrations, and coping with future needs, Red Hat Storage research results, June 2016
6
FLEXIBILITY IS CRUCIAL
7
STORAGE IS EVOLVINGOPEN, SOFTWARE-DEFINED STORAGE
Standardized, unified, open platforms
Standard Computersand Disks
Ceph Gluster +++
Control Plane (API, GUI)
Custom GUI
Proprietary Hardware
Proprietary Software
Custom GUI
Proprietary Hardware
Proprietary Software
USER USER USER
ADMIN
ADMIN
ADMIN
Custom GUI
Proprietary Hardware
Proprietary Software
USER ADMIN
Stan
dard
Hard
ware
Open
So
urce
Softw
are
TRADITIONAL STORAGEComplex proprietary silos
WHAT IS SOFTWARE-DEFINED STORAGE?
9
WHAT IS SOFTWARE-DEFINED STORAGE?
SERVER-BASED CENTRALIZED CONTROL OPEN ECOSYSTEM
10
SERVER-BASED STORAGEThe use of software and standard hardware to provide services traditionally provided by single-purpose storage appliances (similar to server virtualization, which uses software to emulate servers), providing increased agility and efficiency
Appliance
USER
Appliance
USER
Appliance
USER
Distributed Cluster of Services
USER USER USER
12
SAN/NAS IS ON THE DECLINE
• Storage capacity shipped inside servers has grown since 2010, and is now the majority
• Capacity shipped in SAN/NAS is declining, reflecting the use of server-based storage technologies by web-scale firms
Changing workloads drive the need for flexible, economical server-based storage.
WW DEPLOYED CAPACITY (TB)100%
80%
60%
40%
20%
0%
2010 2011 2012 2013 2014 2015 2016
EXTERNAL CAPACITY INTERNAL CAPACITY
13
WHAT IS SOFTWARE-DEFINED STORAGE?
CENTRALIZED CONTROLSERVER-BASED OPEN ECOSYSTEM
14
STORAGE ORCHESTRATIONThe ability to provision, grow, shrink, and decommission storage resources on-demand and programmatically, providing increased control and integration of storage into a software-defined data center
Web Console API Command LineA browser interface
designed formanaging distributed
storage
A full API for automation and
integration with outside systems
A robust, scriptable command-line
interface for expert operators
Full life cycle management for distributed, software-defined data services
PROVISION INSTALL CONFIGURE TUNE MONITOR
15
WHAT IS SOFTWARE-DEFINED STORAGE?
OPEN ECOSYSTEMSERVER-BASED CENTRALIZED CONTROL
WHY SOFTWARE-DEFINEDSTORAGE MATTERS
17
WHY THIS MATTERS
PROPRIETARYHARDWARE
Common,off-the-shelf hardware
Lower cost, standardized supply chain
18
INDUSTRY STANDARD HARDWAREStandardization makes storage more convenientCustomers can build clusters using standard hardware from existing vendors that’s perfect for their workload.
• Clusters can be performance-optimized, capacity-optimized, or throughput-optimized
• Need capacity? Add more disks. Too slow? Add more servers.
• Clusters can become larger or smaller with no downtime
19
WHY THIS MATTERS
PROPRIETARYHARDWARE
SCALE-UPARCHITECTURE
Common,off-the-shelf hardware
Scale-outarchitecture
Lower cost, standardized supply chain
Increased operational flexibility
20
PERFORMANCE THAT SCALESPerformance should scale up as capacity does
Software-defined storage intelligently uses hardware to provide performance at very large scale.
• Traditional appliances perform better when they are empty than they do when they are full of disks
• Performance in software-defined storage clusters improves as clusters get larger, not the other way around
• Intel, SanDisk, Fujitsu, and Mellanox regularly contribute performance optimizations
PERF
ORM
ANCE
21
WHY THIS MATTERS
PROPRIETARYHARDWARE
SCALE-UPARCHITECTURE
HARDWARE-BASEDINTELLIGENCE
Common,off-the-shelf hardware
Scale-outarchitecture
Software-basedintelligence
Lower cost, standardized supply chain
Increased operational flexibility
More programmability, agility,and control
22
THE ROBUSTNESS OF SOFTWARESoftware can do things hardware can’t
Storage services based on software are more flexible than hardware-based implementations
• Can be deployed on bare metal, inside containers, inside VMs, or in the public cloud
• Can deploy on a single server, or thousands, and can be upgraded and reconfigured on the fly
• Grows and shrinks programmatically to meet changing demands
23
WHY THIS MATTERS
PROPRIETARYHARDWARE
SCALE-UPARCHITECTURE
HARDWARE-BASEDINTELLIGENCE
CLOSED DEVELOPMENTPROCESS
Common,off-the-shelf hardware
Scale-outarchitecture
Software-basedintelligence
Open developmentprocess
Lower cost, standardized supply chain
Increased operational flexibility
More programmability, agility,and control
More flexible, well-integratedtechnology
24
• Contributions from Intel, SanDisk, CERN, and Yahoo
• Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events
• Over 11M downloads in the last 12 months
• Increased development velocity, authorship, and discussion has resulted in rapid feature expansion
OPEN ECOSYSTEMS
97 AUTHORS/MO
2,453 COMMITS/MO
260 POSTERS/MO
33 AUTHORS/MO
97 COMMITS/MO
138 POSTERS/MO
25
DISRUPTION IN THE STORAGE INDUSTRY
PUBLIC CLOUD STORAGE
TRADITIONALAPPLIANCES
SOFTWARE-DEFINED STORAGE
better
faster
more
less
weaker
limited
better
faster
less
more
stronger
broad
COST EFFICIENCY
PROVISIONING
VENDOR LOCK-IN
SKILL REQUIRED
GOVERNANCE
DEPLOYMENT OPTIONS
27
THE RED HAT STORAGE PORTFOLIO
Gluster management
Ceph data service
Gluster data service
Ceph management
OPE
N S
OU
RCE
SOFT
WAR
EST
AND
ARD
HAR
DW
ARE
Share-nothing, scale-out architecture provides durability and adapts to changing demands
Self-managing and self-healing features reduce operational overhead
Standards-based interfaces and full APIs ease integration with applications and systems
Supported by the experts at Red Hat
28
OVERVIEW: RED HAT GLUSTER STORAGE
• Purpose-built as a scale-out file store with a straightforward architecture suitable for public, private, and hybrid cloud
• Simple to install and configure, with a minimal hardware footprint
• Offers mature NFS, SMB and HDFS interfaces for enterprise use
Intuit uses Red Hat Gluster Storage to provide flexible, cost-effective storage for its industry-leading financial offerings
Nimble file storage for petabyte-scale workloads
Container-Native Storage• Persistent storage• Containerized storageEnterprise File Sharing• Media streaming• Active ArchivesEnterprise VirtualizationRich Media and Archival
CUSTOMER HIGHLIGHT: INTUIT
TARG
ET U
SE
CASE
S
29
STRENGTH: DEPLOYMENT FLEXIBILITY
HOSTS VIRTUAL MACHINE
S
PRIVATE CLOUD
CONTAINERSPUBLIC CLOUD
HYBRID CLOUD
TARGET WORKLOAD: CONTAINERS
30
Deploys storage alongside applications in containersLowers TCO by increasing utilization of resourcesUnifies container and storage orchestrationAllows for rapid adjustments to compute/storage ratio
CONTAINERNGINIX
CONTAINERMASTE
R
Node 2 Node 3
Node 1
Node 4
NGINIX CONTAINER
POSTGRES CONTAINER
31
OVERVIEW: RED HAT CEPH STORAGE
• Built from the ground up as a next-generation storage system, based on years of research and suitable for powering infrastructure platforms
• Highly tunable, extensible, and configurable, with policy-based control and no single point of failure
• Offers mature interfaces for block and object storage for the enterprise
Cisco uses Red Hat Ceph Storage to deliver storage for next-generation cloud services
Powerful distributed storage for the cloud and beyond
Cloud Infrastructure• VM storage with
OpenStack® Cinder, Glance Keystone, Manila, and Nova
• Object storage for tenant apps
Rich Media and ArchivalS3-compatible object storage
CUSTOMER HIGHLIGHT: CISCO
TARG
ET U
SE
CASE
S
32
STRENGTH: PETABYTE SCALE
33
STRENGTH: MASSIVELY DISTRIBUTED DATA
TARGET WORKLOAD: OPENSTACK
34
Allows for instantaneous parallel creation of VMs at massive scaleIntegrates easily and tightly with OpenStack Cinder, Glance, Nova, Keystone, and ManilaOffers instant backup capabilitiesProvides persistent object, file, and database storage for applications
TARGET WORKLOAD: OBJECT STORAGE
35
Stores unstructured data at web scale, using standard hardware Works with standard object APIs for a wide range of compatibility Spans multiple regions with no single point of failure Ideal for active archives, big data archives, and content libraries
36
STORAGE AT RED HAT
PHYSICAL VIRTUAL PRIVATE CLOUD CONTAINERS PUBLIC CLOUD
IDC – The economics of software defined storageComparable features and performance at significantly lower cost of traditional storage appliances
http://red.ht/
2eoTdB6IDC study finds Red Hat Ceph Storage
and Red Hat Gluster Storage to be 53% and 39% less expensive,
respectively, than comparable NAS solutions from traditional storage
vendors, across five years
GARTNER NAMED RED HAT A VISIONARY IN DISTRIBUTED FILE SYSTEMS AND OBJECT STORAGE
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from https://engage.redhat.com/gartnermagic-quadrant-storage-s-201610121525.
• Red Hat Storage recognized as a Visionary by Gartner in their first Magic Quadrant for Distributed File Systems and Object Storage.
• Red Hat Storage positioned furthest and highest in both Completeness of Vision and Ability to Execute in the Visionaries quadrant.
Red Hat Ceph Storage provides us with a high-performing system that can scale to meet the significant storage needs required for genomic sequencing.”
THOMAS CONNOR, PH.D.SENIOR LECTURER, CARDIFF UNIVERSITY AND CO-INVESTIGATOR, CLIMB.
“
ACHIEVING STORAGE NEEDS TO SUPPORT MEDICAL BREAKTHROUGHSBUSINESS CHALLENGEFree cloud-based compute, storage, and analysis tools for academic microbiologists in U.K. Shared IT infrastructure to facilitate collaboration among geographically dispersed research community Easy to use solution to manage 100s of TBs, scale on demand to PBsSOLUTIONRed Hat Ceph Storage and Red Hat Consulting
BENEFITSObject storage to meet large-scale storage requirements of medical research
Greater efficiency in storing enterprise data and managing data growth
Scalability to 1.5PB of raw object storage per site
Cloud Infrastructure for Microbial Bioinformatics
Throughput Optimized
SSD, HDD in standard / dense chassis
Use Case: Rich Media
41
CEPH IS NOT JUST SCALE OUT CAPACITY
IOPS Optimized
NVMe SSD in SLED chassis
Use Case: MySQL
Cost / Capacity Optimized
HDD in dense / ultra-dense chassis
Use Case: Active Archives
High MB/s throughputLarge, sequential IO
Read / write mix
High IOPS / GBSmaller, random IO
Read / write mix
Low cost / GBSequential IOWrite mostly