Sergey Goncharov Solution Architect, Red Hat sgonchar@redhat · STORAGE ORCHESTRATION Storage...
Transcript of Sergey Goncharov Solution Architect, Red Hat sgonchar@redhat · STORAGE ORCHESTRATION Storage...
Why software defined storage matters?Sergey Goncharov
Solution Architect, Red [email protected]
AGENDA● Storage and Datacenter evolution
● Red Hat Storage portfolio● Red Hat Gluster Storage
● Red Hat Ceph Storage● Red Hat Storage roadmap and future
● Q/A
3
WHAT IS SOFTWARE-DEFINED STORAGE?
SERVER-BASED CENTRALIZED CONTROL OPEN ECOSYSTEM
4
THE DATACENTER IS EVOLVING
DEVELOPMENTMODEL
APPLICATIONARCHITECTURE
DEPLOYMENTAND PACKAGING
APPLICATIONINFRASTRUCTURE
STORAGE
Waterfall
Agile
DEVOPS
Monolithic
N-Tier
MICROSERVICES
Virtual Services
CONTAINERS
Bare Metal
Hosted
HYBRID CLOUD
Data Center
Scale Out
SOFTWARE-DEFINED STORAGE
Scale Up
5
SERVER-BASED STORAGE
Server-based storage is the use of software and standard hardware to provide services traditionally provided by single-purpose storage appliances (similar to server virtualization, which uses software to emulate servers), providing increased agility and efficiency.
Appliance
USER
Appliance
USER
Appliance
USER
Distributed Cluster of Services
USER USER USER
6
STORAGE ORCHESTRATION
Storage orchestration is the ability to provision, grow, shrink, and decommission storage resources on-demand and programmatically, providing increased control and integration of storage into a software-defined data center.
Web Console API Command Line
A browser interface designed for
managing distributed storage
A full API for automation andintegration with outside
systems
A robust, scriptable command-lineinterface for expert operators
Full life cycle management for distributed, software-defined data services
PROVISION INSTALL CONFIGURE TUNE MONITOR
7
GROWING INNOVATION COMMUNITIES
• Contributions from Intel, SanDisk, CERN, and Yahoo
• Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events
• Over 11M downloads in the last 12 months
• Increased development velocity, authorship, and discussion has resulted in rapid feature expansion
97 AUTHORS/MO
2,453 COMMITS/MO
260 POSTERS/MO
33 AUTHORS/MO
97 COMMITS/MO
138 POSTERS/MO
8
THE FUTURE OF STORAGEOPEN, SOFTWARE-DEFINED STORAGE
Standardized, unified, open platforms
Standard Computersand Disks
Ceph Gluster +++
Control Plane (API, GUI)
Custom GUI
Proprietary Hardware
Proprietary Software
Custom GUI
Proprietary Hardware
Proprietary Software
USER USER USER
ADMIN ADMIN ADMIN
Custom GUI
Proprietary Hardware
Proprietary Software
USER ADMIN
Sta
nd
ard
Ha
rdw
are
Op
en
So
urce
So
ftwa
re
TRADITIONAL STORAGEComplex proprietary silos
9
THE RED HAT STORAGE PORTFOLIO
Gluster management
Ceph data service Gluster data service
Ceph management
OP
EN
SO
UR
CE
SO
FT
WA
RE
STA
ND
AR
DH
AR
DW
AR
E
Share-nothing, scale-out architecture provides durability and adapts to changing demands
Self-managing and self-healing features reduce operational overhead
Standards-based interfaces and full APIs ease integration with applications and systems
Supported by the experts at Red Hat
10
HOW STORAGE FITS
RED HAT
STORAGE
PHYSICAL VIRTUALIZATION PRIVATE CLOUD CONTAINERS PUBLIC CLOUD
RED HAT GLUSTER STORAGEOpen source, software-defined storage for unstructured file data at
petabyte scale
Media, video Machine, Log Data
GeoSpatial Persistent Storage
Documents
RED HAT GLUSTER STORAGE ADVANTAGES
OPEN
SCALABLE
ACCESSIBLE
MODULAR
ALWAYS-ON
Open, software-defined distributed file and object store
No Metadata Server
Multi-Protocol the Same Data
No Kernel Dependencies
High-Availability across data, systems and applications
• Based on GlusterFS open source community project• Uses proven local file system (XFS)• Data is stored in native format
• Uses an elastic hashing algorithm for data placement• Uses local filesystem’s xattrs to store metadata • Nothing shared scale-out architecture
• Global name space• NFS, SMB, object, HDFS, Gluster native protocol• Posix compliant
• GlusterFS is based on filesystem in userspace (FUSE)• Modular stackable arch allows easy addition of features ...without being tied to any kernel versioniant
• Synchronous replication with self-healing for server failure • Asynchronous geo-replication for site failure
GLUSTER ARCHITECTUREDistributed scale out storage using industry standard hardware
NFS CIFS FUSE
SERVERS WITH LOCAL DISKS
Aggregates systems to one cohesive unit and presents using common protocols
WHY RED HAT CEPH STORAGE FOR OBJECTS?
• Proven at Web-Scale for Object Storage
• Flexible Storage for Your Applications
• Data Protection, Reliability & Availability Enterprises Demand
• Open, Community Based Software Defined Storage for Object Storage
• Cost Effective Object Storage to Help You Maximize Your Storage Budget
15
CEPH ARCHITECTURE
APPLICATION HOST / VM CLIENT
RGW
A web services gateway for object storage, compatible
with S3 & Swift
RBD
A reliable, fully-distributed block device with cloud
platform integration
CEPHFS*
A distributed file system with POSIX semantics &
scale-out metadata
LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOS
A software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
* TECH PREVIEW
NEW INTEGRATED MANAGEMENT WITH RED HAT STORAGE CONSOLE 2
• Easy to use graphical interface to manage storage cluster life cycle
• Ansible-based deployment tools for installation, importation, and granular configuration from CLI or GUI
• Monitoring and troubleshooting with statistical and graphical information about cluster components
RADOS. RGW, RBD, CALAMARI
RED HAT STORAGE CONSOLE 2
CLUSTER OF STANDARD SERVERS AND MEDIA
CLIENTSADMINS
ROAD MAP: RED HAT GLUSTER STORAGE
Gluster 3.7, RHEL 6, 7 Gluster 3.8, RHEL 6, 7 Gluster 4, RHEL 7
TODAY (v3.1) V3.2 (H1-2016) FUTURE (v4.0 and beyond)C
OR
EFI
LES
EC
MG
MT
CO
RE
FILE
SE
CM
GM
T
CO
RE
FILE
SE
CM
GM
T
ERASURE CODING
BIT ROT DETECTION
TIERING
SNAP SCHEDULE
ACTIVE/ACTIVE NFSV4
SMB 3 (BASIC SUBSET)
SELINUX
SSL ENCRYPTION( IN-FLIGHT)
DEVICE MANAGEMENT
GEO-REPLICATION,SNAPSHOTS
DASHBOARD
INODE QUOTAS
FASTER SELF-HEAL
CONTROLLED REBALANCE
SMB 3(ADVANCED FEATURES)
MULTI-CHANNEL
AT-REST ENCRYPTION
DYNAMIC PROVISIONING OF RESOURCES
COMPRESSION
DEDUPLICATION
HIGHLY SCALABLE CTRL PLANE
NEXT-GEN REPLICATION DISTRIBUTION
pNFS
QOS
CLIENT SIDE CACHING
NEW UI
GLUSTER REST API
CORE • Bluestore (tech preview)• Performance optimizations
CORE • Bluestore general availability• Data checksums• End-to-end SSL• Improved multi-threading
RED HAT CEPH STORAGE 2 (today)
MANAGEMENT• Red Hat Storage Console 2• Lifecycle management• Ansible support
OBJECT• Support for global clusters• AD, LDAP, Keystone v3• Expanded API compatibility• NFS gateway (tech preview)
BLOCK• Mirroring
RED HAT CEPH STORAGE 2.1 (Q4 2016)
OBJECT• AWS Secure Token Service (STS)
BLOCK• iSCSI
UPSTREAM ACTIVITY (in the future)
FILE• Active/active metadata servers• Snapshots• Hypervisor integration
OBJECT• Metadata search• Tiering to public cloud• Object-level encryption• Compression
BLOCK
• Distributed client-side cache• Improved iSCSI support