Introducing Cloudian HyperStore 6.0
-
Upload
cloudian -
Category
Technology
-
view
473 -
download
12
Transcript of Introducing Cloudian HyperStore 6.0
![Page 1: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/1.jpg)
Cloudian HyperStore 6.0Changing the economics of StorageRelease : April 12 th 2016
![Page 2: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/2.jpg)
FINALIST
History of Innovation
3
2011 2012 2013 2014 2015 2016
HyperStore 1.5• Software• Scale-out• Peer-to-peer• Multi Tenancy• Quality of Service
(QoS)
HyperStore 2.3• Software• S3 API• Multi Region Storage
HyperStore 3.0• Software• Compression• Virtual Appliance
HyperStore 2.4• Software• Server Side
Encryption• Billing & Chargeback• Citrix Cloud Platform
HyperStore 4.0• Software• Erasure Coding• OpenStack• NFS• Cloud Archive
HyperStore 5.0• Software &
Appliances• Server Side
Encryption• Billing & Chargeback• Citrix Cloud Platform
HyperStore 5.1• Smart Data• Hadoop• Smart Support
HyperStore 5.2• Smart Data Policies• Faster Repair
HyperStore ‘forever live’ 3000• Hot plug everything• Seamless Scale• Petabyte Scale
HyperStore 6.0• Operations @scale• Durable @scale• Tuning @scale
![Page 3: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/3.jpg)
New Economics of Storage
Hardware/Software (62%)$1245
Operations FTE (26%)$511
• In 2016 estimated that 1 FTE can manage 344TB• Software Defined Storage has dramatically lowered storage costs to $122/TB/Year• Operational Costs now significantly outweigh the Acquisition related costs
Other $253 (12%)$2009/TB/Yr TCO
Hardware/Software (14%) $122
Operations FTE (58%)$511
Other $253 (28%)$886/TB/Yr TCO
Source : Gartner 2016 IT Key Metrics Data
Cloudian HyperStore 1c per GB/month
6.0 Focus
![Page 4: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/4.jpg)
Introducing @scale Storage
5
Operations @scale New Operations Console for one click management Fully automated add/remove of nodes Non disruptive rolling upgrades
Durable @scale Always repaired, Always verified Dynamic Object Routing for automated failure avoidance Simple Disaster Recovery with Cross Region Replication
Tuning @scale Visual Storage Analytics reports to automatically identify hot spots Object ‘GPS’ to locate objects
Key announcement messages1. Double the management capacity for your Storage Administrators2. Continuous and automated failure resolution for data durability3. Proactive low-cost management for Petabyte Scale storage
NEW
NEW
April 12th
NEW
![Page 5: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/5.jpg)
Operations @scale : New UI
![Page 6: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/6.jpg)
Operations @scale : New UI
• Instant 360 View• Check Health• View Sys Info
![Page 7: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/7.jpg)
Operations @scale : Datacenters
• One screen for hundreds of nodes• Instantly view health of nodes• Add nodes with one click• Cluster dynamically rebalances
![Page 8: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/8.jpg)
Operations @scale : Usage
• View capacity consumption• View system performance
![Page 9: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/9.jpg)
Operations @scale : Capacity Explorer
• Analyze your data uses across globe• Plan for future based on your demand
![Page 10: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/10.jpg)
Operations @scale : Place data based on your Business Req
![Page 11: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/11.jpg)
Operations @scale : Turnkey Appliance View
• Identify Nodes types• Blink Suspect Servers• Blink Suspect Drives
![Page 12: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/12.jpg)
Tuning @scale : “Object GPS”
• Locate any object parts• Object digests and timestamps validation
![Page 13: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/13.jpg)
Operations @scale : Rolling Upgrade
• Automates Upgrade/Patch installation - touchless
• Uses Puppet framework for distribution and management
• No downtime – node-by-node rolling upgrade
✓ Peer-to-peer system = no SPOF
Distributed Everything – Data, MetaData, Configuration
✓New version- Old version
Node 1 Node 2 Node 3t0
t1
t2
t3
Node 4 Node 5 Node 6
Node 1 Node 2 Node 3 Node 4 Node 5 Node 6
Node 1 Node 2 Node 3 Node 4 Node 5 Node 6
Node 1 Node 2 Node 3 Node 4 Node 5 Node 6
![Page 14: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/14.jpg)
Durable @scale : Proactive Repair
Problem If the node/disk/network failed
today and IOs that were incomplete will have to be identified by scanning the whole system and making a lot of Disk IOs
VERY slow process
Exposes customer for days to data loss in event of double failures
Solution When an IO fails due to
network/node or disk failure we keep hints as to what failed
Now when we need to run repair we know what exactly needs repair
Very FAST process
Drastically Reduce exposure window from days to hours
![Page 15: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/15.jpg)
Durable @scale : Rebuild Analytics
• View Data Rebuild info• View Cluster Rebalance info
![Page 16: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/16.jpg)
To protect against unplanned outagesDurable @scale : Smart Redirect
Introducing Local cache copy
ReplicationClient
Node 1 Node 2 Node 3 Node 4
Cassandra Local Cache
1- req In
2- Node1 Down
3- Node 2 makes Extra Copy stored in local cache4- Node 2 keeps hint in Cassandra
5- req Success
Copy2 Copy1
![Page 17: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/17.jpg)
To protect against unplanned outages
Introducing Hot Spares (k+m+L)
No Local Cache, only hints Repair handled by proactive
repair If customer has 20 nodes it can
only fail 2 nodes - (k+m+L) fixes this
Erasure Coding
Node 1
Node 4 Node 2
Node 3
Node 5 Node 6`
Node 7 Node 8
Client
Durable @scale : Smart Redirect
![Page 18: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/18.jpg)
Durable @scale : Smart Disk Balancing
1. Scenario 1 :Disk Imbalance If we notice an imbalance it will change the tokens pointing from “highly used disk” to “low used
disk”
2. Scenario 2 : Disk failure New data automatically routes to newly assigned resources
t0
t1
t0
t1
Auto Dynamic MappingStatic Mapping
![Page 19: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/19.jpg)
Durable @scale : Repair-On-Read
NODE 1 NODE 2 NODE N+1
Replica Replica
Read��
��
🔧on Read
• All replicas of the object are checked on a read request• Missing/out-of-date replicas are automatically replaced or updated
Object hasChanged
Out-of-Date replicas are replaced or updated
��
🔧on Read
![Page 20: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/20.jpg)
Durable @scale S3 Cross Region Replication
• Restore Script restores object in a new bucket
• Restore scripts generates source list as a CSV file
• No CMC support in 6.0
![Page 21: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/21.jpg)
S3 Compatibility
Ceph S3 testsTotal tests: 416OPEN 4 (3 bugs open: POST filename subst., minor ACL)PASS 386PASS* 26 (Illegal character case handled at Jetty)
![Page 22: Introducing Cloudian HyperStore 6.0](https://reader036.fdocuments.us/reader036/viewer/2022081416/5878e3181a28abfa038b4eaf/html5/thumbnails/22.jpg)
Introducing @scale Storage
23
Operations @scale New Operations Console for one click management Fully automated add/remove of nodes Non disruptive rolling upgrades
Durable @scale Always repaired, Always verified Dynamic Object Routing for automated failure avoidance Simple Disaster Recovery with Cross Region Replication
Tuning @scale Visual Storage Analytics reports to automatically identify hot spots Object ‘GPS’ to locate objects
Key announcement messages1. Double the management capacity for your Storage Administrators2. Continuous and automated failure resolution for data durability3. Proactive low-cost management for Petabyte Scale storage
NEW
NEW
April 12th
NEW