Ceph, Xen, and CloudStack: Semper Melior
-
Upload
patrick-mcgarry -
Category
Technology
-
view
592 -
download
5
description
Transcript of Ceph, Xen, and CloudStack: Semper Melior
![Page 1: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/1.jpg)
Ceph, Xen, and CloudStack:Semper Melior
Xen User Summit| New Orleans, LA | 18 SEP 2013
![Page 2: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/2.jpg)
2
•Patrick McGarry•Community monkey•Inktank / Ceph•/. > ALU > P4•@scuttlemonkey•patrick@inktankcom
Accept no substitutes
C’est Moi
![Page 3: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/3.jpg)
3
•Ceph in <30s•Ceph, a little bit more•Ceph in the wild•Orchestration•Community status•What’s Next?•Questions
The plan, Stan
Welcome!
![Page 4: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/4.jpg)
4
On commodity hardware
Ceph can run on any infrastructure, metal or virtualized to provide a cheap and powerful storage cluster.
Object, block, and file
Low overhead doesn’t mean just hardware, it means people too!
Awesomesauce
Infrastructure-aware placement algorithm allows you to do really cool stuff.
Huge and beyond
Designed for exabyte, current implementations in the multi-petabyte. HPC, Big Data, Cloud, raw storage.
…besides wicked-awesome?
What is Ceph?
Software All-in-1 CRUSH Scale
![Page 5: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/5.jpg)
5
Find out more!Ceph.com
…but you can find out more
Use it todayDreamhost.com/cloud/DreamObjects
Get SupportInktank.com
That WAS fast
![Page 6: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/6.jpg)
6
OBJECTS VIRTUAL DISKS FILES & DIRECTORIES
CEPHFILE SYSTEM
A distributed, scale-out filesystem with POSIX
semantics that provides storage for a legacy and
modern applications
CEPHGATEWAY
A powerful S3- and Swift-compatible gateway that brings the power of the Ceph Object Store to modern applications
CEPHBLOCK DEVICEA distributed virtual block device that delivers high-
performance, cost-effective storage for virtual machines and legacy applications
CEPH OBJECT STORE
A reliable, easy to manage, next-generation distributed objectstore that provides storage of unstructured data for applications
![Page 7: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/7.jpg)
7
![Page 8: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/8.jpg)
8
![Page 9: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/9.jpg)
9
• CRUSH– Pseudo-random placement
algorithm– Ensures even distribution– Repeatable, deterministic– Rule-based configuration
• Replica count• Infrastructure topology• Weighting
![Page 10: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/10.jpg)
1010 10 01 01 10 10 01 11 01
10
10 10 01 01 10 10 01 11 01 10
hash(object name) % num pg
CRUSH(pg, cluster state, rule set)
![Page 11: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/11.jpg)
1110 10 01 01 10 10 01 11 01
10
10 10 01 01 10 10 01 11 01 10
![Page 12: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/12.jpg)
12
CLIENT
![Page 13: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/13.jpg)
13
![Page 14: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/14.jpg)
14
![Page 15: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/15.jpg)
15
![Page 16: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/16.jpg)
16CLIENT
??
![Page 17: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/17.jpg)
17
…with Marty Stouffer
Ceph in the Wild
![Page 18: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/18.jpg)
18
No incendiary devices please…
Linux Distros
![Page 19: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/19.jpg)
19
Object && BlockVia RBD and RGW (Swift API)
Our BFF
IdentityVia Keystone
More coming!Work continues with updates in Havana and Icehouse.
OpenStack
![Page 20: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/20.jpg)
20
BlockAlternate primary, and secondary
Community maintained
CommunityWido from 42on.com
More coming in 4.2!Snapshot & backup supportCloning (layering) supportNo NFS for system VMsSecondary/Backup storage (s3)
CloudStack
![Page 21: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/21.jpg)
21
A blatent ripoff!
Primary Storage Flow
•The mgmt server never talks to the Ceph cluster
•One mgmt server can manage 1000s of hypervisors
•Mgmt server can be clustered
•Multiple Ceph clusters/pools can be added to CloudStack cluster
![Page 22: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/22.jpg)
22
A pretty package
A commercially packaged OpenStack solution back by Ceph.
RADOS for Archipelago
Virtual server management software tool on top of Xen or KVM.
RBD backed
Complete virtualization management with KVM and containers.
BBC territory
Talk next week in Berlin
So many delicious flavors
Other Cloud
SUSE Cloud Ganeti Proxmox OpenNebula
![Page 23: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/23.jpg)
23
Since 2.6.35
Kernel clients for RBD and CephFS. Active development as a Linux file system.
iSCSI ahoy!
One of the Linux iSCSI target frameworks. Emulates: SBC (disk), SMC (jukebox), MMC (CD/DVD), SSC (tape), OSD.
Getting creative
Creative community member used Ceph to back their VMWare infrastructure via fibre channel.
You can always use more friends
Project Intersection
Kernel STGT VMWareLove me!
Slightly out-of-date. Some work has been done, but could use some love.
Wireshark
![Page 24: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/24.jpg)
24
CephFS
CephFS can serve as a drop-in replacement for HDFS.
Upstream
Ceph vfs module upstream samba.
CephFS or RBD
Reexporting CephFS or RBD for NFS/CIFS.
MOAR projects
Project Intersection
Hadoop Samba GaneshaRecently Open Source
Commercially supported product from Citrix. Recently Open Sourced. Still a bit of a tech preview.
XenServer
![Page 25: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/25.jpg)
25
Support for libvirtXenServer can manipulate Ceph!
Don’t let the naming fool you, it’s easy
Blktap{2,3,asplode}Qemu; new boss, same as the old boss(but not really)
What’s in a name?Ceph :: XenServer :: LibvirtBlock device :: VDI :: storage volPool :: Storage Repo :: storage pool
Doing it with Xen*
![Page 26: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/26.jpg)
26
Thanks David Scott!
XenServer host arch
Xapi, XenAPI
xenopsd S M adapters
libvirtlibxl ceph ocfs2libxengue
stlibxc qemu
xen
Client(CloudStack, OpenStack, XenDesktop)
![Page 27: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/27.jpg)
27
Come for the blockStay for the object and file
No matter what you use!
Reduced OverheadEasier to manage one cluster
“Other Stuff”CephFS prototypesfast development profileceph-devellots of partner action
Gateway Drug
![Page 28: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/28.jpg)
28
Squash HotspotsMultiple hosts = parallel workload
But what does that mean?
Instant ClonesNo time to boot for many images
Live migrationShared storage allows you to move instances between compute nodes transparently.
Blocks are delicious
![Page 29: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/29.jpg)
29
Flexible APIsNative support for swift and s3
And less filling!
Secondary StorageComing with 4.2
Horizontal ScalingEasy with HAProxy or others
Objects can juggle
![Page 30: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/30.jpg)
30
Neat prototypesImage distribution to hypervisors
You can dress them up, but you can’t take them anywhere
Still earlyYou can fix that!
Outside usesGreat way to combine resources.
Files are tricksy
![Page 31: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/31.jpg)
31
Where the metal meets the…software
Deploying this stuff
![Page 32: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/32.jpg)
32
Procedural, Ruby
Written in Ruby, this is more of the dev-side of DevOps. Once you get past the learning curve it’s powerful though.
Model-driven
Aimed more at the sysadmin, this procedural tool has a very wide penetration (even on Windows!).
Agentless, whole stack
Using the built-in OpenSSH in your OS, this super easy tool goes further up the stack than most.
Fast, 0MQ
Using ZeroMQ this tool is designed for massive scale and fast, fast, fast. Unfortunately 0MQ has no built in encryption.
The new hotness
Orchestration
Chef Puppet Ansible Salt
![Page 33: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/33.jpg)
33
Canonical Unleashed
Being language agnostic, this tool can completely encapsulate a service. Can also handle provisioning all the way down to hardware.
Dell has skin in the game
Complete operations platform that can dive all the way down to BIOS/RAID level.
Others are joining in
Custom provisioning and orchestration, just one example of how busy this corner of the market is.
Doing it w/o a tool
If you prefer not to use a tool, Ceph gives you an easy way to deploy your cluster by hand.
MOAR HOTNESS
Orchestration Cont’d
Juju Crowbar ComodIT Ceph-deploy
![Page 34: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/34.jpg)
34
All your space are belong to us
Ceph Community
![Page 35: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/35.jpg)
35
![Page 36: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/36.jpg)
36
Up and to the right!
Code Contributions
![Page 37: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/37.jpg)
37
Up and to the right!
Commits
![Page 38: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/38.jpg)
38
Up and to the right!
List Participation
![Page 39: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/39.jpg)
39
This Ceph thing sounds hot.
What’s Next?
![Page 40: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/40.jpg)
40
An ongoing process
While the first pass for disaster recovery is done, we want to get to built-in, world-wide replication.
Reception efficiency
Currently underway in the community!
Headed to dynamic
Can already do this in a static pool-based setup. Looking to get to a use-based migration.
Making it open-er
Been talking about it forever. The time is coming!
Hop on board!
The Ceph Train
Geo-Replication Erasure Coding Tiering Governance
![Page 41: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/41.jpg)
41
Quarterly Online Summit
Online summit puts the core devs together with the Ceph community.
Not just for NYC
More planned, including Santa Clara and London. Keep an eye out: http://inktank.com/cephdays/
Geek-on-duty
During the week there are times when Ceph experts are available to help. Stop by oftc.net/ceph
Email makes the world go
Our mailing lists are very active, check out ceph.com for details on how to join in!
Open Source is Open!
Get Involved!
CDS Ceph Day IRC Lists
![Page 42: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/42.jpg)
42
http://wiki.ceph.com/04Development/Project_IdeasLists, blueprints, sideboard, paper cuts, etc.
http://tracker.ceph.com/All the things!
New #ceph-devel
Splitting off developer chatter to make it easier to filter discussions.
http://ceph.com/resources/mailing-list-irc/Our mailing lists are very active, check out ceph.com for details on how to join in!
Patches welcome
Projects
Wiki Redmine IRC Lists
![Page 43: Ceph, Xen, and CloudStack: Semper Melior](https://reader036.fdocuments.us/reader036/viewer/2022081412/545414dbaf795987748b59d0/html5/thumbnails/43.jpg)
43
Comments? Anything for the good of the cause?
WEBSITECeph.com
SOCIAL@[email protected]/cephstorage