Turning object storage into vm storage

15
Turning Object Storage into Virtual Machine Storage The open source Storage Router

description

Object Storage is today the standard to build scale-out storage. But due to technical hurdles it is impossible to run Virtual Machines directly from an Object Store. Open vStorage is the layer between the hypervisor and Object Store and turns the Object Store into a high performance, distributed, VM-centric storage platform.

Transcript of Turning object storage into vm storage

Page 1: Turning object storage into vm storage

Turning Object Storage intoVirtual Machine Storage

The open source Storage Router

Page 2: Turning object storage into vm storage

The Rise of Object Storage

Source: http://blog.oxygencloud.com/2013/09/16/after-10-years-object-storage-investment-continues-and-begins-to-bear-significant-fruit/

Page 3: Turning object storage into vm storage

Object Storage

• Amazon Simple Storage Service (S3) started the Object Store momentum

• ... but new players are taking over:– Scality– Ceph– Swiftstack– Amplidata– DataDirect Networks– ...

• Used to store files but not as primary storage for Virtual Machines

Page 4: Turning object storage into vm storage

Object Storage Benefits

• Build scale-out storage by adding more servers

• High reliability by duplicating content across nodes

• Easy to manage (no need to know about RAID or volumes)

• Standard API as all solutions support S3 API, no vendor lock-in

• Cost-effictive: mix SSDs (small but fast) and SATA (slow but large)

Page 5: Turning object storage into vm storage

Object Storage <> Hypervisor

• Object Storage– Objects– Buckets– S3 API

• Hypervisor– Files– Folders– Volumes– Block level storage

What is needed is a technology whereby Virtual Machines can use Object Stores instead of a SAN and get the benefits of the low cost

and scale-out capabilities of Object Stores

Page 6: Turning object storage into vm storage

Object Storage Challenges

• The issues to use Object Storage as VM Storage:– Eventual consistency (the CAP Theorem)

– Latency & performance• VMs require low latency and high performance• Object Stores are developed to contain lots of data

(large disks and low performance)• Additional latency as Object Store is on the Local LAN instead of

attached to the server

– Different Management Paradigms• Object Stores understand Objects <> Hypervisors understand VMs

Page 7: Turning object storage into vm storage

Distributed File System?

• Distributed File Systems such as GlusterFS and Ceph are not an answer:– The CAP Theorem also applies to these (consistency <> performance)– They were not designed for VMs (VM Snapshots?, cloning?)

• Even new File Systems (copy-on-write) can’t give an answer

None of the File Systems today have been designed to link Virtual Machines and Object Storage.

Page 8: Turning object storage into vm storage

The answer: Open vStorage

Open vStorage turns Object Storage into Virtual Machine usable primary Storage!

Page 9: Turning object storage into vm storage

What is Open vStorage

Open vStorage is an open source “Grid Storage Router" installed as a virtual machine on a host or a cluster of hosts to create a VM-centric, clustered, reliable, scale out and high performance storage system for virtual machines out of Storage Backends such as Object Stores.

Page 10: Turning object storage into vm storage

An Overview

ESXi / KVM ESXi / KVM ESXi / KVM

Scale-outVM VM

VM VM

VM VM

VM VM

VM VM

VM VM

Unified Namespace

...Swift

SSDSSD

OpenvStorage

SSDSSD

OpenvStorage

SSDSSD

OpenvStorage

Page 11: Turning object storage into vm storage

Key aspects are

• Unified namespace presented as NFS to all Hosts• Open vStorgae leverages server flash as acceleration• Flexibility in storage backends• Redundancy and Fault Tolerance in Software• Asymmetric Scale Out Architecture• VM-centric approach• Multi-hypervisor• Open-source: Apache License version, Version 2,0

Page 12: Turning object storage into vm storage

Solving Eventual Consistency

TIM

E

LBA 1: A

LBA 2: B

LBA 3: C

LBA 4: D

LBA 5: E

LBA 1: F

LBA 1: A

LBA 6: G

LBA 2: C

LBA 3: D

LBA 4: E

...

SSD or PCI Flash

LBA 1: A

LBA 2: B

LBA 3: C

LBA 4: D

LBA 5: E

SCO 1

LBA 1: F

LBA 1: A

LBA 6: G

LBA 2: C

LBA 3: D

SCO 2

LBA 4: E

...

SCO 3

Transfer SCOs once they are full to the Object Store at slow paceEach write is appended

to the current SCO

Swift

...

Page 13: Turning object storage into vm storage

How does it work ...

VMVMDK

NFS

VMConfig

Open vStorage VSA

DistributedDatabase

Volume

VMVMDK

VMConfig

Volume

Storage Backend (Ceph, SwiftStack, ...)

Open vStorage VSA

VMVMDK

VMConfig

Volume

Volume is only available on 1 Host!

Page 14: Turning object storage into vm storage

Open vStorage: Summary

• Uses an always append approach• Push Storage Container Objects to the Object Store once it is full

(at slower pace)• Each Virtual Machine with the Open vStorage software has its own

NFS server and exports a different file system instance• But each Host is tricked into believing that it accesses a single

unified namespace shared across all the Hosts (so vMotion is supported)

Page 15: Turning object storage into vm storage

Download the Open vStorage (it is free) and give it a try!

http://doc.openvstorage.com/

Give it a try ...