在这里写上你的标题

副标题文字副标题文字

作者名字 /日期

Island: Local Storage plugin for Cinder

Presented at OpenStack Summit, San Diego

2012-09-22

Hui Cheng/RongZhe Zhufreedomhui@gmail.com | zrzhit@gmail.com

00

01

02

03

04

05

写上你的文字你的文字

Content

1. Why

2. Design

3. Snapshot

4. Discussions

https://blueprints.launchpad.net/cinder/+spec/local-storage-volume

=00

01

02

03

04

05

写上你的文字你的文字

Why

Network bottleneck

Re-mirroring storm

Low performanceThe high performance of 12 disks RAID10

Incremental snapshot

Snapshots store in swift

High durability

Incremental snapshot

Snapshots store in S3

写上你的文字你的文字

1000Mb NetworkEvery x86 server have 12 disks

AWS EBS

Our Environment

Island

=

Design – Island in Cinder

Users

Nova API Volume API

NovaRPC

ComputeManager

Volume Manager

Island

KVM Guest

NovaDB Swift API

cinderRPC

CinderDB

Swift Storage

KEY

REST

AMQP

SQL

POSIX

Snapshot in Swift

images

Host

Island Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift

=

00

01

02

03

04

05

Design – Incremental Snapshot in Swift

Host

Swift

VM

image

snap1

1 2 3 4 1 2 2 3

snap2

snap3

Snapshots are saved incrementally. Each subsequent snapshot will only save modified data block to swift. A table of contents points to the latest versions of each data block. Each data block have a reference counter.

VM

image

VM

image

=

00

01

02

03

04

05

写上你的文字你的文字

Design – Island Architecture

IslandDriver

Snapshot

Vdisk ObjectStorage

QCOW2 VDMK Swift NFS Glusterfs

create_volumedelete_volumecreate_volume_from_snapshotcreate_snapshotdelete_snapshotattach_volumedetach_volume……

get_snapshotput_snapshotdelete_snapshot

get_objectput_objectdelete_objectset_object_metaget_object_meta

create_vdiskdelete_vdiskcreate_snapshotdelete_snapshotread_vdiskwrite_vdiskget_update_blocksread_vdisk_meta

=

00

01

02

03

04

05

Design - Process

Create volumeDo nothing

Create volume from snapshotDo nothing

Delete volumeDo nothing

Delete snapshotIsland delete snapshot in swift

=

00

01

02

03

04

05

写上你的文字你的文字

Design - Process Create the image of the

volume

Check the volume

Update volume[‘host’]

Download snapshot and wirte it to the

image

Download the last snapshot and wirte it to the

image

old volume

new volume

Attach volume

new volume form snapshot

00

01

02

03

04

05

Design - Process

Create snapshot

Create snapshot in image

Delete old snapshot in image

Upload incremental snapshot to swift

Detach volume

Create a new snapshot DB item, set snapshot[‘volume_id’] as the volume

Upload incremental snapshot to swfit

Create snapshot in image

Delete the image of the volume

=

00

01

02

03

04

05

写上你的文字你的文字

Snapshot – How to Create/Read Snapshot in Image

Create internal snapshot in imageBecause there is not a qemu monitor command to create internal snapshot in image when the image was attached. So, we add a new qemu monitor command to create point-in-time snapshot in image.

Accessing QCOW2 image in pythonQCOW2 image format use 2-level Lookup Table http://people.gnome.org/~markmc/qcow-image-format.htmlWe implement qcow2 python library to read/write qcow2 image data and metadata.

=

00

01

02

03

04

05

写上你的文字你的文字

Snapshot – How to Implement Incremental Snapshot

We need to find out updated data blocks between two snapshots.

Compare current snapshot pointer table and previous snapshot pointer

table in image

Find updated data blocks

Upload updated data blocks to objectstorage

Upload snapshot pointer table to objectstorage

Future: modify qcow2 driver in qemu, mark updated data blocks in pointer table of snapshot in image.

=

00

01

02

03

04

05

写上你的文字你的文字

Discussions

The storage capacity of a x86 server is limit, but if many instances in a server and many volume are attached to these instances, it will be a issue.

The attach volume action is slower than other plugin.

Optimization?

Make Island as a independent system?