Welcome to theS3 Server Hackathon!

Cloud Field Days #1 © Scality 20163

Scality RING: Automatize storage for Digital Business

The Scality RING is object-based software-defined storage for the cloud.

We run on standard x86 servers and create a giant pool of storage.

We protect the data and provide 100% reliable, high performance access for any capacity-driven application.

FILE OBJECT OPENSTACK

Cloud Field Days #1 © Scality 20164

Agenda for today

Presentation of the hackathon, judges and logistics

S3 Server tech Overview - https://github.com/scality/s3

Seagate Kinetic drives Overview

Teams and projects assignement!

The Judges!

The Prizes!

How are we judging?

We know the code will not be beautiful!

We’re looking for a functional prototype

It’s OK not to handle quirky corner cases!

How are we judging?

Innovation - We’re looking for creative solutions

Market potential - It has to solve a real problem!

Execution - A working demo ☺

Each team has:

A name,

A mentor,

A clear defined project that’s achievable,

Developers and a leader,

Each team will do a 5 minutes pitch + demo on Sunday

Logistics!

–––

Open source S3 Server!

S3 Adoption keeps rising

© Scality 201614

▪ Open source object storage serverhttps://github.com/scality/s3

▪ Written in Node.js▪ Single instance running in a docker

container▪ Uses docker volumes for persistent storage▪ Same code as Scality’s RING S3 interface

What is S3 Server?

© Scality 201615

▪ Developers can install and develop S3-based apps locally

▪ Enterprises can host a local test/dev environment to learn about object storage

▪ Enterprises can host a small, local object storage system in production

Scality S3 Server: From 0 to S3 in Under 5 Minutes

S3 Serve

r

S3 Serve

r

S3 Serve

r

Backup Application

S3

The teamSF & Paris

© Scality 2016

Scality S3 Server achitecture

Protocol StackBuckets, Objects, MPU (REST API)

AUTH BUCKETS DATA

LevelDB*

Scality RING v6

Kinetic IP drives

Docker Volume

* Persisted in Docker Volume

Scality RING v6Vault

JSON*

S3 API

S3 Server

Public Cloud

© Scality 201618

S3 server released under a Apache-2.0 on Github3.3K downloads on dockerhub!

18

Seagate KineticKinetic development and the basics

Kinetic On SMR (Shingled Magnetic Recording)

What is Kinetic

Kinetic is a drive architecture in which the drive is a key/value service with Ethernet connectivity

Put/Get/Delete API over TCP/SSL

https://www.openkinetic.org/

SMRData recording technology used in drives to increase storage density and overall storage capacity

https://en.wikipedia.org/wiki/Shingled_magnetic_recording

Why Kinetic on SMRIncrease storage capacity

Same Put/Get/Delete API (SMR is transparent to users/applications)

Kinetic Development OverviewKinetic Simulator

Functionally equivalent to Kinetic Drives.

https://github.com/Kinetic/kinetic-java

Build

git clone https://github.com/Kinetic/kinetic-java.git mvn clean package

Run (one or more instances) java -jar kinetic-simulator/target/kinetic-simulator-"Version"-SNAPSHOT-jar-with-dependencies.jar or./bin/startSimulator.sh

More Information Related to Buildhttps://github.com/Kinetic/kinetic-java/blob/master/README.md

Preferred IDE: Eclipse (Mars and up)http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/neon1a

Notes On Kinetic K/V KeysKeys are sorted lexicographically. e.g.

http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/primitives/UnsignedBytes.html#lexicographicalComparator%28%29

https://github.com/Kinetic/kinetic-java/blob/master/kinetic-simulator/src/main/java/com/seagate/kinetic/simulator/persist/leveldb/KineticComparator.java

Use sequential keys for better throughput (not required for later releases)

An example to generate sequential keys

https://github.com/Kinetic/kinetic-java/blob/master/kinetic-test/src/test/java/com/seagate/kinetic/KVGenerator.java

Hackathon Group Key Space Each group is recommended to use its own assigned key space (prefix) to avoid conflict for shared drives. For example, group 1 is assigned prefix of “00”, group 2 is assigned prefix of “01”., etc. Eg:

All keys in group 1 would be in the form of “000”, “001”, “002” ..., etc.

All keys in group 2 would be in the form of “010”, “011”, “012” ..., etc.

All keys in group 3 would be in the form of “020”, “021”, “022” ..., etc.

Scality/Kinetic Hackathon

Thank You!

Thank you!