Introducing DS3 - a RESTful Interface to Deep Storage
14
Interfaces to Deep Storage
-
Upload
insidehpc -
Category
Technology
-
view
277 -
download
2
description
In this video, Molly Rector and David Trachy from Spectra Logic describe the new DS3 RESTful interface to Deep Storage. "Designed by Spectra Logic, DS3 is the first native REST-based interface to deep storage which enables the easy archival of large amounts of bulk data. It is a data transport and communication interface that allows software clients to direct and manage "bulk" storage read or write operations to and from deep storage. DS3 utilizes the standard S3 command set plus additional commands specifically designed to optimize the transport of data objects to and from tape. These additional commands enable modern architecture frameworks to easily and efficiently communicate with massively scalable deep storage technologies. The DS3 interface utilizes the traditional REST client/server architecture and high-level PUT and GET commands to move objects to and from deep storage. The interface is media agnostic such that over long data retention periods data can be migrated from older storage types to newer ones transparently." Learn more: http://www.spectralogic.com/index.cfm?fuseaction=products.displayContent&CatID=2222
Transcript of Introducing DS3 - a RESTful Interface to Deep Storage
Interfaces to Deep Storage
REST (Representa4onal State Transfer)
• REST is the primary set of principles used on the Web • Cloud architectures are RESTful • A REST architecture uses a client and server model • REST architectures use simple commands to offer high value services
REST Behind the Scene S3 REST Command: PUT
S3 REST Command: GET
REST Behind the Scene S3 REST Command: PUT
S3 REST Command: GET
Example REST Interfaces
• Simple Storage Services (S3) • OpenStack • Atmos • Cloud Data Management Interface (CDMI)
S3 (Simple Storage Service)
Created by Amazon, S3 simplifies web-‐scale storage and compu4ng development. – Defines a cloud storage web service – Proven technology and wide adop4on (2+ trillion objects) – Many clients and client developers
S3 + + Sequen4al Media Support
Removable Media Support =
A REST/S3 interface
David Trachy Senior Director Emerging Markets
What is DS3?
• A Defined Web Service • Uses HTTP(s) • RESTful (the way the web works) • Inherently WAN compa4ble • Modeled a\er Amazon S3
Why Amazon Simple Storage Service?
• Defines a cloud storage web service • Most popular cloud storage interface – Proven technology (2+ trillion objects) – Many clients and client developers
Why We Like about S3
• Security model • Data integrity • Encryp4on support • Par4al data object restore • Meta-‐data per data object • Infinite scale-‐out
S3 Concepts
• Object – A hunk of data (ex: file, medical image) – “Put” or “Get” object from S3 – Objects are typically files from a file system – Small or large objects are supported
• Bucket – A large collec4on of objects
What did we add to get DS3?
• For sequen4al storage medium – Bulk Put, write a large number of objects – Bulk Get, read a large number of objects
• For removal storage medium – Export Bucket, Eject all pieces of media associated with a bucket
DS3 Availability
• Public announcement – Today, October 8, 2013 @ 12:30 pm MST – No embargo on DS3 or Vision informa4on
• Availability – Immediately to development community
