17/01/13

Storage @ ScaleQuest for " My#ical Silver Bullet

Wednesday, 30 January 131. Introduce ourselves2. Tell what we do3. Give rundown on flow4. Why storage is a key piece

Stora% &sues & Solutions...

Wednesday, 30 January 13Background: Meta:! - Seeming non-overlap between processing and delivery - Meta-management! ! - Versioning! ! - State/lifecycle! ! - Tagging/Variants Backend:! - Different QoS and SLAs:! ! - Processing needs high writes! ! - Delivery needs high reads! ! - Archival needs cheap! - File-size varianceNeed for separating the meta and storage - our meta >> block level metaSolution: Storage as a framework

Self-healing*

Support of various content types

(Video, music, text etc etc)

Scales to 100s of petabytes

Adaptive read/writescaling*

Supports 1000s of Concurrent clients

Metadata Management & Versioning

QOS & SLA Policy management

One Stora% Solution to rule #em all...

We will not utter any more of Mordor's dark language here....

Ops & Admin Tools

Wednesday, 30 January 13Problem areas

1) Metadata management - Left Side2) Storage - Right side

Clients

Stora% Sy'em in Shir(..

Metadata Server

Storage Service

Client Client Client

NFS MountHTTP Service

HTTP API

ClientClient

Wednesday, 30 January 131) Metadata server

1.1) Versioning 1.2) Tagging

1.3) Lifecycle (Archive, Delete..., )2) Storage Service

2.1) Deployed over ZFS, OpenSolaris, Storage Controller with 16 RAID zones 2.2) Manage files over computable layouts in filesystem

2.3) Duplicate detection & de-duplication3) Clients

3.1) NFS mount 3.2) HTTP client for path transparency

4) Problems 4.1) Capacity scaling

4.2) HA 4.3) Coupling with client systems/applications

Handoff to phaneesh

Stora% Sy'em in Rivendell...

Metadata Servers

HTTP Service

Clients

Client Client Client ClientClient

Storage Infrastructure

HTTP API

Storage Service 1

Storage Service 2

Storage Service 3

Storage Service 3

Optimized for READ

Optimized for WRITE

Optimized for CAPACITY

Optimized for X

Discovery Service

Read Cache Read Cache Read Cache Read Cache Read Cache Read Cache Read Cache

HTTP READ

HTTP WRITE

QOS

Wednesday, 30 January 131) Discovery Service

1.1) Publish capabilities & QOS characteristics 1.2) Maintain availability information

1.3) Leader election (Most optimal for queries returning multiple instances)2) Metadata Server

2.1) Use discovery metadata: QOS3) Storage Infrastructure

3.1) Hybrid 3.2) Optimized for a specific application, usecase, operation

4) Read cache - LRU, Top X files

Storage Infrastructure

StorageStats Query

Palantir Stone sees it all...

Policy & QOS

Metadata Repository

StorageAPI

Discovery Service

Versions

Metadata

Lifecycle

Storage Service 1 Storage Service n

Clients

Client Client Client ClientClient

Metadata

Storage Stats

Actual I/O

Storage System Query

Service Request Service response

Wednesday, 30 January 131) Metadata - Versioned

2) Lifecycle tracking3) Policies -> Referenced/managed with metadata

Discuss call flow

1) Write2) Read

3) MoveQOS -> Write/Move (Lifecycle change)

Read -> Will use QOS applied during Write/Move

Every#ing & commo)ty in Rivendell...

7

X X X•Quad Core•8 GB or 16 GB RAM•2T X 12 SATA

•Quad Core•8 GB or 16 GB RAM•2T X 12 SATA

•Quad Core•8 GB or 16 GB RAM•2T X 12 SATA

Wednesday, 30 January 13

1) Easily replaceable2) Optimized configurations for specific operations, applications3) Economics @ scale

Meanwhile In Dexter's lab...

•5 Node Setup (2 Clients, 3 Storage Nodes)•Desktop grade machines•Connected to a 1G 8-port switch•Various deployment/configurations•Standard read/write load tests using dd, iozone etc..

Subjects•GlusterFS•Ceph

Wednesday, 30 January 131) GlusterFS

1.1) Distributed mode characteristics 1.2) Distributed-replicated

1.3) Striping - Read charecteristics, Write charecteristics

2) Ceph??

Self-healing*

Support of various content types

(Video, music, text etc etc)

Scales to 100s of petabytes

Adaptive read/writescaling*

Supports 1000s of Concurrent clients

Metadata Management & Versioning

QOS & SLA Policy management

One Stora% Solution to rule #em all...

We will not utter any more of Mordor's dark language here....

Ops & Admin Tools

Wednesday, 30 January 131) Problems that we need not solve

2) Solutions that are generic to LOB3) Specific to digital content

Questions?

Wednesday, 30 January 13

Wednesday, 30 January 13