Sepp Stieger- Technology Watch Fujitsu Webtalk Storage...

0 © Fujitsu 2014

Webtalk Storage Trends

Sepp Stieger- Technology Watch Fujitsu

Webtalk Channel TechCommunity

New Storage Technologies– which ones are hype and reality?

How soon will these new trends impact your data center?

1 © Fujitsu 2014

Storage Trends

Trends / Disruptions

Storage System Challenges

Key Requirements

Fujitsus ETERNUS CD10000 Technology

OpenStack

2 © Fujitsu 2014

Storage Industry Distruption

Flash/ SSD

Cloud

Services

commercial-

on premise

Software

Defined

Convergence

e.g. Mobile Phones

Distributed

Architectures

Driving Transformation on Premise

Source: IDC 04/14

3 © Fujitsu 2014

IDC 3rd Platform: Opportunities at the intersection of Mobile, Cloud, Social and Big Data

From 2013 through 2020, 90% of IT industry growth will be driven by 3rd Platform technologies that, today, represent just 22% of ICT spending

Services will be build on innovative mash-ups of cloud, mobile devices/apps, social technologies, big data/analytics, and more

Data Center Transforming Converged systems will account for over 1/3 of

enterprise cloud deployments by 2016

Software-defined networks will penetrate 35% of Ethernet switching in the data center

Growing importance of mega DC, Service Source: IDC 12/12

4 © Fujitsu 2014

And the others?

This page is intentionally left blank. Please listen.

7 © Fujitsu 2014

Modular scalability of capacity and performance up to 100s of PB

Zero planned or unplanned downtime

Technology refreshes without system downtime or migration

Self-optimizing

Efficient management of exponential data growth

Lower total costs of capacity

The new unified data access (Block, File, Object, future access formats)

Key requirements for future storage architectures

10 © Fujitsu 2014

ETERNUS CD10000 Technology

11 © Fujitsu 2014

Next Generation Storage from Fujitsu

Copyright 2014 FUJITSU

ETERNUS CD10000

ETERNUS

CD10000

Unlimited

Scalability

Open

Standards

Cost

optimized

The new

unified

Immortal

System

Zero

Downtime

12 © Fujitsu 2014

Immortal System

Node1 Node2 Node(n)

+

Adding nodes

with new generation

of hardware

……… +

Adding nodes

New Node1

Non-disruptive add / remove / exchange of hardware (disks and nodes)

Mix of nodes of different generations, online technology refresh

Very long lifecycle reduces migration efforts and costs

13 © Fujitsu 2014

Node2

Zero downtime

Node1 Node2 Node(n)

Data is automatically distributed over disks and nodes in a self optimizing way

2,3,4.. data copies (replicas) protect against disk and node failures (instead of RAID)

In case of a disk or node failure lost data copies are automatically rebuilt

A very fast node interconnect enables a fast rebuild

Maintenance without any interruption

14 © Fujitsu 2014

Unlimited Scalability

Cluster of storage nodes

Capacity scales by adding storage nodes

Each nodes has integrated compute

power adding also performance

Three node types differing in capacity

and performance attributes –any mix

possible

Theoretically scalable to exabytes of data

1st version of CD10000 is released for

max. 224 nodes enabling a max. capacity

of 50 Petabyte

Basic node 12 TB Performance node 35 TB Capacity node 252 TB

15 © Fujitsu 2014

How Nature Overcomes the Central Access Bottleneck

Swarm of birds or fish Source: wikipedia

16 © Fujitsu 2014

How ETERNUS CD10000 Overcomes the Central Access Bottleneck

Disks assume the role of fish in

the storage swarm

every disk is represented by an

OSD (Object Storage Device)

Clients can directly access the

OSDs (disks, fish ) without a

central instance

19 © Fujitsu 2014

From Scalability to Reliability

CLIENT 1 VMs CLIENT 2 VMs

Redundant Client Interconnect (IP based)

Redundant Cluster Interconnect (IP based)

CLIENT 3 VMs CLIENT 4

VMs

Stor Node

Distributed Redundant Storage

Intelligent data Distribution

across all nodes and spindles

= wide striping (64KB – 16MB)

Redundancy with replica

= 2, 3 … 8

Thin provisioning

Fast distributed rebuild

Availability, Fault tolerance

Disk, Node, Interconnect

Automatic rebuild

Distributed HotSpare Space

Transparent Block, File access

Reliability and Consistency

Scalable Performance

Pure PCIe-SSD for extreme

Transaction processing

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Block Object Block Block

20 © Fujitsu 2014

From Reliability Back To Scalability


Block



CLIENT 3 VMs

Object

CLIENT 4 VMs

Block

Stor Node

Distributed Redundant Storage

Add nodes without

reconfiguration

For more Performance

For more Capacity

Automatic rebalancing

avoids hot spots PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Block

Rebalance

Rebalance

Rebalance

Rebalance

21 © Fujitsu 2014

Seamless Technology Hardware Refresh


Block



CLIENT 3 VMs

Object

CLIENT 4 VMs

Block

Add storage nodes based

on new technology

without reconfiguration

Seamlessly phase out old

technology storage nodes

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Stor Node

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

Block

Rebalance

Rebalance

Restore redundancy

Restore redundancy

22 © Fujitsu 2014

DRS HA/DR Design, up to 80km


Object

Redundant Client Interconnect (IP)

Redundant Cluster Interconnect (IP)

CLIENT 3 VMs

Block

CLIENT n VMs

Object

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Block



DR Node

(at 3rd site)

2 replica will be placed locally

The 3rd/4th replica will go to the remote site

On a site failure, all data will be available

at the remote site to minimize the RTO

Recovery Time Objective

23 © Fujitsu 2014

DRS HA/DR Design, up to 80km


Object



CLIENT 3 VMs

Block

CLIENT n VMs

Object

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Block



DR Node

(at 3rd site)

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

Stor Node

SSD

SSD

SAS

SAS

SAS

SAS

… 16 ×

Storage

Nodes

… 16 ×

Storage

Nodes

24 © Fujitsu 2014

Unified Storage Mgmt. for Block, Object, & File

Librados A library allowing

apps to directly

access RADOS,

with support for

C, C++, Java,

Python, Ruby,

and PHP

Ceph Object

Gateway

(RGW) A bucket-based

REST gateway,

compatible with

S3 and Swift

Ceph Block

Device

(RBD) A reliable and fully-

distributed block

device, with a Linux

kernel client and a

QEMU/KVM driver

Ceph File

System

(CephFS) A POSIX-compliant

distributed file

system, with a Linux

kernel client and

support for FUSE

App App

Object

Host / VM

Virtual Disk

Client

Files & Dirs

Ceph Storage Cluster (RADOS) A reliable, autonomous, distributed object store comprised

of self-healing, self-managing, intelligent storage nodes

33 © Fujitsu 2014

Software Defined Storage ETERNUS CD10000 Application Areas

34 © Fujitsu 2014

Application Areas for Scale-Out Storage

Cloud Services

Service Providers

Telcos

Scientific Computing

Universities

Research Institutions

Life Sciences, Chemical Industry, Automotive, Oil &

Gas, …

Multi Media

Video Surveillance

Broadcasting

35 © Fujitsu 2014

Cloud

Cloud Services

Focus on IaaS virtualization platforms with OpenStack /

KVM

Sync & Share

S3 compatible Object Store

Cloud service / Telco provider with fast growing

expansion strategy based on Open Souce

software

Enterprise companies with in-house cloud service and

additional focus on providing their own services for external

customers

36 © Fujitsu 2014

Scientific Computing

Universities, Research Institutions, Industries

depending on scientic computing (Life Sciences (e.g.

Genomic Sequencing), Weather Forecasting,

Climate Research, Automotive, Seismic Analysis, Oil

& Gas, …)

High performance storage for HPC

Mid to long term storage of larger data quantities

(historical data) addressing 500TB+ capacities

file system

object storage

37 © Fujitsu 2014

Multi Media

Video Surveillance

Media and Broadcasting companies

Fast online streaming services of PB`s of video and audio data

Realtime rendering and streaming process of high quality video data

Fast growing data environment and secure of consistent availability

Secure handling of PB data volumes - always available

HA- no downtime scenario

38 © Fujitsu 2014

Cloud Services

Public cloud offering with high degree of standardisation

Private cloud offering with significant portions of Open Source / Linux

Customers who are open for new infrastructure concepts (OpenStack, Ethernet

based storage access)

Customers interested in storing mid to long term data via an object oriented

interface (e.g. Amazon S3)

Scientific Computing / Multi Media

larger quantities of unstructured data

significant portions of inactive data

high throughput for sequential data

39 © Fujitsu 2014

OpenStack

40 © Fujitsu 2014

What is OpenStack?

Open Source Cloud Platform

Simple to implement

powerful implementation frameworks

Massively scalable

Fastest growing cloud platform

1000+ Developers

Biggest industry support

180+ Participating Organisations

Platin

Gold

Corporate

…

…

41 © Fujitsu 2014

OpenStack Cloud Layers

OpenStack Architecture

Physical Server (CPU, Memory, SSD, HDD) and Network

Base Operating System (Linux)

OAM

-dhcp

-Deploy

-LCM

Hypervisor

KVM (ESXi,

Hyper-V)

Compute (Nova)

Network

(Neutron) +

plugins

Dashboard (Horizon)

Billing Portal

OpenStack

Cloud APIs

RADOS

Block

(RBD)

S3

(Rados-GW)

Object (Swift) Volume (Cinder)

Authentication

(Keystone) Images (Glance)

EC2 API

Metering (Ceilometer)

42 © Fujitsu 2014

OpenStack Cloud Layers

OpenStack and ETERNUS CD10000

Physical Server (CPU, Memory, SSD, HDD) and Network

Base Operating System (Linux)

OAM

-dhcp

-Deploy

-LCM

Hypervisor

KVM (ESXi,

Hyper-V)

Compute (Nova)

Network

(Neutron) +

plugins

Dashboard (Horizon)

Billing Portal

OpenStack

Cloud APIs

RADOS

Block

(RBD)

S3

(Rados-GW)

Object (Swift) Volume (Cinder)

Authentication

(Keystone) Images (Glance)

EC2 API

Metering (Ceilometer)

Fujitsu

Storage

ETERNUS CD10000

43 © Fujitsu 2014

45 © Fujitsu 2014

Typical usage areas

Data volumes

500TB and beyond

Exponential and

unknown data growth

Typical usage areas

(Cloud) service providers

Telecommunications providers

Financial institutions

R&D, public institutions, universities Companies with large R&D activities

Public institutions with

huge document repositories

Media, broadcasting /

streaming companies

Business analytics tasks

needing fast access to large

scale (historical) data

46 © Fujitsu 2014

(Cloud) service providers

Typical usage scenarios

1 2 3 4 5

IaaS providers

Collaboration, Email, Sharepoint, DropBox services

(National) telco providers who are expanding in IT services

Large enterprises companies with in-house cloud services

IT organizations extending their DC usage in order to provide services for external customers as well

Providers with open source platform strategy (KVM, OpenStack, etc.)

47 © Fujitsu 2014


Companies, public institutions, universities etc. with large online archives 1 2 3 4 5

Need online fast access to large amounts of current and historical data,

e.g. research or legal documents, contracts

Use object data for efficient archiving

Reliable online storage for 500TB of data and beyond –when data migration to tape is not applicable

48 © Fujitsu 2014


Industry specific service providers (e.g. midsize banking, insurance companies) 1 2 3 4 5

High availability and a fast online store

Disaster recovery site for storage

Fast access for customer data and profiles

Zero downtime requirements

49 © Fujitsu 2014


Media and broadcasting/streaming companies 1 2 3 4 5

Fast online streaming services with petabytes of video and audio data / mediastores

Real-time rendering and streaming process of high quality video data

Creation of fast copies for media with high access frequency

Fast growing data environment; secure consistent availability

Secure handling of petabyte data volumes

Zero downtime requirements

50 © Fujitsu 2014


Big data/business analytics 1 2 3 4 5

Fast access to high volumes of historical data

High I/O performance to upload data into analytical servers for number crunching

Reduction of storage costs by using highly distributed data on disks instead of using SSDs

Using one storage system for production and analytics enabling so that current data can be analyzed

without slowing down productive environments

51 © Fujitsu 2014

Summary: typical usage scenarios

ETERNUS CD10000 for enterprise environments

High capacity demand

Unpredictable data growth

Keep high volumes even of historical data online

52 © Fujitsu 2014

ETERNUS CD10000 – Example Configurations

OpenStack and VDI

Video Surveillance and Broadcasting

DropBox, Synch&Share

OpenStack, VDI, DropBox, Synch&Share

53 © Fujitsu 2014

ETERNUS CD10000 example configuration (1)

Client VMs Client VMs

librbd



Client VMs

librbd

Client VMs

librbd

Perf Node

OpenStack compute, VDI

8 × Performance nodes

Raw: 8 × 6TB PCI-SSD = 48TB

Raw: 8 × 48TB SAS = 384TB

SSD as Cache pool with replica

= 3 ~16TB usable

2.5” SAS as Tiered pool with

EC ~275TB usable

Bandwidth ↑

IOPS ↑

Perf Node Perf Node Perf Node Perf Node Perf Node Perf Node Perf Node

librbd

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

Cache Pool Tiered Pool Capacity Pool

54 © Fujitsu 2014


Client Apps Client Apps

Object



Client Apps

Object

Cap.Node

Dropbox, Sync & Share

4 × Capacity nodes

Raw: 4 × 0.8TB PCIe-SSD

Journal

Raw: 4 × 240TB SATA = 960TB

3,5” HDD as Capacity pool with

EC ~680TB usable

Bandwidth →

IOPS ↓

Cap.Node Cap.Node Cap.Node

Object

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS


55 © Fujitsu 2014


Client VMs Client VMs

librbd



Client Apps

File

Client Apps

File

Perf Node

OpenStack compute, VDI

Dropbox, Sync & Share

4 × Performance,

4 × Capacity nodes

Raw: 8 × 6TB PCI-SSD = 48TB

Raw: 4 × 48TB SAS = 192TB

Raw: 4 × 240TB SATA = 960TB

SSD as Cache pool with replica

= 3 ~16TB usable

SAS 2.5” as Tiered pool with EC

~135TB usable

3,5” HDD as Capacity pool with

EC ~680TB usable

Bandwidth ↑

IOPS ↑

Perf Node Perf Node Perf Node Cap.Node Cap.Node CapaNode Cap.Node

librbd

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

PCI SSD

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

NL-SAS

SA NL-SAS TA

NL-SAS

NL-SAS

NL-SAS

NL-SAS


56 © Fujitsu 2014

Next Generation Storage from Fujitsu

ETERNUS CD10000

ETERNUS

CD10000

Unlimited

Scalability

Open

Standards

Cost

optimized

The new

unified

Immortal

System

Zero

Downtime

57 © Fujitsu 2014

Unlimited Scalability

Cluster of storage nodes

Capacity scales by adding storage nodes

Each nodes has integrated compute

power adding also performance

Three node types differing in capacity

and performance attributes –any mix

possible

Theoretically scalable to exabytes of data

1st version of CD10000 is released for

max. 224 nodes enabling a max. capacity

of 50 Petabyte

Basic node 12 TB Performance node 35 TB Capacity node 252 TB

Sepp Stieger- Technology Watch Fujitsu Webtalk Storage...

Documents

Transcript of Sepp Stieger- Technology Watch Fujitsu Webtalk Storage...