Scalable Storage Management with NVMe-oF · 2019 Storage Developer Conference. © Intel. All Rights...

2019 Storage Developer Conference. © Intel. All Rights Reserved. 1

Scalable Storage

Management with NVMe-oF

Piotr Wysocki, Rafał Bogdanowicz

Intel


Agenda

Intel® RSD Overview

NVMoF management in Intel® RSD

Intel® RSD storage service evolution

Intel® RSD 2.5 Redfish/Swordfish and SPDK mappings


Code availability

All discussed code available on GitHub:

https://github.com/intel/intelRSD

It will be presented on SDC Workshop

https://github.com/intel/intelRSD


1. Source: Quantifying Datacenter Inefficiency: Making the Case for Composable Infrastructure, IDC, Document #US42318917, 2017. 2. Source: Disaggregated Server Architecture Drives Data Center Efficiency and Innovation, Shesha Krishnapura, Intel Fellow and Intel IT CTO, 2017

Today’s data center Challenges Intel®Rack Scale Design

Current Infrastructure

• Fixed ratio of compute, storage, and accelerator resources

• Expensive refresh & scale out

• Outdated software interface

• Cumbersome hardware provisioning process

Composable

Disaggregated

Interoperable

Data Center Agility, Built on Open Standards

“an industry-aligned architecture for composable, disaggregated infrastructure built on modern, open standards.”

DecreaseCosts

IncreaseAgility

50 %Efficiency

IT Operations1

35 People Hours Per

Rack Update2

45 %Utilization

Of Equipment1

https://h20195.www2.hpe.com/v2/Getdocument.aspx?docname=a00005183enw

https://www.intel.com/content/www/us/en/it-management/intel-it-best-practices/disaggregated-server-architecture-drives-data-center-efficiency-paper.html


Storage Sled

Compute

Compute

Compute

Compute

Network

Accelerator Sled

Intel Pod Manager

Composed Node 2

Orchestration

App 1 App 2 App 3

Composed Node 1

Intel® RSD Key Attributes

ComposableDisaggregated Interoperable

Single-Pane-of-Glass Management

Vendor A Hardware

Vendor B Hardware

Vendor C Hardware

Vendor D Hardware

Open Standard API

Compose hardware resources “on the fly”

Buy less up front andSave money over time

Choose the best now without vendor lock-in

OEMs with solutions based on RSD

Intel®Rack Scale Design


Storage Sled

Compute

Compute

Compute

Compute

Network

Accelerator Sled

Intel Pod Manager

Composed Node 2

Orchestration

App 1 App 2 App 3

Composed Node 1

Benefits of Disaggregation and Composability

ComposableDisaggregated

Compose hardware resources “on the fly”

Buy less up front andSave money over time

Resource poolingMaximize utilization ofhigh-value assets andimprove agility withdynamic composability

Modular RefreshIndependently scale

and upgrade resources

with better lifecycle

Management

Operational CostsImprove Power UsageEffectiveness (PUE) andstreamline operationsand HW management


Intel® RSD – Composability

Orchestration Software

App1 App1 App1Compose hardwareresources “on the fly”for specific workloads

Intel® RSD softwarefunctions include:

Resoureces DiscoveryAutomatically discover and storehardware characteristics andlocation for all your resources

Node Composition Dynamically compose compute,storage, and other resources tomeet workload specific demands

Telemetry Data Monitor data center efficiencyand detect, diagnose, and helppredict resource failures


Intel® RSD – Storage Disaggregation

Storage Sled

Compute

Compute

Compute

Compute

Network

Accelerator Sled

Disaggregation

Save money over timewith modular refresh Great scalability


Intel® Rack Scale Software stack

Rack Scale POD Manager

POD Wide AssetManagement

POD Wide Composed Nodes Management

POD Wide StorageManagement

POD Wide FabricManagement

Cloud or Data Center Manager

OpenStack3rd Party

Datacenter MgrDIY

Microsoft DIY

POD ManagerFunctionality

POD ManagerFunctionality

VMware

Rack Scale POD Management API

Pooled Systems

Compute Module

Network Module

Fabric Module

Storage Module

Storage Services

Storage Pools

Logical Volumes

Storage Endpoints

Rack Manager

Power Thermal

Chassis

Rack Scale Pooled Systems API Chassis Management APIStorage Services API

Memory Module

FPGA Module

Accelerator Module

Network Services

Network Infrastructure

Network Protocols

Networking Services

Network Services API


Intel® Rack Scale Storage Services

Core ManagementApplication

Op

era

tin

g S

yst

em

(Lin

ux* )

Asset Manager CMDB

Redfish/Swordfish API

Storage Management

Storage Assets (Physical and Logical)

Generic Assets Management Interface (JSON-RPC)

iSCSI CEPH

Targets Logical Devices Physical Devices

LVMNativeLinux

NativeLinuxDisk

Mgmt

NVMeover

Fabrics


POD Manager

Discovery

Service

Storage

Server

Client host

(initiator)

Ethernet

Management

network

Data

network

Intel® RSD components in NVMoF

1 2


Intel® RSD 2.3+

Partitions over NVMe

PSME REST Server

PSME SPDK Agent

SPDK (nvmf_tgt)

JSON-RPC

GAMI

Redfish Swordfish

Intel® RSD 2.4+

Storage Performance

Development Kit

Intel® RSD Storage Service evolution

Intel® RSD 1.2+

iSCSI Storage Service


Why SPDK?

SPDK BDEVSubsystem

NVMe-oFTarget

SPDK NVMe PCI Driver

NVMe PCIe Controller(s)

SPDK NVMe Module (NVMe BDEV)

SPDK Logical Volume Store (LVS)

SPDK Logical Volumes (LVOL BDEV)

NVMf Subsystems

Namespaces HostsThe Storage Performance Development Kit – a set of libraries providing:

• High performance

• Scalability

• Low latency

• Efficient use of CPU and memory resources

• Modularity


Service Root

Chassis Systems Fabrics

Ch1

Storage Services

CS1 NVMeoF SS1

D1

Zones

Z1

Ethernet Interfaces

NIC1

NIC2

Endpoints

Initiator1 Target1

DrivesVolumes

Storage Pools

V1 V2 SP1

resource

navigation link

Storage Service and Common Fabric Model

(Redfish + Swordfish)


/Chassis/{id}/Drives/{d-id}NVMe Block Device (BDEV)

Logical Volume Store (LVS)

Logical Volume (LVOL)

NVMf Subsystem

/StorageServices/{id}/StoragePools/{sp-id}

/StorageServices/{id}/Volumes/{v-id}

/Fabrics/{id}/Endpoints/{target-id}

/Fabrics/{id}/Endpoints/{target-id}

/Fabrics/{id}/Endpoints/{initator-id}NVMf Subsystem + Host

/Fabrics/{id}/Zones/{zone-id}

Intel® RSD 2.5 NVM over Fabrics

SPDK to Redfish/Swordfish mapping


Initial discovery for all resources (GET) get_bdevs

get_lvol_stores

get_nvmf_subsystems


Redfish/Swordfish actions to SPDK mapping


Volume create (POST) construct_lvol_bdev

snapshot_lovl_bdev

clone_lvol_bdev

destroy_lvol_bdev

Volume snapshot (POST)

Volume clone (POST)

Volume delete (DELETE)

Volume extend (PATCH) resize_lvol_bdev



snapshot_lvol_bdev


Endpoint create (POST)

Endpoint delete (DELETE)

Zone create (POST)

Zone s endpoints modification (PATCH)

Zone delete (DELETE)

nvmf_subsystem_create

nvmf_subsystem_add_listener

nvmf_subsystem_add_ns

delete_nvmf_subsystem

nvmf_subsystem_add_host

nvmf_subsystem_add_host

nvmf_subsystem_remove_host

nvmf_subsystem_remove_host




Questions?

Thank you

Scalable Storage Management with NVMe-oF · 2019 Storage Developer Conference. © Intel. All Rights...

Documents

Transcript of Scalable Storage Management with NVMe-oF · 2019 Storage Developer Conference. © Intel. All Rights...