G2M Research Multi-Vendor Webinar #3: NVMe Over Fabrics...

RESEARCH

G2M Research Multi-Vendor Webinar #3: NVMe Over Fabrics (NVMe-oF) Best PracticesNovember 6, 2018

2 © 2018 G2M Communications, Inc.. All rights reserved.RESEARCH

Webinar Agenda

9:00-9:05 Ground Rules and Webinar Topic Introduction(G2M Research)

9:06-9:25 Sponsoring Vendor presentations on topic (5 minute each) 9:26-9:39 Key Question 1 (2-minute question; 3 minutes response per

vendor) 9:40-9:41 Audience Survey 1 (2 minutes) 9:42-9:55 Key Question 2 (2-minute question; 3 minutes response per

vendor) 9:56-9:57 Audience Survey 2 (2 minutes) 9:58-10:11 Key Question 3 (2-minute question; 3 minutes response per

vendor)10:12-10:13 Audience Survey 3 (2 minutes) 10:14-10:18 Audience Q&A (8 minutes) 10:19-10:20 Wrap-Up

RESEARCH

G2M Research Introductionand Ground RulesMike HeumannManaging Partner, G2M Research


Tom SpencerSr. Director, ProductMarketingSolarflarewww.solarflare.com

Panelists

Host/Emcee: Mike HeumannManaging PartnerG2M Researchwww.g2minc.com

John KimDirector, StorageMarketingMellanoxwww.mellanox.com

Laurent HendrichsSenior ProductLine ManagerBroadcomwww.broadcom.com

Praveen MidhaDirector, MarketingServer ConnectivityMarvellwww.marvell.com

http://www.solarflare.com/

http://www.g2minc.com/

http://www.mellanox.com/

http://www.broadcom.com/



What (and Why) is NVMe over Fabric

NVMe over Fabric (NVMe-oF) extends the NVMe block storage transport, which is normally native to PCI Express (PCIe), over various data center fabrics

NVMe-oF is important because it eliminates the protocol overhead inherent in SCSI-based storage protocols such as iSCSI (Ethernet) and Fibre Channel Protocol (FC)

NVMe-oF is a standard that is managed by NVMe.org (the NVM Express trade group)


NVMe over Fabric (NVMe-oF) Definitions and Flavors

NVMe transports running over Ethernet:– NVMe over RDMA over Converged Ethernet (NVMe-oRoCE)

– NVMe over iWARP (NVMe-oiWARP)

– NVMe over TCP (NVMe-oTCP)

NVMe transports running over Fibre Channel:– NVMe over Fibre Channel (NVMe-oFC)

NVMe transports running over InfiniBand:– NVMe over InfiniBand (NVMe-oIB)

7© 2018 Mellanox Technologies | Confidential

November 2018

Mellanox NVMe-oF Market Update


Mellanox NVMe-oF Leadership - Public Customers


NVMe-oF Support Across Mellanox Product LineMore NVMe-oF Features Each Generation

ConnectX-410-100Gb (IB/Eth)

ConnectX-5E10-100Gb (IB/Eth)

PCIe Gen4

ConnectX-610-200Gb (IB/Eth)

BluefieldConnectX-5 + ARM

PCIe 4

Adapters

SoC

ConnectX-310/40/56Gb (IB/ Eth)

PCIe Gen3

2012 2018

Future Mellanox Products

Early Pre-Standard NVMe-oF

Demo NAB April

2015

5M IOPS 9us Latency

SC16 Demo

NVMe-oF Protocol Offload 2017

Single ASIC

NVMe-oF JBOF

Solution

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=w1X-qovzxxZwGM&tbnid=3QebT3qgltyrNM:&ved=0CAUQjRw&url=http://blogs.vmware.com/vcloud/tag/private-cloud&ei=pUhUUfwtytDRBcGlgPAL&psig=AFQjCNGrYZSYsCR6KNT5RxR7Nr2do4oHjg&ust=1364564402070510




NVMe-oF Innovation with Protocol OffloadD

DR

4D

DR

4

PCIe Switch

NVME SSD

NVME SSD

Initiator x86

ConnectX-5

Initiator x86

ConnectX-5

DD

R4

DD

R4

PCIe Switch

NVME SSD

NVME SSD

Initiator x86

ConnectX-5

Initiator x86

ConnectX-5

SOC

6M IOPs, 512B block size 2M IOPs, 4K block side 50% CPU utilization ~15usec latency (without SSD)

8M IOPs, 512B block size 5M IOPs, 4K block side 0.01% CPU utilization ~5usec latency (without SSD)

ConnectX & Intel CPU no offload NVMe-oF offloadTarget Target

2 100Gb Initiators 2 100Gb Initiators


Summary

Mellanox is a leader in NVMe-oF technology with ten plus customers already in production and tens more introducing their product in the near termOur products have supported NVMe-oF from its early beginnings

over RoCE and IB protocols and will soon add TCPWe are continuing to innovate new features for NVMe-oF like our

protocol offload and single ASIC JBOF solutions


Thank You

RESEARCH

BroadcomEliot RosenDirector of Product Marketingwww.broadcom.com


14 Broadcom Proprietary and Confidential. © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.|

The promise of NVMeoF

Perfo

rman

ce

Scalability

FC

NVMe

SAS/ SATA

iSCSI

NVMeoF

Direct Attach Ethernet


NVMeoF adoption status

Mature Silicon Hardware ecosystem

58800

Software ecosystem Early adopters

2016-18: Standardization and ecosystem development

2019: first sizeable NVMeoF deployments

2020: NVMeoF technology of choice for Ethernet networks

NVME AFAAI/Deep Learning

Composable Storage

SPDK


NVMeoF Architectures

Enterprise Cloud

BacksideFabric

Front sideFabric

NVMeJBOF/FBOF

StorageControllers

APPs

APPs

APPs

NVMe Scale Out Expansion

NVMe-oF NVMe-oF

Storage sideFabric

APPs

APPs

APPs

Rack Scale Disaggregation- HyperScale

NVMe-oF

Blocks ofStorage

Blocks ofCompute

Front sideFabric

EnterpriseArrays

APPs

APPs

APPs

Traditional SAN

NVMe-oF


Storage Initiator• Offload services from x86

– Security, Erasure/RAID, Compression etc.• Manage storage separate from host

End to End NVMe-oF Solutions based on NetXtreme E and S Series

Storage Target• Provides efficient integrated solution for

NVMeoF• Models for enterprise scale up and

cloud scale out

FABRICRoCE and TCP/IP

S SeriesSmartNIC

E SeriesPerformance RNIC

or

JBOF Architecture

Storage

PCIe

Sw

itch

NVMe

NVMe

NVMe

NVMe

Stingray™1x100G

Stingray™1x100G

Scale Out Architecture

Con

nect

or

Stingray™

RESEARCH

SolarflareTom SpencerSr. Director, Product Marketingwww.solarflare.com

http://www.solarflare.com/

19 © 2018 Solarflare Communications, Inc. Confidential & Proprietary

Five Reasons Why NVMe Over TCP is Cool

Tom Spencer | Senior Director, Product MarketingSolarflare


NVMe over Fabrics

NVMe-oTCPPlugin

Home


OptaneLatency~10us

0

5

10

15

20

25

30

35

40

45

512 4096

Late

ncy

us

Block Size

LinuxKernelNVMe

SPDK

SPDK+

Onload

LinuxKernel

SPDK

SPDK+

Onload

44%Lower

35%Lower

54%Lower

28%Lower

1. Performance


98% of installed servers connected with TCP

2% of installed servers connected with RDMA

2. Ease of Deployment


3. Runs on Legacy Networks


4. Ubiquity


5. TCP Has Addressed Congestion

BeforeAfter


5 Reasons Why NVMe-oF TCP is Cool

1. Performance

2. Ease of deployment

3. Runs on legacy networks

4. Ubiquity

5. TCP has addressed congestion

RESEARCH

MarvellPraveen MidhaDirector of MarketingServer Connectivity BUwww.marvell.com

http://www.marvell.com/

M a r v e l l C o n f i d e n t i a l28

NVMe Server Software

Server Transport Abstraction

FibreChannel Infiniband FCoERoCEv2 iWARP

Storage Transport Abstraction

NVMe SSDs

TCP

Scaling our NVMe Requires a (Real) Network

Many options, plenty of confusion

Fibre Channel is the transport for the vast majority of today’s all flash arrays

FC-NVMe Standardized in Mid-2017

RoCEv2, iWARP and InfiniBand are RDMA-based but not compatible with each other

NVMe-oF RDMA Standardized in 2016

FCoE fabric is an option

NVMe/TCP - making its way through the standards process


Enabling NVMeoTCP

NVMeoF alternative to iSCSI

NVMe-oF alternative to iSCSI for Storage Systems with PCIe NVMe SSDs

More efficient End-to-End operations as SCSI to NVMetranslations are eliminated

Co-Existence with NVMeoF

Co-exists with other NVMeoFtransport mechanisms

Transport selection may be based on h/w support and/or policy

No Forklift Offload

Enables NVMe-oF I/O operations in existing IP Datacenter environments

Software-only NVMe Host driver with NVMe-TCP transport

Offloaded NVMe-TCP for Target applications


1 2

3

4

5

Universal RDMA + ToE

Ultimate in choice and investment protection with concurrent support for RoCE, RoCEv2 and iWARPand full offload for NVMe/TCP

FC-NVMe

Industry’s most accepted storage transport, now enhanced for NVMe.

Broad Spectrum of Ethernet and Fibre Channel

Connectivity16G and 32GFC , 10/25/40/50GbE

to host the most demanding Storage workloads and deliver

scalability

Seamless Storage Migration

Concurrent offload for NVMe-oF, iSER, iSCSI and FCoE for

Ethernet and FCP and FC-NVMe for Fibre Channel

enables seamless upgrade paths to next gen storage

End to End SolutionsInitiator and Target mode solutions, including SPDK that leverage years of storage and storage networking experience

NVMe-oF

Marvell NVMe-oF Solution


NVMe-oF: Making the “Well-Informed” Choice?

Not “just” about “fabrics” performance Use CasesCulture and Install Base


Panel Question #1

Even though all of the major storage companies now support NVMe media in their AFAs, NVMe over Fabric is still in its infancy in AFAs. What do you think it will take for NVMe-oF to take off in AFAs?– Mellanox– Broadcom– Solarflare– Marvell


Audience Survey Question #1

Has your organization deployed or are you planning to deploy NVMe-oF based storage in the near future? (check one):

• We currently have NVMe-oF based storage deployedin our organization: %

• We are planning on deploying NVMe-oF based storage within the next 12 months: %

• We are planning on deploying NVMe-oF based storage between 12 months and 24 months from now: %

• We are currently evaluating NVMe-oF based storage ,but we have not made a decision to deploy them yet: %

• We currently do not have a use case to deployNVMe-oF based storage in our organization: %

• I am unsure of my organization’s plans to deployNVMe-oF based storage: %


Panel Question #2

What do you think the primary applications with be that utilize NVMe-oF? When do you expect to see widespread deployment of these applications on NVMe-oF?– Broadcom– Solarflare– Marvell– Mellanox



What are your primary concerns when considering deployment of NVMe-oF (check all that apply)?

• We aren’t sure that we need NVMe-oF’s performance advantagesin the near future %

• Cost of NVMe-oF vs current storage networking solutions %

• Unsure of what use cases NVMe-oF makes sense for %

• NVMe-oF technology immaturity (standards, interoperability, etc.) %

• NVMe-oF ecosystem immaturity (arrays, switches, managementapplications, etc.) %

• Other %


Panel Question #3

How important will “smart NICs” be for NVMe-oF, and where are they best used?– Solarflare– Marvell– Mellanox– Broadcom



What use cases do you see NVMe-oF utilized for in your organization over the next twelve months? (check all that apply):

• Databases/database applications: %

• Unstructured data/object storage: %

• Real-time analytics: %

• High-performance computing and modeling: %

• DevOps: %

• Other %

• None: %


Audience Q&A

RESEARCH

Thank You For Attending

RESEARCH

G2M Research Multi-Vendor Webinar #3: NVMe Over Fabrics...

Documents

Transcript of G2M Research Multi-Vendor Webinar #3: NVMe Over Fabrics...