Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK...
Transcript of Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK...
![Page 1: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/1.jpg)
Jim Harris
Principal Software Engineer
Data Center Group
![Page 2: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/2.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 2
By Edmund Garman from Salem, Oregon, USA - Waikiki, Honolulu PanoramaCC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=39574339
![Page 3: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/3.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 3
Agenda• Community Growth
• Feature Growth
• Looking Forward
![Page 4: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/4.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 4
SPDK Patch Commit StatisticsVersion Range Total
CommitsNon-Intel Committers
Non-IntelCommit %
v1.0 .. v16.12 1360 13 2.6%
v16.12 .. v18.01 2696 31 4.6%
v18.01 .. v19.01 3083 51 14.7%
![Page 5: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/5.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 5
SPDK Patch Commit StatisticsVersion Range Total
CommitsNon-Intel Committers
Non-IntelCommit %
v1.0 .. v16.12 1360 13 2.6%
v16.12 .. v18.01 2696 31 4.6%
v18.01 .. v19.01 3083 51 14.7%
2018+65% Non-Intel Committers
3x Non-Intel Commits
![Page 6: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/6.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 6
Community Growth
+30% YoY+218% since Jan’17
![Page 7: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/7.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
QoS
Linux nbdRDMA
RDMA
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
7
SPDK 18.04
![Page 8: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/8.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
QoS
Linux nbdRDMA
TCP
RDMA
TCP
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
8
SPDK 18.04
![Page 9: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/9.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 9
NVMe/TCPNVMe TP ratified November 2018
SPDK added TCP transport for
NVMe driver
NVMe-oF target
Supports alternative TCP stack implementations
![Page 10: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/10.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
10
SPDK 18.04
![Page 11: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/11.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 11
CryptoHardware Accelerators
• Intel Quickassist
Optimized Software
• ISA-L
DPDKFramework
NVMe bdev
NVMe Controller
Namespace
Logical Volume Store
lvolbdev
lvolbdev
lvolbdev
cryptobdev
cryptobdev
cryptobdevKEYKEYKEY
cryptodev
![Page 12: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/12.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
12
SPDK 18.04
![Page 13: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/13.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 13
CinderProvision ephemeral storage using SPDK
• NVMe-oF target
• Logical volumes
• Dynamic configuration via JSON-RPC
Kubernetes integration in progress
![Page 14: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/14.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
14
Open ChannelFTL
SPDK 18.04
![Page 15: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/15.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 15
Open Channel BLOCK FTLHost FTL enabling smart data placement
• Based on OC2.0 specification
Block FTL support added to bdev nvme module
Long term goal: Zoned Namespace API
With ZNS/OC adapters
![Page 16: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/16.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
Recently Added Features
iSCSI
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
16
Open ChannelFTL
OCF
OCFCache
SPDK 18.04
![Page 17: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/17.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 17
Open Cas FrameworkBlock Caching Library
Open Sourced in Q4’18
SPDK integration added v19.01
![Page 18: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/18.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
Recently Added Features
iSCSI
spdk-cli
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
18
Open ChannelFTL
OCF
OCFCache
SPDK 18.04
![Page 19: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/19.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 19
Spdk-cli
![Page 20: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/20.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 20
Looking ForwardUsage Models
Rack Scale Architecture
Smart NICs
New SPDK Features
![Page 21: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/21.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
SPDK Architecture
iSCSI
spdk-cli
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
21
CompressOpen Channel
FTL
OCF
OCFCache
SPDK 18.04
![Page 22: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/22.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 22
Block compression
22
DPDKFramework
compressbdev
compressdev
PMDK
lvolbdev
DPDK Framework for compression algorithms
Accelerators or Optimized SW
PMDK for persistent memory access
For compression metadata
SPDK logical volumes for storage
For realizing capacity savings
![Page 23: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/23.jpg)
SPDK, PMDK & VTune™ Amplifier Summit
Drivers
StorageServices
StorageProtocols
iSCSI Target
NVMe-oF*Target
SCSI
vhost-scsiTarget
NVMe
NVMe Devices
Blobstore
FabricsInitiator
Intel® QuickDataTechnology
Driver
Block Device Abstraction (bdev)
Ceph RBD
Linux AIO
Logical Volumes
3rd Party
NVMe
PCIe
In Progress
vhost-blkTarget
BlobFS
Integration
RocksDB
Ceph
Tools
fio
PMDKblk
virtio(scsi/blk)
VPP TCP/IP
QEMU
Cinder
QoS
Linux nbdRDMA
Crypto
TCP
RDMA
TCP
SPDK Architecture
iSCSI
spdk-cli
vhost-nvmeTarget
virtio
virtio-PCIe
vhost-user
nvme-cli
Added since 18.04
23
CompressOpen Channel
FTL
OCF
OCFCache
SPDK 18.04
io_uring
![Page 24: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/24.jpg)
SPDK, PMDK & VTune™ Amplifier Summit 24
Io_uring
Linux logo by: [email protected] Larry Ewing and The GIMP
New Linux kernel interface for asynchronous I/O
Targeted for Linux 5.1
Significant efficiency improvements compared to Linux AIO
SPDK io_uring bdev module
![Page 25: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/25.jpg)
SPDK, PMDK & Vtune™ Summit 25
Future WorkSmart Scheduling
Block Indirection
Zoned Namespaces
Replication and Parity
Even better performance!
![Page 26: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/26.jpg)
SPDK, PMDK & Vtune™ Summit 26
Call to ActionTake advantage of talks and demos to learn more about SPDK
and PMDK and VTuneTM Amplifier
Join the SPDK Community
![Page 27: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/27.jpg)
![Page 28: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/28.jpg)
SPDK, PMDK & Vtune™ Summit 28
Backup
![Page 29: Jim Harris Principal Software Engineer Data Center GroupSPDK+State+of+… · OCF OCF Cache SPDK 18.04 io_uring. SPDK, PMDK & VTune™ Amplifier Summit 24 Io_uring Linux logo by: lewing@isc.tamu.edu](https://reader036.fdocuments.us/reader036/viewer/2022071512/6132c45adfd10f4dd73aa93a/html5/thumbnails/29.jpg)