Introduction to RDMA Storage Systems
description
Transcript of Introduction to RDMA Storage Systems
![Page 1: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/1.jpg)
Mr. P. K. Gupta Sandeep Gupta 051416Roopak Agarwal 051309
![Page 2: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/2.jpg)
Road Map:
Problem StatementIntroduction Storage Systems And Storage Devices
Brief Introduction To SAN and NAS DMA and RDMA
How Does RDMA WorksRDMA Application
Future Work
![Page 3: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/3.jpg)
![Page 4: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/4.jpg)
Problem Statement
• Network Bandwidth: 40%/yr• CPU Power: x2 every 18 months• Storage Capacity x2/yr
• Memory and I/O bandwidth?
![Page 5: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/5.jpg)
A Server-to-Storage Bottleneck
Source: Brocade
![Page 6: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/6.jpg)
![Page 7: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/7.jpg)
Storage Area Network (SAN) Storage accessed over a Fibre Channel switching
fabric, using encapsulated SCSI. Block level storage system Fibre-Channel SAN IP SAN
• Implementing SAN over well-known TCP/IP
• iSCSI: Cost-effective, SCSI and TCP/IP
![Page 8: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/8.jpg)
NAS• Network Attached Storage(NAS)• Storage accessed over TCP/IP, using industry
standard file sharing protocols like NFS, HTTP, Windows Networking Provide File System FunctionalityTake LAN bandwidth of Servers
![Page 9: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/9.jpg)
![Page 10: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/10.jpg)
04/20/23 10
Motivation for RDMA
• Motivation Take advantage of the RDMA protocol suite & RNIC hardware.
When RDMA technology advances, so does iSCSI.
Zero-copy, direct data placement can be achieved with generic protocols.
• High-level goal: Run “most efficiently” on generic RNICs.
![Page 11: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/11.jpg)
Direct Memory Access (DMA)
• Direct Memory Access (DMA) is a capability provided that allows data to be sent directly from an attached device to the memory on the computer's motherboard.
![Page 12: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/12.jpg)
Remote Direct Memory Access (RDMA)
RDMA is a concept whereby two or more computers communicate via Direct memory Access directly from the main memory of one system to the main memory of another .
![Page 13: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/13.jpg)
![Page 14: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/14.jpg)
DMA RDMA
![Page 15: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/15.jpg)
What Makes RDMA Advantageous ?
• There is no CPU cache or context switching overhead needed to perform the transfer.
• Transfers can continue in parallel with other system operations.
![Page 16: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/16.jpg)
Some Objections to RDMA
• More complex API than TCP/stream interface– A new API is not required (but worthwhile for some applications)– Sockets Direct Protocol (SDP)
• Hardware vendors must all agree for approach to succeed in the market– –Extremely valid, but worth the cost
• Security concerns about opening memory on the network– Hardware enforces application buffer boundaries. – Makes it no worse than existing security problem with a 3rdparty inserting data
into the TCP data stream.
![Page 17: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/17.jpg)
RDMA Applications• Clustered Database , Application Servers : They
generate a large number of short messages , typically sent for synchronization and locking among nodes, so latency affects the degree to which they can scale.
• High performance Computing • Storage Infrastructure
![Page 18: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/18.jpg)
Future Work
• RDMA over WAN
• iSCSI extension for RDMA (iSER)
![Page 19: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/19.jpg)
iSCSI Extension for RDMA (iSER) :
• Protocol that maps the iSCSI protocol, available from RDMA
• Only data transfer in iSCSI protocol is changed
• Reduces TCP/IP overhead
![Page 20: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/20.jpg)
04/20/23 20
iSER Big picture
• Layered on RDMAP protocol.
• iSCSI still requires direct TCP access for bootstrapping (login).– Transition to RDMA
• Same iSCSI well-known port
SCSI
iSCSI
TCP
Datamover Interface
iWARPprotocolsuite
RNICRNIC
iWARPVerbs
Generic RDMA over TCP/IP
1
RDMAP
DDP
MPA
iSER
2
A TCP-based implementation is shown as an example.
![Page 21: Introduction to RDMA Storage Systems](https://reader035.fdocuments.us/reader035/viewer/2022081419/56813fef550346895daafb13/html5/thumbnails/21.jpg)
Any Queries?