A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16]...
Transcript of A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16]...
![Page 1: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/1.jpg)
A Double-Edged Sword: Security Threats and Opportunities
in One-Sided Network Communication
Shin-Yeh Tsai Yiying Zhang
![Page 2: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/2.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A
Server
![Page 3: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/3.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
Send→ Server
![Page 4: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/4.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50Key-50
Send→ Server
![Page 5: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/5.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50Key-50
←Reply Server
![Page 6: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/6.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
SET Key-100
Key-50
User B
Send→ Server
![Page 7: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/7.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
Send→ Server
![Page 8: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/8.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
←Reply Server
![Page 9: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/9.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
![Page 10: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/10.jpg)
CPU
Traditional (Two-sided communication)
�2
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
![Page 11: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/11.jpg)
CPU
Traditional (Two-sided communication)
�2
User A
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
![Page 12: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/12.jpg)
CPU
Traditional (Two-sided communication)
�2
GET Key-50User A
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
Key-50
![Page 13: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/13.jpg)
CPU
Traditional (Two-sided communication)
�2
GET Key-50User A
User B
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
Key-50
![Page 14: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/14.jpg)
CPU
Traditional (Two-sided communication)
�2
SET Key-100
GET Key-50User A
User B
Memory
User A GET Key-50
SET Key-100
Key-50
Key-100User B
One-sided communication
MemoryCPU
Server
Key-50
Key-100
![Page 15: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/15.jpg)
�3
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 16: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/16.jpg)
�3
[ATC ’13]Pilaf
[NSDI ’14]FaRM
[SIGCOMM ’14]HERD
[SOSP ’15]DrTM
[SoCC ’17]APUS
[SOSP ’15]FaRM + Xact
[ASPLOS ’15]Mojim
[EuroSys ’16]DrTM+R
[VLDB ’16]RSI
[OSDI ’16]FaSST
[ATC ’16]Cell
[OSDI ’16]Wukong
[SoCC ’17]Hotpot
[ATC ’17]Octopus
[VLDB ’17]NAM-DB
[OSDI ’18]DRTM+H
[SOSP ’17]LITE [SOSP ’17]
KV-Direct
[FAST ’19]Orion
[SYSTOR '19]Storm
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 17: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/17.jpg)
�3
[ATC ’13]Pilaf
[NSDI ’14]FaRM
[SIGCOMM ’14]HERD
[SOSP ’15]DrTM
[SoCC ’17]APUS
[SOSP ’15]FaRM + Xact
[ASPLOS ’15]Mojim
[EuroSys ’16]DrTM+R
[VLDB ’16]RSI
[OSDI ’16]FaSST
[ATC ’16]Cell
[OSDI ’16]Wukong
[SoCC ’17]Hotpot
[ATC ’17]Octopus
[VLDB ’17]NAM-DB
[OSDI ’18]DRTM+H
[SOSP ’17]LITE [SOSP ’17]
KV-Direct
[FAST ’19]Orion
Performance[SYSTOR '19]
Storm
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 18: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/18.jpg)
�3
[ATC ’13]Pilaf
[NSDI ’14]FaRM
[SIGCOMM ’14]HERD
[SOSP ’15]DrTM
[SoCC ’17]APUS
[SOSP ’15]FaRM + Xact
[ASPLOS ’15]Mojim
[EuroSys ’16]DrTM+R
[VLDB ’16]RSI
[OSDI ’16]FaSST
[ATC ’16]Cell
[OSDI ’16]Wukong
[SoCC ’17]Hotpot
[ATC ’17]Octopus
[VLDB ’17]NAM-DB
[OSDI ’18]DRTM+H
[SOSP ’17]LITE [SOSP ’17]
KV-Direct
[FAST ’19]Orion
Performance[SYSTOR '19]
Storm
Scalability
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 19: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/19.jpg)
�3
[ATC ’13]Pilaf
[NSDI ’14]FaRM
[SIGCOMM ’14]HERD
[SOSP ’15]DrTM
[SoCC ’17]APUS
[SOSP ’15]FaRM + Xact
[ASPLOS ’15]Mojim
[EuroSys ’16]DrTM+R
[VLDB ’16]RSI
[OSDI ’16]FaSST
[ATC ’16]Cell
[OSDI ’16]Wukong
[SoCC ’17]Hotpot
[ATC ’17]Octopus
[VLDB ’17]NAM-DB
[OSDI ’18]DRTM+H
[SOSP ’17]LITE [SOSP ’17]
KV-Direct
[FAST ’19]Orion
Performance[SYSTOR '19]
Storm
ScalabilityUsability
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 20: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/20.jpg)
�3
[ATC ’13]Pilaf
[NSDI ’14]FaRM
[SIGCOMM ’14]HERD
[SOSP ’15]DrTM
[SoCC ’17]APUS
[SOSP ’15]FaRM + Xact
[ASPLOS ’15]Mojim
[EuroSys ’16]DrTM+R
[VLDB ’16]RSI
[OSDI ’16]FaSST
[ATC ’16]Cell
[OSDI ’16]Wukong
[SoCC ’17]Hotpot
[ATC ’17]Octopus
[VLDB ’17]NAM-DB
[OSDI ’18]DRTM+H
[SOSP ’17]LITE [SOSP ’17]
KV-Direct
[FAST ’19]Orion
Performance
What about Security?
[SYSTOR '19]Storm
ScalabilityUsability
RDMA
Omni-Path
NVMeOF
GPUDirect
Gen-Z
![Page 21: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/21.jpg)
Outline• Introduction and Background
• Vulnerabilities in One-Sided Communication
• Vulnerabilities in One-Sided Hardware
• Opportunities in One-Sided Communication
• Conclusion
�4
![Page 22: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/22.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
• WRITE accountability
![Page 23: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/23.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
SET Key-50
• WRITE accountability
![Page 24: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/24.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
SET Key-50
Server: Who SET the (corrupted) record?
• WRITE accountability
![Page 25: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/25.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
SET Key-50
Server: Who SET the (corrupted) record?
• WRITE accountability
• READ accountability
![Page 26: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/26.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
GET Key-100
SET Key-50
Server: Who SET the (corrupted) record?
User B
• WRITE accountability
• READ accountability
![Page 27: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/27.jpg)
Vulnerability 1: Lack of Accountability
�5
Memory
User ACPU
GET Key-100
SET Key-50
Server: Who SET the (corrupted) record?
Server: Who GET the (corrupted) record?User B
• WRITE accountability
• READ accountability
![Page 28: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/28.jpg)
Vulnerability 2: Denial of Service
�6
Attack hardwareMetadata1 Metadata2
• Hard to trace attackers
• Can overload NICs easily
![Page 29: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/29.jpg)
Attacker
Vulnerability 2: Denial of Service
�6
Attack hardwareMetadata1 Metadata2
• Hard to trace attackers
• Can overload NICs easily
![Page 30: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/30.jpg)
Attacker
Vulnerability 2: Denial of Service
�6
Attack hardwareMetadata1 Metadata2
• Hard to trace attackers
• Can overload NICs easily
![Page 31: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/31.jpg)
Attacker
Vulnerability 2: Denial of Service
�6
Attack hardwareMetadata1 Metadata2
• Hard to trace attackers
• Can overload NICs easily
![Page 32: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/32.jpg)
Discussion and Defense• Adding intermediate layer at the sender side
�7Library NIC CPU
MemoryClient
Client NIC CPU
Memory
Intermediate Layer
![Page 33: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/33.jpg)
Discussion and Defense• Adding intermediate layer at the sender side
• Enhancing SmartNIC at the receiver side
�7
SmartNIC
Client NIC CPU
Memory
Client NIC CPU
MemorySoC
Library NIC CPU
MemoryClient
Client NIC CPU
Memory
Intermediate Layer
![Page 34: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/34.jpg)
Outline• Introduction and Background
• Vulnerabilities in One-Sided Communication
• Vulnerabilities in One-Sided Hardware
• Opportunities in One-Sided Communication
• Conclusion
�8
![Page 35: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/35.jpg)
One- and Two-Sided Hardware
�9
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 36: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/36.jpg)
One- and Two-Sided Hardware
�9
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 37: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/37.jpg)
One- and Two-Sided Hardware
�9
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 38: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/38.jpg)
One- and Two-Sided Hardware
�9
1. Address mapping 2. Permission checking 3. Resource isolation
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 39: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/39.jpg)
One- and Two-Sided Hardware
�9
1. Address mapping 2. Permission checking 3. Resource isolation
Memory
CPU UserKernel
One-sided CommunicationOne-Sided
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 40: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/40.jpg)
One- and Two-Sided Hardware
�9
1. Address mapping 2. Permission checking 3. Resource isolation
Memory
CPU UserKernel
One-sided CommunicationOne-Sided
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
![Page 41: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/41.jpg)
One- and Two-Sided Hardware
�9
1. Address mapping 2. Permission checking 3. Resource isolation
Memory
CPU UserKernel
One-sided CommunicationOne-Sided
Memory
BerkeleySocket
CPU User
Kernel
Two-Sided
Memory Region 1. rkey/lkey 2. Address
![Page 42: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/42.jpg)
Vulnerability 3 - Predictable Hardware Managed Keys
�10
Virtual addr + rkey
Physical addr
PTE Translation
rkey
/lkey
Val
ue
0
1M
2M
3M
nth-MemoryRegion Registered0 1000 2000 3000 4000 5000
ConnectX-3ConnectX-4ConnectX-5
![Page 43: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/43.jpg)
Vulnerability 3 - Predictable Hardware Managed Keys
�10
Virtual addr + rkey
Physical addr
PTE Translation
rkey
/lkey
Val
ue
0
1M
2M
3M
nth-MemoryRegion Registered0 1000 2000 3000 4000 5000
ConnectX-3ConnectX-4ConnectX-5
![Page 44: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/44.jpg)
Vulnerability 3 - Predictable Hardware Managed Keys
�10
Virtual addr + rkey
Physical addr
PTE Translation
rkey
/lkey
Val
ue
0
1M
2M
3M
nth-MemoryRegion Registered0 1000 2000 3000 4000 5000
ConnectX-3ConnectX-4ConnectX-5
![Page 45: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/45.jpg)
Vulnerability 3 - Predictable Hardware Managed Keys
�10
Virtual addr + rkey
Physical addr
PTE Translation
rkey
/lkey
Val
ue
0
1M
2M
3M
nth-MemoryRegion Registered0 1000 2000 3000 4000 5000
ConnectX-3ConnectX-4ConnectX-5
![Page 46: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/46.jpg)
Vulnerability 4 - Side Channel in NICs
�11
ConnectX-5, 1KB READ request latency
Virtual addr + rkey
Physical addr
PTE Translation Pe
rcen
tile
0
10
20
30
Latency (us)0 1 2 3 4 5 6 7 8
HitMiss-PageTableEntriesMiss-MemoryRegionInfo
![Page 47: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/47.jpg)
Vulnerability 4 - Side Channel in NICs
�11
ConnectX-5, 1KB READ request latency
Virtual addr + rkey
Physical addr
PTE Translation Pe
rcen
tile
0
10
20
30
Latency (us)0 1 2 3 4 5 6 7 8
HitMiss-PageTableEntriesMiss-MemoryRegionInfo
![Page 48: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/48.jpg)
Vulnerability 4 - Side Channel in NICs
�11
ConnectX-5, 1KB READ request latency
Virtual addr + rkey
Physical addr
PTE Translation Pe
rcen
tile
0
10
20
30
Latency (us)0 1 2 3 4 5 6 7 8
HitMiss-PageTableEntriesMiss-MemoryRegionInfo
![Page 49: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/49.jpg)
Side-Channel Attacks in RDMA (Pythia, USENIX Sec ‘19)
�12
Server Machine
RDMA Network
RNIC SRAM
Main Memory CPU
Client Machine
Attacker
Client Machine
Victim
QP MR PTE
PCIePTEMRData Ac
cess
Pro
babi
lity
(%)
0
20
40
60
80
100
Timeline (ms)0 20 40 60 80 100 120 140
VictimAttacker
![Page 50: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/50.jpg)
Discussion and Defense• Generate memory registration keys cryptographically
�13
Sequential to Random
![Page 51: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/51.jpg)
Discussion and Defense• Generate memory registration keys cryptographically
• Isolate on-board resources for different clients
�13
On-boardResource
Sequential to Random
![Page 52: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/52.jpg)
Discussion and Defense• Generate memory registration keys cryptographically
• Isolate on-board resources for different clients
• Enhancing SmartNIC at the receiver side
�13
On-boardResource
SmartNIC
Client NIC CPU
Memory
Client NIC CPU
MemorySoC
Sequential to Random
![Page 53: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/53.jpg)
Outline• Introduction and Background
• Vulnerabilities in One-Sided Communication
• Vulnerabilities in One-Sided Hardware
• Opportunities in One-Sided Communication
• Conclusion
�14
![Page 54: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/54.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
![Page 55: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/55.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
ORAM READ/WRITE
![Page 56: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/56.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
ORAM READ/WRITE
![Page 57: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/57.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
ORAM READ/WRITE
![Page 58: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/58.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
ORAM READ/WRITE
![Page 59: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/59.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
ORAM READ/WRITE
![Page 60: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/60.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
One-sided READ
ORAM READ/WRITE
![Page 61: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/61.jpg)
Opportunity of One-sided Communication
�15
ORAM Access Server
One-sided READ
ORAM READ/WRITE
![Page 62: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/62.jpg)
Opportunity of One-sided Communication
�16
ORAM Access Server
K% One-sided READ
(1-K)% ORAM READ100% ORAM WRITE
![Page 63: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/63.jpg)
Opportunity of One-sided Communication
�17
ORAM Access Server
X% One-sided READ
(1-X)% ORAM READ100% ORAM WRITE
Thro
ughp
ut (K
OPS
)
0
10
20
30
K% of One-Sided READ Operations0 25 50 75 100
![Page 64: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/64.jpg)
Conclusion• Security concerns of one-sided communication
• Tradeoffs between Performance and Security
• Hardware Vendor, Software Developers, and Datacenter
�18
![Page 65: A Double-Edged Sword: Security Threats and Opportunities ... · RSI [OSDI ’16] FaSST [ATC ’16] Cell [OSDI ’16] Wukong [SoCC ’17] Hotpot [ATC ’17] Octopus [VLDB ’17] NAM-DB](https://reader035.fdocuments.us/reader035/viewer/2022081614/5fc7a20b32634908943f3e3f/html5/thumbnails/65.jpg)
Conclusion• Security concerns of one-sided communication
• Tradeoffs between Performance and Security
• Hardware Vendor, Software Developers, and Datacenter
�18