Post on 07-Oct-2020
1
Toward Scaling Hardware Security Module forEmerging Cloud Services
Juhyeng Han*, Seongmin Kim*, Taesoo Kim † , Dongsu Han
KAIST †Georgia Tech
* The first two authors contributed equally to this work.
Hardware Security Modules (HSMs)
2
• Root of trust for various key management services (KMS)• Their root keys should be stored in HSMs
• Secure physical separation and protection• Satisfies security regulation requirements such as FIPS 140-2
Host HSM
Requestcryptographic operations
Response(e.g., Digital signature)
Physical separation
Root keys
Hardware Security Modules (HSMs)
3
• Root of trust for various key management services (KMS)• Root keys should be stored in HSMs
• Secure physical separation and protection• Satisfies security regulation requirements such as FIPS 140-2
Host HSM
Requestcryptographic operations
Response(e.g., Digital signature)
Physical separation
Physical separation required(U.S. and Canadian security standard)
Tamper-evidentTamper-resistant
FIPS140-2
Root keys
Demands for Scalable Security Services
4
Microservices
Edge computing
Financial technology
Innovation in emerging cloud
industries
Increase of secure network
transactions
User-to-Service
Service-to-Service
Demands for scalable security
services
More cryptographic operations
Low latency & High throughput
Multiple user/key isolation
Problem: Limited Scalability of HSMs
5
HSMMultipleservices
Lots of requestDedicated hardware
Signing speed: 10,000 tps (RSA-2048)Price: $29,900
Network
Performancebottleneck!
Problem: Limited Scalability of HSMs
6
Expensive solution!
Many on-premises HSMsMultipleservices
Lots of request
Network
Problem: Limited Scalability of HSMs
7
Price: $1,250 per month (IBM Cloud HSM)
Multipleservices
Lots of request
Network
Cloud HSM
Expensive solution!
Expensive solution!
Problem: Limited Scalability of HSMs
8
Price: $1,250 per month (IBM Cloud HSM)
Multipleservices
Lots of request
Network
Cloud HSM
Can we efficiently scale out HSMs for key management services?
Enclave
Encryptedcode/data
SGX CPU
System MemorySGX-equipped server
Alternative Approach
9
• Leverages commodity Trusted Execution Environment (TEE) instead of HSMs[S. Chakrabarti et al. “Intel® SGX Enabled Key Manager Service with OpenStack Barbican.” arXiv preprint arXiv:1712.07694, 2017.]
Malicious OS or Hypervisor
Limitation of the Alternative Approach
10
• Leverages commodity Trusted Execution Environment (TEE) instead of HSMs[S. Chakrabarti et al. “Intel® SGX Enabled Key Manager Service with OpenStack Barbican.” arXiv preprint arXiv:1712.07694, 2017.]
Does not provide physical separation & protection
Enclave
Encryptedcode/data
SGX CPU
System MemorySGX-equipped server
Approach : Combining HSMs with TEE-based KMS
11
• Achieves cost-efficient scalability with SGX technology• Maintains security level of physical separation with HSMs• SGX enclaves and HSMs collaborate for key management
SGX-equipped server HSM
Physical separation
PCIe/Network communication
MultipleSGX EnclavesSGX CPU
instructions
Collaborative KMS
Deployment Assumption & Threat Model
12
Root-privilegedattacker
Untrusted Platform
Microservices(KMS clients)
HSM (Trusted)
Multiple SGX Enclaves
(Trusted)
Root keys(Root-of-trust)
Physical separation
KMS request
Fake Enclave
Invalid access
Challenge 1 : Scaling Performance
13
• Frequent private key operation requests to HSMs can incur performance bottleneck.
Untrusted Platform
Microservices(KMS clients)
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
Physical separation
Challenge 1 : Scaling Performance
14
• Frequent private key operation requests to HSMs can incur performance bottleneck.
Untrusted Platform
① Frequent short-living authentication requests
Microservices(KMS clients)
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
Physical separation
Heavy private key operations
Challenge 1 : Scaling Performance
15
• Frequent private key operation requests to HSMs can incur performance bottleneck.
Untrusted Platform
Microservices(KMS clients)
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
Physical separation① Frequent short-living authentication requests
Heavy private key operations
Performance bottleneck
Challenge 1 : Scaling Performance
16
• Frequent private key operation requests to HSMs can incur performance bottleneck.
Untrusted Platform
② Symmetric key operation requests
Microservices(KMS clients)
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
Physical separation① Frequent short-living authentication requests
Heavy private key operations
Challenge 2 : Validation between Enclaves and HSMs
17
Untrusted Platform
RequestResponse
• KMS clients, SGX enclaves and HSMs should trust each others• Lack of validation mechanism between SGX enclaves and HSMs
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
Physical separation
Microservices(KMS clients)
HSM
Multiple SGX Enclaves
Root keys(Root-of-trust)
RequestResponse
Physical separation
18
• KMS clients, SGX enclaves and HSMs should trust each others• Lack of validation mechanism between SGX enclaves and HSMs
Trust?
Fake Enclave
MITM
Trust?
Invalid access
Untrusted Platform
Microservices(KMS clients)
Challenge 2 : Validation between Enclaves and HSMs
Design Goals of ScaleTrust
19
1. Scalable performanceEnhances performance by scaling out and does not make an HSM a performance bottleneck
2. Cost-effectiveness Cost-efficiently scales out for key management services
3. Security Preserves a chain-of-trust from an HSM to clients
Design Overview
20
Untrusted Platform
Physical separation
HSM
Root key pair(Root-of-trust)
Untrusted Platform
Trusted Host
BootstrappingEnclave
KMS EnclavesMicroservices
(KMS clients)
Design Overview
21
Untrusted Platform
Physical separation
HSM
Root key pair(Root-of-trust)
Untrusted Platform
Trusted Host
BootstrappingEnclave
KMS EnclavesMicroservices
(KMS clients)
KMS request
Design Overview
22
Untrusted Platform
Physical separation
HSM
Root key pair(Root-of-trust)
Untrusted Platform
Trusted Host
BootstrappingEnclave
Microservices(KMS clients)
KMS Enclaves
PKCS#11 API calls
KMS request
Design Overview
23
Untrusted PlatformUntrusted Platform
Trusted Host
BootstrappingEnclave
Microservices(KMS clients)
KMS Enclaves
KMS request
Physical separation
Root key pair(Root-of-trust)
Derived keys
PKCS#11 API calls
HSM
Design Overview
24
Untrusted PlatformUntrusted Platform
KMS EnclavesMicroservices
(KMS clients)Derived keys
PKCS#11 API calls
Root public key
KMS request
Physical separation
Root key pair(Root-of-trust)
HSM
Offline key deployment
(Trusted)
BootstrappingEnclave
Trusted Host
Secure bootstrapping
25
Secure bootstrapping ① : An HSM generates
a root key pairs
Microservices(KMS clients)
KMS Enclaves
Untrusted Platform
BootstrappingEnclave
Trusted Host Physical separation
HSM
Root key pair(Root-of-trust)
Secure bootstrapping
26
Secure bootstrapping ② : The HSM shares root public
key with bootstrapping enclave
Microservices(KMS clients)
Offline key deployment
(Trusted)
KMS Enclaves
Untrusted Platform
BootstrappingEnclave
Trusted Host Physical separation
HSM
Root key pair(Root-of-trust)
Secure bootstrapping
27
Secure bootstrapping ③ : The bootstrapping enclave
attests KMS enclaves
RemoteattestationBootstrapping
Enclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Untrusted Platform
Secure bootstrapping
28
Secure bootstrapping ④ :The bootstrapping enclave
shares the public key
Key deploymentBootstrapping
Enclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Untrusted Platform
Secure bootstrapping
29
Secure bootstrapping ⑤ : The KMS enclaves attest the
HSM and build secure channels
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Secure channel
Untrusted Platform
Secure bootstrapping
30
Secure bootstrapping : A fake enclave cannot build a secure channel with the HSM
Fake Enclave
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Untrusted Platform
Remoteattestation
Attestation on SGX Instances
31
Attestation on enclaves ① : When the client first request
to KMS server, it allocates KMS enclaves for the client.
KMS request
Allocated enclaves
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
Untrusted Platform
KMS Enclaves
Attestation on SGX Instances
32
Attestation on enclaves ② : After a new KMS enclave is created, the bootstrapping
enclave attests it.
RemoteattestationBootstrapping
Enclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Secure channel
Untrusted Platform
Attestation on SGX Instances
33
Attestation on enclaves ③ : Also, the client performs
remote attestation to verifythe KMS enclave.
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
PKCS#11 API calls
Secure channel
Untrusted Platform
Remote attestation
Attestation on SGX Instances
34
Attestation on enclaves ④ : After the remote attestation,
the client sends encrypted KMS requests to the enclave
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
Untrusted Platform
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
PKCS#11 API calls
Secure channel
Remote attestationSecure channel
KMS request
Attestation on SGX Instances
35
Untrusted PlatformAttestation on enclaves : A fake enclave cannot build a communication channel with
the client
Fake Enclave
Remoteattestation
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
PKCS#11 API calls
BootstrappingEnclave
Trusted Host
Microservices(KMS clients)
KMS request
Hierarchical Design for Scaling
36
Root-of-trust
Scalable security services
KMS requests
Physical separation
KMS Enclave
Root key pair(root-of-trust)
HSM
Microservices(KMS clients)
Hierarchical Design for Scaling
37
Root-of-trust
Scalable security services
KMS requests
Physical separation
KMS Enclave
Root key pair(root-of-trust)
HSM
Derivedkeys
Microservices(KMS clients)
Root key operation requests
Hierarchical Design for Scaling
38
Microservices(KMS clients)
Physical separation
KMS Enclaves
Root-of-trust
Root key operation requests
Scalable security services
KMS requests
Root key pair(root-of-trust)
Frequent cryptographic requests
Derivedkeys
HSM
JSON Web Token (JWT) for Microservice
39
JWT client
JWT auth server
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
JSON Web Token (JWT) for Microservice
40
JWT clientRefresh token request
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
JWT auth server
JSON Web Token (JWT) for Microservice
41
JWT clientRefresh token request
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
JWT auth server
Creates and signs the refresh token
Refresh token key pair
R
JSON Web Token (JWT) for Microservice
42
JWT clientAccess token request
JWT auth server
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
Verifies the refresh token and sends a new access token
R
A
R
Refresh token key pair
RWeb server
A
Application Case Study : JWT Management
43
JWT client
Physical separation
KMS Enclaves
Refresh token key pair
HSM
JWT auth serverRefresh token request
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
Refresh token request
Application Case Study : JWT Management
44
JWT client
Physical separation
KMS Enclaves
Refresh token key pair
HSM
JWT auth serverRefresh token request
R
Public key of refresh token
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
Refresh token request
R
Application Case Study : JWT Management
45
JWT client
Physical separation
KMS Enclaves
Refresh token key pair
HSM
JWT auth server
Validate the refresh token
Access token requestR
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
Application Case Study : JWT Management
46
JWT client
Physical separation
KMS Enclaves
Refresh token key pair
HSM
JWT auth server
Validate the refresh token
Access token request
A
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
Application Case Study : JWT Management
47
JWT client
Physical separation
KMS Enclaves
Refresh token key pair
HSM
JWT auth serverAccess token request
Refresh token request
A
R : Refresh token(Lifetime: few hours)
: Access token(Lifetime: more than a week)
A
R
R
Preliminary Evaluation
48
SGX-equipped server
SoftHSM
• Environment setup• CPU: Quad-core Intel Xeon E3-1280 v6 (SGX-enabled)• Intel SGX Linux SDK version 2.5• We use SoftHSM to emulate an HSM device.• Each enclave and HSM performs the same SHA-256 with RSA-2048 signing
Enclave calls
PKCS#11 API calls
RSA key pair
RSA key pair
KMS Enclaves
JWT client
Token requests
Preliminary Evaluation: Latency Improvement
49
• Scaling out KMS enclaves for latency improvement
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500
CD
F
Response time (ms)
0.95
HSM
1 Enclave
2 Enclaves
4 Enclaves
Preliminary Evaluation: Cost-effective Scaling
50
Approach for KMS Equipment Performance
(RSA-2048 sign) Price tps/$
ScaleTrust(on-premisesSGX machine)
Xeon E3-1280 v6 CPU (Quad, 4.2 GHz) 3,600 tps $500 7.2
On-premises HSMs-only Luna SA A790 HSM 10,000 tps $29,900 0.33
ScaleTrust(in Azure cloud)
Xeon E-2176G CPU (Quad, 4.7 GHz)
> 3,600 tps(estimated)
$500per month
> 7.2for a month
Cloud HSM (Azure HSM) Luna SA A790 HSM 10,000 tps
$5000+ $3,541
per month
1.17 for a month
*tps = transactions per second
Future work
51
• Evaluation with a real HSM device
Physical separation
HSM
Root key pair(Root-of-trust)
KMS Enclaves
Untrusted Platform
PKCS#11 API calls
Future work
52
• Physical separation by Intel VCA (SGX card)
Intel VCA card
SGX node1
Enclave1
SGX node2 SGX node3
Enclave1
Enclave1Host
SGX node manager
MMIO region
Enclave2
Enclave2
PCIe communication
Conclusion
53
• We explore new design space to address the limitedscalability of HSMs by combining TEE technology
• ScaleTrust preserves chain-of-trust from an HSM to clients
• ScaleTrust utilizes HSMs and SGX enclaves in a hierarchical model to relieve the burden of HSMs
• Our JWT case study shows that ScaleTrust can be applied to key management for microservices.
Thank You