Eleos: Exit-Less OS Services for SGX Enclaves

Meni OrenbachMarina MinkinPavel Lifshits

Mark Silberstein

Accelerated Computing Systems Lab

Haifa, Israel

22 May@Systor' 2017 Meni Orenbach, Technion 2

What do we do?Improve performance: I/O intensive & memory demanding SGX enclaves

Why?Cost of SGX execution for these applications is high

How?In-enclave System Calls & User Managed Virtual Memory

ResultsEleos vs vanilla SGX

2x Throughput: memcached & face verification serversEven for 5x available enclave memory

Available for Linux, Windows*

(*) Without Eleos, these applications crash in Windows enclaves

● Background● Motivation● Overhead analysis● Eleos design● Evaluation

SGX enclaves are already here!

● Secured execution environment● Reversed sandbox● Small TCB● Private code & data

– Confidentiality

– Integrity

– Freshness

● Only CPU is trusted

Operating system

ApplicationEnclave Enclave

– Confidentiality

– Integrity

– Freshness

Operating system

– Confidentiality

– Integrity

– Freshness

Operating system

– Confidentiality

– Integrity

– Freshness

Operating system

– Confidentiality

– Integrity

– Freshness

Operating system

– Confidentiality

– Integrity

– Freshness

Operating system

Lets look atHow to secure server applications with enclaves

Background: Lifetime of a secured server

Untrusted (Host & OS) Trusted (Enclave)

Untrusted memoryUnsecured access

Dedicated SGX memLimited to: 128 MB

Secured access

Wait for networkrequests

Hostapp

Decrypt requests

Enter enclave

Hostapp

Decrypt requests

Enter enclave

Process requests

Hostapp

Decrypt requests

Enter enclave

Process requests

Hostapp

Encrypt responses

Send responses

Decrypt requests

Enter enclave

Process requests

Exit enclave

Hostapp

Encrypt responses

SGX enclaves should be fast

● ISA extensions● Implemented in HW & Firmware● Same CPU HW● In-cache execution suffers no overheads

SGX enclaves should be fast

● ISA extensions● Implemented in HW & Firmware● Same CPU HW● In-cache execution suffers no overheads

However...

Executing a Key-Value Store in enclave is slower

64 MB 512 MB0

Memory footprint

Throughput: Slowdown factor

64 MB 512 MB0

Memory footprint

Crashesin Windows

Send responses

Overhead analysis

Enter enclave

Exit enclave

Hostapp

Decrypt requests 150 cycles/32B

Process requests *100 cycles/32B

Encrypt responses *150 cycles/32B

Overhead analysis

Enter enclave

Exit enclave

Hostapp

Send responses

Enter enclave

Exit enclave

Hostapp Decrypt requests

150 cycles/32B

~3,300cycles

Overhead analysis

Enter enclave

Exit enclave

Hostapp

Send responses

Enter enclave

Exit enclave

150 cycles/32B

Process requests*100 cycles/32B

Encrypt responses*150 cycles/32B

~3,300cycles

~3,800cycles

Overhead analysis

Enter enclave

Exit enclave

Hostapp

Send responses

Enter enclave

Exit enclave

150 cycles/32B

~3,300cycles

~3,800cycles

Exits causes indirect costs:1.5X – 5X slower execution

FlexSC [OSDI'10] syscall analysis

Overhead analysis

Enter enclave

Exit enclave

Hostapp

Send responses

Enter enclave

Exit enclave

150 cycles/32B

~3,300cycles

~3,800cycles

Exits causes indirect costs:1.5X – 5X slower execution

FlexSC [OSDI'10] syscall analysis

Eleos does better!

64 MB 512 MB0

40SGX Eleos

Memory footprint

Eleos does better!

64 MB 512 MB0

40SGX Eleos

Memory footprint

How does Eleos achieve this?

Eleos: Exit-less services

Exit-less system calls with RPC infrastructure

Exit-less SGX paging

Eleos: Exit-less services

Exit-less system calls with RPC infrastructure

Exit-less SGX paging

Background: SGX paging

System mem

SGX mem

Dedicated memoryEnclave code & data

Limited to 128 MB

System memsecret_foo():...*p = 1; SGX mem

EnclaveTrusted

Untrusted

System memsecret_foo():...*p = 1; SGX mem

HardwareAddress translation

EnclaveTrusted

Untrusted

System memsecret_foo():...*p = 1;

Encrypted

SGX mem

Page table

EnclaveTrusted

Untrusted

System memsecret_foo():...*p = 1;

Encrypted

SGX mem

Page table

Swapped-out

EnclaveTrusted

Untrusted

System mem

Faulthandler

secret_foo():...*p = 1;

Encrypted

SGX mem

Page table

Swapped-out

EnclaveTrusted

UntrustedSGX-driver

System mem

Faulthandler

secret_foo():...*p = 1;

Encrypted

Integrityvalidation

Decrypted

SGX mem

Page table

Swapped-out

EnclaveTrusted

UntrustedSGX-driver

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Fast path

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Since SGX memory is smallpaging is not as rare as in native applications

What are the overheads?

Fast path

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

SGX paging overheads

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Enclaveresume

Enclaveexit

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Enclaveresume

Enclaveexit

Indirect costs

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Enclaveresume

Enclaveexit

Overaheads: Untrusted softwaremanages enclave memory

Indirect costs

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

Enclaveresume

Enclaveexit

Overaheads: Untrusted softwaremanages enclave memory

Indirect costs

Wanted: In-enclave virtual memory management

No more exits!

Ideal in-enclave VM management

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

SGX mem

Page table

EnclaveTrusted

SGX driverUntrusted

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

SGX mem

Page table

EnclaveTrusted

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

SGX mem

Page table

EnclaveTrusted

No availablehardware

System mem

Faulthandler

secret_foo():...*p = 1;*(++p) = 2;

SGX mem

Page table

EnclaveTrusted

SoftwareAddress translation

SUVM: Secured user-space VM

System mem

Faulthandler

secret_foo():s_ptr<int> p = suvm_malloc(1024);...*p = 1; SGX mem

Page table

EnclaveTrusted

System mem

Faulthandler

secret_foo():s_ptr<int> p = suvm_malloc(1024);...*p = 1; SGX mem

Page table

EnclaveTrusted

Template class: SecuredPointer.

System mem

Faulthandler

secret_foo():s_ptr<int> p = suvm_malloc(1024);...*p = 1;

Encrypted

SGX mem

Page table

EnclaveTrusted

System mem

Faulthandler

Encrypted

SGX mem

Page table

EnclaveTrusted

Swapped-out

System mem

Faulthandler

Encrypted

SGX mem

Page table

EnclaveTrusted

Swapped-out

System mem

Faulthandler

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

Swapped-out

Integrityvalidation

System mem

Faulthandler

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

Swapped-out

Integrityvalidation

Control pathin-enclave

System mem

Faulthandler

secret_foo():s_ptr<int> p = suvm_malloc(1024);...*p = 1;*(++p) = 2;

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

System mem

Faulthandler

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

System mem

Faulthandler

Encrypted

Decrypted

SGX mem

Page table

EnclaveTrusted

Fast pathNo page table

Lookup!

Wait...Software based VM management?

Based on software address translation on GPUs, ActivePointers [ISCA'2016]

SUVM key contributions

● Multi-threaded

Compared to SGX:

Fast path: up to 20% overheads

Slow path: Eliminates costs of exits

1 Thread 4 ThreadsREAD 5.5x 7xWRITE 3.5x 5.9x

Throughput speedup

Software address translation offers new optimizations

● Customized page size● Customized eviction policy● Multi-enclave memory coordination● Write-back only dirty pages● Sub-page direct access to backing store

Virtual Machineballooning

Biometric Identity checking server

Face verificationserver

Workloadgenerator

450MB DB(5X SGX mem)

10Gb NIC

1 2 40

Eleos Native

Server threads

Biometric Identity validating serverSpeedup compared to vanilla SGX

1 2 40

Eleos Native

Server threads

1 2 40

Eleos Native

Server threads

Eleos scales better than vanilla-SGX:Saves inter-processor-interrupts

1 2 40

Eleos Native

Server threads

Eleos scales better than vanilla-SGX:Saves inter-processor-interrupts

Saturate 10Gb network

Memcached

WorkloadGenerator(memaslap)

GET( )

~75 LOC modificationfor SUVM

MemcachedGraphene LibOS [Eurosys'2014]

500MB DB(5.5X SGX mem)

10Gb NIC

1 Thread 4 Threads0

Eleos (500MB DB) vanilla SGX (20MB DB)

Server threads

MemcachedSpeedup compared to vanilla SGX (500 MB)

No SGX Faults

1 Thread 4 Threads0

Eleos (500MB DB) vanilla SGX (20MB DB)

Server threads

MemcachedSpeedup compared to vanilla SGX (500 MB)

Disclaimer: Eleos+Graphene is 3x slower than native

No SGX Faults

Take aways

● Eleos eliminates enclave exits costs● Eleos available for Windows and Linux

– Makes memory demanding applications available on Windows today

● Eleos takes a modularize approach– Memory demanding app? Link to SUVM

– I/O intensive app? Link to RPC

– Maintaining small TCB

Traditional SGX:Host-centric OS services

Enclave

Operating System

Enclave

Operating System

Getdata

Enclave

Operating System

Getdata

DataUnavailable

Enclave

Operating System

Fetch data

Getdata

DataUnavailable

Enclave

Operating System

Fetch data

Getdata

DataUnavailable

Eleos Insight:Enclave-centric OS services

Enclave

Getdata

Fetch data

In-enclaveServices

Take aways (2)

● Eleos adapts 'accelerator-centric management'– System calls: GPUfs [ASPLOS'13], GPUnet [OSDI'14]

– Virtual memory: ActivePointers [ISCA'16]

● We can do more!– Asynchronous DMA host copies

– Non-blocking enclave launches

More information at:

“SGX Enclaves as Accelerators" [Systex'16]

Thank you

Code is available at:https://github.com/acsl-technion/eleos

shmeni@tx.technion.ac.il

Backup slides

Eleos: Exit-Less OS Services for SGX Enclaves · 2017-05-18 · 22 May@Systor' 2017 Meni Orenbach,...

Transcript of Eleos: Exit-Less OS Services for SGX Enclaves · 2017-05-18 · 22 May@Systor' 2017 Meni Orenbach,...

Eleos: Exit-Less OS Services for SGX Enclaves · 2017-05-18 · 22 May@Systor' 2017 Meni Orenbach,...

Documents

Transcript of Eleos: Exit-Less OS Services for SGX Enclaves · 2017-05-18 · 22 May@Systor' 2017 Meni Orenbach,...

Farsight Enclave Rules

Nostalji Enclave

RODAS ENCLAVE

THIRU MARGADARSHI ENCLAVE

Mobile Application Design and Architecture - atlogys.com · ritika@atlogys.com Address: R-8, Nehru Enclave,New Delhi 110019, ... ð•Suspend and resume or even exit application if

Hradyansh Enclave brochure

Enclave Common API Reference - 01.org · Intel® Software Guard Extensions (Intel® SGX) Enclave Common Loader API Reference - 8 - 4 Enclave Management APIs 4.1 Enclave Get Information

2 Enclave Court

The Enclave Eco.pdf

One Enclave

Enclave Collateral

Eleos: Exit-Less OS Services for SGX EnclavesTrusted SGX driver Untrusted Enclave resume Enclave exit Overaheads: Untrusted software manages enclave memory Indirect costs 22 May@Systor'

Bahria Enclave Prices

EnClave 303

ENCLAVE CULTURAL.pdf

detroit City School District - Michigan€¦ · Exit 16 Exit 225 Exit 59 Exit 15 Exit 224B Exit 58 Exit 224 Exit 14 Exit 223 Exit 57 Exit 222B Exit 222A Exit 13 Exit 56 Exit 222 Exit

detroit City School District - Michigan · Exit 16 Exit 225 Exit 59 Exit 15 Exit 224B Exit 58 Exit 224 Exit 14 Exit 223 Exit 57 Exit 222B Exit 222A Exit 13 Exit 56 Exit 222 Exit 222

El Enclave 1

Portuguese Enclave

ACGU Enclave