Netslice : Enabling Critical Network Infrastructure with Commodity Routers

Netslice: Enabling Critical Network Infrastructure with Commodity

RoutersProf. Hakim Weatherspoon, Cornell University

Joint with Tudor Marian, Ki Suh Lee

TRUST Autumn 2010 Conference, Stanford UniversityNovember 10, 2010

Commodity Datacenters Datacenters are becoming a commodity Unit of replacement

Datacenter in a box: already set up with commodity hardware & software (Intel, Linux, petabyte of storage)

Plug network, power & cooling and turn on typically connected via optical fiber may have network of such datacenters

Commodity Datacenters

Titan tech boom, randy katz, 20083 11/10/2010Critical Network Infrastructure, by Hakim

Weatherspoon

IBM Visit, Critical Infrastructure, by Hakim Weatherspoon

Network Of Globally Distributed Datacenters Cloud Computing—Datacenters interconnected via fiber

Long Fat Networks (LFN) or λ-networks Packet processors and extensible routers—middleboxes

Increase functionality, performance, reliability, and security of network E.g. DPI, IDS, PEP, protocol accelerators, overlay routers, multimedia

servers, security appliances, and network monitors, etc

packet processor

middleboxes 11/10/20104

Network Of Globally Distributed Datacenters Packet processors

Maelstrom [NSDI’08,TONS’10], SMFS [FAST’09] FEC,TCP-Split, De-duplication, Network-sync OS abstractions for packet processing in user-space

Netslice Lambda networks

Cornell NLR Rings [DSN10] SDNA/BiFocals [IMC’10]

Maelstrom

Netslice

SDNA/Bifocals

TCP-SplitSMFS

Cornell NLR Rings

5 Critical Network Infrastructure, by Hakim Weatherspoon

11/10/2010

Network Of Globally Distributed Datacenters Packet processors

Maelstrom [NSDI’08,TONS’10], SMFS [FAST’09] FEC,TCP-Split, De-duplication, Network-sync OS abstractions for packet processing in user-space

Netslice Lambda networks

Cornell NLR Rings [DSN10] SDNA/BiFocals [IMC’10]


11/10/2010

Challenges

Large traffic volume processed / second (10Gb/s) Typical packet processors realized in hardware Trade off flexibility, programmability for speed

Goal: Improve datacenter communication Packet processors and extensible routers Abundance of commodity servers readily available

Commodity hardware

SoftwareProprietaryspecializedhardware


11/10/2010

Takeaway Raw socket cannot take advantage of

multicore/multiqueue Need new OS abstraction to take advantage of

parallelism 9.7G 9.7G9.7G

2.25G


11/10/2010

Outline—Packet Processing Abstractions

The case for user-space packet processors

What’s wrong with the raw socket in multicore environment? Hardware and software overheads Contention and lack of application control of resources

Need new OS abstraction to take advantage of parallelism Netslice

Evaluate Group intruduction Conclude


11/10/2010

The Case Against Low-level Packet Processors

Idiosyncrasies of memory allocator Small virtual address spaces Inability to swap out pages Limit on contiguous memory chunks

Execution contexts and preemptive precedence Interrupt, bottom half, task/user context

Synchronization primitives Tightly coupled with execution contexts (e.g. can I

block?) Lack of development tools Lack of fault isolation

A bug in the kernel is lethal

Hardware

Operating SystemKernel

NetworkStack

ApplicationUser-space

Application


11/10/2010

Opportunity: exploit hardware parallelism

Overheads Contention, contention, contention! Memory wall OS Design Overheads

System calls Context switches Scheduling Blocking

High-level Packet Processors: Where Have All My Cycles Gone?

CPU

Memory


11/10/2010

Contention: Amdahl’s Law Bounds maximum expected parallelism speedup

Fraction P of a program parallelized to run on N CPUs The speedup is

Serial fraction(1-P)

Parallel fraction (P)

Program:

1Serial execution + Parallel execution=


11/10/2010

Contention: Cache-coherent Architecture Effects of cache coherency & memory accesses

Cores read and write blocks of data concurrently Commodity system: Xeon X533 with 4MB L2 cache


11/10/2010

Contention: Peripheral NIC Slow cores fast network interface cards (NICs)

More cores exhibit contention and overheads Hardware transmit/receive multi-queue support

tx/rx queues


11/10/2010

Software overheads in theConventional Network Stack Raw socket: all traffic from all NICs to user-space Hardware and software are loosely coupled Applications have no (end-to-end) control over

resources

tx/rx queues

NetworkStack

ApplicationRaw socket

NetworkStack

NetworkStack

NetworkStack

NetworkStack

NetworkStack

NetworkStack

NetworkStack

NetworkStack

ApplicationApplication

Application



ApplicationNetwork

Stack


11/10/2010

Software overheads in theConventional Network Stack API too general, hence complex network stack

too bloated Raw sockets, end-point sockets, files: all the same Path taken by a packet is unnecessarily expensive Hides information from applications

Limited functionality: least common denominator API Inefficient API: issue one system call per packet

NetworkStack

ApplicationRaw socket

Application ApplicationTCP socket UDP socket

AF_UNIX socketApplication

File

API

TCP socketApplication

ApplicationFile access


11/10/2010

Netslice—user-space packet processor Give power to the application

Packet processing in user-space Four-pronged approach (high level)

Contention prevention Spatially partition hardware

End-to-end control Provide fine-grained control over hardware

Streamline path for packets Export a rich, efficient, backwards compatible API

Hardware

Operating SystemKernel

NetworkStack Netslice

ApplicationUser-space

Application


11/10/2010

Netslice Spatial Partitioning Contention prevention Independent (parallel) execution contexts

Split each Network Interface Controller (NIC) One NIC queue per NIC per context

Group and split the CPU cores Implicit resources (bus and memory bandwidth)

Temporal partitioning(time-sharing)

Spatial partitioning(exclusive-access)


11/10/2010

Netslice Spatial Partitioning Example 2x quad core Intel Xeon X5570 (Nehalem)

Two simultaneous hyperthreads – OS sees 16 CPUs Non Uniform Memory Access (NUMA)

QuickPath point-to-point interconnect Shared L3 cache


11/10/2010

Fine-grained Hardware Control End-to-end control App controls NIC queue and CPU slice allocation

NIC hardware interrupt routing & NIC queue Kernel execution context User-space execution context Tight coupling of software and hardware components


11/10/2010

Streamlined Path for Packets Inefficient conventional network stack

One network stack “to rule them all” Performs too many memory accesses Pollutes cache, context switches, synchronization,

system calls, blocking API


11/10/2010

Netslice API Expresses fine-grained hardware control Flexible: based on ioctl

ioctl(fd, NETSLICE_CPUMASK_GET, &mask); sched_setaffinity(getpid(), sizeof(cpu_set_t),

&mask.u_peer); Backwards compatible (read/write)

fd = open("/dev/netslice-1", O_RDWR); read(fd, iov, IOVS)

Efficient: batch send/receive (read/write) Amortize overhead of

protection domain crossing


11/10/2010

Experimental Setup

R710 packet processors dual socket quad core 2.93GHz Xeon X5570

(Nehalem) 8MB of shared L3 cache and 12GB of RAM

6GB connected to each of the two CPU sockets Two Myri-10G NICs

R900 client end-hosts four socket 2.40GHz Xeon E7330 (Penryn) 6MB of L2 cache and 32GB of RAM23 Critical Network Infrastructure, by Hakim

Weatherspoon11/10/2010

Netslice Evaluation

Compare against state-of-the-art RouteBricks in-kernel, Click & pcap-mmap user-space

Additional baseline scenario All traffic through single NIC queue (receive-livelock)

What is the basic forwarding performance? How efficient is the streamlining of one Netslice? What is the benefit of batching?

How is Netslice scaling with the number of cores? Can build high-speed complex packet

processors?24 Critical Network Infrastructure, by Hakim Weatherspoon

11/10/2010

Simple Packet Routing End-to-end throughput, MTU (1500 byte) packets

Error bars (always present) denote standard error of mean

9.7G 9.7G9.7G

2.25G

7.6G 7.5G

5.6G

74% of kernel

1/11 ofNetslice


11/10/2010

Single Netslice user and kernel context CPU placement There are several choices


11/10/2010

Linear Scaling with CPUs

# of CPUs used # of CPUs used

• IPsec with 128 bit key—typically used by VPN– AES encryption in Cipher-block Chaining

mode 9.1G8G


11/10/2010

Netslice Implementation of Maelstrom

In-kernel reference version: 8432 lines of C Netslice version: 1197 lines of user-space C

Forwarding throughput: 8952.04±37.25 Mbps Maelstrom/Netslice goodput: 6993.69±35.7 Mbps

27.27% FEC overhead (for r=8, c=3) 6993.69Mbps × (1 + c ⁄ (r+c)) = 8901 Mbps


11/10/2010

10Gbps and beyond Netslice

Flexible API and spatial partitioning Nehalem CPUs

FSB not the bottleneck any longer Multiqueue NICs

Each core carves a private slice of every NIC Batching

Userspace multi-read / multi-write instead of ossified conventional read / write

Traditional tricks Pin down memory, minimize LLC contention, etc.


11/10/2010

Conclusion Network layer is fundamental to Datacenter

operations Packet processors enhance network functionality and

performance

Improve network performance with software packet processors running on commodity servers in userspace

OS support to build packet processing applications Harness implicit parallelism of modern hardware to scale Solution completely portable; kernel module load at

runtime30 Critical Network Infrastructure, by Hakim

Weatherspoon11/10/2010

Paper Trail Theme: “Datacenter Middleboxes” BiFocals/SDNA in IMC-2010 NLR study in DSN-2010 SMFS in FAST-2009 Maelstrom (FEC) in TONS-2010 and NSDI-2008 FWP, NSDI-2008 Poster Session

More at http://fireless.cs.cornell.edu


11/10/2010

Questions


11/10/2010

Netslice : Enabling Critical Network Infrastructure with Commodity Routers

Documents

Transcript of Netslice : Enabling Critical Network Infrastructure with Commodity Routers