Linux in Mission-Critical, High-Availability, High- …...• Leader in financial services...

Copyright © 2013 Solarflare Communications, Inc. All Rights Reserved.

May 2013

Linux Enterprise End User Summit

Linux in Mission-Critical, High-Availability, High-Reliability Environments

Copyright © 2013 Solarflare Communications, Inc. Proprietary and Confidential, Slide 2

Solarflare

•  High-performance software & low-latency hardware for 10GbE networking

•  Accelerate mission-critical applications for traders & exchanges

•  Leader in financial services (Exchanges, Banks, Proprietary Trading Firms)

•  Partnerships with Arista, Cisco, IBM, Juniper, Red Hat, Oracle, VMware

•  Over 700 customers

“Solarflare’s product, EnterpriseOnload is a robust, rigorously tested and fully supported solution that addresses our demanding support and service level requirements. In addition to providing the highest-performance, lowest-latency hardware, Solarflare’s unique and innovative application acceleration software can be used to deploy quickly without any need to re-write our applications.” Andrew Bach Senior Vice President of Network Services for NYSE Euronext


Linux in Mission Critical Environments

•  The path of Linux in FSI through 2000s –  from blades/grids to highest performance/lowest latency critical automated

trading servers –  (and everything in between)

•  Sparked by addition of features needed for enterprise environments (clustering, data replication, enterprise level support)

•  … while maintaining high performance (at least single threaded!)… •  (winning the latency race among general purpose operating systems)


Linux in the enterprise

•  Further virtualization in the future, in order to cut cost and simplify maintenance

•  BUT, we need to maintain performance •  … and low latency profiles

–  Latency sensitive applications have mostly resisted virtualization efforts (thus far)

•  Two paths – bare metal installations for ultra low latency and virtualization for everything else

•  Traditionally thought of as mutually exclusive

•  BUT … is it???


Application Acceleration Middleware

•  Application acceleration –  Reduce latency by 50% –  Increase message rates by 3x –  Application direct HW access –  Reduce interrupts, context

switches, copies

•  Seamless integration –  Sockets / POSIX API –  No software modifications –  No specialized protocols –  Standard Ethernet

•  Open source GPLv2


OpenOnload: Lowest Latency

•  Ultra-low latency without modifying SW or HW

•  Supports DDIO –  IO/Cache integration

* Sandy Bridge platform: Single i5-2500k 3.3GHz

1/2 RTT Latency SFN6122F UDP 2.3 µsec TCP 2.4 µsec

Peak Packet Rate SFN6122F Dual Port 20Mpps


OpenOnload® performance


Virtualized networking

‘Classic’ virtualized networking -  Hypervisor emulates network hardware -  Flexible! -  But, performance is poor -‐  Limits the workloads that can be virtualized -  Uses CPU cycles


Native Networking Performance

Typical virtualization (with SR-IOV)

-  Network driver binds directly to the PCIe VF

-  Near native networking performance -  BUT, no longer virtual

-  No migration


Solarflare plug-in model

Plug-in SR-IOV hypervisor bypass

-  Maintains traditional software path -  PLUS VF path -  Near native performance -  Keep full virtualization features

(migration) -  RSS within guest VMs -  127 VFs, 1024 VNICs


Optimized for Virtualization Performance

vNICvNIC

vNICvNIC

vNICvNIC

vNICvNIC

vNIC

•  Low latency architecture –  Cut-through forwarding –  Stateless protocol offloads

•  Scalable hardware increases VM density –  Up to 2048 vNICs & 254 VFs

•  Individual Tx/Rx Queue pairs, MSI-X interrupts (vNIC) –  Competitors scale to 32 or 128 vNICs/VFs

•  Scalable performance –  Packet processing scales with number of cores & VMs –  Core scaling with NetQueue, VMQ, SR-IOV

•  Enterprise feature set –  VLANs, Teaming –  PXE and iSCSI booting


VMVM

NIC

VM VM VM

driver

TCP/IP

driver

TCP/IP

driver

TCP/IP

driver

TCP/IP

driver

TCP/IP

KVM Host

VNICVNICVNICVNICVNICVNIC

PF VF VF VF

VNICVNICVNICVNICVNICVNIC VNICVNICVNIC

Bridge

Improving KVM Performance & Scalability

•  Core scaling enhances performance

–  Spreads I/O across all CPU cores

–  Improves 10GbE performance scalability

–  VM / vNIC core alignment –  Mitigates hypervisor becoming

I/O bottleneck

•  SR-IOV acceleration –  Near native performance –  VM direct access to VF / vNIC

•  More vNICs & VFs = better scaling across all VMs

–  Solarflare = 2048 vNICs


•  Top two highest published SPECvirt_sc2010 results

•  Identical processor configuration •  Different network adapters •  Top results for 2-way & 4-way

Record Breaking VM Density – SPECvirt_sc2010

Benchmark Configuration Benchmark Result

Server Processor Sockets/ Cores

Network Adapter Score # VMs Date

Published HP ProLiant DL980 G7

Intel® Xeon® E7-4870 (2400 MHz)

8 / 80 Solarflare SFN5122F

8956 552 Dec-2011

HP ProLiant DL980 G7

Intel Xeon E7-4870 (2400 MHz)

8 / 80 Intel X520 7424 456 Oct-2011

Solarflare with SR-IOV delivers record performance with 21%

increase!

* Based on best performing system published at www.spec.org/virt_sc2010 as of 23 July 2012. SPEC® and the benchmark SPECvirt_sc® are registered trademarks of the Standard Performance Evaluation Corporation.


•  Top three SPECvirt_sc2010 results for 2, 4, 8 processor configurations •  Industry-leading SPECvirt_sc2010 scores achieved with Red Hat

Enterprise Linux (RHEL) using KVM and Solarflare Adapters

SPECvirt Benchmarks

2352 2442

4603 4682

7424

8956

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

RHEL 6.2 (KVM)Intel x520

RHEV 3 (KVM)Solarflare


RHEL 6.1 (KVM)Solarflare


RHEL 6.1 (KVM)Solarflare

2 processors 4 processors 8 processors

Solarflare with SR-IOV delivers record performance with 21% increase!

Comparison based on best performing systems published at www.spec.org as of 25 June 2012. SPEC® and the benchmark SPECvirt_sc® are registered trademarks of the Standard Performance Evaluation Corporation. For more information about SPECvirt_sc2010, see www.spec.org/virt_sc2010/.


2.79 3.365.8

8.1510.1

16.6

9.3

17.4 18.3

9.32

16.8

33.4

0

5

10

15

20

25

30

35

40

I/O Perform

ance (G

bps)

Solarflare Hybrid SR-‐IOV Performance Advantage

No SR-‐IOV

With SR-‐IOV

0

5

10

15

20

25

30

35

40

1 VM 4 VMs

No SR-‐IOV

With SR-‐IOV

Hybrid SR-IOV Accelerates Virtualization Performance

Traffic Unidirectional Bi-directional Bi-directional Unidirectional Bi-directional Bi-directional

10GbE ports 1 1 2 1 1 2

* Solarflare test results

•  Solarflare scales and delivers near line-rate performance


Summary

•  Software/hardware architecture is key –  Hypervisor & OS bypass eliminate bottlenecks –  Plug-in architecture retains virtualisation benefits

•  Hypervisor bypass can be even more effective than OS bypass –  Enables a wider range of applications to be virtualised

•  This technology enables enterprises to virtualize performance sensitive applications

–  Including latency sensitive applications!

Linux in Mission-Critical, High-Availability, High- …...• Leader in financial services...

Documents

Transcript of Linux in Mission-Critical, High-Availability, High- …...• Leader in financial services...