Linux As A Network OS

Dinesh G DuttShrijeet MukherjeeNolan LeakePradosh Mohapatra

09/16/13 2Linuxcon 2013

Networking circa 2008

Flickr photo from [http://www.flickr.com/photos/duncanh1/]

09/16/13 3Linuxcon 2013

Data Centers have thrown a monkey wrench in the world of networking whether it be how networks are managed, the new technologies they have brought to the fore such as SDN or network virtualization

09/16/13 4Linuxcon 2013

Modern data center networks are based around the following ideas:

New breed of applications which are L3-aware and workaround network failuresIP-based networksAutomated management

• Configuration and monitoring•Rapid spin-up and spin-down of

networks

09/16/13 5Linuxcon 2013

Operating Systems define how you manage the individual boxes and thereby the system

09/16/13 6Linuxcon 2013

Traditional Router/Switch Oses:Complex routing/switching featuresStructured as a black box:– No well-defined API

Closed development modelAntediluvian management tool chainVery slow spin-up and spin-down of networks

09/16/13 7Linuxcon 2013

Modern data centers usually run GNU/Linux as the server OS:

Well established and open APISophisticated management tool chain

• Including scriptingVibrant community fueling innovationExcellent networking support

09/16/13 8Linuxcon 2013

Networking support includes the expected and more:

09/16/13 9Linuxcon 2013

In other words: GNU/Linux is a great fit as the OS for not just servers but even routers and switches in the modern data center

09/16/13 10Linuxcon 2013

So What ? What advantages does this provide ?

09/16/13 11Linuxcon 2013

What Linux as the network OS enables: Open routing and switching platformUnified management tool chainNew ways of solving problemsNeatly sidesteps a bunch of problemsA potential to return IP networking to its roots: “rough consensus, working code”

09/16/13 12Linuxcon 2013

Open Routing/Switching implies a more participatory role for everyone to develop networking, transparency for troubleshooting and understanding

09/16/13 13Linuxcon 2013

Quagga

Kernel STP, mstpd

lldpd, open-lldp

Routing Suites

Bridging

Discovery

Monitoring Net-snmp, collectd, ganglia

Open Routing/Switching Platform

09/16/13 14

Datacenter 1Datacenter 1

Tools to manage servers such as Chef, Puppet, Ansible etc. can also be used to manage the network

09/16/13 15Linuxcon 2013

netplug, ifplugdFlexlink

Keepalived or a script to configure virtual MAC on bridge

HSRP/VRRP

New ways of solving the problem

09/16/13 16Linuxcon 2013

With Linux as the network OS, we neatly sidestep issues that arise from the traditional router OS' being a black box:

Technologies such as netconfAPIs for programming network

09/16/13 17Linuxcon 2013

What's missing ?Hardware acceleration of the networking forwarding path

09/16/13 18

One Way Of Hardware Accelerating

Linux KernelLinux Kernel RoutingTable

RoutingTable

ARPTableARPTable

Ethernet InterfacesEthernet

Interfaces

switchdswitchdRouting ProtocolsRouting

Protocols

CPU, RAM, Flash, etcCPU, RAM, Flash, etc

SwitchDriver

SwitchDriver

Switch Silicon

Switch Silicon

Customer ApplicationsCustomer

Applications

Switch ports show up as virtual interfaces (swp0, swp1 …)• typical tools - ifconfig, route, arp, ip, brctl, ethtool, tcpdump, etc

Kernel FIB/ARP table are synchronized with HW• userspace can send/receive packets and insert routes normally

Advanced operations use specialized APIs• expose hardware acceleration beyond stock Linux routing/bridging

09/16/13 19Linuxcon 2013

Evolving Linux further:Consistent way to map data structures to hardware

•Netlink's publish functionality not available with netfilter, for eg.

Provide mechanisms to allow commands to fail if backend hardware install fails

Linuxcon 201309/16/13 20

Linux as the embedded OS: process and memory mgmt

Embedded OS with process and memory mgmt

No real OS, while loop

Monolithic OS Third Party Real-time OS

Linux-based OS

Evolution Of Network OS

Eg: IOS, CatOS

Proprietary routingAnd switching stack

Eg: ION Eg: NX-OS, EOS Cumulus Linux

Linux OS

Linux as Network OS:Native routingand switching

Proprietary routingAnd switching stack

Proprietary routingAnd switching stack

09/16/13 21

It's taken me all my life to learn what not to play.

- Dizzy Gillespie

09/16/13 22Linuxcon 2013

Imagine a world where every router ran GNU/Linux

09/16/13 23Linuxcon 2013

Thank You For Listening!

ddutt@cumulusnetworks.com