Post on 21-Apr-2018
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 2
OUTLINE
• Background: • A Linux distribuKon is more than lots of packages • SupporKng new plaRorms can be a PITA
• Proposal: • OCP should host its own Linux distribuKon for switches • “DistribuKon” == exisKng distro + added switch bits and config • Users and developers benefit from common repository
• Benefits for users • Benefits for developers • Going forward:
• Big Switch willing to contribute code/scripts to bootstrap
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 3
BACKGROUND – DISTRIBUTION
• “Linux” proper is just the kernel • “DistribuQon” is everything else
• e.g., RedHat, Ubuntu, Slackware, Gentoo, etc. • Libc, compiler, user space binaries • ConfiguraKon, file system layout, startup scripts • Package management, full-‐featured boot loader
• A lot of work goes into making a good distribuQon • Default configuraKons, daemons • Q/A (lots and lots)
• Lots of possibiliQes for niche distribuQons • e.g., embedded environments differ from server • Bootloaders vs. full fledged systems
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 4
BACKGROUND – PLATFORM SUPPORT
• Server and switch plaUorms have many idiosyncrasies • Litany of lible devices we never think of… • USB, GPIO, flash, PCI, serial, RTC, EEPROM, DMA, Crypto chips • MPIC -‐ MulKple programmable interrupt controller • … all at plaRorm-‐specific memory locaKons
• x86-‐based standards shield us from low-‐level plaUorm details • Vendor must write a BIOS for each plaRorm, e.g., ACPI standard • OperaKng systems (e.g., Linux) discovery devices via BIOS
• But switch plaUorm ecosystem is not as evolved • Includes switch specific devices, like I2C, GPIOs, etc. • Manual map/inventory of hardware à memory address via
Device Tree Source (DTS) files
SomeKmes we forget, but the boot process is horrible
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 5
BACKGROUND – RISK TOWER OF BABEL
OCP PlaRorm V1
Hardware Layer
Device Tree #1 Initrd #1
PlaRorm Dependent
Fedora Linux Kernel
PlaRorm Independent
Stack #1 Stack #2 Stack #3
STP + MLAG
OCP PlaRorm V2
Device Tree #2 Initrd #2
Std. Debian Linux Kernel
OpenFlow daemon
White box vendor
Device Tree #3 Initrd #3
BusyBox Linux Kernel
Quagga + hooks Switch Agent(s)
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 6
PROPOSAL: PICTORIAL VERSION
OCP PlaRorm V1
STP + MLAG
OCP PlaRorm V2
Unified Device Tree Repository Unified Driver Repository
Standard Debian, tools, etc. Stock Linux Kernel + any patches
OpenFlow daemon
White box vendor
Quagga + hooks
Keep differenKaKon in switch agents
Come together around the common bits
Maximize hardware abstracKon
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 7
PROPOSAL – OCP SWITCH DISTRIBUTION
• OCP should make its own Linux-‐based switch distribuQon • Low effort -‐ repackage well-‐known distro, e.g., Debian • Tuned for switches, but with full-‐featured tools • CollecKon of the right DTS files for common switch plaRorms • CollecKon of the right kernel modules
• Running “the things you want and nothing else” • Yes: NTP, syslog, SNMP server, • No (by default): NetworkManager, avahi, etc.
• All switch vendors have effecQvely do this already • There is lible (no?) Intellectual Property in the distribuKon • Vendors differenKate themselves by daemons that run on top
of the distribuKon
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 8
PERSPECTIVE RELATIVE TO ONIE Different tools for different use cases – but maximize code reuse
ONIE First boot Loader
~3MB
Normal Full-‐featured Boot Loader (w/BusyBox)
Main Network OS Image (.swi)
(w/real binaries)
~16MB
~160 MB
Proposed OCP DistribuKon Github.com/
onie/onie
Common kernel and DTS files??
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 9
BENEFITS FOR USERS
• Help foster an OSS ecosystem sandbox • Easy OS binary to download and play with
• Vendor agnosQc common Linux plaUorm • Deploy your non-‐switch tools on any box
- e.g., Chef/puppet/custom • Manage the switch like any other server
• Central repository for DTS files • Less fricKon to support new plaRorms • Easy hardware validaKon
• Possible “app store” for vendor-‐specific add-‐ons • Deploy and swap daemons with `apt-‐get` • Probably a long way off, but sKll interesKng
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 10
BENEFITS FOR VENDORS
• All vendors already have their own distribuQons • Informal check: most are based off of Debian Wheezy • No significant space for differenKaKon, might as well
standardize • Reduce engineering effort
• Reduce the effort to support new plaUorms • Open up the ecosystem – good for everyone • Central repository for hardware vendors to test their drivers
• Normalize hardware compaQbility lists
Secondary importance, but why we’re proposing this
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 11
GOING FORWARD – CONTRIBUTION FROM BSN
• Contribute enQre Linux distribuQon to open source/OCP • Based off of Debian Wheezy
• Pre-‐made binaries for x86 and PowerPC • Tuned for embedded switches
• Non-‐BusyBox à real tools for main OS, BusyBox for loader • Extra drivers for flash, GPIOs, I2C, ethernet drivers, etc. • A custom kernel patches, e.g., enhanced I2C support
• Linux drivers and DTS files for many common plaUorms • Quanta LB8D, LB9, LB9A, LY2, LY2R, LY5 • Accton 5652 • More coming
• Full-‐featured boot loader – ONIE is just for first Qme install
A developer workflow as much as a distribuKon
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 12
SWITCHLIGHT ARCHITECTURE
Controller
Linux Kernel ASIC
Lib C ASIC SDK
SSH Fan Control NTP Syslog SNMP
CLI
Indigo
ASIC Driver
OpenFlow Agent
Loxi
Legend
3rd Party Open
BSN Open
BSN Binary
3rd Party Closed
x86
Vswitch Kernel
Vswitch Driver
Switch Light
DistribuKon decoupled from
OpenFlow Agent
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 13
REFERENCES
• More on Device Trees • hbp://www.devicetree.org/Main_Page • hbp://www.celinux.org/elc08_presentaKons/glikely-‐-‐device-‐
tree.pdf • x86 Boot Process and BIOS Standards
• hbp://duartes.org/gustavo/blog/post/how-‐computers-‐boot-‐up • hbp://www.acpi.info/presentaKons/ACPI_Overview.pdf
• ONIE • Code: github.com/onie/onie • Docs: hbp://onie.github.io/onie
©2013 B IG SWITCH NETWORKS , INC . WWW.B IGSWITCH .COM 14
CONCLUSION
• Linux distribuQons for switches are a fair bit of work • Drivers, device trees, configuraKon • Full-‐featured loader, real tools
• Lots of opportunity and benefit to share code • Good for users and vendors
• Reduces effort/fricKon for community hardware support • Vendor agnosKc base for sandbox
• Open up the ecosystem • BSN is offering to seed the project with parts of the SwitchLight
distribuQon • (Safely decoupled from the OpenFlow daemon)