Big data, better networks

v

Big Data with Cumulus

David Sinn

April 22, 2015

Cumulus Networks: Making Networks Accessible

§ Vision §  Bringing the Linux Revolution to Networking

§ Mission §  To enable high capacity networks that are easy to deploy

and affordable helping customers realize the full promise of the software-defined data center.

cumulusnetworks.com 2

Big Data in the Data Center

§ Super-compute §  Clusters of x86 servers. §  Ultra-low latency §  Large scale east/west traffic

§ Distributed Storage/MapReduce §  Dynamic workloads, so locality is difficult

§ Costly Legacy Deployments


Traditional BigData Deployments

§ Layer-2 based §  Constrained east/west §  High port costs §  Variable Latency §  Host resiliency demands §  Closed protocols


L3: A better design

§ ECMP based Fat Tree §  East/west bandwidth

§ Better Failure Handling

§ Scalable: No Bottlenecks

§ Simple Feature Set

§ Open Protocols!


LEAF

SPINE

Big Data Improvements

§ Stop worrying about locality §  Full bandwidth is available between any pair of servers §  Location becomes irrelevant for performance §  Keep replicas as far away from each other as possible

§ Faster recovery §  Plenty of core bandwidth to re-replicate after failure §  Host HA becomes a lower level concern


Unified Management

§ Converged administration §  Same automation tools for managing servers now

available for the network

cumulusnetworks.com

Layer 3 Fabric

Servers

Switches

7

Cumulus Innovation Focus – Apps for the networking infrastructure

Don’t re-invent the wheel, but smoothen the ride §  ONIE – Unmanned Install §  ZTP – Zero Touch Provisioning §  PTMd – Cable fault detection §  Unnumbered Interfaces – IP management §  Network Virtualization – Minimize physical changes §  Dual Home hosts – Simplified Networks

8

NETWORK OS

HARDWARE

APP APP APP

hardware

Cumulus Linux

routing ganglia bridging

hardware

operating system

app app app

CumulusNetworks.com

Automation Workflow

§ Installation Framework §  PXE & ONIE to install the OS’s §  Anisble to install the applications and configure the

nodes

Cumulus Networks and Partner Confidential 9

bare metal server

Server OS

app app app

BIOS and PXE

bare metal switch

Network OS

app app app

Boot Loader and ONIE

Similar to installing a server OS using PXE

Network OS Boot Loader and ONIE

boot into

Boot Loader and ONIE

boot into

ONIE looks for and installs network OS image

Demo

§ Cumulus Customer Workbench §  2-Spine + 2 Leaf

•  OSPF Unnumbered with PTM §  2-Ubuntu Servers

•  Hortonworks Hadoop •  HDFS & YARN •  Server 1: master + worker •  Server 2: worker


Ansible

§ Example Automation Process

§ Network §  Define topology §  Install license §  Configure OSPF Unnumbered & PTM §  Start networking


Ansible Continued

§ Server Setup §  Create users, setup networking §  Install packages and dependencies §  Deploy site specific Hadoop Config §  Start HDFS & YARN

§ Enjoy Your BigData Cluster


Demo

§ Let’s switch to the demo


The Data Center Network with Cumulus Linux

CumulusNetworks.com 14

Open Affordable Responsive

Unlock Your Choices.

§  Disaggregation of hardware and software

§  Choice of best-in-class platforms and tools

§  Inherently flexible – with no vendor lock-in

§  Vast resources in the Linux community

Serve the Business Better.

§  Rapid, accurate services with automation – provisioning in minutes instead of days

§  Orchestration via server tools

§  Accelerated innovation via DevOps, open source, and broad solution ecosystem

Save as You Grow.

§  Scalable, affordable capacity in layer 2, layer 3 or layer 3 overlay networks

§  >50% CapEx savings with industry-standard hardware running Cumulus Linux

§  Opex savings through automation, use of server tools and simpler expansion

© 2015 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.

§ Thank You!


Unleashing the Power of Open Networking

Big data, better networks

Technology

Transcript of Big data, better networks