Post on 01-Sep-2018
How Open NX-OS enables more Open, Extensible, Modular and
Flexible DatacentersShane Corban Product Manager
PSODCT-2030
• Data Center Trends
• Customer Requirements Driving Change
• Open NX-OS Introduction
• Open Bootloaders
• Open Automation Tool Integration
• Open Interfaces
• Open Programmability Tool Choice
• Conclusion
Agenda
• At the end of the session, the participants should be able to:
• Articulate Cisco’s Open-NXOS feature set and capabilities
• Dispel the perception in the SDN space that Cisco and NX-OS is not extensible, modular and programmable
• Understand how this Open Framework provides both the feature richness of traditional NX-OS along with the extensibility/modularity and an extensive programmatic toolset
Session Objectives
Next-Generation Data Center TrendsDeliver Services At Speed, At Scale, At Lowest Cost
Capex & Opex
OptimizationScalable
Architecture
Public Sector
Compliance
Cloud Service Providers
Agility Scale
Fast
Services
Open Innovations
Web2.0 Enterprise/Financial
Workload
MobilitySecurity
Driving Infrastructure TransformationEnabling Business Innovation Velocity
Simplify
Infrastructure
Management
Modular, Multi-
vendor
Interoperability
Adaptable
Operating System
Automation Innovation Consistency
Open NX-OS consistent across
both ToR and ModularExtensibilityAuto Deployment
OptionsOpen Application
IntegrationProgrammability
Tool Choice
DevOps
Enabling
POAP NXAPI
Yocto
SDK
Standard Open InterfacesOpen Interfaces
Automation and Visibility
Adaptable NXOS
AdaptableSDK
Programmable BootStrap and ProvisioningBootStrap/ Provisioning
Package and Application Management
Native Application Integration
PXE
OPEN NXOS – Enhancements across all NX-OS Infrastructure Layers
Data
Models
Server Management Tools
OPEN NX-OS - Extensible, Open, Programmable
Open NX-OS: Infrastructure Layer Enhancements
OPEN BOOTLOADERS & PROVISIONING
OPEN PACKAGE/APPLICATION INTEGRATION
OPEN INTERFACES
OPEN OBJECT BASED API’s (NX-API, Model Driven)
Open NX-OS consistent across
both ToR and Modular
Open NXOS
Open NX-OS Bootloaders & ProvisioningiPXE
VTEP VTEP VTEP VTEP
• Leverage existing compute deployment
infrastructure (PXE/iPXE) for
operationalizing NX-OS
• Deploy NX-OS from a web server via
HTTPS or TFTP server with support for
both IPv4 and IPv6
• NX-OS CLI option added to select boot
option either <bootflash(default) > or
<pxe>
Boot Server(DHCP &
HTTP/TFTP)
NX-OS Image Repository
DHCP
DISCOVER(v4/v6)
IP Address &
File/Image URL
TFTP GET
FILE/HTTP
URL
http://n9k-
dk9….bin..
Validate
Image
Checksum &
Boot
Open NX-OS: Infrastructure Layer Enhancements
OPEN BOOTLOADERS & PROVISIONING
OPEN PACKAGE/APPLICATION INTEGRATION
OPEN INTERFACES
OPEN OBJECT BASED API’s (NX-API, Model Driven)
Open NX-OS consistent across
both ToR and Modular
Open NXOS
Open NX-OS Package Management via YUM/RPMLXC and Native Daemons
• Ability to third party packages in Secure Guestshell or natively in NX-OS kernel
• Install all third party applications (Puppet/Chef, etc) as RPMs
• Daemon managed via standard Linux interfaces
• Built-in support for YUM package manager
• Patching and upgrade using standard rpm/yum workflows
• NX-OS processes(BGP) can be upgraded/patched via “yum update”
Package as RPM
C app with
standard Linux
constructs
Open Embedded
64 bit Build
Environment
Cisco/Local
Repository
RPM local
repository
RPM upload
YUM Install
Linux Daemon
Linux Kernel
• Raw Socket
• Netdevs
• Libpcap
init.d
Monitoring
server
ASIC
Build Server Target Switch
Open NX-OS Custom Application Integration• Third party or custom developed applications deployed:
• Natively in NX-OS Linux
• In isolated Secure CentOS7 Guestshell Environment
• Custom Application Building/Integration:
• Download Yocto 1.2 SDK toolchain, available openly from
Cisco CCO or www.yocto.org
• Install SDK on any Linux distribution server in your
datacenter - Fedora/Ubuntu/CentOS
• Build/Make application source/scripts, and package and
deploy using yum install on infrastructure
Deploy Application using Standard Linux Methods:
• YUM install custom agent RPM start script in “/etc/init.d/app
start” or “service app start”
Local
Repository
Server(YUM)
Yum install
app.rpm
http://repo-
server/app.rpm
Respository
Third Party
Protocol Apps
Devops
OrchestrationAutomation
Monitoring/
Analytics
Third Party/Custom
Applications
tcollector
64 Bit Yocto Based Linux Kernel
Switch Hardware
Routing/Switching
Packages
Kernel Route Kernel ARP Kstack/NetdevsKernel
Space
User
SpaceBase NX-OS
OpenLLDP
Open NX-OS Third Party Application IntegrationSoftware Architecture
Puppet/Chef Master Server
Native Linux Service
/etc/init.d/puppet.d & chef.d
NX-OS
Cisco Puppet/Chef Agent
Open NX-OS Puppet/ChefPuppet/Chef Agent: Architecture
NX-APICisco Puppet/Chef
Module(Incl Utility
GEMs)
Linux Software
Repository
Server Yum/RPM install
puppet/chef.rpm
• Cisco Puppet Agent RPM/software
package posted to Puppetforge and Open
Sourced to Github
• Install Cisco Puppet Module on Puppet
Master
• Yum install Puppet Agent rpm on switches
• Switch Agent periodically will poll
Puppet/Chef Master for updated
catalog/cookbooks and attempt to
converge switch to desired state
Open NX-OS Puppet/ChefCisco Chef & Puppet Agent Types/Provider Support
Camden Chef/Puppet Agent Types/Providers
cisco_vtp
cisco_tacacs_server
cisco_tacacs_server_host
cisco_snmp_server
cisco_snmp_community
cisco_snmp_group
cisco_ospf
cisco_ospf_vrf
cisco_vlan
cisco_bgp*
cisco_bgp_vrf*
cisco_interface
cisco_interface_ospf
cisco_interface_vlan
• Agents RPM installed natively on switch, using agent RPM or
within isolated guestshell environment
• Supported Agent Types/Providers for Camden
• Cisco Network Element Chef/Puppet module code will be
published on github, CCO, and Chef Supermarket, Puppetforge
websites
• Resource & Provider Code
• Provider Utility (Device Objects, Node Object, Object->CLI
mapping etc)
• Transport Mgr Utility (aka NXAPI Utility)
• Agent is extensible beyond what we support by default by
using the utility classes.
• Agent is also extensible by passing CLI using
cisco_command_config resource:
• cisco_command_config { " feature-portchannel1":
• command => " interface port channel1\n
• description nwk1-0106-ic4-gw1|Po2407\n
• no switchport\n
• ip address 17.148.35.225/31\n }
Kernel (cgroup, LSM)
NX-OS root file system
Native Linux
Processes
Native Linux
ProcessesBash Bash
Native Linux
Processes
Native Linux
Processes
Native Linux
Processes
Guest root file systemPkg-1.rpm Pkg-2.rpm
Pkg-2.rpm Pkg-3.rpm
Ns=globalNs=global Ns=guestshell Ns=guestshell Ns=guestshellNs=global Ns=guestshell
Native Shell, RPM +
Containers
• Secure common distribution CentOS7 environment in which customer may install their own custom
applications
• Use “guestshell resize” command to restrict CPU/memory/rootfs resources available to Guest
Shell
Open NX-OS: Third Party Application IntegrationSecure Guest Shell
Pkg-4.rpm
Open NX-OS: Infrastructure Layer Enhancements
OPEN BOOTLOADERS & PROVISIONING
OPEN PACKAGE/APPLICATION INTEGRATION
OPEN INTERFACES
OPEN OBJECT BASED API’s (NX-API, Model Driven)
Open NX-OS consistent across
both ToR and Modular
Open NXOS
Open NX-OS Linux InterfacesBash Access • Leverage Linux command toolkit for monitoring
configuration and troubleshooting
• # tcpdump -w file.pcap -i eth1-1
• Use ethtool to display detailed interface statistics:
• #ethtool –S eth2-1
• Use ifconfig to change mtu for an interface to jumbo MTU:
• #ifconfig eth2-1 mtu 9000
• Use ip route to add a static route for a given interface:
• #ip route add 203.0.113.0/24 via 198.51.100.2 dev eth2-1
• Leverage bash for NX-OS scripting automation
• vsh –c “show interface brief” | grepup | awk/sed
Open Interfaces: NX-OS Kernel Stack Interfaces Representing VRF context via Linux Name Space
Use
r S
pa
ce
ASIC-1 ASIC-M
Eth1/1 Eth M/NEth1/2
Portchannel1SubIntf
Switch Ports
Lin
ux
Ke
rne
l
• Forwarding information within the ‘VRF’ context can be accessed via a corresponding Linux Name Space
• setns, ip-netns to change VRFs and add routes within a given namespace.
NetDevices
ASIC-2 . . .
Eth1/1Linux Networking Stack
Hard
wa
re
Default
Name
Space
Default
VRF
Name
Space
Red
VRF
Red
Name
Space
Orange
VRF
Orange
Name
Space
Purple
VRF
Purple
Ap
ps
Linux Networking Apps Monitoring/Visibility Automation/Provisioning
ASIC-3
Open NX-OS: Infrastructure Layer Enhancements
OPEN BOOTLOADERS & PROVISIONING
OPEN PACKAGE/APPLICATION INTEGRATION
OPEN INTERFACES
OPEN OBJECT BASED API’s (NX-API, Model Driven)
Open NX-OS consistent across
both ToR and Modular
Open NXOS
Open NX-OS ProgrammabilityNX-API Developer Sandbox
• Tool provides a convenient way
for network engineers to get up
to speed with scripting and
automation via web browser
interface
• Available on all Nexus
platforms.
• CLI commands embedded in
structured input and output
(JSON/XML) via HTTP/HTTPS
• Use “feature nxapi” to enable
access on the platform
• Automate at scale using REST API
access to Nexus object store
• Automate operations leveraging object
create/update/delete operations
• Benefits of Model Based automation
• Software productivity
• Software maintainability
• Software quality and code reuse
• Asynchronous model driven programmability:
• BGP, VLAN, LACP, ACL, QoS, UDLD, CDP,
MAC, DHCP, DNS, RBAC, AAA,SVI,
Logging, NTP, VRRP
• Object Model Specification available at
http://developer.cisco.com
• Push based model for event reporting,
leveraging websockets interface.
CLI RESTJSON
RPCSNMP
BGP LACP ACL QoSVLAN
Sys
Dom-yDom-x
BGP ACL
• …
MIT
Config
Faults &
Events
Stats
Data Models (Logical / Concrete)
Operational
Data
DME Processes
Data Management Engine
Object Store REST API(HTTP/HTTPS)
Open NX-OS ProgrammabilityModel Driven Automation
Object Based ProgrammabilityBGP Configuration
CLI POST Request (Pre Camden) POST Request BGP Object (Camden)
router bgp 11
router-id 1.1.1.1
POST http://Switch-IP/ins
{'content-type':'application/json-
rpc'}.json()
{ "jsonrpc": "2.0",
"method": "cli",
"params": {
"cmd": "config t",
"version": 1 }, "id": 1},
{ "jsonrpc": "2.0",
"method": "cli",
"params": {
"cmd": "router bgp 11",
"version": 1 },"id": 1},
{ "jsonrpc": "2.0",
"method": "cli",
"params": {
"cmd": "router-id
1.1.1.1",
"version": 1}, "id": 2}]
POST http://Switch-
IP/api/mo/sys/bgp/inst.json
{ "bgpInst" : {
"children" : [{
"bgpDom" : { 11
"attributes" : {
"name":"default",
"rtrId" : "1.1.1.1"
}
}
}
]
}
}
Open-NXOS Reference Links
Software Link
Chef
Agent(Supermarket)
http://supermarket.chef.io
Puppet
Agent(Puppetforge)
http://forge.puppetlabs.com
Third Party Agents
Repository(Cisco Repo)
http://engci-
maven.cisco.com/artifactory/enxos
-thirdparty-yum/
Nexus 3/9K GiT
Repository (Scripting
Examples, etc)
http://github.com/datacenter/nexus
9000
SDK for developing
custom application
RPMs
www.yocto.org
Complete Your Online Session Evaluation
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.
• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions
Thank you