Post on 14-Apr-2018
#clmel
vCPE and Network Function Virtualisation for Enterprises
BRKVIR-2605
Matthias Falkner, Distinguished Engineer, Technical Marketing
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Abstract
Network Function Virtualisation is gaining increasing traction in the industry based on the promise of reducing both CAPEX and OPEX using COTS hardware. This session introduces the use-cases for NfV for Enterprise network architectures, such as virtualising branch routers, LISP nodes, IWAN deployments, or enabling enterprise hybrid cloud deployments. The sessions also discusses the technology of NfV from both a system architecture as well as a network architecture perspective. Particular focus is given on understanding the impact of running routing functions on top of hypervisors, as well as the placement and chaining of network functions. Performance of virtualised functions is also discussed.
BRKVIR-2605 Cisco Public© 2015 Cisco and/or its affiliates. All rights reserved.
Agenda
• Introduction & Motivation
• Deployment Models and Characteristics
• The Building Blocks for NfV(today)
• NfV Trade-offs and Research Topics
• Conclusion
8
Introduction and Motivation
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Network Functions Virtualisation (NFV)
Announced at SDN World Congress, Oct 2012
• AT&T
• BT
• CenturyLink
• China Mobile
• Colt
• Deutsche Telekom
• KDDI
• NTT
• Orange
• Telecom Italia
• Telstra
• Verizon
• Others TBA…
10
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
…NFV decouples the network functions such as NAT, Firewall, DPI, IPS/IDS, WAAS, SBC, RR etc. from proprietary hardware appliances, so they can run in software. …..It utilises standard IT virtualisation technologies that run on high-volume service, switch and storage hardware to virtualise network functions..…..It involves the implementation of network functions in software that can run on a range of industry standard server hardware, and that can be moved to, or instantiated in, various locations in the network as required, without the need for installation of new equipment.
What is NfV? A Definition
Sources:
https://www.sdncentral.com/which-is-better-sdn-or-nfv/
http://portal.etsi.org/nfv/nfv_white_paper.pdf
Service
Orchestration
NFVSDN X86
compute
11
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
A. Perceived Benefits for NfV - Architecture
Motivation Description
Reduction of the number of network
elements to manage and deploy
• Integration of network functions into a single system reduces the number
of appliances / NE to manage / configure
• Fewer hardware types to deploy / plan for
Service Elasticity • Deployment of VMs much faster than appliances
• Easy scale up / scale down of services
• Flexible service portfolio (mixing VNFs)
Operational efficiencies through
virtualisation
• Can leverage virtualisation advantages from data centre (vMotion,
dynamic resource scheduling, power management etc) also for VNFs
Reduced complexity for High
Availability
• VMs have a smaller failure domain.
• Stateless deployments become more acceptable, so less complexity
through stateful redundancy deployments
• ISSU simplified by deploying a NEW VM and failing over
12
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
B. Perceived Benefits for NfV - CAPEX
Motivation Description
Deployment of standard x86-based
servers
• Servers considered cheaper than routers / appliances
• Servers already deployed in branch / DC / PoP
Deployment of best-of-breed • Separation of network functions allows best-of-breed services
• eliminates vendor lock-in
• Encourages openness and competition among software vendors
• CAPEX reduction through competition
Cost reduction through economies of
scale
• Deployment of huge server farms in DCs can lead to better resource
utilisation
Simplified Performance Upgrades • Capability to increase performance without forklift upgrades
13
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
C. Perceived Benefits for NfV - OPEX
Motivation Description
Reduction of branch visits • Changes / upgrades in the service can be made in software
• No longer need to swap appliances on-site for service upgrades,
appliance failures
Automated network operations • Virtualisation places focus on automation and elasticity, thus reducing
management
Flexible VNF-based operation • Software upgrades can be done independently per VNF
• VNFs can be placed flexibly in branch, PoP or DC
Elimination / reduction of organisational
boundaries
• IT and network operations align
14
Deployment Models and Characteristics
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
NfV in the Enterprise – Taxonomy by Function
Enterprise NfV
Network Control
Control Plane vRR, vWLC, vMC, vMS/MRs…
Orchestration, Management &
Policy
Transport
On-premiseBasic
IWAN
CloudPrivate cloud
Public cloud
Network Functions /
Services
On-premise
Router-integrated server
Router + external Server
Server-based (vRouter + VNFs)
CloudPrivate cloud
Public cloud
Hybrid
18
Virtualisation of Control Plane Functions
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Shared Services
1. NfV Virtualisation Models: Control Plane Functions
• Virtualisation of Control plane functions
– Route Reflectors
– PfR MC
– LISP MS/MR
– WLC
– …
• Can be on-premise or in larger Enterprise WAN PoPs or in the cloud
– Assuming VNFs are reachable by IP
• CSR 1000v offers functional and operational consistency
– Virtualised IOS XE
WAN
Campus
vWLC vRR
vMS/MR vMC
20
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Example: vRR with CSR 1000v
• CSR 1000v offers full IOS XE route-reflector functionality
ASR1001 &
ASR1002-X
(8GB)
ASR1001 &
ASR1002-X
(16GB)
CSR1000v
(8GB)
CSR1000v
(16GB)
RP2 (8GB) RP2 (16GB)
ipv4 routes 7M 13M 8.5M 24.8M 8M 24M
vpnv4 routes 6M 12M 8.1M 23.9M 7M 18M
ipv6 routes 6M 11M 7.4M 21.9M 6M 17M
vpnv6 routes 6M 11M 7.3M 21.3M 6M 15M
BGP sessions 4000 4000 4000 4000 8000 8000
VMs
SP Aggregation
Customer
Premise
SP Core
Data Centre
vRR
21
Virtualising Branch Functions
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Virtualisation of Branch Functions
• Current Branch infrastructure often contains physical appliances that complicate architecture
• Typical Appliances vary by branch size– Remote office (1-5 users): firewall
– Small (5-50 users): switched infrastructure, small call control, firewall, IPS/IDS
– Medium (50-100 users): redundancy, local campus, call control, firewall, IPS, IDS, WAAS
– Large (100+ users): redundancy, local campus, call control, firewall, IPS, IDS, WAAS
• …In addition to end-points (Phones, Printers, local storage…)
WAN
Campus /
DC
Branch
CUBECUBE
Branch Appliances
• Router: Routing, ACL, NAT, SNMP..
• Switch: port aggregation
• Services realised with appliances
• Full redundancy
• Could be multi-vendor (Best of breed)
Fib/DSL/Cab.
Fib/DSL/Cab.
23
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Branch Virtualisation – On premise Options
Branch
Router + virtualised L4-7 services
• Router performs transport functions (Routing, ACL, NAT,
SNMP..)
• Services virtualized on external server
• Optional redundancy
• VNFs Could be multi-vendor (Best of breed)
F/D
Branch
Fully virtualised Branch
• Physical router replaced by x86 compute
• Both transport and network services virtualised
• Optional redundancy
• VNFs could be multi-vendor (Best of breed)
F/D
1
2
3
WAN
WAN
Branch
Router + integrated L4-7 services
• E.g. ISR + UCS-E
• Router performs transport functions
• Services (Firewall, WAAS..) virtualised on UCS-E
• Optional redundancy
F/D
WAN
24
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
LISP VPN Gateway
• Challenge
– Extending architecture globally
– Simplicity & flexibility
• Virtual Router Benefits
– Faster provisioning
– Hardware availability
– Rapid reaction to global demand
• CSR 1000v Advantages
– Full-service router supporting LISP and MPLS VPN
– Can be coupled with Encryption
– Consitent L3VPN feature set (IOS XE)
– QoS transparency
LISP – MPLS
Gateway
LISP overlay
Real-WorldExample
25
Cloud Virtualisation
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cloud Virtualisation Categories
• Virtualised routers first deployed in cloud environments
– Bind cloud applications into the enterprise network infrastructure
• New use cases
– Virtualisation of L3 transport for small branches (SOHO)
– Hybrid branch virtualisation environments
27
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Application Visibility in the Amazon Cloud
• Cloud network enhanced by sophisticated routing functionality
– Secure connectivity to cloud (encryption)
– VPC to VPC connectivity
– Application Visibility
– WAAS
VPCs are part of enterprise network
End-to-end Cisco network (including
AWS Cloud)
Application Visibility
Remote Sites
& Employees
Enterprise
Data Centre
Public
InternetVPC2
VPC1
VPCs are part of enterprise network
End-to-end Cisco network (including
AWS Cloud)
Application Visibility
28
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
WAN
DC
Campus
WAN
DC
Campus
Branch Virtualisation: Cloud Options
L2 Private-cloud Branch – 1:1
• Small branches with low throughput and
no WAAS, Encryption, HA requirements
• Switch: transport, Storm control, L2 COS
• Routing & Services: done in PoP or in SP
DC running on UCS (at PoP or in DC)
• Single tenant, but optionally single-or multi-
site
Routing, QoS,
FW, NAT..
Branch
Branch Routing, QoS,
FW, NAT..
F/D
L3 Private-cloud Branch – 1:1
• L3 router remains in branch but performs
minimal functions
• L4-7 services virtualised in the private
cloud
• Branch router tightly coupled with virtual
router in the private cloud for services
Routing, QoS,
FW, NAT..
Branch
Branch FW, NAT..
F/D
4
5
Suitability for applications with stringent bandwidth / delay /
jitter requirements?
29
The Building Blocks for NfV (Today)
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Architecture Building Blocks Enterprise NfV
• A transport network
• Physical Hardware
– X86 servers
– NfV-capable routers
• Virtual Network Functions
– Virtual Routers, Firewalls, NATs…
• Hypervisors / Containers
• Orchestration and Management
• Service Chaining (Optional)
Branch 1
Policy
Orchestration & Management
PHY PHYHost OS
VM1 VM2
PnP
LCMHypervisor
VSwitch
DC
…
WAN Branch N
VMx
31
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
ETSI NfV Reference Architecture
Source: http://www.etsi.org/deliver/etsi_gs/nfv/001_099/002/01.01.01_60/gs_nfv002v010101p.pdf32
Execution reference points Main NFV reference pointsOther reference points
Computing
Hardware
Storage
Hardware
Network
Hardware
Hardware resources
Virtualisation LayerVirtualised
Infrastructure
Manager(s)
VNF
Manager(s)
VNF 2
OrchestratorOSS/BSS
NFVI
VNF 3VNF 1
Virtual
ComputingVirtual Storage Virtual Network
NFV Management and Orchestration
EMS 2 EMS 3EMS 1
Service, VNF and Infrastructure Description
Or-Vi
Or-Vnfm
Vi-Vnfm
Os-Ma
Se-Ma
Ve-Vnfm
Nf-Vi
Vn-Nf
Vl-Ha
Management
Orchestration
Virtual Network Functions
Hypervisor
Compute Hardware
Virtual Network Functions
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Management
& Orchestration
Voice &
Video
Security
Network
Infrastructure
Available VNFs from Cisco for Enterprise (Sample)
Deep Packet
Inspection
(vSCE)
Web Security
(vWSA)
E-Mail Security
(vESA)
Identity Services
Engine
(vISE)
DMVPN
(CSR1Kv)
SSL VPN
(CSR1Kv)
Virtual ASA
Firewall
(ASAv)
NAT
(CSR1Kv)
Virtual Zone
Based Firewall
(CSR1Kv)
IPSec and SSL
VPN
(ASAv)
vNGIPS
(SourceFire)
IPSec VPNs (Flex,
Easy, GET)
(CSR1Kv)
Virtual Router
CE / CPE
(CSR1Kv)
Nexus 1000V
Virtual
Route Reflector
(CSR1Kv, XRv)
CML / VIRL
Wireless LAN
Controller
(WLC/MSE)
Network Analysis
Module (NAM)
Wide Area
Application Service
(WAAS)
AppNav and AVC
(CSR1Kv)
DHCP
(CSR1Kv)
IP SLA
(CSR1Kv)
VXLAN (L2,L3),
OTV, VPLS, LISP
(CSR1Kv)
Virtual
PE/ IP Router
(CSR1Kv)
Cisco VDS-IS
Cisco Unified
Coms Manager,
Presence, Unity
Unified Contact
Centre, CC
Express
CUBE
(CSR1Kv)
Roadmap
Video
Conferencing
(MSE8K)
Enterprise Network
Controller (APIC-
EM)
Prime Performance
Manager, Prime
Analytics
Prime Network
Registrar, IP
Express
Prime
Access Registrar
Prime Fulfillment,
Order Fulfillment
Prime Home
Cisco Prime
Infrastructure,
Provisioning
Prime
Collaboration
Prime Network
Service ControllerUCS Director
Prime Service
Catalog
Intelligent
Automation for
Cloud (IAC)
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cisco Virtual Network Functions
• Adaptations from physical systems / solutions
• Feature and operational consistency between physical and virtual systems
– E.g. CSR 1000v and ASR 1000 / ISR 44xx are all based on the SAME IOS XE
• Exposure of APIs (REST)
• Flexible Licensing models (perpetual, Smart Licensing, Cisco ONE)
• Flexible Performance
– ASAv: {100Mbps, 1Gbps, 2Gbps}
– CSR 1000v: {10Mbps, 50Mbps, 100Mbps, 250 Mbps, 500Mbps, 1Gbps, 5 Gbps, 10Gbps}
– vSCE: 5 Gbps
35
BRKSEC-2762
BRKARC-2010
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cisco CSR 1000V – Virtual IOS XE NetworkingCisco IOS Software in Virtual Form-Factor
Virtualised Networking with Rapid Deployment and Flexibility
IOS XE Cloud Edition
• Selected features of IOS XE based on targeted use cases
Infrastructure Agnostic
Not tied to any server or vSwitch, supports ESXi, KVM, Xen, AMI
Throughput Elasticity
• Delivers 10Mbps to 20 Gbps throughput, consumes 1 to 8 vCPU
Multiple Licensing Models
• Term, Perpetual
Programmability
• RESTful APIs (leverages OnePK) for automated managementServer
Hypervisor
Virtual Switch
VPC/ vDC
OS
App
OS
App
CSR 1000V
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Architecture (CSR 1000v) - Virtualised IOS XE
Virtualised IOS XE
Generalised to work on any x86 system
Hardware specifics abstracted through a virtualisationlayer
Control Plane and Data Plane mapped to vCPUs
Bootflash: NVRAM: are mapped into memory from hard disk
No dedicated crypto engine – we leverage the Intel AES-NI instruction set to provide hardware crypto assist.
Boot loader functions implemented by GRUB
Packet path within CSR 1000v
1. Ethernet driver (ingress)
2. Rx thread
3. PPE Thread (packet processing)
4. HQF Thread (egress queueing)
5. Ethernet driver (egress)
Control PlaneForwarding Plane
vNICvCPU vMemory vDisk
Physical Hardware
CPU Memory Disk NIC
Hypervisor (VMware / Citrix / KVM)
Chassis Mgr.
Forwarding Mgr.
IOS
Chassis Mgr.
Forwarding Mgr.
FFP Client / Driver
FFP code Linux Container
37
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Technology Package IOS-XE Features
IPBase
Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS
High Availbility: HSRP, VRRP, GLBP
Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS
Basic Security: ACL, AAA, RADIUS, TACACS+
Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF
SECIPBase Plus…
Multicast: IGMP, PIM
Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN
AppX
IPBase Plus…
Advanced Networking: L2TPv3, BFD, MPLS, VRF, VXLAN
Application Experience: WCCPv2, AppXNAV, NBAR2, AVC, IP SLA
Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS
AX ALL FEATURES
CSR 1000v Feature Support and Technology PackagesREFERENCE
38
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cisco ASAv Firewall and Management Features
Cisco® ASA 9 Feature Set
Cisco
ASAv10
ASAv30
Removed clustering and
multiple-context mode
Parity with all other Cisco ASA platform features
10 vNIC interfaces and VLAN tagging
Virtualisation displaces multiple-context and clustering
SDN (Cisco APIC) and traditional (Cisco ASDM and CSM)
management tools
Dynamic routing includes OSPF, EIGRP, and BGP
IPv6 inspection support, NAT66, and NAT46/NAT64
REST API for programmed configuration and monitoring
Cisco TrustSec® PEP with SGT-based ACLs
Zone-based firewall
Equal-Cost Multipath
Failover Active/Standby HA model
39
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Protection Across the Attack Continuum with FirePOWERv
• Virtual machine discovery
• Enforce application policy
• Access control to segment security zones
• Visibility into virtual network communications
• Protect VMs even as the migrate across hosts
• Intrusion prevention without hairpinning
• Single pane-of-glass across physical and virtual networks
• Automated response via Integration with platform security controls
BEFOREDiscover
Enforce
Harden
AFTERScope
Contain
Remediate
Attack Continuum
Detect
Block
Defend
DURING
41
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
FirePOWERv Virtual Defence Centre• Deployed as virtual appliance
• Inline or passive deployment
• Full NGIPS Capabilities
• Add-on capability
• Control
• Advanced Malware Protection
• URL Filtering
• Deployed as virtual appliance
• Manages up to 25 sensors
• physical and virtual
• single pane-of-glass
DC
Virtual IPS Appliances
42
Hypervisors
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
CSR 1000v and Hypervisor Processing Relationships
• Example: 3 CSR VMs scheduled on a 2-socket 8-core x86
– Different CSR footprints shown
• Type 1 Hypervisor
– No additional Host OS represented
• HV Scheduler algorithm governs how vCPU/IRQ/vNIC/VMKernelprocesses are allocated to pCPUs
• Note the various schedulers
– Running ships-in-the-night
Pro
ce
ss
Qu
eu
e
HV Scheduler
Core0
pCPU1 pCPU2 pCPU3 pCPU4
pCPU5 pCPU6 pCPU7 pCPU8
vCPU12
vCPU03
vNICn2
VM Kernel1
Core1
pCPU1 pCPU2 pCPU3 pCPU4
pCPU5 pCPU6 pCPU7 pCPU8
vSwitch
VM1(4vCPU CSR 1000v)
CS
RIOSFman /
CManPPE HQF Rx
vCPU01 vCPU1
1 vCPU31 IRQ1 vNIC1
1 VM Kernel1
PP
E
vCPU21 vNICn
1
Guest OS Scheduler
Pkt Scheduler
VM2(1vCPU CSR 1000v)
vCPU02 IRQ2 vNIC1
2 VM Kernel2vNICn2
CS
RIOSFman /
CManPPE HQF Rx
Guest OS Scheduler
Pkt Scheduler
VM3 (2vCPU CSR 1000v)
vCPU03 IRQ3 vNIC1
3 VM Kernel3vNICn3
CS
RIOSFman /
CManPPE HQF Rx
Guest OS Scheduler
vCPU13
Pkt Scheduler
45
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Virtual Switches / Bridges
• Virtual switches ensure connectivity between physical interfaces and Virtual Machines
• Can have multiple vSwitches per host
• May have L2 restrictions
• May impact performance
46
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cisco Nexus 1000V
Hypervisor
Modular Switch
…
Linecard-N
Supervisor-1 (Active)
Supervisor-2 (StandBy)
Linecard-1
Linecard-2
Ba
ck P
lane
VEM-NVEM-1 VEM-2
VSM: Virtual Supervisor Module
VEM: Virtual Ethernet Module
VSM-1 (active)
VSM-2 (standby)
Virtual Appliance
Network
Management
Server
Admin
NX-OS
Control Plane
NX-OS
Data Plane
Hypervisor Hypervisor
47
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco PublicOpenStack Controller
Cisco
Nexus
1000V
VEM
Cisco
Nexus
1000V
VEM
Cisco
Nexus
1000V
VEM
VM VM VM VMVM VM VM VMVM VM VM VM
Cisco Nexus 1000V VSM
Virtual Supervisor Module (VSM)
• Virtual or Physical appliance running Cisco NXOS (supports Hi-availability)
• Performs management, monitoring, and configuration
• Tight integration with management platforms
Virtual Ethernet Module (VEM)
• Enables advanced networking capability on the hypervisor
• Provides each virtual machine with dedicated “switch port”
• Collection of VEMs : 1 virtual network Distributed Switch
KVM KVMKVM
Server Server Server
Cisco Nexus 1000V Deployment Scenario
48
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
VMWare Scheduler Details
• ESXi scheduler responsible for allocating pCPU to
– vCPU threads
– IRQs
– VM kernel Management
– i/o threads
• Default ESXi behaviour: allocate equal shares of pCPU cycles to each vCPU
– Design goal is fairness across VMs
– vCPU threads have a scheduling entitlement
– If thread has is below its execution entitlement, scheduling priority is raised
– Scheduler also accounts for entitlements and allocations
– Maximisation of pCPU utilisation may have negative throughput effects for VMs (e.g. cache thrash)
• ESXi can also co-schedule related threads (‘relaxed co-scheduling)
– Avoids synchronisation latency for related processes
49
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
KVM Scheduler Notes
• Each VM appears as a regular linux process to the host OS
Processes can be given relative priorities to influence the scheduler
Implements real-time scheduling extensions
• Linux schedulers generally time-share between processes
Process with the highest current priority gets scheduled onto CPU
Supports dynamic process priorities
Supports pre-emption
• Uses a ‘Completely Fair scheduler (CFS)’ under KVM+RedHat
Includes control groups to allow minimum resource allocations to processes
• Support for live VM Migration
50
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Hypervisors vs. Linux Containers
Hardware
Operating System
Hypervisor
Virtual Machine
Operating
System
Bins / libs
App App
Virtual Machine
Operating
System
Bins / libs
App App
Hardware
Hypervisor
Virtual Machine
Operating
System
Bins / libs
App App
Virtual Machine
Operating
System
Bins / libs
App App
Hardware
Operating System
Container
Bins / libs
App App
Container
Bins / libs
App App
Type 1 Hypervisor Type 2 Hypervisor Linux Containers (LXC)
Containers share the OS kernel of the host and thus are lightweight.
However, each container must have the same OS kernel.Containers are isolated,
but share OS and, where
appropriate, libs / bins.
51
I/O Architecture
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
x86 machine
Host-OS /
KVM
Qemu /
v-Host
tap
vSwitch (OVS) / Linux bridge
NIC driver
Guest-OS
Virtio-net
Guest-OS
Virtio-net
Qemu /
v-Host
tap
AppAppAppAppAppApp
Virtualising I/O – KVM Architecture Example
Hypervisor virtualises the NIC hardware to the multiple VMs
Hypervisor scheduler responsible for ensuring that I/O processes are served.
There is a single instance of physical NIC hardware, including queues, etc.
many to one relationship between the VM’s vNIC and the single physical NIC
One vHost/VirtIO thread used per configured interface (vNIC)
May become a bottleneck at high data rates
NIC port
54
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
• KVM with OVS consumes a vHost thread per configured VM interface
• The vHost thread is very CPU intensive, requires dedicated physical core
• On 16-core server, can only get 3 CSR1000v (2vCPU, 2 i/f each)
– Cores for CSR: 6
– Cores for VTF: 2
– Cores for vHost: 6
– Free: 2
• Should be considered when service chaining
Hypervisor Traversal Tax: Example KVM with OVS
x86 machine
NIC
Host-OS / KVM
Guest-OS
Open vSwitch
layer-2 sorter / switch / classifier
PF
PF driver
VTF
Virtio-net
Qemu /
vHOST
tap
Guest-OS
Virtio-net
Qemu /
vHOST
tap
CSR
Guest-OS
Virtio-net
Qemu /
vHOST
tap
CSR
Guest-OS
Virtio-net
Qemu /
vHOST
tap
CSR
Guest-OS
Virtio-net
Qemu /
vHOST
tap
CSR
Hypervisor traversal
tax = 8/16 = 50%
May not be
fully utilised!
55
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
x86 machine
NIC
Host-OS / KVM
Guest-OS Guest-OS Guest-OS
driver driver
I/O Optimisations: Direct-map PCI (PCI pass-through)
• Physical NICs are directly mapped to a VM
Bypasses the Hypervisor scheduler layer
PCI device (i.e. NIC) no longer shared among VMs
Typically, all ports on the NIC are associated with VM
Unless NIC supports virtualisation
• Caveats:
Limits the scale of the number of VMs per blade to ‘number of physical NICs per system’
Breaks live migration of VMs
AppAppAppAppAppApp
AppAppApp
driver
NIC NIC
56
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
I/O Optimisations: Single Root IO Virtualisation - SR-IOV with PCIe pass-through• Allows a single PCIe devices to appear to be
multiple separate PCIe devicesNIC supports virtualisation
• Enables network traffic to bypass software switch layers
• Creates physical and virtual functions (PF/VF)PF: full featured PCIe
VF: PCIe without configuration resources
Each PF/VF gets a PCIe requestor ID s.t. IO memory management can be separated between different VFs
Number of VFs dependent on NIC (O(10))
• Ports with the same (e.g. VLAN) encap share the same L2 broadcast domain
• Requires support in BIOS/Hypervisor x86 machine
NIC
Host-OS / KVM
Guest-OS Guest-OS Guest-OS
layer-2 sorter / switch / classifier
VF VF VF PF
VF driver VF driver VF driver
AppAppAppAppAppApp
AppAppApp
SR-IOV
Master
Driver
57
Management & Orchestration
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
CSR 1000v VM Instantiation & Bring-up - Overview
• CSR 1000v VM Instances can be instantiated using the following methods (with possible hypervisor dependencies)– VMWare ESXi: vSphere
– KVM / Xen: Openstack
– Public cloud: Marketplace
• Image Management– VMWare ESXi: vCloud Director
– KVM / Xen: Openstack Glance
– Public cloud: Marketplace
• An new Configuration OVF Tool (COT) is also provided for Cisco VMs
• License management – Smart licensing / Cisco ONE
60
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cisco Prime Infrastructure and CSR 1000v
• Prime Infrastructure supports cross-platform lifecycle management
– Lifecycle -> Design / Deploy / Operate / Report
– Support for templating, workflows
– State-of-the-art GUI with configurable dashboards / dashlets
• Prime Infrastructure NOT helping on managing the servers / hypervisors
– No support for generic server hardware configuration / monitoring
– No hypervisor monitoring / configuration support
• Same Functionality at device level as ISR 44xx and ASR1000
• Prime Network Services Controller (NSC) also supported for Hierarchical, Multi-Tenant Network Services management
61
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Prime Infrastructure 2.2 GUI Preview
Prime Infrastructure
Lifecycle Mega-Menus
Prime Infrastructure Configuration
Element Options
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Prime Service Catalog
(PSC)User Self-
Service Portal
NFV
Orchestrator
VM and Service
Lifecycle Manager
SDN sub-
system /
SDN
Controller
SDN Virtual
Forwarder
VM & Storage
Orchestrator
DCI
REST API REST API
MP-BGPRestconf
/YangN
etc
onf/
Yang
Or
CLI
Prime Order Fulfillment
or SP’s OSS/BSS
OpenStack APIs
VNF
VNFx86 Server
ESC API VTM API
OpenStack
Elastic Services
Controller (ESC)
Virtual Topology
Controller (VTC)
Service
Assurance
A Framework
enabled by
multiple products
& architecture
OVS
VTF
Cisco NfV Orchestration SolutionOSS
Network Services Orchestrator (NSO)
(Foundation Based on Tail-f NCS)
REST API /
JCloud (Future)
SP WAN
64
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cloud Service OrchestrationOrchestration
WorkflowCatalogPortal / UI / API
VM/Storage Control
Network Control
Network Service Control
Se
rvic
e
Cre
atio
n
Serv
ice
Monitorin
g
Serv
ice
Config
IP
Contr
ol
DC
Netw
ork
Contr
olle
r
WA
N
Contr
olle
r
…
NfV Example Workflow1. Request received
2. Catalog item
3. Defines workflow
4. Workflow calls Service Creation to set up service VMs
5. Service Creation calls to Openstack to set up VMs
6. Openstack sets up VMs
7. Workflow calls to Service Config function to set up services
8. Service Config configures services
9. Workflow calls DC network controller
10. DC network controller configures overlay network
11. Service monitoring tracks availability and performance of service
12. Service Creation manages service elasticity and high availability
Infrastructure
Physical
Network
Virtual
Network
Compute
Storage
Virtual Services
1112
10
1
2 3
4 7 9
5
6
8
68
Service Chaining
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco PublicUnderlay
Service Chaining Architecture Components
• Virtualisation facilitates a service chaining paradigm– Allows for significant architectural shifts
• Components1. Service classifier (SC): filter flows that enter a particular chain of service functions
2. Loadbalancer (LB): enable elastic capacity expansion / contraction in case the aggregate traffic volume for a service function exceeds its capacity
3. Service function (Sf): VMs or hardware that execute a one or more service functions in the chain
4. Transport Protocol used to carry packets between the loadbalancer / classifier and the service functions
5. Underlay network to get packets from one service function to the next
Service3Service2Service1
Service4Service1SC/LB1,2
3
4
5
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Benefits of Service Chaining
• Policy-based execution of service functions: see example on next slide
• Best of breed service functions– Can stitch together service functions from different vendors who each in turn deliver
best-of breed
• Independent troubleshooting and management– Each service function can be configured / managed / upgraded independently
– Allows for different departments to be responsible for a particular service function
– Elastic capacity expansion / contraction of service functions
• Leverage benefits of virtualisation– Fast and flexible introduction / expansion of services
– VM Moves
– Optimal placement of service functions
– Server hardware upgrades
• Flexible service ordering71
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Try rendering a business policy like …
All traffic between the Internet & Web front end servers apply:
De/Encryption with highest throughput / low latency and least $$ cost
Copy all “mobile” only transactions to a Big Data analytics system
Perform the copy at most optimal point ($$ cost & least latency impact)
Send all traffic through a SLB+WAF & and IDS
Additionally, deploy this policy with other caveats like:
Service functions are both virtual and physical and vendor neutral
Compute & service elasticity; compute mobility
Practically impossible today!
Why we must Evolve Service Chaining
InternetElastic
SSL
Elastic
LB +
WAF
Elastic
IDS
Elastic
Web FE
Elastic
Copy
Elastic
Analytics
Mobile
72
NfV Trade-Offs and Research Topics
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Main Trade-off and Research Areas
1. Cost of NfV solution as a function of performance
2. Trading-off performance for virtualisationflexibility
– Tuning performance may impact virtualisationelasticity
3. Architectural Considerations
– Capacity planning Service Function Chains?
– Orchestration solution?
– High-Availability requirements?
74
CAPEX / OPEX
PerformanceArchitecture
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Cost / Performance Trade-offs
• CAPEX Viability for virtualisation may require a minimum VM-packing density on a server
– How many VMs can be deployed simultaneously to achieve a certain CAPEX goal?
– Particularly applicable for Cloud deployment architectures
• What are cost effective deployment models?
– Mixing of application VMs and VNFs on the same hardware?
– Single-tenant / Multi-tenant?
– Hypervisor type?
– Hyperthreading?
– SLA guarantees and acceptable loss rates?
– High-availability requirements and architectures?
75
Architectural Considerations
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
WAN
Differences Between Cloud and Branch NfV Use-Cases
• Focus on cloud orchestration and virtualisation features
• Mix of applications and VNFs may be hosted in the cloud
• Horizontal scaling -> smaller VM footprints
• Dynamic capacity & usage- / term-based billing
• Focus on replacing hardware-based appliances
• Typically smaller x86 processing capacity in the branch
• NfV applications (Firewall, NAT, WAAS..) may consume large proportion of available hardware resources
– larger VM footprints
• Cloud orchestration and automation has to be distributed over all branches
– integration with existing OSS desirable for migration
UCS
VDI VDI
DB
ERP
Win WinDPI
UCS
IPSFirewall
WAN
DC
Branch
UCS
VDI VDI
DB
ERP
Win WinDPI
UCS
VDI VDI
DB
ERP
Win WinDPI
77
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
• Deployment of multi-tenant VMs can significantly improve the business case
– Leverage multi-tenancy feature set in IOS XE on CSR 1000v
• Leverages different footprint sizes of CSR 1000v, for example
– Deploy small footprint for single-branch & large footprint for multi-branch
• BUT:
– comes with a different operational model (Need to consider multi-tenancy for on-boarding a new branch)
– Has different failure-radius implications
Single-Branch vs. Multi-Branch VM Deployments
Branch 1
WAN DC / Cloud
Branch N
Branch 1
WAN DC / Cloud
Branch N
79
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
CSR 1000v as Multi-tenant vCPE - Example
Profile 1 (multi-tenant)
1vCPU CSR – 400 Mbps
200 VRF’s @ 5Mbps/VRF
QOS, DHCP Server, Static Route, IP SLA,
SNMP
Profile 2 (single-tenant)
1vCPU CSR - 50Mbps
QOS, DHCP Server, OSPF, IP SLA,
IGMPv2, PIM SM, SNMP, ACL2
Number of VM instances / server chassis 20 44
Number of branches / VNF instance 40 1
Total number of branches / server blade 800 44
Total aggregate bandwidth / server chassis 8 Gbps 2.2 Gbps
• Multi-tenant CSR 1000v deployed for 5 Mbps ‘vanilla’ branches requiring 5 Mbps each
• Single-tenant CSR 1000v deployed for high-end branches requiring 50 Mbps each
– Note that the 44 VM scenario (Profile 2) is oversubscribed, however the max bandwidth per VM requirement is only 50Mbps
80
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
VNF High-Availability Architecture Considerations
• Traditional Networking: make all critical network services highly-available
• Active-Standby or Active-active redundancy models
• Stateful redundancy for NAT, Firewall (i.e. stateful services)
• Adds architectural complexity
– HSRP, NSR, Stateful HA features…
• Does a virtualised environment need HA?
– Depends on PIN
• Branch: YES
• Cloud: MAYBE
– Can rely on reload / re-boot of VMs as this happens much faster
– Function of VM scope (cf. single-branch VNFs)
83
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Multiple Single-feature VMs (star)
Service Chaining HA
• Many of the functions considered for service chaining are stateful
• VM HA not sufficient
e.g. VMWare HA relies on replay, so crash of the active VM also crashes the standby
• Require a mechanism to copy VM state and deploy and active-active or active-standby model
CSR will support stateful inter-VM HA for NAT/ZBFW in IOS XE 3.13
Need to collect requirements for feature-independent HA architecture
• Impacts placement of active / standby VMs
Same DC? Different DCs?
• LB needs to be re-programmed upon failure
S1act S2act
LB
S2sbyS1sby
Multiple Single-feature VMs (Circular)
S1act S2act
LB
S2sbyS1sby
Single Multi-feature VMs
(S1+S2)act
LB
(S1+S2)sby
S2sby
S2act
S2sby
S2act
(S1+S2)act
(S1+S2)sby
Animated Slide
84
Performance Aspects for VNF Deployments
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Performance Aspects for VNF Deployments
• Throughput / SLAs for VNFs are determined by a multitude of factors
– System architecture, in particular I/O
– Hypervisor type (VMWare ESXi, KVM, Microsoft HyperV, Citrix XEN..)
• Throughput can be increased significantly by hypervisor tuning
• Need to determine
– How many VMs to run on a server blade
– Acceptable frame loss rates
86
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
• VMWare ESXi and KVM schedulers can perform in the same order of magnitude with tuning
– BUT: need to apply tuning recommendations, especially for KVM
– Most impactful tuning: I/O Optimisations (e.g. VM-Fex, SR-IOV)
• KVM shows bottlenecks when untuned
– descriptor ring restriction in KVM limits performance improvements for larger vCPUVMs
Hypervisor Impacts on Performance
CEF ACL NAT Qos Firewall IPSec(100tunnel-SHA)
8vCPU/4GB 2006 1918 1410 2013 1181 1073
4vCPU/4GB 2228 2014 1296 2190 944 510
2vCPU/2.5GB 1895 1466 1188 1554 936 279
1vCPU/2.5GB 999 622 588 622 548 183
0
500
1000
1500
2000
2500
Throughp
ut(M
bps)
ESXiiMIXThroughput(Mbps)with0.01%lossRateacrossFootprints-Uni-D,IOSXE3.13
UCSC200M2:2xIntelXeon2690SandyBridge2.90Ghz8cores,16threads
CEF ACL NAT QOS Firewall IPSec(100tunnel-SHA)
4vCPU/4GB 898 811 720 745 404 187
2vCPU/4GB 851 778 681 712 391 189
1vCPU/4GB 845 761 680 701 380 179
0
100
200
300
400
500
600
700
800
900
1000
Throughput(M
bps)
KVM+UbuntuiMIXThroughput(Mbps)with0.01%lossRateacrossFootprints,IOSXE3.13
UCSC200M2:2xIntelXeon2690SandyBridge2.90Ghz8cores,16threads
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Use a Direct path I/O technology (SR-IOV w/ PCIe pass-through) with CPU tuning below! Otherwise the following configurations are recommended:
KVM Performance Tuning Recommendations
Tuning
Recommendation
Details / Commands Tuning
Disable Hyperthreading Can be done in BIOS CPU
Find I/O NUMA Node cat /sys/bus/pci/devices/0000:06:00.0/numa_node
Enable isolcpus run command “numactl -H” CPU
Pin vCPUs ‘sudo virsh vcpupin test 0 6’ CPU
Set CPU in performance Mode run /etc/init.d/ondemand stop. CPU
Set Procsessor into pass-
through
virsh edit <vm name>
add this line <cpu mode='host-passthrough' />
CPU
Disable IRQ Balance run “service irqbalance stop”. CPU
NUMA-aware VM edit vm config by virsh edit <VM name>.
<vcpu placement='static' cpuset='8-15'>1</vcpu>
CPU
IRQ Pinning find specific nic interrupt number from /proc/interrupts. set affinity to other core than
pinned cpu than for CPU and vHost pinning
CPU
89
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
KVM Performance Tuning Recommendations (cont.)
Tuning
Recommendation
Details / Commands Tuning
Pin vHost processes ‘sudo taskset -pc 4 <process Number>’,
Where <process Number> is found using ‘ps -ef | grep vhost’
I/O
Change vnet txqueue length to
4000
Default tx queue length is 500
‘sudo ifconfig vnet1 txqueuelen 4000’
I/O
Turn off TSO, GSO, RSO, ‘ethtool -K vnet1 tso off gso off gro off’ I/O
Disable KSM echo 0 > /sys/kernel/mm/ksm/run
NOTE: these settings may impact the number of VMs that can be instantiated on a server / blade
90
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Quantitative Impact of various hypervisor tuning steps
Sample Results of Different Performance Improvements
defaultw/Hyperthreading HyperthreadingOff vCPUPinningonly Txqueuelenof4000only
Txqueuelenof4000+vCPUPinning+vhostpinning+txo,rxooff+Hyper
threadingOff
AverageThroughputMbps 100% 145% 174% 509% 952%
0%
100%
200%
300%
400%
500%
600%
700%
800%
900%
1000%
AverageThrough
put(M
bps)
SampleImpactwithdifferentHypervisorTuningsKVM+Ubuntu1.0withOVS,2vCPUCSR1000v,XE3.12Engineeringimage,IMIXtraffic,UCS2202.7GHz,0.01FLR
91
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
SR-IOV Virtualisation Caveats
• vSphere vMotion
• Storage vMotion
• vShield
• NetFlow
• VXLAN Virtual Wire
• vSphere High Availability
• vSphere Fault Tolerance
• vSphere DRS
• vSphere DPM
• Virtual machine suspend and resu
• Virtual machine snapshots
• MAC-based VLAN for passthrough virtual functions
• Hot addition and removal of virtual devices, memory, and vCPU
• Participation in a cluster environment
• Network statistics for a virtual machine NIC using SR-IOV passthrough
• The following features are not available for virtual machines configured with SR-IOV:
92
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
• Only works for 1vCPU VMs
• Fault Tolerance is not supported or incompatible in combination with
• Snapshots
• Storage vMotion
• Linked Clones
• VM Backups
• Virtual SAN
• Symmetric multiprocessor VMs
• Physical raw disk mapping
VMWare ESXi Fault Tolerance Caveats
• Paravirtualised guests
• NIC Passthrough
• Hot-plugging devices
• Serial or parallel ports
• IPv6
• …
93
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
HT Oversubscription
Running Multiple VMs on a Server Blade
• Throughput is typically NOT additive as the number of VMs increases on a server blade
– Major bottleneck is I/O
– Hyperthreading and oversubscription effects
– Cache thrashing
• I/O hypervisor bypass techniques improve multi-VM system throughput
– BUT: may impact virtualisation features (Vmotion etc.)
94
0%
100%
200%
300%
400%
500%
600%
700%
800%
1 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
1 vCPU VMs
Ag
gre
ga
te T
hro
ug
hpu
t M
pbs
Near linear
performance increase
as VMs are added
due to VMFex with
Direct Path
Hypervisor CPU
contentionVM
Oversubscription
B200 M2, 12 Cores, 2.67 Ghz
VM/FEX & Direct Path
ESXI 5.1
IP Packets CEF IMIX
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
0
20
40
60
80
100
120
140
160
180
200
0.00
0.40
0.80
1.20
1.60
2.00
2.40
2.80
3.20
3.60
4.00
4.40
4.80
5.20
5.60
6.00
6.40
6.80
7.20
7.60
8.00
8.40
8.80
9.20
9.60
10.00
10.40
10.80
11.20
11.60
Norm
alizedThrough
put(%
toBaselin
e)
%TrafficLossAccepted
Throughputasafunc onofacceptableTrafficLoss(%)
Esxi
1%reference
Log.(Esxi)
Loss Rate Interpretation - Background
• Performance results vary depending on what acceptable frame loss is defined. Typical definitions for loss rates (FLR) range from
– Absolutely 0 packets lost -> Non-drop Rate
– 5 packets lost
– 0.1% of PPS lost
• Small relaxation of FLR definition can lead to significant higher throughput
• Typically FLR Test data reported for 5 packet loss (to account for warmup) with multiple consecutive 1 minute runs
95
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Determination of Desired Frame Loss Rate
• Throughput can be affected by definition of acceptable loss rates
• Tests measure % of dropped traffic for various traffic loads
– Offer traffic load -> observe loss -> reduce offered load until desired loss rate reached
• BUT: Difficult to get consistent data across multiple runs.
• How to interpret the right loss-rate?
• Example:
– Highest rate at which LR of 0.01% appears -> 475 Mbps
– Lowest rate below which LR of 0.01% is ALWAYS observed -> 374 Mbps
– Loss rate ‘violations’ at {445, 435, 414, 384} Mbps
Sample Data
REFERENCE
97
Conclusion
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Summary
• Introduction & Motivation
• Deployment Models and Characteristics
• The Building Blocks for NfV (today)
• NfV Trade-offs and Research Topics
99
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Key Conclusions
1. Network Function Virtualisation is rapidly maturing and enabling first use-cases TODAY for enterprise network functions
– Virtualisation of control plane functions
– Cloud-based network services
2. NfV enables new architectural approaches leading to potential CAPEX and OPEX savings
– Unclear Benefit from replacement of existing transport infrastructure solutions for the sake of it
– Orchestration and Management put into the spotlight
3. Architectural details both at the system and network level need to be well understood and examined
– E.g. Service Chaining
100
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Call to Action
• Visit the World of Solutions for
– Cisco Campus – CSR Demo, APIC-EM
– Walk in Labs
– Technical Solution Clinics
• Meet the Engineer
• Lunch time Table Topics
• DevNet zone related labs and sessions
101
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Continue Your Education
• Demos in the Cisco Campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
102
Q & A
© 2015 Cisco and/or its affiliates. All rights reserved.BRKVIR-2605 Cisco Public
Give us your feedback and receive a
Cisco Live 2015 T-Shirt!
Complete your Overall Event Survey and 5 Session
Evaluations.
• Directly from your mobile device on the Cisco Live
Mobile App
• By visiting the Cisco Live Mobile Site
http://showcase.genie-connect.com/clmelbourne2015
• Visit any Cisco Live Internet Station located
throughout the venue
T-Shirts can be collected in the World of Solutions
on Friday 20 March 12:00pm - 2:00pm
Complete Your Online Session Evaluation
Learn online with Cisco Live!
Visit us online after the conference for full
access to session videos and
presentations. www.CiscoLiveAPAC.com