Post on 11-Jul-2018
© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicLATAM Data Center Design and Deployment Seminars 2009 1
Data CenterData CenterDesign and DeploymentDesign and Deployment
SeminarsSeminarsLatin America - 2009Latin America - 2009
ArgentinaArgentina BrasilBrasil ChileChile PeruPeru MexicoMexico
© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicLATAM Data Center Design and Deployment Seminars 2009 2
Server Virtualization Networking
&Unified Computing
Design and DeploymentSeminars 2009
Carlos PereiraCarlos PereiraData Center Consulting Systems Engineer – Latin America
carlos.pereira@cisco.com
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 3LATAM Data Center Design and Deployment Seminars 2009
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 4LATAM Data Center Design and Deployment Seminars 2009
Engineering
event effect
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 5LATAM Data Center Design and Deployment Seminars 2009In Peru
In Peru and Argentina
In Argentina, Brasil, Chile, Peru and Mexico
In Argentina, Brasil, Chile, Peru and Mexico
In Argentina, Brasil, Chile, Peru and Mexico
In Argentina, Brasil, Chile, Peru and Mexico
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 6LATAM Data Center Design and Deployment Seminars 2009
Data Center Architecture StrategyThe Data Center Evolution
Mainframe
Phase 1 IT Relevance and C
ontrol
Application Architecture EvolutionCentralized
Phase 2
Client-Server and Distributed Computing
Decentralized
Phase 3
Service Oriented and Web 2.0 Based
Virtualized
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 7LATAM Data Center Design and Deployment Seminars 2009
•Many under utilized servers•Cable sprawl•High power, cooling costs•High CAPEX•For $1 spent on server capex ~$5 spent opex
Today
Server Virtualization – key DC Trend Efficient utilization, Reduce Cable & Power Costs with 10GbE
Joint work between Network, Server & Storage teams is key
•Cable sprawl•Power, cooling costs•Less number of access layer Ethernet ports
Access Layer
ServerFibre-
ChannelEthernet
SAN BSAN ALAN
4 x 1GE
Virtualization Step1
GE
VMVM VMVMVMVM VMVM
VMVM VMVMVMVM VMVM
HypervisorServer hardware
Access Layer
ServerFibre-
ChannelEthernet
SAN BSAN ALAN
10GE
•GE to 10GE in access layer•Less interfaces – reduced Cable sprawl•Savings from power and cooling
Virtualization Step2
10 GE
VMVM VMVMVMVM VMVM
VMVM VMVMVMVM VMVM
HypervisorServer hardware
Access Layer
ServerUnified IO
SAN BSAN ALAN
•Unified I/O - LAN & SAN consolidation•Reduce NICs, HBAs,•Reduce cabling•More Savings from power and cooling•Lower capex
Virtualization Step3
10 GE/FCOE
VMVM VMVMVMVM VMVM
VMVM VMVMVMVM VMVM
HypervisorServer hardware
Servers
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 8LATAM Data Center Design and Deployment Seminars 2009
X
Distributed Resource Scheduling (DRS)
• Balance workloads• Right-size hardware• Optimize real time
High Availability (HA)• Restart immediately when H/W or OS fail
• Protect all apps
On-demand Capacity• Scale without disruption• Reconfigure on the fly• Provision new server
Operational BenefitsThe Agile & Resilient Data Center
Server Virtualization and its Implications Put New Pressure on The Network
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 9LATAM Data Center Design and Deployment Seminars 2009
However …However …
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 10LATAM Data Center Design and Deployment Seminars 2009
Any Workload, Any Server, Anywhere
Transparency and Balance to Virtualization
Network
ServerServer +Virtualization
NetworkUnifiedFabric Server +
Virtualization
Network Architecture defined Server OperationsServer Virtualization changed Network Architectures
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 11LATAM Data Center Design and Deployment Seminars 2009
With virtualization, VMs have a transparent view of their resources…
Transparency in the Eye of the Beholder
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 12LATAM Data Center Design and Deployment Seminars 2009
…but its difficult to correlate network and storage back to virtual machines
Transparency in the Eye of the Beholder
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 13LATAM Data Center Design and Deployment Seminars 2009
Scaling globally depends on maintaining transparency while also providing operational consistency
Transparency in the Eye of the Beholder
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 14LATAM Data Center Design and Deployment Seminars 2009
Some of the “networking issues” associated
Problems:VMotion
• VMotion may move VMs across physical ports—policy must follow
• Impossible to view or apply policy to locally switched traffic
• Cannot correlate traffic on physical links—from multiple VMs
VLAN101
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 15LATAM Data Center Design and Deployment Seminars 2009
Why the Network is Changing
Desire for VM-level access-layer policy
Virtualization is driving higher link utilization
More demanding role of network (i.e. DRS)
Current approaches lead to inconsistent network policies
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 16LATAM Data Center Design and Deployment Seminars 2009
Current View of the Access Layer
Typically provisioned as trunk to the server running ESX
No visibility to individual traffic from each VM
Unable to troubleshoot, apply policy, address performance issues
Boundary of network visibility
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 17LATAM Data Center Design and Deployment Seminars 2009
VN-Link View of the Access Layer
Nexus 1000V and VN-Link provide visibility to the individual VMs
Policy can be configured per-VM
Policy is mobile within the ESX cluster
Boundary of network visibility
Nexus 1000V Nexus 1000V Distributed Virtual SwitchDistributed Virtual Switch
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 18LATAM Data Center Design and Deployment Seminars 2009
Cisco’s Data Center Switching portfolio
Catalyst 4948-10GE
CBS 3100Blade Switches
Catalyst 4900M
Catalyst 6500
Nexus 5000Nexus 7000
Nexus 2148T
with VSS =
=
=with Service Modules
Nexus 1000v
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 19LATAM Data Center Design and Deployment Seminars 2009
Data Center Architecture EvolutionNexus – Optimized for the Virtualized Data Center
DC Virtual Access
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 20LATAM Data Center Design and Deployment Seminars 2009
CBS 31xx Blade
Nexus 5000 & Nexus 2000Top-of-Rack
Nexus 7000 End-of-Row
Nexus 5000& FCoETop-of-Rack
Access Layer
Nexus 700010GbE Agg
MDS 9500Storage
Catalyst 6500End-of-Row
CBS 31xxMDS 9124eNexus blade (*)
10GbE and 4/8Gb FC Server Access10Gb DCE / FCoE Server Access
1GbE Server Access
Nexus 700010GbE Core
Virtualized Data Center InfrastructureGigabit Ethernet
10 Gigabit Ethernet
10 Gigabit DCE
4/8Gb Fiber Channel
10 Gigabit FCoE/DCE
(*) future
SAN BSAN A
Cisco UCS
Cisco Catalyst 6500DC Services
Aggregation LayervPCvPC
vPCvPCCore Layer
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 21LATAM Data Center Design and Deployment Seminars 2009
Understood! Understood! Makes fully sense Makes fully sense … …
… … but, what is that and but, what is that and how does it work ?how does it work ?
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 22LATAM Data Center Design and Deployment Seminars 2009
VMWare vSphere / ESX Server Components
VMWare ESX Server
vSwitchvSwitch
vmnics
vnics
Virtual Machine
Software virtual switch
VMware ESX is a “bare-metal” hypervisor that partitions physical servers in multiple virtual machines
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 23LATAM Data Center Design and Deployment Seminars 2009
VMware ESX 3.x Networking Components
VMs
vmnic0
vmnic1
vNIC
vNIC
Virtual Ports
VM_LUN_0007
VM_LUN_0005
vSwitch0
vSwitch
VMNICS =Uplinks
Per ESX Server Configuration
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 24LATAM Data Center Design and Deployment Seminars 2009
MAC1
VM1
Ethernet1/1
MAC2
VM2
Destination MAC Port
MAC1 1/1
MAC2 1/1
Forwarding Table
?
DMAC = MAC2DMAC = MAC2
Why is a Virtual Switch needed in the first place ?
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 25LATAM Data Center Design and Deployment Seminars 2009
Destination MAC Port
MAC1 1/1
MAC2 1/1
Forwarding Table
VM1
Ethernet1/1
MAC2
VM2
vSwitch or Nexus 1000v
Virtual SwitchingVirtualized Servers Need “VN-link” Technology
MAC1
=Nexus1kv
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 26LATAM Data Center Design and Deployment Seminars 2009
VN-Link (or Virtual Network Link) is a term which describes a new set of features and capabilities that enable VM interfaces to be individually identified, configured, monitored, migrated and diagnosed.
VN-Link requires platform support for Port Profiles, Virtual Ethernet Interfaces, vCenter Integration, and Virtual Ethernet mobility.
Cisco VN-Link
The term literally refers to a VM specific link that is created between the VM and Cisco switch. It is the logical equivalent & combination of a NIC, a Cisco switch interface and the RJ-45 patch cable that hooks them together.
Hypervisor
VNIC VNIC
VETH VETH
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 27LATAM Data Center Design and Deployment Seminars 2009
vNetwork – 3rd Party Virtual SwitchesEnterprise networking vendors can provide their own implementations of the virtual switch leveraging the vNetwork switch API interfaces
Enables support for 3rd party networking capabilities, including monitoring and management of the virtual network
vSwitch
CU
RR
ENT
vSwitch vSwitch
vNetw
ork vNetwork Platform
vNetwork Distributed Switch
vNetwork Platform
Third Party Switch Products
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 28LATAM Data Center Design and Deployment Seminars 2009
vNetwork Distributed Switch (DVS)Both Cisco and VMWARE provide DVS functionalities
http://www.vmware.com/products/vnetwork-distributed-switch/
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 29LATAM Data Center Design and Deployment Seminars 2009
VN-Link With the Cisco Nexus 1000V
Cisco Nexus 1000VCisco Nexus 1000VSoftware Based Industry’s first 3rd-party vNetwork
Distributed Switch for VMware vSphere Built on Cisco NX-OS Compatible with all switching platforms Maintain vCenter provisioning model
unmodified for server administration; allow network administration of virtual network via familiar Cisco NX-OS CLI
Policy-Based Policy-Based VM ConnectivityVM Connectivity
Mobility of Network & Mobility of Network & Security PropertiesSecurity Properties
Non-DisruptiveNon-Disruptive Operational Model Operational Model
vSphere
NexusNexus1000V1000V
Nexus 1000VNexus 1000V
VMVM VMVM VMVM VMVM
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 30LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000V Architecture
Virtual Supervisor Module (VSM) Virtual or Physical appliance
running Cisco OS (supports HA) Performs management, monitoring,
& configuration Tight integration with VMware
Virtual Center
Virtual Ethernet Module (VEM) * Enables advanced networking
capability on the hypervisor
Provides each VM with dedicated “switch port”
Collection of VEMs = 1 Distributed Switch
Cisco Nexus 1000V Enables: Policy Based VM Connectivity Mobility of Network & Security
Properties Non-Disruptive Operational Model
vCenter
VMW ESX
Server 1
VMware vSwitch VMW ESX
Server 2
VMware vSwitch VMW ESX
Server 3
VMware vSwitch
VM #1
VM #4
VM #3
VM #2
VM #5
VM #8
VM #7
VM #6
VM #9
VM #12
VM #11
VM #10
VEM VEM VEMNexus 1000V
** VSM virtual appliance will run on either ESX 3.5 or 4.0 * VEM requires vSphere / ESX 4.0
Cisco VSMs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 31LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000V Components
Cisco VEM
VM1 VM2 VM3 VM4
Cisco VEM
VM5 VM6 VM7 VM7
Cisco VEM
VM9 VM10 VM11 VM12
Virtual Ethernet Module(VEM) Replaces Vmware’s virtual switch Enables advanced switching capability
on the hypervisor Provides each VM with dedicated
“switch ports”
vCenter Server
Virtual Supervisor Module(VSM) CLI interface into the Nexus 1000V Leverages NX-OS 4.04a Controls multiple VEMs as a single
network device
Cisco VSMs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 32LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000VFaster VM Deployment
VMW ESX
Server
VMW ESX
Server
Cisco Nexus 1000V
VM #1
VM #4
VM #3
VM #2
VM #5
VM #8
VM #7
VM #6
VM Connection Policy Defined in the network Applied in Virtual Center Linked to VM UUID
Defined PoliciesWEB Apps
HR
DB
Compliance
Cisco VN-Link—Virtual Network LinkPolicy-Based
VM ConnectivityNon-Disruptive
Operational ModelMobility of Network
& Security Properties
vCenter
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 33LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000VRicher Network Services
VMW ESX
Server
VMW ESX
Server
Cisco Nexus 1000V
VM #5
VM #8
VM #7
VM #6
VM #4
VM #3
VM #2
VM #1
VM #4
VM #3
VM #2
VM #1
VN-Link Property Mobility VMotion for the network Ensures VM security Maintains connection stateVirtual
Center
VMs Need to Move VMotion DRS SW Upgrade/Patch Hardware Failure
Policy-Based VM Connectivity
Non-DisruptiveOperational Model
Mobility of Network & Security Properties
VN-Link: Virtualizing the Network Domain
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 34LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000VIncrease Operational Efficiency
VMW ESX
Server
VMW ESX
Server
Cisco Nexus 1000V
VM #5
VM #8
VM #7
VM #6
VM #4
VM #3
VM #2
VM #1
Network Benefits Unifies network mgmt and ops Improves operational security Enhances VM network features Ensures policy persistence Enables VM-level visibility
Policy-Based VM Connectivity
Non-DisruptiveOperational Model
Mobility of Network & Security Properties
VN-Link: Virtualizing the Network Domain
Virtual Center
Server Benefits Maintains existing VM mgmt Reduces deployment time Improves scalability Reduces operational workload Enables VM-level visibility
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 35LATAM Data Center Design and Deployment Seminars 2009
Network & Server Administrator View
Consistent Workflow: configure network port profiles with Cisco CLI, continue to select Port Groups when configuring a VM from VMware vSphere Client
Network AdministratorCisco CLI or DCNM GUI
Server AdministratorVMware vSphere Client
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 36LATAM Data Center Design and Deployment Seminars 2009
Example: Port Profile (Nexus 1000v VSM view)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 37LATAM Data Center Design and Deployment Seminars 2009
Example: Port Profile (vCenter View)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 38LATAM Data Center Design and Deployment Seminars 2009
Example: Port Profile (VM View)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 39LATAM Data Center Design and Deployment Seminars 2009
Network Intelligence Extending Inside The Virtualized Servers
• Top of Rack switches• End of Row switches• Fabric extenders
• Embedded blade switches for leading server vendors• Pass-through modules
• Soft-switches• VN-Link for VM-aware networking
Rack Servers Blade Servers Virtualized servers
Consistent Networking Services and Management Model for the Data Center Network Infrastructure, Physical and Virtual.
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 40LATAM Data Center Design and Deployment Seminars 2009
What about Microsoft and What about Microsoft and Xen-based virtualization Xen-based virtualization
solutions ?solutions ?
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 41LATAM Data Center Design and Deployment Seminars 2009
Windows Server 2008
VM 2VM 1
“Designed for Windows” Server Hardware
Windows hypervisor
VM 3
Parent Partition Child Partitions
User Mode
KernelMode
Ring -1MgmtNIC 1
iSCSI NIC 2
VSP
VSP
VSwitch 1NIC 3
VSwitch 2NIC 4
Applications Applications Applications
VM Service
WMI Provider
VM Worker Processes
Windows Kernel VSC Windows
Kernel VSC LinuxKernel VSC
VMBus VMBus VMBusVMBus
Hyper-V Setup: Networking & iSCSI
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 42LATAM Data Center Design and Deployment Seminars 2009
Virtual Switch Configura?onParent Par??on LAN Configura?on
Microsoft Hyper-V: Networking configuration
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 43LATAM Data Center Design and Deployment Seminars 2009
Hyper-V Setup: Networking & iSCSI
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 44LATAM Data Center Design and Deployment Seminars 2009
Xen networking (1/2)
Xen Virtual Ethernet interfaces
Xen creates, by default, seven (07) pairs of "connected virtual ethernet interfaces" for use by dom0.
Think of them as two ethernet interfaces connected by an internal crossover ethernet cable.
veth0 is connected to vif0.0, veth1 is connected to vif0.1, etc., up to veth7 -> vif0.7.
source: wikipedia
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 45LATAM Data Center Design and Deployment Seminars 2009
Xen networking (2/2)
Every time a domU instance is created, it is assigned a new domain id number. The first domU will be id #1, the second will be #2, etc.
For each new domU, Xen creates a new pair of "connected virtual ethernet interfaces", with one end in domU and the other in dom0.
For example, domU #5's eth0 is attached to vif5.0.
The default Xen configuration uses bridging within domain 0 to allow all domains to appear on the network as individual hosts.
source: wikipedia
© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicLATAM Data Center Design and Deployment Seminars 2009 46
Nexus1000v Design and
Implementation
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 47LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000vDistributed Virtual Switch
Fabric Function
Linecards Equivalent
vCenter
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
Hypervisor
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
Virtual EthernetModule
Virtual SupervisorModule
N1k-VSM# sh module
Mod Ports Module-Type Model Status1 1 Supervisor Module Cisco Nexus 1000V active *2 1 Supervisor Module Cisco Nexus 1000V standby3 48 Virtual Ethernet Module ok4 48 Virtual Ethernet Module ok
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 48LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000VVirtual Interface
VMWare ESX Server
veth = Virtual Machine port (vnic)
veth3 veth7 veth68
VEM - Module 3
N1k-VSM# sh interface virtual Port Adapter Owner Mod Host
Veth3 Net Adapter 1 Ubuntu VM 1 pe-esx1Veth7 Net Adapter 1 Ubuntu VM 2 pe-esx1Veth68 Net Adapter 1 Ubuntu VM 3 pe-esx1
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 49LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000v Ethernet Interface
VMWare ESX Server
VEM - Module 3
VMWare ESX Server
VEM – Module 4
eth3/1
eth3/2
eth4/1
eth4/2
eth = uplink port on the ESX Server
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
WS-C6504E-VSS#sh cdp neighborsDevice ID Local Intrfce Platform Port ID
N1k-VSM Gig 1/1/1 Nexus1000 Eth 3/1N1k-VSM Gig 2/1/2 Nexus1000 Eth 3/2N1k-VSM Gig 1/8/1 Nexus1000 Eth 4/1N1k-VSM Gig 2/8/2 Nexus1000 Eth 4/2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 50LATAM Data Center Design and Deployment Seminars 2009
MAC Learning
Each VEM learns independently and maintains a separate MAC table
VM MACs are statically mapped
Other vEths are learned this way (vmknics and vswifs)
No aging while the interface is up
Devices external to the VEM are learned dynamically
Cisco VEM
VM3 VM4
Eth4/1
Cisco VEM
VM1 VM2
Eth3/1
VEM 3 MAC Table
VM1 Veth12 Sta?cVM2 Veth23 Sta?cVM3 Eth3/1 DynamicVM4 Eth3/1 Dynamic
VEM 4 MAC Table
VM1 Eth4/1 DynamicVM2 Eth4/1 DynamicVM3 Veth8 Sta?cVM4 Veth7 Sta?c
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 51LATAM Data Center Design and Deployment Seminars 2009
Definition of Port-profile
switchportswitchport access vlan 10switchport mode access
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 52LATAM Data Center Design and Deployment Seminars 2009
Network Administrator view
N1k-VSM# sh port-profile name Ubuntu-VM
port-profile Ubuntu-VM
description:
status: enabled
capability uplink: no
capability l3control: no
system vlans: none
port-group: Ubuntu-VM
max-ports: 32
inherit:
config attributes:
switchport mode access
switchport access vlan 95
no shutdown
assigned interfaces:
Vethernet2
Vethernet4
Port-Profile as viewed from the Network and Server Administrator
Server admin view
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 53LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus1kv Provide Separation of Network and Server Roles
Server AdministratorServer Administrator Network AdministratorNetwork Administrator
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 54LATAM Data Center Design and Deployment Seminars 2009
What makes the Virtual Switch “Distributed”?
ESX servers that are under the same Nexus 1kv VSM share the same Port-Profile Configuration
When a new Port-Profile is defined it gets automatically propagated to all the ESX servers (VEMs) that are the VSM
In this example ESX1 and ESX2 are under VSM1 and share the green and red Port-Profile
ESX3 and ESX4 are under VSM2 and share the Blue and Yellow Port Profile
3 41 2
VSM1 VSM2
Port ProfilesPort Profiles Port ProfilesPort Profiles
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 55LATAM Data Center Design and Deployment Seminars 2009
Prior to DVS Ensuring Port-Group Consistency was a Manual Process
Each ESX host is configured individually for Networking
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 56LATAM Data Center Design and Deployment Seminars 2009
VMotion Requires the Destination vSwitch to have the same Port-Groups/Port-Profiles as the originating ESX host
Prior to DVS you had to manually ensure that the same Port-Group existed on ESX Host 1 as ESX Host 2
VM4
vmnic0
VM5
ESX Host 2
VM6VM1 VM2
ESX Host 1
VM3
vSwitch
Rack10Rack1
vmnic1
vSwitch
vmnic0 vmnic1
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 57LATAM Data Center Design and Deployment Seminars 2009
“Distributed” Virtual Switching facilitates VMotion Migration
VMW ESX
Server 2
VMW ESX
Server 1
VEM
VM #4
VM #3
VM #2
VM #1
VM #4
VM #3
VM #2
VM #1
VMs Need to Move VMotion DRS SW Upgrade/Patch Hardware Failure
VEM
Port Profiles
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 58LATAM Data Center Design and Deployment Seminars 2009
Configuring Access-Lists, Port Security, SPAN, etc… without Nexus1kv is Complicated
Is VM#1 on Server 1? Or on which server, on which switch do I put the ACL?
ACL need to be specify the IP address of the VM else you risk to drop both VM1 and VM3 traffic
SPAN will get all traffic from VM1, VM2, VM3, VM4!! You need to filter that!!
Port Security CAN’T be used
VMW ESX
Server 1
VM #4
VM #3
VM #2
VM #1
ACLs (complicated)
SPAN (realistically can’t be used)
Port Security needs to be disabled
vSwitch
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 59LATAM Data Center Design and Deployment Seminars 2009
You can use Access-Lists, Port Security, SPAN, etc… WITH Nexus1kv
Is VM#1 on Server 1? It doesn’t matter ACL “follows” the VM
SPAN will get only the traffic from the virtual Ethernet Port
Port Security ensures that VMs won’t generate fake MAC addresses
VMW ESX
Server 1
VEM
VM #4
VM #3
VM #2
VM #1
ACLs specific to a Port-Group
SPAN on a virtual ethernet port
Port Security
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 60LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000 DVSNexus 1000 DVS
vNIC Security
VMs can be secured in multiple ways:
VLANs
ACLs
Private VLANs
Port-Security
VM #4
VM #3
Server
VM #2
VM #1
vnics
vmnic
IEEE 802.1q trunk
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 61LATAM Data Center Design and Deployment Seminars 2009
Hypervisor Hypervisor
PromiscuousPort
PromiscuousPort
Community‘A’
Community‘B’
IsolatedPorts
Primary VLAN
Community VLAN
Community VLAN
Isolated VLAN
Only One Subnet
x
Private VLANs can be extended across ESX servers by using the Nexus1kv
Promiscuous ports receive and transmit to all hosts
Communities allow communications between groups
Isolated ports talk to promiscuous ports only
x
x
.11 .12 .13 .14 .15 .16 .17 .18OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 62LATAM Data Center Design and Deployment Seminars 2009
SPAN traffic to a Catalyst 6500 or a Nexus 7k where you have a sniffer attached
Hypervisor
OS
App
OS
App
OS
App
OS
App
Hypervisor
OS
App
OS
App
OS
App
OS
App
Hypervisor
OS
App
OS
App
OS
App
OS
App
Capture here
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 63LATAM Data Center Design and Deployment Seminars 2009
Ease of ProvisioningMaking Blade Servers Deployment Faster
1 Physically Add a new blade (or replace an old one)
2 Go to vCenter, add host to cluster
3 Done:
the new blade is in production
All port-groups appear
Nexus 1000vNexus 1000v
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 64LATAM Data Center Design and Deployment Seminars 2009
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
vSphere / ESX Server NIC Teaming
VMWare ESX Server
vSwitch – Module 3
VMWare ESX Server
vSwitch – Module 4
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 65LATAM Data Center Design and Deployment Seminars 2009
Loop Prevention without STP
Cisco VEM
VM1 VM2 VM3 VM4
Cisco VEM
VM5 VM6 VM7 VM7
Cisco VEM
VM9 VM10 VM11 VM12
BPDU are dropped
Eth4/1 Eth4/2
BPDUX
X
No Switching From Physical NIC to NIC
Local MAC Address Packets Dropped on
Ingress (L2)
X
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 66LATAM Data Center Design and Deployment Seminars 2009
VEM Forwarding
VMWare ESX Server
VEM - Module 3
No Spanning tree
BPDU are dropped
MAC A MAC B MAC C
VEM MAC Table
MAC A
MAC B
MAC C
Local MAC Adress are switched locally
Everything else send to upstream switch
BPDUBPDU
DMAC : C DMAC : X
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 67LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000VNIC Teaming and Load-Balancing
VMWare ESX Server
The Nexus 1000V load balance based on 16 different parameters
N1k-VSM(config)# port-channel load-balance ethernet ? dest-ip-port Destination IP address and L4 port dest-ip-port-vlan Destination IP address, L4 port and VLAN destination-ip-vlan Destination IP address and VLAN destination-mac Destination MAC address destination-port Destination L4 port source-dest-ip-port Source & Destination IP address and L4 port source-dest-ip-port-vlan Source & Destination IP address, L4 port and VLAN source-dest-ip-vlan Source & Destination IP address and VLAN source-dest-mac Source & Destination MAC address source-dest-port Source & Destination L4 port source-ip-port Source IP address and L4 port source-ip-port-vlan Source IP address, L4 port and VLAN source-ip-vlan Source IP address and VLAN source-mac Source MAC address source-port Source L4 port vlan-only VLAN only
VSM
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 68LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000VvPC Host Mode
VMWare ESX Server
VEM
The Nexus 1000V detect the upstream Cisco switch and create automatically - using CDP - a port-channel bundling all the links to the same switch
OS
App
OS
App
OS
App
VSM
N1k-VSM#sh cdp neighborsDevice ID Local Intrfce Platform Port ID
N1k-VSM Eth 3/1 WS-4900-1 Gig 1/1 N1k-VSM Eth 3/2 WS-4900-1 Gig 1/2
N1k-VSM Eth 3/3 WS-4900-2 Gig 1/1 N1k-VSM Eth 3/4 WS-4900-2 Gig 1/2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 69LATAM Data Center Design and Deployment Seminars 2009
Nexus 1000v with VSS or vPC
VMWare ESX Server
VEM - Module 3
VMWare ESX Server
VEM - Module 4
Nexus 1000v leverage cdp to create an etherchannel as soon as the same upstream switch is seen on the VEM uplink.
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 70LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 1000V Scalability
A single Nexus 1000V• 66 modules (2x Supervisors and 64x Ethernet Modules)
Virtual Ethernet Module: • 32 physical NICs
• 256 virtual NICs
Limit Per Nexus 1000V• 512 Port Profiles
• 2048 physical ports
• 8,192 virtual ports (vmknic, vswif, vnic)
Virtual Supervisor - StandbyVirtual Supervisor - Standby
VEMVEM
VEMVEM
VEMVEM
VEMVEM
VEMVEM
VEM VEM
VEMVEM
VEMVEM
VEMVEM
VEMVEM
Virtual Supervisor - ActiveVirtual Supervisor - Active
Nexus 1000V
© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicLATAM Data Center Design and Deployment Seminars 2009 71
Nexus1000v Design Examples
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 72
VSM VM Placement
Cisco VEM
VM1 VM2 VM3
Cisco VEM
VM4 VM5 VM6 VM7
Cisco VEM
VM8 VM9 VM10
VSM - Virtual Appliance Primary and Secondary VSMs should
remain on separate machines VMware anti-affinity rules can be
applied
VSM-VA-1 VSM-VA-2
VSM - Performance Requires 2GB dedicated RAM (not
shared) 1Ghz vCPU VSM should not be VMotioned
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 73
Two NIC Configuration
Cisco VEM
VM Data
Po1SG0 SG1
C P
VMK SC
Access Layer ConfigurationTrunk port
No EtherChannel
VEM ConfigurationSource Based Hashing
N1K Port ChannelSingle PC (vPC-HM)VM Data, Service Console, VM Kernel, Control and
Packet
Use CaseSmall 1Gb servers (rack or blade)10Gb (Ethernet or FCoE)
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 74
Four NIC Configuration
Cisco VEM
VM Data
Po2SG0 SG1
C P
VMK SC
Access Layer ConfigurationTrunk port
No EtherChannel
VEM ConfigurationSource Based Hashing
N1KV Port Channel 1vPC-HMVM Data
Use CaseMedium 1Gb servers (rack or blade)Need to separate VMotion from Data
Po1SG0 SG1
N1KV Port Channel 2vPC-HMService Console, VM Kernel, Control and Packet
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 75
Four NIC Alternative-1 Configuration
Cisco VEM
VM Data
C P
VMK SC
Access Layer ConfigurationTrunk port
Single EtherChannels spanning both switchs
VEM ConfigurationFlow Based Hashing
Port Channel 1Standard EtherChannelVM Data, Service Console, VM Kernel, Control and
Packet
Use Case‘Clustered’ Switches (7K vPC, 6K VSS, 3K VBS)Maximizes VM bandwidthShared links for VMotion and Data
Po1
Clustered Switches
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 76
Four NIC Alternative-2 Configuration
Cisco VEM
VM Data
Po2
C P
VMK SC
Access Layer ConfigurationTrunk port
Two EtherChannels spanning each switch
VEM ConfigurationFlow Based Hashing
N1KV Port Channel 1Standard EtherChannelVM Data
Use Case‘Clustered’ Switches (7K vPC, 6K VSS, 3K VBS)Still maintains separation of Data and VMotion
Po1 N1KV Port Channel 2Standard EtherChannelService Console, VM Kernel, Control and Packet
Clustered Switches
© 2009 Cisco Systems, Inc. All rights reserved.N1KV_Technical_v1 77
Six NIC Configuration
Cisco VEM
VM Data
C P
VMK SC
Access Layer ConfigurationTrunk port
Separate EtherChannels from each switch to Po1 only
VEM ConfigurationFlow Based Hashing
N1KV Port Channel 1vPC-HMVM Data
Use CaseHigh performance serversGreater than 1Gb VM bandwidthSeperate links for VMotion and Data
Po1SG0 SG1
Po2 N1KV Port Channel 2vPC-HMService Console, VM Kernel, Control and Packet
SG0 SG1
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 78LATAM Data Center Design and Deployment Seminars 2009
That’s cool !That’s cool !
1.1.Can you do this in Can you do this in hardware as well ?hardware as well ?
2.2.What are the next What are the next steps ?steps ?
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 79LATAM Data Center Design and Deployment Seminars 2009
YES!VN-Link with Network Interface Virtualization
Nexus Switch with VN-LinkHardware Based
Extend Network Interface Virtualization to adaptors and hypervisors
Allows scalable hardware-based implementations through hardware switches
Standards-based initiative: Cisco & VMware proposal to specify “Network Interface Virtualization”
Combines VM and physical network operations into one managed node
VMW ESX
VM #4
VM #3
ServerVM #2
VM #1
VN-Link
Nexus
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 80LATAM Data Center Design and Deployment Seminars 2009
One of the reasons behind it:The proliferation of Control Points
Introduction of blade switches and rack switches
Evolution of the access layer has led to an increasingly complex management environment and switch topology
Introduction of hypervisors
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 81LATAM Data Center Design and Deployment Seminars 2009
Introducing Network Interface Virtualization (NIV)
Logically collapse access layer to simplify management and architecture
Proposal to
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 82LATAM Data Center Design and Deployment Seminars 2009
Cisco Nexus 2000First Product Based On Network Interface Virtualization
Nexus 7000
Nexus 5000
Nexus 2000
Aggregation
Access
AvailableNow
• Consistent NX-OS across all switches
• Simplified Management• GbE/10GbE Flexibility
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 83LATAM Data Center Design and Deployment Seminars 2009
Server Connectivity Evolution – Present
Shift towards server virtualization
Multiple VMs inside each physical server, connected by virtual switches
Rapid proliferation of logical elements that need to be managed
Feature parity issues between virtual and physical elements
Separate management of physical ( ) and logical ( ) elementsSeparate management of physical ( ) and logical ( ) elements
VMsvNICs
VSwitch
VMsvNICs
VSwitch
VMsvNICs
VSwitch
VMsvNICs
VSwitch
Management Challenges
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 84LATAM Data Center Design and Deployment Seminars 2009
Server Connectivity Evolution
Virtual Interfaces within VMs and physical interfaces are now visible to the switch
Both network configuration and policy enforcement for these interfaces can now be driven from the switch
This allows consolidated management of physical and virtual elements
Consolidated management of physical ( ) and logical elementsConsolidated management of physical ( ) and logical elements
VSwitch VSwitch
With Network Interface Virtualization: Consolidated Management
VMsvNICs
VSwitch
VMsvNICs
VSwitch
VMsvNICs
VMsvNICs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 85LATAM Data Center Design and Deployment Seminars 2009
VirtualizationPlatform
ComputePlatform
Network Platform
Resource Scaling
Unified Fabric
Dynamic Prov
Process Automation
Business Service Management
PerformanceManagement
Hypervisor Optimization
AutomatedProvisioning
Cisco Unified Computing System delivers the next step in the Data Center 3.0 vision by uniting network, computing, and virtualization resources into a seamless system.
The resulted integration: Unified Computing
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 86LATAM Data Center Design and Deployment Seminars 2009
“Virtual Data Center” Evolution Path
Unified Computing System
Consolidation Virtualization Automation Utility Cloud
Data Center Networking
Unified Fabric
Unified Computing
Enterprise Cloud
Inter - Cloud
LocationFreedom
HWFreedom
ProvisioningFreedom
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 87LATAM Data Center Design and Deployment Seminars 2009
The Unified Computing Journey
Unified Fabric
• Wire once infrastructure
• Low-latency lossless
• Virtualization aware
Unified Computing
• Consolidated Fabric & I/O
• Stateless• Vn-tagging• Management
Data Center 3.0
• Business service focused
• Resilient• Distributed • Standards-
based
Unified Virtual
Machines
• VN - Link• Application
Mobility
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 88LATAM Data Center Design and Deployment Seminars 2009
Unified Computing Building BlocksUnified Fabric Introduced with the Cisco Nexus Series
Physical Wire once infrastructure
(Nexus 5000) Fewer switches, adapters,
cables
Ethernet FibreChannel
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 89LATAM Data Center Design and Deployment Seminars 2009
Unified Computing Building BlocksUnified Fabric Introduced with the Cisco Nexus Series
Physical Wire once infrastructure
(Nexus 5000) Fewer switches, adapters,
cables
Virtual VN-Link (Nexus 1000v) Manage virtual the same as
physical
Virtual
Physical
Ethernet FibreChannel
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 90LATAM Data Center Design and Deployment Seminars 2009
Unified Computing Building BlocksUnified Fabric Introduced with the Cisco Nexus Series
Physical Wire once infrastructure
(Nexus 5000) Fewer switches, adapters,
cables
Virtual VN-Link (Nexus 1000v) Manage virtual the same as
physical
Scale Fabric Extender (Nexus 2000) Scale without increasing points
of management
Virtual
Physical
Ethernet FibreChannel
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 91LATAM Data Center Design and Deployment Seminars 2009 91
Mgmt Server
Cisco Unified Computing solutionMgmt ServerEmbed management
Unify fabrics
Optimize virtualization
Remove unnecessary switches,
adapters,
management modules
Less than 1/2 the support infrastructure for a given workload
Mgmt Server
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 92LATAM Data Center Design and Deployment Seminars 2009 92
Mgmt Server
Cisco Unified Computing solutionA single system that encompasses:Network: Unified fabric
Compute: Industry standard x86
Storage: Access options
Virtualization optimized
Unified management modelDynamic resource provisioning
Efficient ScaleCisco network scale & services
Fewer servers with more memory
Lower costFewer servers, switches, adapters, cables
Lower power consumption
Fewer points of management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 93LATAM Data Center Design and Deployment Seminars 2009 93
Cisco Unified Computing solution
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 94LATAM Data Center Design and Deployment Seminars 2009 94
SAN B
Cisco Unified Computing solutionSingle, scalable integrated system
Network + compute virtualization
Dynamic resource provisioning
Mgmt SAN ALAN
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 95LATAM Data Center Design and Deployment Seminars 2009
UCS ManagerEmbedded– manages entire system
UCS Fabric Interconnect20 Port 10Gb FCoE40 Port 10Gb FCoE
UCS Fabric ExtenderRemote line card
UCS Blade Server ChassisFlexible bay configurations
UCS Blade Server Industry-standard architecture
UCS Virtual AdaptersChoice of multiple adapters
UCS Building Blocks
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 96LATAM Data Center Design and Deployment Seminars 2009
UCS ManagerEmbeddedManages entire system
UCS Fabric Interconnect20 Port 10Gb FCoE40 Port 10Gb FCoE
UCS Fabric ExtenderRemote line card
UCS Blade Server ChassisFlexible bay configurations
UCS Blade ServerIndustry-standard architecture
UCS Virtual AdaptersChoice of multiple adapters
Cisco UCS and Nexus Technology
VN-LinkNexus 1000V
CNAs with FCoE
Nexus 2000Fabric Extender
Nexus 5000Unified Fabric
Nexus ProductsUCS Components
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 97LATAM Data Center Design and Deployment Seminars 2009
Cisco Unified Computing System (UCS) – Physical
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 98LATAM Data Center Design and Deployment Seminars 2009
Top of Rack Interconnect(40 or 20 10GE ports) + (2 or 1 GEM uplink slots)
ChassisUp to 8 half width blades or 4 full width blades
Fabric ExtenderHost to uplink traffic engineeringUp to 80Gb Flexible bandwidth allocation
Adapter – 3 optionsCisco Virtualized adapter Compatibility CNAs (Emulex and QLogic) – Native FC +
Intel OplinIntel Oplin - (10GE only)
Compute Blade
Blade Enclosure
x86 Computer x86 Computer
X
II
x8x8x8x8
B
MGMT
SS
B
X X X X X
CC
A
G G
G G
SAN
G
R
A
G
G G
G
R
G
PM P
SANLAN
FabricInterconnect
FabricInterconnect
FabricExtender
FabricExtender
Compute Blade(Half slot)
Adapter
Compute Blade(Full slot)
AdapterAdapter
Cisco Unified Computing System (UCS) – Physical
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 99LATAM Data Center Design and Deployment Seminars 2009
Enclosure, Fabric Switch, & Blades (Front)
6U Enclosure
1U or 2U Fabric Switch
Up to eight per enclosure
(Optional)
Up to four per enclosure
Mix blade types
Ejector Handles
Full width server blade
Half width server blade
Hot Swap SAS drive
Redundant, Hot Swap Power Supply Redundant, Hot Swap Fan
Redundant, Hot Swap Power Supply
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 100LATAM Data Center Design and Deployment Seminars 2009
Rear View of Enclosure and Fabric Switch
Redundant Fabric Extender
RedundantHot SwapFan Module
Fan Handle
10GigE Ports Expansion Bay
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 101LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters Options
Converged network adapters (CNA)
Ability to mix and match adapter types within a system
Automatic discovery of component types
Virtual Machine Aware: Virtualization and
Consolidation
Existing Driver Stacks Proven 10GbE Technology
CostCompatibilityVirtualization
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 102LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters: Interface Views
10 GigE Backplane interfaces to IOMsPhysical Interfaces
vHBAs & vNICs will be bound to these physical interface
Intel Oplin will not have HBA component. Could run FCoE software stack
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 103LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters: CLI view Required to scope to correct chassis/blade/adaptor rtp-6100-B# scope adapter 1/5/1
Note: Only one adaptor on the half slot bladertp-6100-B# scope adapter 1/5/2 Error: Managed object does not exist
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 104LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters: vHBA Detail IdentificationVendor
Provisioned WWN and if bound to Profile
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 105LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters: Ethernet vNIC details
Ethernet stats
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 106LATAM Data Center Design and Deployment Seminars 2009
Cisco Unified Computing System (UCS) – Logical
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 107LATAM Data Center Design and Deployment Seminars 2009
Unified Computing Key Value Propositions:Drivers for use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 108LATAM Data Center Design and Deployment Seminars 2009
Server Attributes / Configuration Points 1/3
ServerServerIdentity (UUID)
Adapters
Number
Type: FC, Ethernet
Identity
Characteristics
Firmware
Revisions
Configuration settings
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 109LATAM Data Center Design and Deployment Seminars 2009
NetworkNetworkUplinks
LAN settings
vLAN, QoS, etc…
SAN settings
vSAN
Firmware
Revisions
ServerServerIdentity (UUID)
Adapters
Number
Type: FC, Ethernet
Identity
Characteristics
Firmware
Revisions
Configuration settings
Server Attributes / Configuration Points 2/3
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 110LATAM Data Center Design and Deployment Seminars 2009
StorageStorageOptional Disk usageSAN settingsLUNsPersistent BindingFirmwareRevisions
NetworkNetworkUplinks
LAN settings
vLAN, QoS, etc…
SAN settings
vSAN
Firmware
Revisions
ServerServerIdentity (UUID)
Adapters
Number
Type: FC, Ethernet
Identity
Characteristics
Firmware
Revisions
Configuration settings
Server Attributes / Configuration Points 3/3
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 111LATAM Data Center Design and Deployment Seminars 2009
Traditional Server Deployment
Server Administrator: Configure management LAN Upgrade firmware versions
– Chassis, BMC, BIOS, adapters Configure BIOS settings Configure NIC settings Configure HBA settings Configure boot parameters
Storage Administrator: Configure LUN access
– Masking, binding, boot LUN Configure switch
– Zoning, VSANs, QoS
Network Administrator: Configure LAN access
– Uplinks, VLANs Configure policies
– QoS, ACLs
Perform tasks for each server
Inhibits “pay-as-you-grow” incremental deployment– Needs admin coordination every time– May incur downtime during deployments
Complex server replacement, upgrade, migration process– Most of these tasks need to be performed for replacement server
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 112LATAM Data Center Design and Deployment Seminars 2009
Definable Attributes Disks & usage
Network
Type: FC, Ethernet, etc.
Number
Identity
Characteristics
LAN settings
vLAN, QoS, etc…
SAN settings
LUNs
vSAN & Persistent Binding
Firmware
Revisions
Configuration settings
Identity (BIOS)
UCS Server Profile Opt-in Choices Fixed AttributesProcessors
Memory Capacity
Bandwidth Capacity
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 113LATAM Data Center Design and Deployment Seminars 2009
NetworkNetworkUplinks
LAN settings
vLAN
QoS
etc…
Firmware
Revisions
StorageStorageOptional Disk usageSAN settingsLUNsPersistent BindingSAN settingsvSANFirmwareRevisions
UCS Service Profile
ServerServerIdentity (UUID)
Adapters
Number
Type: FC, Ethernet
Identity
Characteristics
Firmware
Revisions
Configuration settings
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 114LATAM Data Center Design and Deployment Seminars 2009
• Blade identities can be duplicated, automatically moved and deployed, and failed-over to another blade
• Firmware and bios included• “Stateless” environment• Significant process/labor savings
Service Profile• Encapsulation of HW state – MAC, WWN, Firmware, BIOS
Service Profile Service Profile
Service ProfileService Profile
UCS Service Profile
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 115LATAM Data Center Design and Deployment Seminars 2009
UCS Service ProfilesHardware “State” Abstraction
Separate firmware, addresses, and parameter settings from server hardware
Separate access port settings from physical ports
Physical servers become interchangeable hardware components
Easy to move OS & applications across server hardware
BMC FirmwareMAC AddressNIC FirmwareNIC Settings
Drive Controller F/WDrive Firmware
UUIDBIOS FirmwareBIOS Settings
Boot Order
WWN AddressHBA FirmwareHBA Settings
State abstracted from hardware
LAN Connectivity SAN ConnectivityOS & Application
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 116LATAM Data Center Design and Deployment Seminars 2009
Don’t I get this already from VMware?Hypervisors & Hardware State
Hardware State Virtualization
Server Virtualization(VMware, Xen, HyperV, etc.)
BMC FirmwareMAC AddressNIC FirmwareNIC Settings
Drive Controller F/WDrive Firmware
UUIDBIOS FirmwareBIOS Settings
Boot Order
WWN AddressHBA FirmwareHBA Settings
HYPERVISOR
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
Server virtualization & hardware state abstraction are independent of each other
Hypervisor (or OS) is unaware of underlying hardware state abstraction
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 117LATAM Data Center Design and Deployment Seminars 2009
UCS Service Profiles End to End Configure of Full UCS HW Stack
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 118LATAM Data Center Design and Deployment Seminars 2009
Server Upgrades: Within a UCS
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Disassociate server profile from old server
Associate server profile to new server
Old server can be retired or re-purposed
Old Server New Server
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 119LATAM Data Center Design and Deployment Seminars 2009
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Upgrades: Across UCS Instances
1. Disassociate server profiles from servers in old UCS System
1. Migrate server profiles to new UCS system
1. Associate server profiles to hardware in new UCS system
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Old UCS System New UCS System
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 120LATAM Data Center Design and Deployment Seminars 2009
Server Upgrades:Across UCS Instances
1. Disassociate server profiles from servers in old UCS system
1. Migrate server profiles to new UCS system
1. Associate server profiles to hardware in new UCS system
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Old System New System
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 121LATAM Data Center Design and Deployment Seminars 2009
Server Upgrades:Across UCS Instances
1. Disassociate server profiles from servers in old UCS system
1. Migrate server profiles to new UCS system
1. Associate server profiles to hardware in new UCS system
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: finance-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Old System New System
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 122LATAM Data Center Design and Deployment Seminars 2009
Dynamic Server Provisioning
Server Name: web-server-01UUID: 56 4d cd 3f 59 5b 61…MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LANFirmware: xx.yy.zz
Server Name: app-server-01UUID: 65 d4 cd f3 59 5b 16…MAC : 08:00:69:02:01:16WWN: 5080020000076789Boot Order: SAN, LANFirmware: xx.yy.zz
Profiles for Web Servers Profiles for App Servers
Apply appropriate profile to provision a specific server type
Same hardware can dynamically be deployed as different server types
No need to purchase custom configured servers for specific applications
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 123LATAM Data Center Design and Deployment Seminars 2009
Total Servers: 18
Today’s Deployment:Provisioned for peak capacity
Spare node per workload
Blade
Blade
Blade
Blade
Blade
Blade
Blade
Blade
Web Servers
Blade
Blade
Blade
Blade
Blade
Oracle RAC
Blade
Blade
Blade
Blade
Blade
VMware
Server Profiles - Reduce Overall Server CAPEX
Blade
Blade
Blade
Blade
Web Servers
Blade
Blade
Blade
Oracle RAC
Blade
Blade
Blade
VMware
HA Spare
BurstCapacity
Hot SpareBurst Capacity SpareNormal use Blade
Blade
Blade
Total Servers: 14
Blade
With Server Profiles:– Resources provisioned as needed– Same availability with fewer spares
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 124LATAM Data Center Design and Deployment Seminars 2009
Physical Servers
Server Profiles
Run-time association
Server NameUUIDMACWWN
Boot infoLAN ConfigSAN Config
Server NameUUIDMACWWN
Boot infoLAN ConfigSAN Config
Server NameUUID, MAC,WWN
Boot infofirmware
LAN, SAN ConfigFirmware…
Dynamic Management
Server profilesAbstracts server characteristics from
the physical server hardware
Pre-defined and pre-created server identities
Default is shipped hardwareStored in switch
“Associated” with a physical serverManual or policy-driven
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 125LATAM Data Center Design and Deployment Seminars 2009
Stateless Computing
Server attributes no longer tied to physical hardware
Not just identity
Seamless server mobility
Within switch domain
Network boot (LAN or SAN)Boot order and devices are part of server profile
Local disks can be used for temp, swap, etc.
Scrubbed between use (optional)
SAN LAN
Chassis-1/Blade-5
Chassis-9/Blade-2
Server Name: LS-AUUID: 56 4d cd 3f 59 5b 61… MAC : 08:00:69:02:01:FCWWN: 5080020000075740Boot Order: SAN, LAN
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 126LATAM Data Center Design and Deployment Seminars 2009
Unified Computing Key Value Propositions:Drivers for use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 127LATAM Data Center Design and Deployment Seminars 2009
Blade C
hassis 10GE/FCoE
LANIPC
Unified Fabric
SAN
Blade
Blade
Unified Fabric Fewer switches
Fewer adapters
All I/O types available in each chassis
10GE & FCoE
LAN, SAN, IPC
Easier to manage
Blades can work with any chassis
Small network domain
Today’s Approach All fabric types have
switches in each chassis
Repackaged switches
Complex to manage
Blade-chassis configuration dependency
Costly
Small network domain
Blade
Blade
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 128LATAM Data Center Design and Deployment Seminars 2009
Fabric extender• Manage oversubscription2:1 to 8:1• FCoE from blade to fabric switch• Customizable bandwidth
High performance backplane• 2x 40G total bandwidth per half slot - 8 lanes of 10G (half-slot)- 16 lanes of 10G (full-slot)• Redundant data and management paths• Support auto discover of all component
Backplane and Fabric Extender
Backplane Fabric Extender
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 129LATAM Data Center Design and Deployment Seminars 2009
UCS: Overall System (Rear)
Uplinks
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 130LATAM Data Center Design and Deployment Seminars 2009
Unified Compute Key Value Propositions:Drivers for use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 131LATAM Data Center Design and Deployment Seminars 2009
What is SR-IOV about? Single Root IO Virtualization (SR-IOV) allows “virtualizing” the 10 GigE link (via the PCI-Express bus) into multiple
“virtual links”.
SR-IOV is a PCI-Sig standard
In other words you can create multiple “vmnics” each with its own bandwidth allocation
Server
VM1
vnic
VM2
vnic
Virtual Switch
vmnic
VM3
vnic
VM4
vnic
Virtual Switch
vmnic
pNIC: 10 Gbps
This is what SR-IOV enables
This could be Nexus 1000v
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 132LATAM Data Center Design and Deployment Seminars 2009
UCS Adapters Options
CostCostCompatibilityCompatibilityVirtualizationVirtualization
PCIe x16
10GbE/FCoE
vNICs
Eth
0
FC
1
QP
2
FC
3
Eth
127
10GbE/FCoE
PCIe Bus
FC10GbE Software FCoE
“Free” SAN Access for Any Ethernet Equipped
Host
Existing Driver Stacks
VM I/O Virtualization and Consolidation
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 133LATAM Data Center Design and Deployment Seminars 2009
Virtualized adapter designed for both single-OS and VM-based deployments
Provides mobility, isolation, and management from the networkSecureTransparent to hosts
Cut-through architecture High Performance2x 10GbLow latency High BW IPC support
128 vNICsEthernet, FC or SCSI500K IOPSInitiator and Target mode
Cisco UCS Virtualized Adapter
PCIe x16
10GE/FCoE
MAC 0 MAC 1
Eth
0
FC
1
SCSI
2
FC
3
Eth
127
User Defineable
vNICs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 134LATAM Data Center Design and Deployment Seminars 2009
Enables Passthrough I/O
Palo
Host IOMMU
Device Driver Device Driver Device Driver
DeviceManager
Guest OS Guest OS Guest OS
Virtualization Layer
vNIC vNIC vNIC
vNICs appear as independent PCIe devices
Centrally manageable and configurable
Hot-pluggable Virtual NICs
Different types: Eth, FC, SCSI, IPC
Guest drives device directly
Use Cases:
I/O Appliances
High Performance VMs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 135LATAM Data Center Design and Deployment Seminars 2009
Compute Blade
Network Interface Virtualization adapter Vary nature and number of PCIe interfacesEthernet, FC, SCSI, IPC Up to 128 different PCIe devicesHot-pluggable - only appear when definedPCI-Sig IOV compliant Part of Server Array fabricCentrally managed and configured
NIV AdapterFC FC Eth Eth Eth FC IPCSCSISCSIEthEth EthEth
OS
Cisco UCS Virtualized Adapter
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 136LATAM Data Center Design and Deployment Seminars 2009
User Configuration – Example
Class Name FC Gold Ethernet BE
COS Value 3 1 0
Drop/No-Drop No-Drop Drop Drop
Strict Priority No No No
Bandwidth/Weight 1 (20%) 3 (60%) 1 (20%)
Global System Class Definitions
FC Traffic High PriorityEthernet
Best EffortEthernet
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 137LATAM Data Center Design and Deployment Seminars 2009
Unified Compute Key Value Propositions:Drivers for use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 138LATAM Data Center Design and Deployment Seminars 2009
Blade OverviewCommon Attributes
Differences
Half-width blade Full-width blade
2 x Intel Nehalem-EP processors
2 x SAS hard drives (optional)
Blade Service processor
Blade and HDD hot plug support
Stateless blade design
10Gb CNA and 10GbE adapter options
Half-width blade
12 x DIMM slots 48 x DIMM slots
1 x dual port adapter 2 x dual port adapters
4x memory
Full-width blade
2x I/O bandwidth
4x the memory
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 139LATAM Data Center Design and Deployment Seminars 2009
2 socket Nehalem-EP blade
48 x DDR3 DIMMs
2 x Mezzanine Cards
2 x Hot swap disk drives
Up to 384GB per 2 socket blade
Transparent to OS and applications
Reduced server costs–Purchase fewer servers for memory-bound applications
Reduced power and cooling costs
Reduced software costs–Most software is licensed on a per-socket basis
Full-Height Blade
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 140LATAM Data Center Design and Deployment Seminars 2009
Expanded Memory Blade
8GB
8GB
8GB
8GB
8GB
8GB
Nehalem-EP Processor
Slot 16Slot 17Slot 18Slot 19
Slot 20Slot 21
Channel 0(green)
Channel 1(blue)
Channel 2(red) 8GB
8GB
Slot 22Slot 23
8GB
8GB
8GB
8GB
8GB
8GB Slot 8Slot 9
Slot 10Slot 11
Slot 12Slot 13
8GB
8GB
Slot 14Slot 15
8GB
8GB
8GB
8GB
8GB
8GB Slot 0Slot 1Slot 2Slot 3
Slot 4Slot 5
8GB
8GB
Slot 6Slot 7
Nehalem-EP Processor
Slot 16Slot 17Slot 18Slot 19
Slot 20Slot 21
Channel 0(green)
Channel 1(blue)
Channel 2(red)
Slot 22Slot 23
Slot 8Slot 9
Slot 10Slot 11
Slot 12Slot 13Slot 14Slot 15
Slot 0Slot 1Slot 2Slot 3
Slot 4Slot 5Slot 6Slot 7
32GB
32GB
32GB
32GB
32GB
32GB
Physical View
Logical View
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 141LATAM Data Center Design and Deployment Seminars 2009
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 142LATAM Data Center Design and Deployment Seminars 2009 142
Key Characteristics & Benefits Resource Expansion: Savings
Memory Capacity & Pricing
Capacity Speed DIMMs Cost* DIMMs Cost* Savings64 GB 1066 Mhz 4x 8GB & 8x 4GB $7,860 32x 2GB $3,840 51%96 GB 1066 Mhz 12x 8GB $15,780 48x 2GB $5,760 63%
192 GB 1066 Mhz 12x 16GB** $78,900 48x 4GB $15,600 80%384 GB 1066 Mhz 12x 32GB** N/A 48x 8GB $63,120 N/A
* DDR2 pricing as of 12/08 – estimated for DDR3 at 12/09** Nonexistent or Nonstandard DIMM (MetaRAM or other). Est. MetaRAM pricing.
1
2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 143LATAM Data Center Design and Deployment Seminars 2009
I/O
CPU
Memory
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 144LATAM Data Center Design and Deployment Seminars 2009
Unified Compute Key Value Propositions:Drivers for use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 145LATAM Data Center Design and Deployment Seminars 2009
LANSAN B
Unified Management (1/2)
SAN A
Blade C
hassis
Blade C
hassis
Blade C
hassis
Blade C
hassis
Two Failure Domains Separate fabrics
Central supervisor, forwarding logic
Distributed Fabric Extenders
Traffic isolation
Oversubscription
Infrastructure Management Centralize chassis managementIntrinsic system management
Single management domain Scalable architecture
10GE/FCoE10GE/FCoE
ChassisManagement
ChassisManagement
ChassisManagement
ChassisManagement
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 146LATAM Data Center Design and Deployment Seminars 2009
Unified Management (2/2)
UCS Manager
Single point of device managementAdapters, blades, chassis, LAN & SAN connectivityEmbedded managerGUI & CLI
Standard APIs for systems managementXML, SMASH-CLP, WSMAN, IPMI, SNMPSDK for commercial & custom implementations
Designed for multi-tenancyRBAC, organizations, pools & policies
XML API
GUI
Custom Portal
Systems ManagementSoftware
Standard APIs
View 1 View 2
CLI
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 147LATAM Data Center Design and Deployment Seminars 2009
UCS Conceptual Overview
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 148LATAM Data Center Design and Deployment Seminars 2009
UCS Resources - Example
• Server Blades
• Adapters
Physical • UUIDs• VLANs• IP Address• MAC Address• VSANs• WWNs
Logical
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 149LATAM Data Center Design and Deployment Seminars 2009
05:00:1B:32:00:00:00:0405:00:1B:32:00:00:00:03
05:00:1B:32:00:00:00:02
01:23:45:67:89:0d01:23:45:67:89:0c
01:23:45:67:89:0b
blade 3blade 2
Resource Pools - Example
blade 1blade 0
01:23:45:67:89:0a
05:00:1B:32:00:00:00:01
Blade pool
MAC pool
Blades
MACs
WWNsWWN pool
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 150LATAM Data Center Design and Deployment Seminars 2009
How They Work together
UCS ServerUCS Server
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 151LATAM Data Center Design and Deployment Seminars 2009
Profiles – Example Servers Virtual Machines Ethernet Adapters Fibre Channel Adapters IPMI Profiles
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 152LATAM Data Center Design and Deployment Seminars 2009
Out-of-the-Box Protocol Support
SMASH CLP
Remote KVM UCS CLI and GUI
UCS XML API
CIM XMLIPMI
SNMP
Serial Over LAN
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 153LATAM Data Center Design and Deployment Seminars 2009
UCS Manager loaded from 6100 switchPoint a browser at IP address of switch
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 154LATAM Data Center Design and Deployment Seminars 2009
UCS Graphical interface
CONTENT PANENAVIGATION PANE
Top directory map tells you where you are in tree
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 155LATAM Data Center Design and Deployment Seminars 2009
Navigation Pane TabsEquipment | Servers | LAN | SAN | VM | Admin
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 156LATAM Data Center Design and Deployment Seminars 2009
Creation Wizards
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 157LATAM Data Center Design and Deployment Seminars 2009
Multi-Tenancy Model (opt-in)Network
ManagementCompany
HR Finance
Facilities
Policies
PoliciesServerServer
ServerServer
ServerServer
ServerServer
ServerServer
ServerServer
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Bla
de C
hass
isB
lade
Cha
ssis
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Bla
de C
hass
isB
lade
Cha
ssis
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Bla
de C
hass
isB
lade
Cha
ssis
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Bla
de C
hass
isB
lade
Cha
ssis
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
Bla
de C
hass
isFa
bric
Ext
ende
r
Fabr
ic E
xten
der
Bla
de C
hass
isB
lade
Cha
ssis
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Fabr
ic E
xten
der
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade
Compute BladeCompute Blade Fabr
ic E
xten
der
Fabr
ic E
xten
der
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 158LATAM Data Center Design and Deployment Seminars 2009
Tenant Portal for Multi-Tenant Deployment Server Array Manager supports
Multiple hierarchical server organizations
Network organization
Infrastructure organization
RBAC and object-level security
Cisco UCS GUI
Designed for enterprise deployment
Provides a global view
Single tenant custom views
Through custom portals
Typically as plugin of an existing data center infrastructure
Server Array
XML API
Custom Portal
California GUI
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 159LATAM Data Center Design and Deployment Seminars 2009
Unified Compute Integration in the Data Center:Use cases
Hardware State Abstraction – Service Profiles
Unified Fabric - FCOE
Virtualized Adapter
Expanded Memory Server
Unified Management
UCS IntegrationUCS Integration
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 160LATAM Data Center Design and Deployment Seminars 2009
320 Servers Time to provision new
applications: days to weeks $20M spent on CapEx $800K spent on power
and cooling (3 year) 3,520 Cables 31 Racks
Legacy System
320 Servers Time to provision new
applications: minutes $12M spent on CapEx $650K spent on power
and cooling (3 year) 480 Cables 12 Racks
Unified Computing System
19%
40%
86%
61%
Savings
Case Study 1 – Large Enterprises
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 161LATAM Data Center Design and Deployment Seminars 2009
40% cost savings in cabling, fiber, patch cords and labor 30% more power available to servers 50%+ physical servers in the same space Up to 28,000 virtual machines versus 7,200 in a legacy
environment of the same size Up to 4 times more virtual machines per kilowatt of power;
minimum of 76 virtual machines are being deployed per kilowatt of power
Unified Computing System Results
Results for Unified Computing System in conjunction with Nexus 7000 and Nexus 5000 5 production business apps including News@Cisco and the Office of the Chairman and CEO
Case Study 2 – Cisco IT
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 162LATAM Data Center Design and Deployment Seminars 2009
Case Study 3: Financial Customer
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 163LATAM Data Center Design and Deployment Seminars 2009
UCS and Nexus in the Data Center
Rack 1
GigE
10GE
Row 1 / Domain 1 / POD 1Rack 1
GigE
10GE
…..
Core Layer
Distribution Layer
10GE
...Rack 1 Rack 12
Access Layer
FEX
Nexus 5000
Nexus 7010
Nexus 7010
1GE to Servers
10GE Servers
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 164LATAM Data Center Design and Deployment Seminars 2009
UCS and Nexus in the Data Center
Rack 1
GigE
10GE
Row 1 / Domain 1 / POD 1Rack 1
GigE
10GE
…..
Core Layer
Distribution Layer
10GE
...Rack 1 Rack 12
Access Layer
FEX
Nexus 5000
Nexus 7010
Nexus 7010
1GE to Servers
10GE Servers
UCS 6100
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
slot 1slot 2slot 3slot 4slot 5slot 6slot 7slot 8
blade1blade2blade3blade4blade5blade6blade7blade8
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 165LATAM Data Center Design and Deployment Seminars 2009
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 166LATAM Data Center Design and Deployment Seminars 2009
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 167LATAM Data Center Design and Deployment Seminars 2009
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 168LATAM Data Center Design and Deployment Seminars 2009
Q and A
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 169LATAM Data Center Design and Deployment Seminars 2009
www.cisco.com/go/datacenter