Cisco - VPC Concepts
Transcript of Cisco - VPC Concepts
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 1
vPC Best Practices with Nexus
SAVBUTMETeam
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2
Complete Sync
Partial Sync5.1(2) 5.25.0(3) 5.1(3)
5.0(3)N2 5.2N15.0(3)N1 5.1(3)N1
5.0(3)U2 5.1(3)U15.0(3)U1
Nexus 7000
Nexus 7000
Nexus 5000
Nexus 3000
E-Rocks
Andaman
Complete sync done at major releases Architectural changes
Major enhancements
Major new features
Partial sync done at minor releases Critical flaws/bugs
Minor new features
Minor enhancements
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
• vPC basic components
• Hardware Specific Considerations
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
• vPC is a Port-channeling concept extending link aggregation to two separate physical switches
• Allows the creation of resilient L2 topologies based on Link Aggregation.
Eliminates the need for STP in the access-distribution
• Provides increased bandwidth
All links are actively forwarding
• vPC maintains independent control planes
• vPC switches are joined together to form a ―domain‖
Virtual Port Channel
L2
SiSi SiSi
Increased BW with vPC
Non-vPC vPC
Physical Topology Logical Topology
vPC domain
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
• vPC peer – a vPC switch, one of a pair
• vPC member port – one of a set of ports (port
channels) that form a vPC
• vPC – the combined port channel between the
vPC peers and the downstream device
• vPC peer link – Link used to synchronize state
between vPC peer devices, must be 10GbE.
Also carry multicast/broadcast/flooding traffic
and data traffic in case of vpc member port
failure
• vPC peer keepalive link – the peer keepalive
link between vPC peer switches. It is used to
carry heartbeat packets
• CFS – Cisco Fabric Services protocol, used
for state synchronization and configuration
validation between vPC peer devices
• Orphan port—Non-vPC member port
vPC peer
keepalive linkvPC peer link
Orphan
Port
Orphan
Port
Orphan
Port
Orphan
Port
SecondaryPrimary
vPC
vPC
member
port
vPC peer
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
• Graceful consistency check:
On the N7k: NXOS 5.2
On the N5k: NXOS 5.0(2)N2(1)
• Per VLAN consistency check:
On the N7k: NXOS 5.2
On the N5k: 5.0(2)N2(1)
• Autorecovery:
On the N7k: NXOS 5.2
On the N5k: NXOS 5.0(2)N2(1)
• Config-sync:
On the N7k: Freetown
On the N5k: NXOS 5.0(2)N2(1)
• vPC on FEX
On the N5k: NXOS 4.2(1)N1(1)
On the N7k: NXOS 5.2
• Orphan Ports shutdown:
On N7k: NXOS 5.2
On N5k: E-Rocks+
• IGMP bulk sync:
On N7k: to be verified
On N5k: starting from NXOS 5.0(3)N1(1a)
• Multicast Optimization on Peer-link:
On N7k: hidden comand as of NXOS 5.1(3) (but not supported)
On N5k: starting from NXOS 5.0(3)N1(1a)
• ARP synchronization:
On N7k: NX-OS 4.2(6) and 5.0(2) (Bogota), fixed in 5.1(1) (Cairo)
On N5k: under investigation for Goldcoast
• vPC peer-switch:
On N7k: 4.2(6), 5.x
On N5k: under investigation for Goldcoast
• FEX preprovisioning:
On N7k: Freetown
On N5k: NXOS 5.0(2)N1(1)
• Dual Layer vPC:
On N7k: TBD
On N5k: Fairhaven
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
• vPC allows a single device to use a port channel across two neighbor switches (vPC peers)
• Eliminate STP blocked ports
• Layer 2 port channel only
• Provide fast convergence upon link/device failure
Port
channel
vPC Peers
Portchannel
vPC Peers
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
• Peer Link carries both vPC data and control traffic between peer switches
• Carries any flooded and/or orphan port traffic
• Carries STP BPDUs, IGMP updates, etc.
• Carries Cisco Fabric Services messages (vPCcontrol traffic)
• Carries ―multicast‖ traffic (more details follow)
• Minimum 2 x 10GbE ports
• ALL VLANS used on vPC PORTS MUST BE PRESENT ON THE PEER-LINK
5020 (config)# interface port-channel 10
5020 (config-if)# switchport mode trunk
5020 (config-if)# switchport trunk allowed <BETTER TO ALLOW ALL VLANS>
5020 (config-if)# vpc peer-link
5020 (config-if)# spanning-tree port type network
vPC Peer
Link
5k01 5k02
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
• Peer Keep-alive provides and out of band heartbeat between vPC peers
• Purpose is to detect and resolve roles if a Split Brain (Dual Active) occurs
• Messages sent on 1 second interval with 5 second timeout
• 3 second hold timeout on peer-link loss before triggering recovery
• Should not be carried over the Peer-Link
• Use the mgmt0 interface in the management VRF
• Can optionally be a dedicated link, 1Gb is adequate (first 16 ports on 5020 are 1/10GE ports)
• 3rd option, use a routed inband connection over L3 infrastructure (using SVI‘s in the default VRF)
dc11-5020-1(config)# vpc domain 20
dc11-5020-1(config-vpc-domain)# peer-keepalive destination 172.26.161.201 source
172.26.161.200 vrf management
Note:
--------:: Management VRF will be used as the default VRF ::--------
Peer Keepalive
can be carried
over the OOB
management
network
int mgmt 0
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
• Peer keep-alive is a routable protocol (both N5K and N7K)
• Primary design requirement is to have a physically different path than all other vPC traffic
• In all cases do not carry the peer-keepalivecommunication over the vPC peer-link
On Nexus 7000 when possible use dedicated VRF and front panel ports for peer-keepalive link (1G is more than adequate).
2nd best is to use the management interfaces
3rd option is to use an upstream L3 network for peer-keepalive
• If using mgmt 0 interfaces do ‘not’ connect the supervisor management interfaces back to back
In a dual supervisor configuration only one management port will be active at a given point in time!
Connect both mgmt 0 ports to the OOB network
Standby Management
Interface
Active Management
Interface
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
• Cisco Nexus 5000 Series
• Peer keepalive:
1st option management port.
2nd option dedicated front panel port in dedicated VLAN.
3rd option upstream L3 network
• Cisco Nexus 7000 Series
• vPC works on all existing I/O modules
• Peer keepalive:
• 1st option dedicated front panel port in dedicated VRF.
• 2nd option is management interface.
• 3rd option upstream L3 network
• M1/F1 cards can be used for vPC
• Peer-link requires 10 GigE cards
• Peer-link should not span M1 and F1, peer-link should be made on either all F1cards or all M1 cards
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Part number ModelvPC Peer-link
(10 GE Only)
VPC Member Port
N7K-M132XP-12
N7K-M132XP-12L ✓ ✓
N7K-M148GT-11
N7K-M148GT-11L ✗ ✓
N7K-M148GS-11
N7K-M148GS-11L ✗ ✓
N7K-M108X2-12L ✓ ✓
N7K-F132XP-15 ✓ ✓
NEXUS 7000 I/O modules
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
M M M F F F F
M-Series Mode
M vPC Peer-link on
M-Series Modules
Mixed Chassis Mode
M vPC Peer-link on
M-Series Modules
F-Series Mode
vPC Peer-link on
F-Series ModulesF
Mixed Chassis Mode
vPC Peer-link on
F-Series Modules (*)F
Recommendation : for mixed chassis mode (F1/M1) with vPC peer-link on F1 ports, use at least 2 M1 LC. This will provide resiliency for L3 features (FHRP, SVI).
M
(*) : command ―peer-gateway exclude-vlan <vlan list>‖ needed for backup routing path over vPCpeer-link
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
• NX-OS 5.1.3 introduces new behavior for handling vPC peer-gateway in mixed chassis mode (M1/F1) :
•Topology with M1 peer-link : IP/ARP packets destined to the remote Active IP/MAC get routed locally•Topology with F1 peer-link : IP/ARP packets destined to the remote Active IP/MAC use the tunneling mechanism
M M M M F F F F
M-Series
ModeMixed Chassis
Mode
F-Series
Mode
Mixed Chassis
Mode
Knob Not Required
Classic behavior of peer-gateway
Knob not Required
Peer Gateway not
required
Knob Required for transit
path/VLAN
IP/ARP Tunneling over Peer
link
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
vPC Peer-link
S1 S2
vPC Primary vPC Secondary
F1 F1
vPC Peer-link
S1 S2
vPC Primary vPC Secondary
F1M1
vPC Peer-link
S1 S2
vPC Primary vPC Secondary
M1M1
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
• With dual-active scenarios
MAC address synchronization is interrupted
IGMP synchronization is interrupted
• There is a 50% likelihood that unicast traffic is flooded and that multicast traffic is dropped
5k01 5k02
3IG
MP
rep
ort
fo
r G
1
2 - Host subscribes to G1
1
4 igmp sync lost
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
• There will be 2 primary switches sending independent BPDUsVPC Port-channels on upstream/downstream switches will be error-disabled by ‗EtherChannel Misconfiguration Guard‘ after ~90 secondshttp://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a008009448d.shtml
• If Nexus 7000/5000 is on the other end of VPC no action from STP as 7000/5000 do not support EtherChannel Guard
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
• When the peer-link is disconnected
• vPC secondary detects primary switch is alive through peer keepalive link
• The secondary vpc peer switch suspends all its vpc member ports in order to avoid traffic drop
• KEEP PEER KEEPALIVEAND PEER-LINKS SEPARATE
vPC
PrimaryvPC Secondary
Po10
5k01 5k02
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
dca-n7k2-vdc2
dc11-5020-1# show running int port-channel 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dc11-5020-2# show running int port-channel 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dca-n7k2-vdc2# sh run interface port-channel 201
version 4.1(5)
interface port-channel201
switchport mode trunk
switchport trunk allowed vlan 100-105
vPC supports standard 802.3ad port channels from upstream and or downstream devices
Recommended to enable LACP
―channel-group 201 mode active‖dc11-5020-2dc11-5020-1
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
dca-n7k2-vdc2
dc11-5020-1# show running int port-channel 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dc11-5020-2# show running int port-channel 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dca-n7k2-vdc2# sh run interface port-channel 201
version 4.1(5)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
• vPC forwards only on locally connected members of the port channel if any exist (same principle as VSS)
• Multiple topology choices
• Square
• Full Meshdc11-5020-2dc11-5020-1
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
• vPC maintains layer 2 topology synchronization via CFS
• Copies of flooded frames are sent across the vPC-Link in case any single homed devices are attached
Frames received on the vPC-Link are not forwarded out vPC ports
2
3
1. Host MAC_A send packet to MAC_C
2. FEX runs hash algorithm to select one fabric uplink
3. N5K-1 learns MAC_A and flood packets to all ports
(in that VLAN). A copy of the packet is sent across
the peer link
4. N5K-2 floods the packet to any port in the VLAN
except the vPC member ports to prevent duplicated
packets
5. N7K-1 and N7K-2 repeat the same forwarding logic
6. N5K-1 updates the the MAC address learned on the
vPC port on N5K-2 via CFS
MAC_C
MAC_A
1
5
5
N5K-1 N5K-2
CFS
6
4
CFS
6
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
• Traffic is forwarded if destination address is known (both switches MAC address tables populated)
• Always forward via a locally attached member of a vPC if it exists
1. Host MAC_C send packet to MAC_A
2. N7K-2 forwards frame based on learned
MAC address
3. N5K-2 forwards frame based on learned
MAC address
MAC_C
MAC_A
N5K-1 N5K-2
1
2
3
N5K-1# sh mac-address-table vlan 101
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+-----
101 001b.0cdd.387f dynamic 0 Po30
101 0023.ac64.dda5 dynamic 30 Po201
Total MAC Addresses: 4
N5K-2# sh mac-address-table vlan 101
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+-----
101 001b.0cdd.387f dynamic 0 Po30
101 0023.ac64.dda5 dynamic 30 Po201
Total MAC Addresses: 4
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
• On loss of all of the locally attached members of the vPC MAC address table is updated to forward frames for the vPCacross the vPC Peer Link
• Note: Po20 is the vpc peer-link
MAC_C
MAC_A
N5K-1 N5K-2
N5K-1# sh mac-address-table vlan 101
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+-----
101 001b.0cdd.387f dynamic 0 Po30
101 0023.ac64.dda5 dynamic 30 Po201
Total MAC Addresses: 4
N5K-2# sh mac-address-table vlan 101
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+-----
101 001b.0cdd.387f dynamic 0 Po20
101 0023.ac64.dda5 dynamic 30 Po201
Total MAC Addresses: 4
1
3
2
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
√ √ X √
x
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
• Both switches in the vPC Domain maintain distinct control planes
• CFS provides for protocol state synchronization between both peers (MAC Address table, IGMP state, …)
• System configuration must also be kept in sync
• Currently there are 2 options to keep configuration consistent:
a manual process with an automated consistency check to ensure correct network behavior
config-sync
Two types of interface consistency checks
Type 1 – Will put interfaces into suspend state to prevent invalid forwarding of packets
Type 2 – Error messages to indicate potential for undesired forwarding behavior
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
• Type 1 Consistency Checks are intended to prevent network failures
• Incorrectly forwarding of traffic
• Physical network incompatibilities
• vPC will be suspended
dc11-5020-2# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
<snip>
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
201 Po201 down failed vPC type-1 configuration -
incompatible - STP
interface port guard -
Root or loop guard
inconsistent
dc11-5020-1# sh run int po 201
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dc11-5020-2# sh run int po 201
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
spanning-tree guard root
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
• Type 2 Consistency Checks are intended to prevent undesired forwarding
• vPC will be modified in certain cases (e.g. VLAN mismatch)
dc11-5020-1# show vpc brief vpc 201
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
201 Po201 up success success 100-104
2009 May 17 21:56:28 dc11-5020-1 %ETHPORT-5-IF_ERROR_VLANS_SUSPENDED: VLANs 105 on Interface port-
channel201 are being suspended. (Reason: Vlan is not configured on remote vPC interface)
dc11-5020-1# sh run int po 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 100
switchport trunk allowed vlan 100-105
vpc 201
dc11-5020-2# sh run int po 201
version 4.1(3)N1(1)
interface port-channel201
switchport mode trunk
switchport trunk native vlan 105
switchport trunk allowed vlan 100-104
vpc 201
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
Global Spanning Tree Parameters need to be
consistent
Global QoS Parameters need to be consistent
Global Parameters
• c-nexus5010-1# show vpc consistency-parameters global
• Legend:
• Type 1 : vPC will be suspended in case of mismatch
• Name Type Local Value Peer Value
• ------------- ---- ---------------------- -----------------------
• QoS 2 ([], [3], [], [], [], ([], [3], [], [], [],
• Network QoS (MTU) 2 (1538, 2240, 0, 0, 0, (1538, 2240, 0, 0, 0,
• Network Qos (Pause) 2 (F, T, F, F, F, F) (F, T, F, F, F, F)
• STP Mode 1 Rapid-PVST Rapid-PVST
• STP Disabled 1 None None
• STP MST Region Name 1 "" ""
• STP MST Region Revision 1 0 0
• STP MST Region Instance to 1
• VLAN Mapping
• STP Loopguard 1 Disabled Disabled
• STP Bridge Assurance 1 Enabled Enabled
• STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
• Allowed VLANs - 1,50 1
• Local suspended VLANs - 50 -
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
• Don‘t forget to keep global configuration in sync
• Any configuration that could cause an error in forwarding (e.g. loop) will disable all affected interfaces
• As an example if you make a change to an MST region you must make it on ‗both‘ peers
• Solution: define MST region mappings from the very beginning of the deployment, for ALL VLANs, the ones that exist as well as the ones that have not yet been created
• Defining a region mapping is orthogonal to creating a VLAN
vPCvPC vPC
mst region
vlans 1-5, 12mst region
vlans 1-5, 10
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
Inconsistency Type Impact Recommendation New Enhancements
VLAN to MST Region mapping mismatch
STP global settings (BA, Loop Guard, Root Guard)
1
Global
Pre-provision and MAP all VLANs on the MST region
Perform STP operations per port
Operate change during maintenance window
Leverage graceful conflict resolution Config Sync
(5.0(2)N1(1) on N5K, Freetown for N7K)
&
Graceful Conflict Resolution
(CSCtf84865,N7K -4.2(8)& 5.2, N5K –
5.0(2)N2(1))
Spanning-tree per interface settings,
switchport type (trunk/versus access)…
Port-channel mode
Per-vPC
Operate change during
maintenance window and/or
leverage graceful conflict
resolution
Quality of Service Configuration
2
Global
Minimum disruption
Per-vPCVLANs configured on vPC
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
• tc-nexus5010-1# show vpc consistency-parameters global
• Name Type Local Value Peer Value
• ------------- ---- ---------------------- -----------------------
• QoS 2 ([], [3], [], [], [], ([], [3], [], [], [],
• []) [])
• Network QoS (MTU) 2 (1538, 2240, 0, 0, 0, (1538, 2240, 0, 0, 0,
• 0) 0)
• Network Qos (Pause) 2 (F, T, F, F, F, F) (F, T, F, F, F, F)
• Input Queuing (Bandwidth) 2 (50, 50, 0, 0, 0, 0) (50, 50, 0, 0, 0, 0)
• Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
• Priority)
• Output Queuing (Bandwidth) 2 (50, 50, 0, 0, 0, 0) (50, 50, 0, 0, 0, 0)
• Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
• With Graceful Resolution only ports on the vPC secondary are ―suspended‖ if a Type-1 global inconsistency occurs
• This limits the impact of configuration changes.
• switch(config)# vpc domain 10
• switch(config-vpc-domain)# [no] graceful consistency-check
• Requires 5.0(2)N2(1) on the Nexus 5k
• Requires 5.2 on the Nexus 7kvPCvPC vPC
mst region
vlans 1-5, 12mst region
vlans 1-5, 10
vPC primary vPC secondary
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39
Check whether STP is enabled or disabled on per-VLAN basis.
VLANs that have mismatched status will be suspended on both switches
Rest of VLANs won‘t be affected
Prior to this change all VLANs are affected
Disable STP on VLAN 5
5.2
5.0(2)N2(1)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40
• Config-sync allows administrators to make configuration changes on one switch and have the system automatically synchronize to its peers.
• This eliminates any user prone errors & reduces the administrative overhead of having to configure both vPC members simultaneously.
• Config-sync and Graceful conflict resolution are complementary features
• Config-sync traffic is carried over the peer keepalive link
vPCvPC vPC
mst region
vlans 1-5, 12mst region
vlans 1-5
+ vlan 12
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
• Global Configurations:
VLANs
ACLs
STP configurations
QOS
• Interface Level Configurations:
Ethernet Interfaces
Port Channel Interfaces
vPC Interfaces
• Which configurations are not synchronized?
Enabling ―Feature‖
vPC domain configuration
FCoE configuration
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42
N5000-1#sh run switch-profile
Switch-profile Apple
sync-peers destination 10.29.170.8
N5000-1(config-if)# config sync
N5000-1(config-sync)# switch-profile Apple
N5000-1(config-sync-sp)# int ethernet
100/1/3
N5000-1(config-sync-sp-if)# switch mode
trunk
N5000-1(config-sync-sp-if)# verify
Verify Successful
N5000-2#sh run switch-profile
Switch-profile Apple
sync-peers destination 10.29.170.7
N5000-1#sh run switch-profile
interface ethernet 100/1/3
switchport mode trunk
N5000-2#sh run switch-profile
interface ethernet 100/1/3
switchport mode trunk
N5000-1#
feature vpc
vpc domain 10
peer-keepalive destination 10.29.170.8
N5000-2#
feature vpc
vpc domain 10
peer-keepalive destination 10.29.170.7
N5000-1(config-if)# config sync
N5000-1(config-sync)# switch-profile Apple
N5000-1(config-sync-sp)# commit
Commit Successful
NOTE: Verify does not push the config to peer, user must issue “commit” for sync to
take place
If sync fails, then the config is in the BUFFER
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43
N5K-1(config-sync-sp-if)# sh switch-profile A buffer
-----------------------------------------------------
Seq-no Command
-----------------------------------------------------
1 interface Ethernet100/1/9
1.1 switchport mode trunk
1.2 switchport trunk allowed vlan 5-10
2 interface Ethernet100/1/10
2.1 switchport mode access
N5K-1(config-sync-sp)# ?
buffer-delete Delete buffered command(s)
buffer-move Move buffered command(s)
N5K-1(config-sync-sp)# buffer-delete 1
N5K-1(config-sync-sp)# sh switch-profile A buffer
-----------------------------------------------------
Seq-no Command
-----------------------------------------------------
2 interface Ethernet100/1/10
2.1 switchport mode access
• Configuration is stored in a buffer until commit is applied.
• User can add/delete/move configuration.
• Once the config has been pushed via commit, it will no longer show up in buffer (it will show up in ―show running-config switch-profile X‖)
• If the commit fails due to mutexcheck or other reasons, the failed configuration still shows in the buffer, you have to explicitly remove it to continue
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44
• Interface Ethernet1/11
• fex associate 100
• switchport mode fex-fabric
• channel-group 100
config-t area
switch-profile area• Interface Ethernet1/11
• shut/no shut
This portion is synchronized
This portion is not
synchronized
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 45
A port-channel may consist of port ethernet 1/1 on n5k01
And erthernet 1/2 on n5k02 FEX A/A has the same FEX configured to both N5ks, so
Preprovisioning has to be configured identically
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46
• If one vPC peer needs to be disconnected completely from the vPCdomain you can still operate the remaining one
• For this you need to leverage the commands ―reload restore‖ and ―autorecovery‖
• Reload restore deals with the split brain scenario allowing a vPC peer to bring up new vPC ports even after a reload
• Autorecovery deals with the sequential loss of peer-link first, and peer-keepalive second, allowing the vPCsecondary to bring up the vPC ports (which were down previously)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47
• VPC needs to be able to talk to the peer (over peer-link) before bringing up VPC port-channels
Negotiate LACP/STP operating roles for the chassis
Wait for per-port peer parameters and handshake to bring up vPC ports
• Performs peer parameters consistency check on each VPC bringup
• Only after VPC port-channels are brought up.
• What if after a full DC outage (both Nexus down), only one switch is coming up ?
• Will not bring up VPCs if after a datacenter outage, only one VPCpeer comes back up
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48
1
Switch3 Switch3
Switch2Switch1
Switch1
2
Switch3
Switch1
3
When adding a new vPC member port, the port goes up
Existing vPCs are brought up
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49
1
Switch3
Switch2Switch1 Switch3
Switch1
2
Switch2
Switch3
Switch1
3
Switch2
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50
S2-SecondaryS1 -Primary
Peerlink down and keepalive working
Secondary shuts vPCs
vPC peer-link
vPC 1
po1
Keepalive
S2-SecondaryS1 -Primary
Primary fails
Po1 is completely shut
vPC peer-link
vPC 1
po1
Keepalive
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51
vPCPrimary
Peerlink down and keepalive down
vPC peer-link
vPC 1
po1
Keepalive
After 3 consecutive keepalive timeouts
Secondary changes role and brings up vPCs
vPC peer-link
vPC 1
po1
Keepalive
vPC Operational Primary
vPCSecondary
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52
• STP for vPCs is controlled by the vPC operationally primary switch and only such device sends out BPDUs on STP designated ports.
• This happens irrespectively of where the designated STP Root is located
• The vPC operationally secondary device proxies STP BPDU messages from access switches toward the primary vPC
Primary
vPC
Secondary
vPC
BPDUs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53
MAC_A MAC_BSW3
SW2
SW4
SW1
vPC1 vPC2
vPC_PL
L2
L3
ECMPECMP
vPC PK-Link
Primary
vPC
Secondary
vPC
Secondary
Root
Primary
Root
vPC peer-link is a
regular STP port
vPC Primary
Switch Source
and controls
STP for vPCs
The secondary vPC
device does NOT
source STP BPDUs on
symmetrical vPCs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54
• Assume the following topology with vPC enabled on the vPC
• If the Primary fails over, the secondary needs to start sending BPDUs
• If the Primary was also the STP root, the secondary also has to overtake the role as a root
• If this process lasts too long, the uplink port on 5k02 may go into BA_Inconsistent state
• Better not use Bridge Assurance with vPC
• Bridge Assurance on peer-link is fine (and is the default)
5k01 5k02
Secondary
becomes
primary and
root
7k01 7k02
Primary /
Root
BPDUs prior
to the failureBA Inconsistent
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55
Primary Secondaryleft# sh span vlan 101
VLAN0101Spanning tree enabled protocol rstpRoot ID Priority 8293
Address 0023.04ee.be01This bridge is the root
...
Bridge ID Priority 8293 (priority 8192)Address 0023.04ee.be01
...
Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- ---------------Po1 Desg FWD 1 128.4096 (vPC) P2pPo100 Root FWD 2 128.4195 (vPC peer-link)
left# sh vpc role | i macvPC system-mac : 00:23:04:ee:be:01 vPC local system-mac : 00:1b:54:c2:42:43
right# sh span vlan 101
VLAN0101Spanning tree enabled protocol rstpRoot ID Priority 8293
Address 0023.04ee.be01This bridge is the root
...
Bridge ID Priority 8293 (priority 8192)Address 0023.04ee.be01
...Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- ---------------Po1 Desg FWD 1 128.4096 (vPC) P2pPo100 Desg FWD 2 128.4195 (vPC peer-link)
In Peer-Switch mode bridge-ID comes from system-mac as opposed to local mac in normal mode
ROOT ROOT
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56
• BA is default enabled on Peer-Link (and recommended to remain enable), not recommended for VPCs unless Peer-Switch feature is used
Without Peer-switch BA should be kept only on Peer-Link (no BA/Loop guard on VPCs)
• Dispute is default enabled (for both RSTP and MST on VPC)
• UDLD [normal mode] is recommended to take out bad links from channels
• BA + UDLD + Dispute (on all interswitch links when using Peer-switch) when all switches support this (nexus7000/5000)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57
• By default on the Nexus 5x00 series, LACP sets a port to ―I state‖ if it does not receive an LACP PDU from the peer. This behavior is different on the Nexus 7000 series where the default is to suspend a port if it doesn‘t receive LACP PDUs.
• For server facing port-channels it is better to allow LACP ports to revert to I-state if the server doesn‘t send LACP PDUs. By doing this the I-state port can operate like a regular Spanning-Tree port. Also this allows immediate server connectivity when it boots up before the full LACPnegotiation has taken place.
• For network facing ports, allowing ports to revert to I-state creates additional Spanning-tree state without any real benefit.
• This behavior can be configured on a per Port-Channel basis with the configuration ―[no] lacp suspend-individual‖ (which is the equivalent of the Catalyst IOS command ―port-channel standalone-disable‖.
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58
• IGMP snooping shares the snooped reports with the peer vPCswitch to help with multicast forwarding
• Forwarding of IGMP protocol packets is tweaked so that IGMPreports received on one vPC switch is also forwarded to the vPC peer. Thus multicast forwarding state remains in sync on both the vPC switches.
• Do NOT DISABLE IGMP Snooping!
• If you need to support Firewalls / Clusters:
Use static IGMP entries OR
Create an IGMP querier!
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59
• vPC maintains dual active control planes and STP still runs on both switches
• IGMP join/leave messages received on one peer is forwarded to another peer via peer link
• IP muticast packets are sent to host through local port
• Non-IP multicast and broadcast packets are flooded
vPC Primary vPC Secondary
vPC SecondaryvPC Primary
IGMP
join/leave
IGMP
join/leave
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60
• So is the multicast traffic going to the peer link?
Yes, but duplicates are avoided by using the vPC loop prevention technique, which should rather be called ―duplicate prevention‖
• And how about orphan ports?
Orphan ports receive traffic because the multicast traffic is always sent over the peer-link
N7k01 N7k02
N5k01 N5k02
1 3
1 2 3 4
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61
• Assuming that there are no orphan ports it is possible to remove multicast traffic from crossing the peer-link with the command
• no ip igmp snooping mrouter vpc-peer-link (Nexus 5k)
• ip igmp snooping vpc peer-link-exclude (hidden command on the Nexus 7k, not supported)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62
N5k-1 N5k-2
IGMP Group sync
• VPC peer-link is considered as mrouterport. Therefore all multicast traffic is flooded over peer-link
• A CLI was introduced in 5.0(3)N1(1) to avoid that. With the CLI multicast traffic is sent to vPC peer-link only when it is necessary, such as, there is singly connected host
• Improving multicast convergence time with peer-link down/up and switch reload
• The CLI is not supported for FEX dual-home topology in 5.0(3)N1(1). The limitation will be removed in upcoming release-5.0(3)N2(1)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63
vPC on the N7k
vPC on the N5k
N7k01 N7k02
N5k01 N5k02
1 2 3 4
• If the peer-link is lost the vPC secondary is going to shut down the vPC member ports
• For single attached hosts, plssee
• CSCtc49559
• and Orphan ports ―suspend‖ feature
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64
Intended for devices that do not support port-channel. Other devices should be dually connected by vPCs (Orphan-port CLI is available only on physical ports, not on port-channels)
Configure single attached devices (like FW or LB) port as orphan-port
When vPC peer-link goes down, vPC secondary peer device shuts all its vPC member ports as well as orphan ports
CE-1
S2-SecondaryS1 -Primary
vPC peer-link
vPC 1
po1
Keepalive
Orphan port
Active or Standby
Active or Standby
S1(config)# int eth 1/1
S1(config-if)# vpc orphan-ports suspend
S2(config)# int eth 1/1
S2(config-if)# vpc orphan-ports suspend
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67
• vPC maintains dual active control planes and STP still runs on both switches
• HSRP active process communicates the active MAC to its neighbour
• Only the HSRP active process responds to ARP requests
• HSRP active MAC is populated into the L3 hardware forwarding tables, creating a local forwarding capability on the HSRP standby device
• Consistent behavior for HSRP, VRRP and GLBP
• No need to configure aggressive FHRP hello timers as both switches are active
HSRP Active HSRP Standby
HW Programmed to forward frames
sent to the FHRP MAC address on
BOTH Switches
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68
It recommended to ‗not‘ use HSRP link tracking in a vPC configuration
Reason: vPC will not forward a packet back on a vPC once it has crossed the peer-link, except in the case of a remote member port failure
Use an L3 point-to-point link between the vPC peers to establish a L3 backup path to the Core in case of uplinks failure
A single point-to-point VLAN/SVI will suffice to establish a L3 neighbor
VLAN 100 VLAN 200
VLAN 100, 200,300
SVI
VLAN 300
SVI
VLAN 300
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 69
• Non-RFC compliant end hosts
Device required to send packets to the MAC address returned in ARP response (HSRP virtual MAC)
Some non-compliant devices use the MAC address of the sender device (Switch physical MAC)
NAS devices (i.e. NETAPP Fast-Path or EMC IP-Reflect) have been found to do this
• vPC Peer Gateway - NX-OS 4.2(1)
Allows a vPC peer to respond both the the HSRP virtual and the real MAC address of both itself and it‘s peer
L3
L2
VLAN 100VLAN 200
“peer-gateway” command
tells the vPC to respond
to the physical MAC
address of its peer
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70
Not enabled by default
After the peer-link comes up perform an ARP bulk sync over CFSoE to the peer switch
Improve Convergence for Layer 3 flows
ARP Synchronization Process
Primary vPC
Secondary vPCS
P
P S
ARP TABLE
IP1 MAC1 VLAN 100
IP2 MAC2 VLAN 200
ARP TABLE
IP1 MAC1 VLAN 100
IP2 MAC2 VLAN 200
IP1 MAC1 IP2 MAC2
SVIs
S1(config-vpc-domain)#
ip arp synchronize
S2(config-vpc-domain)#
ip arp synchronize
Note:
CSCti06907
has
been fixed
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71
Feature Function Availability
VPC interaction with FHRP Both active and standby peer function as
gateway
HSRP VRRP
Peer-gateway L3 forwarding when the DMAC is peer‘s
MAC
vPC delay restore Delay bringing up vPC ports
vPC exclude VLAN CLI to specify SVI interfaces won‘t be
suspended when peer-link fails
ARP synchronization Synchronize ARP between two peer
switches
Roadmap
PIM pre-built-SPT Both N5k joins source tree as PIM last hop
router
PIM dual DR Both N5k can be DR when it is first hop
router
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 72
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73
FE
X21
48T
sta
rtin
g f
rom
4.1
(3)N
1(1
)
FE
X2248,
2232 fro
m
4.2
(1)N
1(1
)
FE
X2248,
2232,
2224
from
4.2
(1)N
1(1
)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74
Fairhaven
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 75
N7K NXOS 5.1(1)
activeactive
N
activeactive
N
Y
activeactive
N
activeactive
N7K NXOS 5.2
activeactive
activeactive
Future
Y
radar
Y
YY N Y Y
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 76
Nexus 5000 Topologies (Nexus 2248TP & 2232PP)
Redundancy model – Dual Switch with redundant fabric
Provides isolation for Storage topologies (SAN ‘A’ and ‘B’)
Port Channel and Pinning supported for Fabric Link
vPC Supported with up to 2 x 8
links
Local Etherchannel with up to 8
links
FCoE Adapters supported on 10G
N2K interfaces
Straight Through
Redundancy model – Single switch with dual ‘supervisor’ for fabric, data control & management planes
No SAN ‘A’ and ‘B’ isolation (VSAN isolation sufficient in the future?)
Dual Homed
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77
Nexus 7000 Topologies (Nexus 2248TP & 2232PP)
Local Etherchannel with up to 8
links
NIC Teaming:TLB/ALB
NXOS 5.2Nexus 2248TP & 2232PP
Fabric links supported on N7K-M132XP-12 & N7K-M132XP-12L
Port Channel only supported for Fabric Links
Local port channel support on 2248 & 2232
No support for DCB and FCoE (parent switch fabric ports not DCB capable yet)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 78
Redundancy model – Dual Switch (each switch supports redundant supervisors)
MCEC Etherchannel with up to 16
links
Nexus 5000 Fairhaven
Redundancy model – Single switch with dual ‘supervisor’, fabric, line card, data control & management planes
MCEC Etherchannel with up to 16
links
Nexus 7000 - vPC – NXOS 5.2
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79
24 FEX
Nexus 2000 Straight-through deployment
n5k01
FEX100
FEX101
FEX102
max 24 with Nexus 5500 = 768 ports
max 4/8 ―fabric links‖
Active/Standby
n5k01
FEX100
FEX101
FEX102
n5k02
FEX120
FEX121
FEX122
max 24 x 2
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 80
Cisco Nexus 2000 Series
Straight-Through vPC
Cisco Nexus 2000
Active-Active
vPC
Primary
FEX120FEX100vPC 1 vPC 2
FEX120FEX100
vPC
Secondary
HIF HIF
HIF HIF
Fabric Links Fabric Links
Peer Keepalive
Peer Link
vPC Member Port
vPC
Primary
vPC
Secondary
up to 8 ports
up to 8 ports
up to 8 ports up to 8 ports
up to 24 PC
per FEX up to 24 PC
per FEX
up to 4 ports up to 4 ports
up to 4 ports up to 4 ports
FEX 2248
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81
Cisco Nexus 2000 Series
Straight-Through vPC
Cisco Nexus 2000
Active-Active
vPC
Primary
FEX120FEX100vPC 1 vPC 2
FEX120FEX100
vPC
Secondary
HIF HIF
HIF HIF
Fabric Links Fabric Links
Peer Keepalive
Peer Link
vPC Member Port
vPC
Primary
vPC
Secondary
up to 8 ports
up to 8 ports
up to 8 ports up to 8 ports
up to 16 PC
per FEX up to 16 PC
per FEX
up to 8 ports up to 8 ports
up to 8 ports up to 8 ports
FEX 2232
Compatible with FCoE IF server uses 2 uplinksDoesn‘t support FCoE, today
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82
• In a Dual Tier vPC configuration FCoE traffic will NOT be load shared across both sets of fabric links
• SAN ‗A‘ and ‗B‘ isolation is maintained
• This may result in un-even sharing of traffic across the multiple fabric links
FCoE + LAN on one set of fabric links
LAN only on the other set of fabric links
• Need to plan for the aggregate traffic capacity
LAN & SAN traffic
SAN BSAN A
LAN traffic
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83
• vPC basic components
• Hardware Specific Considerations
• vPC forwarding rules
• vPC enhancements
• L3 and vPC
• Adding FEX
• Summary designs
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84
vPC on the N7k
N7k01 N7k02
N5k01 N5k02
2/1 2/2 2/1 2/2
2/9 2/10 2/9 2/10
Po51,2
root
logical equivalent
Root
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 85
vPC on the N7k
N7k01 N7k02
N5k01 N5k02
2/1 2/2 2/1 2/2
Po10
2/9 2/10 2/9 2/10
Po51
Peer Link
primary secondary
root
regular STP priority
logical equivalent
Root
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86
2248TPs
5500 or 50x0
7010s with
F1linecards
vPC peer link
Running vPC
only for server
attach ports
x8x8 x8
x8
vPC peer link16 port
HW
Etherchan
nel
16 port
HW
Etherchan
nel
... ...
32 PORTS
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 87
SW01SW02
N5k01 N5k02
2/1 2/2 2/1 2/2
Po10
2/9 2/10 2/9 2/10
Po51
Peer Link
primary secondary
regular STP priority
Root
HSRP primary
Secondary Root
HSRP secondary
Clear access VLANs to
create a Loop Free Topology
logical equivalent
SW01SW02
2/9 2/10 2/9 2/10
F F
Clear access VLANs to
create a Loop Free Topology
Root
HSRP primary
Secondary Root
HSRP secondary
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 88
• Traffic flows are symmetric from access to aggregation
• vPC is still useful to optimize traffic flows from access to aggregation
• All traffic flows through the active HSRP switch, in this case SW01
• Client-to-Server traffic uses both SW01 and SW02
• Peer-link is almost unutilized
SW01SW02
N5k01 N5k02
2/1 2/2 2/1 2/2
Po10
2/9 2/10 2/9 2/10
Po51
Peer Link
Clear access VLANs to
create a Loop Free Topology
Root
HSRP primary
Secondary Root
HSRP secondary
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 89
• Define domains
• Establish Peer Keepalive connectivity
• Create a Peer link
• Create vPCs
• Make Sure Configurations are Consistent / leverage config-sync / configure graceful conflict resolution
N7k01 N7k02
5 6 7 8
N2k01 N2k02
1 2 3 4
• Following steps are needed to build a vPC
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 90
• Ensure domain-id or system-mac differs between Agg pair and Access pair
• Connect the N7ks with redundant peer-links across linecards
• Connect the N5ks with redundant peer-links
• Create a single Port-channel leveraging LACPbetween Aggregation and Access
• Do not forget that putting a VLAN on a vPCrequires that that VLAN be on the Peer-link too
• If you foresee significant multicast traffic, or there is a high percentage of single attached devices, you may want to size the peer-link to match the uplink bandwidth utilization
N7k01 N7k02
1 2 3 4
N5k02N5k01
5 6 7 8
N2k01 N2k02
1 2 3
LACP
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 91
• If you use the ―peer switch‖ functionality, then define Identical Priorities on the Aggregation Layer switches, to make them the root
• Do not use Bridge Assurance
• Keep the default STP priorities on the access layer switches
• IF using MST, Make Sure that VLAN range Configurations are Consistent
• With MST be aware of the NXOS VLAN range and of the Global Type-1 Inconsistencies, hence configure VLAN-to-region mappings from day 1
• Use pathcost method long
• Configure STP port type edge or port type edge trunk
N7k01 N7k02
5 6 7 8
N2k01 N2k02
1 2 3 4
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 92
• Configure HSRP priorities as usual, both peers forward L3 traffic
• Configure vPC delay restore to avoid L3 traffic loss upon reboot
• Create a L3 backup ―link‖ between the N7k
• Configure peer-gateway for firewalls, load balancers, filers
• Configure regular L3 ECMP from the core to the aggregation layer
N7k01 N7k02
1 2 3 4
N5k02N5k01
5 6 7 8
N2k01 N2k02
1 2 3
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 93
N7k01 N7k02
N5k01 N5k02
2/1 2/2 2/1 2/2
2/9 2/10 2/9 2/10
• Make sure to leverage Reload Restore and auto-recovery
• Make sure to have mgmt0 connectivity for config-sync to work (you may want to use the same mgmt0 for vPC peer keep-alive)
• FEX A/A provides redundancy and each HIF
• Config-sync also helps with regular port channels
• FEX pre-provisioning is highly recommended
Thank You