Cisco Fabric Path Doc 2

What is FabricPath and why use it?FabricPath is Layer 2 routing, also known as MAC-in-MAC routing. This is achieved by running IS-IS protocol in the L2 control plane, where it is responsible for building the topology and Shortest Path Tree (SPT). Routing protocols for the win!FabricPath was designed to overcome the limitation of Spanning-Tree Protocol (STP). What limitations?Some that come to mind arepoor convergence, unnecessary flooding and maintenance of full CAM tables. Oh, and did I mention no ability for equal-cost multipathing (ECMP)?!With FabricPath, we have the capability to actively forward on all links. If there is a failure on one of the links, traffic will be redistributed across all the others. When traffic comes into the FabricPath domain, a single lookup is performed to identify the switch closest to the destination, providing optimal flows.Starting TopologyWe will be using this topology for this walkthrough.

The above topology is STP-only. Before we dive into the configuration, lets check out the current VLAN database and Spanning-Tree on N7K3. Well take a look at this again in a few minutes.VLANs and Spanning-TreeN7K3# show vlan

VLAN Name Status Ports---- -------------------------------- --------- -------------------------------1 default active Eth4/13, Eth4/14, Eth4/15 Eth4/1640 VLAN0040 active Eth4/13, Eth4/14, Eth4/15 Eth4/1650 VLAN0050 active Eth4/13, Eth4/14, Eth4/15 Eth4/1660 VLAN0060 active Eth4/13, Eth4/14, Eth4/15 Eth4/1670 VLAN0070 active Eth4/13, Eth4/14, Eth4/15 Eth4/1680 VLAN0080 active Eth4/13, Eth4/14, Eth4/15 Eth4/16

VLAN Type Vlan-mode---- ----- ----------1 enet CE 40 enet CE 50 enet CE 60 enet CE 70 enet CE 80 enet CE

Remote SPAN VLANs-------------------------------------------------------------------------------

Primary Secondary Type Ports------- --------- --------------- -------------------------------------------

N7K3# show spanning-tree vlan 40,50

VLAN0040 Spanning tree enabled protocol rstp Root ID Priority 24616 Address e8ed.f339.4f44 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 24616 (priority 24576 sys-id-ext 40) Address e8ed.f339.4f44 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Eth4/13 Desg FWD 2 128.525 P2p Eth4/14 Desg FWD 2 128.526 P2p Eth4/15 Desg FWD 2 128.527 Network P2p Eth4/16 Desg FWD 2 128.528 Network P2p

VLAN0050 Spanning tree enabled protocol rstp Root ID Priority 24626 Address e8ed.f339.4f44 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 24626 (priority 24576 sys-id-ext 50) Address e8ed.f339.4f44 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Eth4/13 Desg FWD 2 128.525 P2p Eth4/14 Desg FWD 2 128.526 P2p Eth4/15 Desg FWD 2 128.527 Network P2p Eth4/16 Desg FWD 2 128.528 Network P2p FabricPath TopologyOur end-goal will look like this, with FabricPath running between the N7Ks, and Classical Ethernet running to the bottom two switches.

FabricPath ConfigurationInstall the feature-set in the Admin VDCADMIN# conf tADMIN(config)# install feature-set fabricpathNotice this automatically allows the feature-set in the VDCvdc N7K3 id 4 limit-resource module-type m2xl f2e allow feature-set fabricpathEnable the FabricPath feature-set on the switches/VDCsN7K3# conf tN7K3(config)# feature-set fabricpath

N7K4# conf tN7K4(config)# feature-set fabricpath

N7K5# conf tN7K5(config)# feature-set fabricpath

N7K6# conf tN7K6(config)# feature-set fabricpath Notice the default configuration of FabricPathN7K3# show run fabricpath

!Command: show running-config fabricpath!Time: Sun Aug 3 20:01:52 2014

version 6.2(6)feature-set fabricpath

fabricpath domain defaultAll weve done so far is enable the FabricPath feature-set. Notice below that we are already assigned a SID (switch-ID). This is a 12-bit address dynamically assigned via DRAP (Dynamic Resource Allocation Protocol), whichis used for identifying the switch in the FabricPath domain. The system-id is the MAC of the switch or VDC (verify with show vdc internal mac_address_table on the Admin VDC)N7K3# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 1============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------* 370 e8ed.f339.4f44 Primary Confirmed No No

N7K4(config)# show fabricpath sstatic switch-id system-id N7K4(config)# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 1============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------* 327 e8ed.f339.4e44 Primary Confirmed No No

N7K5(config)# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 1============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------* 104 e8ed.f339.4f45 Primary Confirmed No No

N7K6(config)# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 1============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------* 76 e8ed.f339.4e45 Primary Confirmed Yes No As you can see, these SIDs are a little all over the place. We can statically configure these so theyre easier to recognize in the FabricPath domain.Configure Static FabricPath Switch-IDN7K3(config)# fabricpath switch-id 73N7K3(config)# show fabricpath switch-id local Switch-Id: 73System-Id: e8ed.f339.4f44

N7K4(config)# fabricpath switch-id 74N7K4(config)# show fabricpath switch-id localSwitch-Id: 74System-Id: e8ed.f339.4e44

N7K5(config)# fabricpath switch-id 75N7K5(config)# show fabricpath switch-id local Switch-Id: 75System-Id: e8ed.f339.4f45

N7K6(config)# fabricpath switch-id 76N7K6(config)# show fabricpath switch-id local Switch-Id: 76System-Id: e8ed.f339.4e45 Notice we can already run this command to look at the IS-IS adjacencies, which will be used to build our MAC-in-MAC routing topology and shortest path tree.N7K6(config)# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceNothing yet, so lets bring up some interfaces and look again. First well configure FabricPath on all the layer-2 interfaces on N7K3 and N7K4.Configure FabricPath switchportsN7K3(config)# int e4/13-16N7K3(config-if-range)# switchport mode fabricpath

N7K4(config)# int e4/13-16N7K4(config-if-range)# switchport mode fabricpath

2014 Aug 3 20:09:13 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 e8ed.f339.4e44 over Ethernet4/13 - DOWN (New) on MT-02014 Aug 3 20:09:13 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 e8ed.f339.4e44 over Ethernet4/13 - UP on MT-02014 Aug 3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - DOWN (New) on MT-02014 Aug 3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - INIT on MT-02014 Aug 3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - UP on MT-02014 Aug 3 20:09:24 N7K3 %ETHPORT-5-IF_UP: Interface Ethernet4/14 is up in mode fabricpath2014 Aug 3 20:09:24 N7K3 %ETHPORT-5-IF_UP: Interface Ethernet4/13 is up in mode fabricpath2014 Aug 3 20:09:24 N7K3 %L3VM-5-FP_TPG_INTF_UP: Interface Ethernet4/14 up in fabricpath topology 02014 Aug 3 20:09:24 N7K3 %L3VM-5-FP_TPG_INTF_UP: Interface Ethernet4/13 up in fabricpath topology 0Immediately in the logs we can see adjacencies form and topology build. Lets look at our ISIS adjacencies again:N7K3# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K4 N/A 1 UP 00:00:25 Ethernet4/13N7K4 N/A 1 UP 00:00:29 Ethernet4/14

N7K4# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K3 N/A 1 UP 00:00:26 Ethernet4/13N7K3 N/A 1 UP 00:00:24 Ethernet4/14Awesome, we have an adjacency up on both links! Lets take a look at the switch-id table:N7K3# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 2============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------* 73 e8ed.f339.4f44 Primary Confirmed Yes No 74 e8ed.f339.4e44 Primary Confirmed Yes No N7K4# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 2============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+-------------------- 73 e8ed.f339.4f44 Primary Confirmed Yes No * 74 e8ed.f339.4e44 Primary Confirmed Yes No Great, we see eachothers SIDs and System-IDs. Notice the * indicates the local switch.Since all of our layer 2 interfaces are now running FabricPath, is there a need for Spanning-Tree?

N7K3# show spanning-tree No spanning tree instance exists.

N7K4# show spanning-tree No spanning tree instance exists.The switches sure dont think so! And theyre right, since all of our L2 ports are in switchport mode fabricpath, we ensured the switch that there will be no active CE (Classical Ethernet) VLANs on this switch. Check out the CAM table and youll already see some new fields for FabricPath:N7K4# sh mac address-table Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, (T) - True, (F) - False VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------* 40 0000.0c07.ac28 static - F F 73.0.4325* 50 0000.5e00.0132 static - F F 73.0.4325G - e8ed.f339.4e44 static - F F 0.0.0(R)G 40 e8ed.f339.4e44 static - F F sup-eth1(R)G 50 e8ed.f339.4e44 static - F F sup-eth1(R)We now see that traffic to 0000.0c07.ac28 will be FabricPath encapsulated with the frame directedtowards Switch-ID 73, sub-Switch-ID 0 (used in vPC), and Local ID 4325 (FabricPath edge port the frame will be forwarded on). Note: SID and SWID are used interchangeably to represent Switch-ID.Next lets bring up the FabricPath interfaces on N7K5 and N7K6 that are facing N7K3 and N7K4N7K5(config)# int e4/17-18N7K5(config-if-range)# switchport mode fabricpath

N7K6(config)# int e4/17-18N7K6(config-if-range)# switchport mode fabricpath

N7K5# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K3 N/A 1 UP 00:00:28 Ethernet4/17N7K4 N/A 1 UP 00:00:27 Ethernet4/18

N7K6# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K4 N/A 1 UP 00:00:29 Ethernet4/17N7K3 N/A 1 UP 00:00:31 Ethernet4/18

N7K3# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K4 N/A 1 UP 00:00:29 Ethernet4/13N7K4 N/A 1 UP 00:00:30 Ethernet4/14N7K5 N/A 1 UP 00:00:23 Ethernet4/15N7K6 N/A 1 UP 00:00:30 Ethernet4/16

N7K4# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K3 N/A 1 UP 00:00:29 Ethernet4/13N7K3 N/A 1 UP 00:00:26 Ethernet4/14N7K6 N/A 1 UP 00:00:31 Ethernet4/15N7K5 N/A 1 UP 00:00:28 Ethernet4/16We have adjacencies! Next step is to actually configure VLANs to run in FabricPath mode.Configure FabricPath VLANs on N7K3 and N7K4N7K3(config)# vlan 40,50N7K3(config-vlan)# mode fabricpath

N7K4(config)# vlan 40,50N7K4(config-vlan)# mode fabricpath

N7K3# sh vlan

VLAN Name Status Ports---- -------------------------------- --------- -------------------------------1 default active 40 VLAN0040 active Eth4/13, Eth4/14, Eth4/15 Eth4/1650 VLAN0050 active Eth4/13, Eth4/14, Eth4/15 Eth4/1660 VLAN0060 active 70 VLAN0070 active 80 VLAN0080 active

VLAN Type Vlan-mode---- ----- ----------1 enet CE 40 enet FABRICPATH 50 enet FABRICPATH 60 enet CE 70 enet CE 80 enet CE Notice that VLANs 40 and 50 now show FABRICPATH as the mode. The VLANs will now participate in the FabricPath domain and will run conversational MAC learning. This topic is covered well in the articles posted at the top of this blog. In short, with conversational MAC learning, the switch will only learn a MAC address if it already knows the destination MAC address, and only if it is a unicast packet. This saves on CAM resources and optimizes the control plane.Configure FabricPath VLANs on N7K5 and N7K6First, I want to show you what happens when we configure fabricpath VLANs in a vPC environment.N7K5(config)# vlan 40,50N7K5(config-vlan)# mode fabricpath N7K5(config-vlan)# end

N7K5# 2014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel22 are being suspended. (Reason: Vlan is not allowed on Peer-link)2014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel21 are being suspended. (Reason: Vlan is not allowed on Peer-link)2014 Aug 3 20:39:11 N7K5 %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on 10.122.0.209@pts/82014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel20 are being suspended. (Reason: Vlan mode not allowed on vPC)2014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel22 are being suspended. (Reason: Vlan is not allowed on Peer-link)2014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel20 are being suspended. (Reason: Vlan mode not allowed on vPC)2014 Aug 3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel21 are being suspended. (Reason: Vlan is not allowed on Peer-link)Notice the impact when configuring the VLANs without enabling FabricPath on the vPC the VLANs go suspended. Check vPC and youll see that VLANs 40 and 50 are now removed from the Peer-linkN7K5(config-vlan)# sh vpcLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 20 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 2 Peer Gateway : DisabledDual-active excluded VLANs : -Graceful Consistency Check : EnabledAuto-recovery status : Enabled (timeout = 240 seconds)

vPC Peer-link status---------------------------------------------------------------------id Port Status Active vlans -- ---- ------ --------------------------------------------------1 Po20 up 60,70,80

vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans-- ---- ------ ----------- ------ ------------21 Po21 up success success 60,70,80 22 Po22 up success success 60,70,80 Lets go ahead and convert these back to CEN7K5(config-vlan)# no mode fabricpath Configure vPC+ FabricPath VLANs on N7K5 and N7K6 vPC+The first thing we want to do is configure the switch-id which will be used to identify the vPC virtual switch domain in the FabricPath domain. If you do not do this, vPC will throw you an error:2014 Aug 3 20:41:09 N7K5 %VPC-2-VPC_CORE_PORT_FPATH_BUP_FAILED: Failed to bring up vPC+ peer link port port-channel1 in Fabric Path Port Mode - vPC+ Fabric Path switch ID not configuredConfigure the vPC FabricPath switch-IDN7K5(config)# vpc domain 20N7K5(config-vpc-domain)# fabricpath switch-id 20Configuring fabricpath switch id will flap vPCs. Continue (yes/no)? [no] yesNote: --------:: Re-init of peer-link and vPCs started ::-------- N7K6(config)# vpc domain 20N7K6(config-vpc-domain)# fabricpath switch-id 20

N7K5(config-vpc-domain)# show vpcLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 20 vPC+ switch id : 20Peer status : peer link is down (Peer-link is not in fabricpath mode for vPC+) vPC keep-alive status : peer is alive vPC fabricpath status : peer is reachable through fabricpathConfiguration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 2 Peer Gateway : DisabledDual-active excluded VLANs : -Graceful Consistency Check : EnabledAuto-recovery status : Enabled (timeout = 240 seconds)Fabricpath load balancing : DisabledPort Channel Limit : limit to 244

vPC Peer-link status---------------------------------------------------------------------id Port Status Active vlans -- ---- ------ --------------------------------------------------1 Po20 down -

vPC status-------------------------------------------------------------------------------id Port Status Consistency Reason Active vlans vPC+ Attribute-- ---- ------ ----------- ------ ------------ --------------21 Po21 down success success - DF: No, FP MAC: 20.1.65535 22 Po22 down success success - DF: No, FP MAC: 20.1.65535

Our Peer-link is down, this is because we also need to configure the switchport mode on the vpc peer-link port-channel:N7K5(config-vpc-domain)# int po20N7K5(config-if)# switchport mode fabricpath

N7K6(config-vpc-domain)# int po20N7K6(config-if)# switchport mode fabricpath

N7K5# show vpcLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 20 vPC+ switch id : 20Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive vPC fabricpath status : peer is reachable through fabricpathConfiguration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 2 Peer Gateway : DisabledDual-active excluded VLANs : -Graceful Consistency Check : EnabledAuto-recovery status : Enabled (timeout = 240 seconds)Fabricpath load balancing : DisabledPort Channel Limit : limit to 244

vPC Peer-link status---------------------------------------------------------------------id Port Status Active vlans -- ---- ------ --------------------------------------------------1 Po20 up -

vPC status-------------------------------------------------------------------------------id Port Status Consistency Reason Active vlans vPC+ Attribute-- ---- ------ ----------- ------ ------------ --------------21 Po21 up success success - DF: No, FP MAC: 20.11.65535 22 Po22 up success success - DF: No, FP MAC: 20.12.65535 Great, our peer-link is back up! Notice above these two things:1. We now have a Port Channel Limit of 244. This new limit is imposed due to the new sub-switch (sSID) ID used when running vPC+. This feid identifies the actual port-channel interfaces associated with a a particular vPC+ switch pair.2. We now have vPC+ Attributes. 20.11.65535 is the SID.sSID.LID we talked about early. Notice we now have sSIDs.Lets take a look at the FabricPath Switch-ID Table.N7K3# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 6============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+-------------------- 20 e8ed.f339.4e45 Primary Confirmed No Yes 20 e8ed.f339.4f45 Primary Confirmed No Yes * 73 e8ed.f339.4f44 Primary Confirmed Yes No 74 e8ed.f339.4e44 Primary Confirmed Yes No 75 e8ed.f339.4f45 Primary Confirmed Yes No 76 e8ed.f339.4e45 Primary Confirmed Yes No N7K4# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 6============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+-------------------- 20 e8ed.f339.4e45 Primary Confirmed No Yes 20 e8ed.f339.4f45 Primary Confirmed No Yes 73 e8ed.f339.4f44 Primary Confirmed Yes No * 74 e8ed.f339.4e44 Primary Confirmed Yes No 75 e8ed.f339.4f45 Primary Confirmed Yes No 76 e8ed.f339.4e45 Primary Confirmed Yes No N7K5# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 6============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------[E] 20 e8ed.f339.4f45 Primary Confirmed No Yes 20 e8ed.f339.4e45 Primary Confirmed No Yes 73 e8ed.f339.4f44 Primary Confirmed Yes No 74 e8ed.f339.4e44 Primary Confirmed Yes No * 75 e8ed.f339.4f45 Primary Confirmed Yes No 76 e8ed.f339.4e45 Primary Confirmed Yes No N7K6# show fabricpath switch-id FABRICPATH SWITCH-ID TABLE Legend: '*' - this system '[E]' - local Emulated Switch-id '[A]' - local Anycast Switch-idTotal Switch-ids: 6============================================================================= SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED/ ANYCAST --------------+----------------+------------+-----------+--------------------[E] 20 e8ed.f339.4e45 Primary Confirmed No Yes 20 e8ed.f339.4f45 Primary Confirmed No Yes 73 e8ed.f339.4f44 Primary Confirmed Yes No 74 e8ed.f339.4e44 Primary Confirmed Yes No 75 e8ed.f339.4f45 Primary Confirmed Yes No * 76 e8ed.f339.4e45 Primary Confirmed Yes NoWe have Emulated Switch-IDs that identify the vPC+ switches. Youll see a single emulated switch-id with two system-IDs that match the actual vPC peers.Since we did not configure all interfaces on N7K5 and N7K6 as mode fabricpath, we must still run spanning-tree for the classical ethernet ports.

N7K5# sh spanning-tree vlan 40

VLAN0040 Spanning tree enabled protocol rstp Root ID Priority 32808 Address c84c.75fa.6000 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32808 (priority 32768 sys-id-ext 40) Address c84c.75fa.6000 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Po21 Desg FWD 1 128.4116 (vPC) P2p Po22 Desg FWD 1 128.4117 (vPC) P2p RoutingLets take a look at the routing table:N7K3# show fabricpath routeFabricPath Unicast Route Table'a/b/c' denotes ftag/switch-id/subswitch-id'[x/y]' denotes [admin distance/metric]ftag 0 is local ftagsubswitch-id 0 is default subswitch-id

FabricPath Unicast Route Table for Topology-Default

0/73/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:45:07, local1/20/0, number of next-hops: 2 via Eth4/15, [115/40], 0 day/s 00:06:59, isis_fabricpath-default via Eth4/16, [115/40], 0 day/s 00:06:59, isis_fabricpath-default1/74/0, number of next-hops: 2 via Eth4/13, [115/40], 0 day/s 00:40:25, isis_fabricpath-default via Eth4/14, [115/40], 0 day/s 00:40:25, isis_fabricpath-default1/75/0, number of next-hops: 1 via Eth4/15, [115/40], 0 day/s 00:14:25, isis_fabricpath-default1/76/0, number of next-hops: 1 via Eth4/16, [115/40], 0 day/s 00:14:12, isis_fabricpath-defaultN7K4# show fabricpath route...

0/74/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:44:48, local1/20/0, number of next-hops: 2 via Eth4/15, [115/40], 0 day/s 00:06:55, isis_fabricpath-default via Eth4/16, [115/40], 0 day/s 00:06:55, isis_fabricpath-default1/73/0, number of next-hops: 2 via Eth4/13, [115/40], 0 day/s 00:40:21, isis_fabricpath-default via Eth4/14, [115/40], 0 day/s 00:40:21, isis_fabricpath-default1/75/0, number of next-hops: 1 via Eth4/16, [115/40], 0 day/s 00:14:21, isis_fabricpath-default1/76/0, number of next-hops: 1 via Eth4/15, [115/40], 0 day/s 00:14:08, isis_fabricpath-defaultN7K5# show fabricpath route...

0/20/1, number of next-hops: 00/20/11, number of next-hops: 1 via Po21, [80/0], 0 day/s 00:06:50, vpcm0/20/12, number of next-hops: 1 via Po22, [80/0], 0 day/s 00:06:50, vpcm0/75/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:44:30, local1/20/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:06:50, local1/73/0, number of next-hops: 1 via Eth4/17, [115/40], 0 day/s 00:14:16, isis_fabricpath-default1/74/0, number of next-hops: 1 via Eth4/18, [115/40], 0 day/s 00:14:16, isis_fabricpath-default1/76/0, number of next-hops: 1 via Po20, [115/20], 0 day/s 00:06:50, isis_fabricpath-default2/20/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:06:50, localN7K6# show fabricpath route...

0/20/1, number of next-hops: 00/20/11, number of next-hops: 1 via Po21, [80/0], 0 day/s 00:06:45, vpcm0/20/12, number of next-hops: 1 via Po22, [80/0], 0 day/s 00:06:45, vpcm0/76/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:44:10, local1/20/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:06:45, local1/73/0, number of next-hops: 1 via Eth4/18, [115/40], 0 day/s 00:13:58, isis_fabricpath-default1/74/0, number of next-hops: 1 via Eth4/17, [115/40], 0 day/s 00:13:58, isis_fabricpath-default1/75/0, number of next-hops: 1 via Po20, [115/20], 0 day/s 00:06:45, isis_fabricpath-default2/20/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:06:45, localWe can quickly see the ftag/switch-id/subswitch-id routes, and the paths based on the admin distance/metric. Notice on N7K3 that we have two equal-cost paths to N7K4 1/74/0 (FTAG 1 / SID 74 / sSID 0 because no vPC involved). FTAGs are used for multidestination traffic, which we will get to very soon.1/74/0, number of next-hops: 2 via Eth4/13, [115/40], 0 day/s 00:40:25, isis_fabricpath-default via Eth4/14, [115/40], 0 day/s 00:40:25, isis_fabricpath-defaultTraffic EngineeringRemember, FabricPath is Layer-2 routing, and we can use our routing protocol IS-IS to engineer traffic. Lets say we wanted N7K3 to prefer the path over Eth4/14 to get to N7K4. We could increase the metric on Eth4/13 to something higher than 40.N7K3(config)# inte e4/13N7K3(config-if)# fabricpath isis metric 100

N7K3# show fabricpath routeFabricPath Unicast Route Table'a/b/c' denotes ftag/switch-id/subswitch-id'[x/y]' denotes [admin distance/metric]ftag 0 is local ftagsubswitch-id 0 is default subswitch-id


0/73/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 00:46:15, local1/20/0, number of next-hops: 2 via Eth4/15, [115/40], 0 day/s 00:08:07, isis_fabricpath-default via Eth4/16, [115/40], 0 day/s 00:08:07, isis_fabricpath-default1/74/0, number of next-hops: 1 via Eth4/14, [115/40], 0 day/s 00:41:33, isis_fabricpath-default1/75/0, number of next-hops: 1 via Eth4/15, [115/40], 0 day/s 00:15:33, isis_fabricpath-default1/76/0, number of next-hops: 1 via Eth4/16, [115/40], 0 day/s 00:15:20, isis_fabricpath-defaultWe now only have 1 link listed in the route table to SID 74.Multidestination TreesWe know that unicast traffic is L2 routed based on the SID and uses the IS-IS SPT to get to the destination SID. But what about multidestination traffic, such as multicast, broadcast and unknown unicasts? Well, thats handled a little differently.FabricPath automatically builds two separate logical trees for handling multidestination traffic. The first tree is used to handle broadcast and unknown unicasts, the second tree is used to handle multicast traffic. Each tree is assigned a network-wide identity, known as an FTAG.Tree 1 = FTAG 1 = Broadcast and unknown unicastTree 2 = FTAG 2 = MulticastLike Spanning-Tree, each tree has a root that is chosen automatically, based on this criteria:1.Highest root priority 8-bit value between 0-255 (Default is 64)2.Highest System-ID 48-bit VDC MAC address3.Highest Switch-ID 12-bit SIDLets take a look at the current multidestination topology.N7K3# show fabricpath isis topology summFabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/13 Ethernet4/14 Ethernet4/15 Ethernet4/16 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75 Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73Ftag Proxy Root: e8ed.f339.4f45

N7K4# show fabricpath isis topology summary FabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/13 Ethernet4/14 Ethernet4/15 Ethernet4/16 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75 Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73Ftag Proxy Root: e8ed.f339.4f45

N7K5# sh fa i to sFabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/17 Ethernet4/18 port-channel20 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1 [transit-traffic-only], root system: e8ed.f339.4f45, 75 Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73Ftag Proxy Root: e8ed.f339.4f45

N7K6# show fabricpath isis topology summFabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/17 Ethernet4/18 port-channel20 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75 Tree id: 2, ftag: 2 [transit-traffic-only], root system: e8ed.f339.4f44, 73Ftag Proxy Root: e8ed.f339.4f45From the output above, we can see that SID 75 has been chosen as the root for Tree 1, and SID 73 has been chosen as the root for Tree 2. Maybe we want to change this, and have N7K3 be the root of the tree for broadcast traffic, and N7K4 be the root for multicast traffic. Our new multidestination topology would look like this:

To do so, we can change the root priority.FabricPath Root PriorityFirst learning FabricPath, I thought this was an unusual place to configure root-priorty. To easily remember (in case you forget), just do a show run fabricpath and you will see the fabricpath domain default in the configuration at all times. This is a kindly reminder where you need to configure this parameter.N7K3(config)# fabricpath domain defaultN7K3(config-fabricpath-isis)# root-priority 255Notice SID 73 is now the root of tree 1:N7K3(config-fabricpath-isis)# show fabricpath isis top summFabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/13 Ethernet4/14 Ethernet4/15 Ethernet4/16 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1, root system: e8ed.f339.4f44, 73 Tree id: 2, ftag: 2, root system: e8ed.f339.4f45, 75Ftag Proxy Root: e8ed.f339.4f44Configure N7K4 to be the root for multicast tree 2N7K4(config)# fabricpath domain default N7K4(config-fabricpath-isis)# root-priority 254

N7K4(config-fabricpath-isis)# sh fabricpath isis topology summFabricPath IS-IS Topology SummaryFabricpath IS-IS domain: defaultMT-0 Configured interfaces: Ethernet4/13 Ethernet4/14 Ethernet4/15 Ethernet4/16 Max number of trees: 2 Number of trees supported: 2 Tree id: 1, ftag: 1, root system: e8ed.f339.4f44, 73 Tree id: 2, ftag: 2, root system: e8ed.f339.4e44, 74Ftag Proxy Root: e8ed.f339.4f44Another way to look at trees is with the show fabricpath isis trees command. This will actually show you the metricsN7K3# show fabricpath isis treesFabricpath IS-IS domain: defaultNote: The metric mentioned for multidestination tree is from the root of that tree to that switch-id*:directly connected neighbor or linkP:Physical switch-id, E:Emulated, A:Anycast

MT-0Topology 0, Tree 1, Swid routing table20, L1 via Ethernet4/16, metric 4074, L1 via Ethernet4/14, metric 4075, L1 via Ethernet4/15, metric 4076, L1 via Ethernet4/16, metric 40

Topology 0, Tree 2, Swid routing table20, L1 via Ethernet4/14, metric 4074, L1 via Ethernet4/14, metric 075, L1 via Ethernet4/14, metric 4076, L1 via Ethernet4/14, metric 40ECMPWe can also verify the tree roots by looking at our mroute tables and observing the outgoing interfaces. Notice below that on N7K5 we are using E4/17 to get to Tree 1 (N7K3) and using E4/18 to get to Tree 2 (N7K4)N7K5# sh fabricpath mroute ftag 1

(ftag/1, vlan/40, *, *), Flood, uptime: 00:44:49, isis Outgoing interface list: (count: 3) Interface Ethernet4/17, Switch-id 73, uptime: 00:55:38, isis Interface Ethernet4/17, Switch-id 74, uptime: 00:35:53, isis Interface Ethernet4/17, Switch-id 76, uptime: 00:35:53, isis

truncated...

N7K5# sh fabricpath mroute ftag 2

(ftag/2, vlan/40, *, *), Flood, uptime: 00:44:51, isis Outgoing interface list: (count: 3) Interface Ethernet4/18, Switch-id 73, uptime: 00:35:36, isis Interface Ethernet4/18, Switch-id 74, uptime: 00:35:55, isis Interface Ethernet4/18, Switch-id 76, uptime: 00:35:36, isis

truncated...Another fun command is verifying ECMP load-balancing. The default (configurable) load-balancing is shown below. Notice if we change just a single parameter in our flow selector that a different interface is chosen for the ECMP.N7K3# show fabricpath load-balance ECMP load-balancing configuration: L3/L4 Preference: MixedHash Control: SymmetricRotate amount: 1 bytesUse VLAN: TRUE

Ftag load-balancing configuration: Hash Control: SymmetricRotate amount: 1 bytesUse VLAN: TRUE

N7K3(config)# interface Ethernet4/13N7K3(config-if)# no fabricpath isis metric 100

N7K3# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 74 flow-type l3 src-ip 1.1.1.1 dst-ip 2.2.2.2 vlan 40 module 4This flow selects interface Eth4/13

N7K3# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 74 flow-type l3 src-ip 1.1.1.1 dst-ip 2.2.2.3 vlan 40 module 4This flow selects interface Eth4/14FabricPath AuthenticationWhat would a routing protocol be without authentication? We have two forms of authentication with FabricPath. First we have interface authentication, which is the actual hello adjaceny authenticationN7K3(config)# key chain FPKEYN7K3(config-keychain)# key 1N7K3(config-keychain-key)# key-string FPKEYN7K3(config-keychain-key)# exitN7K3(config)# int e4/13-14 N7K3(config-if-range)# fabricpath isis authentication-type md5N7K3(config-if-range)# fabricpath isis authentication key-chain FPKEY

2014 Aug 3 23:04:48 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 N7K4 over Ethernet4/14 - DOWN (Hold timer expired) on MT-02014 Aug 3 23:05:47 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [30986] P2P adj L1 N7K4 over Ethernet4/13 - DOWN (Hold timer expired) on MT-0

N7K3# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K4 N/A 1 LOST 00:05:35 Ethernet4/13N7K4 N/A 1 LOST 00:04:35 Ethernet4/14N7K5 N/A 1 UP 00:00:29 Ethernet4/15N7K6 N/A 1 UP 00:00:28 Ethernet4/16Notice our adjacencies are lost. Lets configure the other side.N7K4(config)# key chain FPKEYN7K4(config-keychain)# key 1N7K4(config-keychain-key)# key-string FPKEYN7K4(config-keychain-key)# exitN7K4(config)# int e4/13-14 N7K4(config-if-range)# fabricpath isis authentication-type md5N7K4(config-if-range)# fabricpath isis authentication key-chain FPKEY

2014 Aug 3 23:07:20 N7K4 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [8149] P2P adj L1 N7K3 over Ethernet4/14 - UP on MT-02014 Aug 3 23:07:26 N7K4 %ISIS_FABRICPATH-5-ADJCHANGE: isis_fabricpath-default [8149] P2P adj L1 N7K3 over Ethernet4/13 - UP on MT-0And were back in business. Notice below that we can see authentication is enabled on the interfaceN7K3# show fabricpath isis interf e4/13Fabricpath IS-IS domain: default Interface: Ethernet4/13 Status: protocol-up/link-up/admin-up Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1 Authentication type MD5 Authentication keychain is FPKEY Authentication check specified Extended Local Circuit ID: 0x1A18C000, P2P Circuit ID: 0000.0000.0000.00 Retx interval: 5, Retx throttle interval: 66 ms LSP interval: 33 ms, MTU: 1500 P2P Adjs: 1, AdjsUp: 1, Priority 64 Hello Interval: 10, Multi: 3, Next IIH: 00:00:02 Level Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID 1 1 1 40 60 Inactive ffff.ffff.ffff.ff-ff Topologies enabled: Level Topology Metric MetricConfig Forwarding 0 0 40 no UP 1 0 40 no UP The next form of authentication we have is FabricPath domain authentication which enforces authentication of the actual IS-IS LSPs. Authentication here will prevent routes from being learned, however, we can still form adjacencies even when the domain authentication is mismatched.N7K3(config)# fabricpath domain defaultN7K3(config-fabricpath-isis)# authentication-type md5 N7K3(config-fabricpath-isis)# authentication key-chain FPKEYNotice authentication is enabledN7K3# show fabricpath isis

Fabricpath IS-IS domain : default System ID : e8ed.f339.4f44 IS-Type : L1 Fabric-Control SVI: Unknown SAP : 432 Queue Handle : 17 Maximum LSP MTU: 1492 Graceful Restart enabled. State: Inactive Last graceful restart status : none Graceful Restart holding time:60 Metric-style : advertise(wide), accept(wide) Start-Mode: Complete [Start-type configuration] Area address(es) : 00 Process is up and running CIB ID: 1 Interfaces supported by Fabricpath IS-IS : Ethernet4/13 Ethernet4/14 Ethernet4/15 Ethernet4/16 Level 1 Authentication type: MD5 Authentication keychain: FPKEY Authentication check specified LSP Lifetime: 1200 L1 LSP GEN interval- Max:8000 Initial:50 Second:50 L1 SPF Interval- Max:8000 Initial:50 Second:50 MT-0 Ref-Bw: 400000 Max-Path: 16 Address family Swid unicast : Number of interface : 4 Distance : 115 L1 Next SPF: InactiveWe have adjacencies, but we can no longer see Switch-IDsN7K3# show fabricpath isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time Interfacee8ed.f339.4e44 N/A 1 UP 00:00:25 Ethernet4/13e8ed.f339.4e44 N/A 1 UP 00:00:33 Ethernet4/14e8ed.f339.4f45 N/A 1 UP 00:00:25 Ethernet4/15e8ed.f339.4e45 N/A 1 UP 00:00:29 Ethernet4/16We no longer have routesN7K3# show fabricpath routeFabricPath Unicast Route Table'a/b/c' denotes ftag/switch-id/subswitch-id'[x/y]' denotes [admin distance/metric]ftag 0 is local ftagsubswitch-id 0 is default subswitch-id


0/73/0, number of next-hops: 0 via ---- , [60/0], 1 day/s 02:15:27, localOnce we configure the other switches with domain authentication, our adjacency tables will populate the SIDs, and our route tables will build.N7K3# sho fab isis adjacency Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:System ID SNPA Level State Hold Time InterfaceN7K4 N/A 1 UP 00:00:33 Ethernet4/13N7K4 N/A 1 UP 00:00:24 Ethernet4/14N7K5N/A 1 UP 00:00:22 Ethernet4/15N7K6N/A 1 UP 00:00:26 Ethernet4/16Full Sample configAs promised, here is a full sample config, N7K3 and N7K4 are configured for authentication. N7K5 and N7K6 are running vPC+ FabricPath.N7K3# sh run fabricpath

feature-set fabricpath

vlan 40,50 mode fabricpathfabricpath switch-id 73

interface Ethernet4/13 fabricpath isis authentication-type md5 fabricpath isis authentication key-chain FPKEY switchport mode fabricpath


interface Ethernet4/15 switchport mode fabricpath

interface Ethernet4/16 switchport mode fabricpathfabricpath domain default authentication-type md5 authentication key-chain FPKEY root-priority 255 N7K4# sh run fabricpath


vlan 40,50 mode fabricpathfabricpath switch-id 74




interface Ethernet4/16 switchport mode fabricpathfabricpath domain default authentication-type md5 authentication key-chain FPKEY root-priority 254 N7K5# sh run fabricpath


vlan 40,50 mode fabricpathfabricpath switch-id 75vpc domain 20 fabricpath switch-id 20

interface port-channel20 switchport mode fabricpath




interface Ethernet4/20 switchport mode fabricpathfabricpath domain default N7K6# sh run fabricpath


vlan 40,50 mode fabricpathfabricpath switch-id 76vpc domain 20 fabricpath switch-id 20

interface port-channel20 switchport mode fabricpath




interface Ethernet4/20 switchport mode fabricpathfabricpath domain defaultHelpful show commandsshow fabricpath isis adjacencyshow fabricpath switch-idshow fabricpath isis database [detail]show fabricpath routeshow fabricpath isis topology summaryshow fabricpath isis trees [multidestination 1|2]show fabricpath mrouteshow key-chainclear fabricpath isis adjacency *

Cisco Fabric Path Doc 2

Documents

Transcript of Cisco Fabric Path Doc 2