LABDCT 2001 (Guide) Nexus.7000

© 2009 Cisco Systems, Inc. All rights reserved 1 of 44

Cisco Nexus 7000 Series LAB

NATALE RUELLO

ROBERT STARMER

Technical Marketing - Data Center Business Unit


Nexus 7000

The Cisco Nexus 7000 Series is a modular data center class series of switching systems

designed for highly scalable end-to-end 10 Gigabit Ethernet networks. The Cisco Nexus 7000

Series is purpose built for the data center and has many unique features and capabilities

designed specifically for the most mission critical place in the network, the Data Center.

Cisco NX-OS, a state-of-the-art operating system, powers the Cisco Nexus 7000 Platform.

Cisco NX-OS is a data center-class operating system built with modularity, resiliency, and

serviceability at its foundation. Drawing on its Cisco IOS and Cisco SAN-OS heritage, Cisco

NX-OS helps ensure continuous availability and sets the standard for mission-critical data

center environments.

Lab Objectives

This instructor-led hands-on lab will introduce the participants to the NX-OS, the operating

system powering the Nexus family switches. The participants will be exposed to the

configuration of some of the new features present in NX-OS. The lab will also focus on some

of the aspects that differentiate NX-OS from the classical IOS.


Lab Procedure

The Lab consists of 10 PODs. Each single POD represents a typical but simplified 3-tier Data

Center design. The core consist of a Catalyst 6500, the aggregation layer consists of two

Nexus’ 7000, while an ESX server and a Nexus 5000 compose the access layer. In more

details, the aggregation layer (on which all the configuration for this lab is performed) is

formed by two N7K-C7010 with one N7K-M148GT-12 and one N7K-M132XP-12 card each.

These two systems run a pre-release version of NX-OS 4.1(3).

A group of two students is assigned to each Pod. Each student will be able to configure his

own Nexus 7000 aggregation device.

During the Lab procedure the students will go through the following steps:

� System Verification

� Management VRF Concept and Basic Connectivity

� CLI Tips

� Role Based Access Control (RBAC)

� Configuration Rollback

� Links Up and Spanning Tree Protocol

� HSRP

� Virtual Port Channel (vPC)

� vPC Failure Scenario

� OSPF

� Stateful Process Restart

� Wireshark

� Virtual Device Contexts (VDCs)


Lab Topology and Access

The diagrams below represent the logical lab setup for the odd and the even pods.

Figure 1 Topology for the odd Pods (1,3,5,7,9)

Figure 2 Topology for the even Pods (2,4,6,8,10)

2/9

2/2 2/1

1/1 1/2

2/1-2

1/13

2/9

Aggregation

N7K-1 N7K-2

C6K-1

N5K

Student 2 Student 1

VMware ESX

2/1-2

1/13

2/10

6K6K6K6K

Access

2/10

Po 10 Po 10

2/25

2/2 2/1

1/1 1/2

2/17-18

1/25

2/25 Aggregation

N7K-1 N7K-2

C6K-1

N5K

Student 2 Student 1

VMware ESX

2/17-18

1/25

2/26

6K6K6K6K

Access

2/26

Po 10 Po 10


The real lab looks quite different ☺… We are actually using the Virtual Device Context

feature.

With your teammate decide which of the two Nexus 7000 aggregation devices you will

be working on. Each POD will be used by a group of two students that will work within the

POD’s Virtual Device Context. All access to your POD devices is via the ESX VMware server

that is available via Microsoft Remote Desktop Client access. Remote desktop access is

defined in the Table 1. In order to connect to the Nexus device:

1. Open the Microsoft Remote Desktop Client on your workstation and point your

machine to the Pod’s VM instance as shown in Table1.

Table 1 POD Access Details

POD Information VM Instance Login/Password

POD1 128.107.222.196 Student1/NXospod1-S1 Student2/NXospod1-S2








POD9 128.107.222.204 Student1/Nxospod9-S1 Student2/Nxospod9-S2

Pod 1 Pod 2

N7K-Aggr N7K-Aggr


POD10 128.107.222.205 Student1/Nxospod10-S1 Student2/Nxospod10-S2

2. For your convenience you will find the puTTY connections on the Desktop. Double

click on the connection of the Nexus 7000 you decided to use (make sure your

teammate will work on the other device). Click YES on the warning message.

Step 1 System Verification PLEASE NOTE: the interfaces referred in most of the output shown in these steps refer

to Pod1. If you are on a different Pod please refer to Figure 1 and Figure 2 on page 4 for the

correspondent interfaces for your Pod.

During the entire duration of this lab we will be just logging into the management interface via

ssh. However it is good to keep in mind that the Nexus 7000 requires console access to

perform the initial configuration of the system. After performing the initial configuration, the

system can be completely managed from the management interface.

Let’s start by checking the system and its configuration.

N7K-1-pod1-S1# show module

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 48 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok 2 32 10 Gbps Ethernet Module N7K-M132XP-12 ok 5 0 Supervisor module-1X N7K-SUP1 ha-standby

6 0 Supervisor module-1X N7K-SUP1 active *

Mod Sw Hw

--- -------------- ------

1 4.1(2.7) 1.0 2 4.1(2.7) 1.3 5 4.1(2.7) 1.0 6 4.1(2.7) 1.0

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 00-1b-54-c1-d1-08 to 00-1b-54-c1-d1-3c JAB122101LL

2 00-1b-54-c1-9a-40 to 00-1b-54-c1-9a-64 JAB1220009J

5 00-22-55-77-5e-e0 to 00-22-55-77-5e-e8 JAB12250199

6 00-22-55-77-5e-50 to 00-22-55-77-5e-58 JAB1225018U

Mod Online Diag Status

--- ------------------

1 Pass

2 Pass

5 Pass

6 Pass

Xbar Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------


1 0 Fabric Module 1 N7K-C7010-FAB-1 ok



Xbar Sw Hw

--- -------------- ------

1 NA 1.0

2 NA 1.0

3 NA 1.0

Xbar MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 NA JAB122300ZH

2 NA JAB122400QQ

3 NA JAB122400QK

* this terminal session

N7K-1-pod1-S1#

Let’s check now the software the system is running.

N7K-1-pod1-S1# show version Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at http://www.opensource.org/licenses/gpl-2.0.php

and http://www.opensource.org/licenses/lgpl-2.1.php

Software

BIOS: version 3.17.0

loader: version N/A

kickstart: version 4.1(3) system: version 4.1(3) BIOS compile time: 03/23/08

kickstart image file is: bootflash:/n7000-s1-kickstart.4.1.3.bin.S7 kickstart compile time: 1/15/2009 12:00:00 [12/20/2008 11:18:14]

system image file is: bootflash:/n7000-s1-dk9.4.1.3.bin.S7 system compile time: 1/15/2009 12:00:00 [12/20/2008 12:53:33]

Hardware

cisco Nexus7000 C7010 (10 Slot) Chassis ("Supervisor module-1X")

Intel(R) Xeon(R) CPU with 4129620 kB of memory. Processor Board ID JAB123501Z7

Device name: N7K-1

bootflash: 2000880 kB slot0: 0 kB (expansion flash)

Kernel uptime is 0 day(s), 23 hour(s), 39 minute(s), 59 second(s)

Last reset at 185087 usecs after Tue Dec 2 04:59:22 2008

NX-OS Version

Storage Devices

Images Location

CPU


Reason: Reset Requested by CLI command reload

System version: 4.1(1.66)

Service:

plugin

Core Plugin, Ethernet Plugin N7K-1-pod1-S1#

Note: NX-OS is composed by two images: a kickstart image that contains the Linux Kernel and a system image that contain most of the NX-OS software components. They both show up in the configuration.

Note: In future release we will be adding other plug-ins, like the “Storage” plug-in for FCoE

Let’s now take a look at the running configuration.

N7K-1-pod1-S1# show running-config

version 4.1(3) <omitted config> vrf context management vlan 1-4 interface Ethernet2/1 interface Ethernet2/2 <omitted interface config> interface Ethernet2/16 interface mgmt0 ip address 192.168.100.20/24

Note: This is the configuration of the first Pod. As explained earlier each Pod runs within a Virtual Device Context (VDC). By using the VDC feature, we can segment the physical Nexus 7000 in multiple logical switches each of which runs in a separate memory space and has visibility only of the hardware resources that it owns, providing total isolation between the VDCs.

The “show running-config” has been improved. One of the improvements consists in the

ability to not only look at the running-config but to also at the defaults values, which do not

show up in the base config. The keyword to be used is “all”.

N7K-1-pod1-S1# show running-config all | begin mgmt0 interface mgmt0

Active Plug-in

These are the interfaces available to your Pod (Virtual Device Context)

Management Interface Config


cdp enable

description

speed auto

duplex auto

no shutdown

ip address 192.168.100.20/24

ip redirects

ip port-unreachable

ip arp gratuitous update

ip arp gratuitous request

line vty

session-limit 32

no exec-timeout

line console

no exec-timeout

terminal length 24

terminal width 80

cfs distribute

no cfs eth distribute

cfs ipv4 mcast-address 239.255.70.83

cfs ipv6 mcast-address ff15::efff:4653

no cfs ipv4 distribute

no cfs ipv6 distribute

ip source-route

ip igmp event-history mtrace size small

ip igmp event-history igmp-internal size small

ip igmp event-history vrf size small

ip igmp event-history events size medium

ip igmp event-history debugs size medium

<omitted output>

Step 2 Management VRF and Basic Connectivity

The management interface is, by default, part of the management VRF. The management

interface “mgmt0” is the only interface allowed to be part of this VRF.

The philosophy beyond Management VRF is to provide total isolation for the management

traffic from the rest of the traffic flowing through the box by confining the former to its own

forwarding table.

In this step we will:

- Verify that only the mgmt0 interface is part of the management VRF

- Verify that no other interface can be part of the management VRF

- Verify that the default gateway is reachable only using the management VRF


N7K-1-pod1-S1# show vrf

VRF-Name VRF-ID State Reason

default 1 Up --

management 2 Up --

N7K-1-pod1-S1# show vrf interface

Interface VRF-Name VRF-ID

mgmt0 management 2 Ethernet1/1 default 1

Ethernet1/2 default 1




<omitted output>

N7K-1-pod1-S1# show vrf management interface

Interface VRF-Name VRF-ID

mgmt0 management 2

Note: The management VRF interface is part of the default configuration and the management interface “mgmt0” is the only interface that can be made member of this VRF. Let’s verify it.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# interface ethernet 2/1 N7K-1-pod1-S1(config-if)# vrf member management % VRF management is reserved only for mgmt0

N7K-1-pod1-S1(config-if)# show int mgmt0

mgmt0 is up

Hardware: GigabitEthernet, address: 0022.5577.5e50 (bia 0022.5577.5e50)

Internet Address is 192.168.100.20/24

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA

full-duplex, 1000 Mb/s

Auto-Negotiation is turned on

1 minute input rate 1264 bits/sec, 1 packets/sec

1 minute output rate 1136 bits/sec, 0 packets/sec

Rx

743 input packets 679 unicast packets 60 multicast packets

4 broadcast packets 70900 bytes

FastEthernet? GigabitEthernet?... no, just “ethernet” interfaces


Tx

567 output packets 542 unicast packets 23 multicast packets

2 broadcast packets 66407 bytes

Try to reach the out-of-bound management network’s default gateway with a ping.

N7K-1-pod1-S1(config-if)# ping 192.168.100.250

PING 192.168.100.250 (192.168.100.250): 56 data bytes

ping: sendto 192.168.100.250 64 chars, No route to host

Request 0 timed out


Request 1 timed out


Request 2 timed out


Request 3 timed out


Request 4 timed out

--- 192.168.100.250 ping statistics ---

5 packets transmitted, 0 packets received, 100.00% packet loss N7K-1-pod1-S1(config-if)#

Note: The ping fails because we are trying to reach a system on the out-of-band management network without specifying the correct VRF.

N7K-1-pod1-S1# ping 192.168.100.250 vrf management

PING 192.168.100.250 (192.168.100.250): 56 data bytes

Request 0 timed out

64 bytes from 192.168.100.250: icmp_seq=1 ttl=254 time=0.887 ms




--- 192.168.100.250 ping statistics ---

5 packets transmitted, 4 packets received, 20.00% packet loss

round-trip min/avg/max = 0.816/0.873/0.943 ms

N7K-1-pod1-S1#

Linux-like output


Step 3 CLI Familiarization

NX-OS CLI is very IOS-like. As you may have already noticed, when configuring the system,

NX-OS gives the user a very IOS look and feel sensation. However there are differences,

which we consider improvements. One of the main differences consists in NX-OS

implementing a hierarchy independent CLI.

Every command can in fact be issued from anywhere in the configuration.


- Verify the CLI hierarchy independence by issuing a ping from different places in the chain

- Verify the CLI piping functionality

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# ping ? *** No matches in current mode, matching in (exec) mode *** <CR>

A.B.C.D or Hostname IP address of remote system

WORD Enter Hostname

multicast Multicast ping

N7K-1-pod1-S1(config)# ping 192.168.100.250 vrf management

PING 192.168.100.250 (192.168.100.250): 56 data bytes


64 bytes from 192.168.100.250: icmp_seq=1 ttl=254 time=0.733 ms <omitted output> --- 192.168.100.250 ping statistics --- 4 packets transmitted, 4 packets received, 0.00% packet loss round-trip min/avg/max = 0.733/0.787/0.874 ms

N7K-1-pod1-S1(config)# int e2/1

N7K-1-pod1-S1(config-if)# ping ?

*** No matches in current mode, matching in (exec) mode ***

<CR>

A.B.C.D or Hostname IP address of remote system

WORD Enter Hostname

multicast Multicast ping

N7K-1-pod1-S1(config-if)# ping 192.168.100.250 vrf management

PING 192.168.100.250 (192.168.100.250): 56 data bytes 64 bytes from 192.168.100.250: icmp_seq=0 ttl=254 time=0.943 ms <omitted output>

N7K-1-pod1-S1(config-if)#

CLI Hierarchy Independent

CLI Hierarchy Independent


Note: You can use the up-arrow and get the command history from the exec mode

Note: Any command can be issued from anywhere within the configuration

The output piping has also been improved and it’s now very similar to the one on Linux

machines.

N7K-1-pod1-S1# show running-config | ?

cut Print selected parts of lines.

egrep Egrep - print lines matching a pattern

grep Grep - print lines matching a pattern

head Display first lines

last Display last lines

less Filter for paging

no-more Turn-off pagination for command output

sed Stream Editor

sort Stream Sorter

tr Translate, squeeze, and/or delete characters

uniq Discard all but one of successive identical lines

vsh The shell than understands cli command

wc Count words, lines, characters

begin Begin with the line that matches

count Count number of lines

end End with the line that matches

exclude Exclude lines that match

include Include lines that match

N7K-1-pod1-S1# sh running-config | grep ?

WORD Search for the expression

count Print a total count of matching lines only

ignore-case Ignore case difference when comparing strings

invert-match Print only lines that contain no matches for <expr>

line-exp Print only lines where the match is a whole line

line-number Print each match preceded by its line number

next Print <num> lines of context after every matching line

prev Print <num> lines of context before every matching line

word-exp Print only lines where the match is a complete word

N7K-1-pod1-S1# sh running-config | grep --v

grep (GNU grep) 2.5.1

Copyright 1988, 1992-1999, 2000, 2001 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The following command will grab the instance of a line with “mgmt0” and print the following 3

lines after that match.

N7K-1-pod1-S1# sh running-config | grep next 3 mgmt0

interface mgmt0 ip address 192.168.100.20/24

Improved CLI Piping


N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# int mgmt 0 N7K-1-pod1-S1(config-if)# [TAB] cdp exit no shutdown

description ip pop vrf

end ipv6 push where

Note: The [TAB] does not only complete the command, but also it shows the available keywords.

N7K-1-pod1-S1(config-if)# ?

cdp Configure CDP interface parameters

description Enter description of maximum 80 characters

end Go to exec mode

exit Exit from command interpreter

ip Configure IP features

ipv6 Configure IPv6 features

no Negate a command or set its defaults

pop Pop mode from stack or restore from name

push Push current mode to stack or save it under name

shutdown Enable/disable an interface

vrf Configure VRF parameters

where Shows the cli context you are in

If you want to know the CLI context you are in use the “where” command.

N7K-1-pod1-S1(config-if)# where

conf; interface mgmt0 admin@N7K-1-pod1-S1%default

Step 4 Role Based Access Control (RBAC)

RBAC stands for “Role Based Access Control”. Upon login, every user gets assigned a “role”

that defines the privileges of the user that gained access to system. NX-OS, through the

RABC feature, provides a very flexible and powerful framework to create ad hoc roles for any

type of user. The roles are groups of rules that permit or deny a set of operations on NX-OS

components.


- Display the default roles

- Display the features and the feature-groups that can be used as part of the role

- Create a new role and apply the role to a newly created user

- Display the newly created role

- Test the role


NX-OS implements 4 default roles for the default VDC. Since the students are logged into

a non-default VDC, only the two VDC default roles will be visible. For completeness the

CLI output below shows all of them but on the students’ Pods only the last two (in bold here)

will be visible.

N7K-1-pod1-S1# show role

role: network-admin

description: Predefined network admin role has access to all commands

on the switch

attribute: global

-------------------------------------------------------------------

Rule Perm Type Scope Entity

-------------------------------------------------------------------

1 permit read-write

role: network-operator

description: Predefined network operator role has access to all read

commands on the switch

attribute: global

-------------------------------------------------------------------


-------------------------------------------------------------------

1 permit read

role: vdc-admin description: Predefined vdc admin role has access to all commands within a VDC instance attribute: local

-------------------------------------------------------------------


-------------------------------------------------------------------

1 permit read-write

role: vdc-operator

description: Predefined vdc operator role has access to all read commands within a VDC instance attribute: local

-------------------------------------------------------------------


-------------------------------------------------------------------

1 permit read

N7K-1-pod1-S1#

Step 4a. Feature and Feature-groups. All users when they login are associated to a

particular role. It can be one of the default pre-configured roles or a user-made role. A role is

a set of rules that define what operations the user can perform on individual CLI commands,

features and feature-groups basis. Feature-groups are essentially groups of related features,

such as the “L3” feature group (defined by default). You can group features in feature-

groups and assign read/read-write permission to the whole group of features. To see the set

of features and the feature groups available to be defined as part of a role, issue the following

commands.

Not visible on your Pod

Not visible on your Pod

Super-user within the Pod

Only show commands for the vdc-operator


N7K-1-pod1-S1# show role feature

feature: aaa feature: access-list feature: arp feature: callhome feature: cdp <omitted output>

N7K-1-pod1-S1# sh role feature-group

feature group: L3

feature: router-bgp

feature: router-eigrp

feature: router-isis

feature: router-ospf

feature: router-rip

N7K-1-pod1-S1#

Step 4b. Create a new role. Creating a role is very easy. We will create a new role that is

allowed to issue all the “show” commands, to check basic connectivity using “ping” and to

configure just the Cisco Discovery Protocol: “cdp”. After creating the role we will define a new

user and associate the role to the newly created user.

N7K-1-pod1-S1# config t N7K-1-pod1-S1(config)# role name nxos N7K-1-pod1-S1(config-role)# ? description Add a description for the role

end Go to exec mode


interface Configure the interface policy for this role




rule Enter the rule number

vlan Configure the vlan policy for this role

vrf Configure the vrf policy for this role


N7K-1-pod1-S1(config-role)# rule 1 permit read N7K-1-pod1-S1(config-role)# rule 2 permit read-write feature cdp N7K-1-pod1-S1(config-role)# rule 3 permit command ping * N7K-1-pod1-S1(config-role)# rule 4 permit command conf t ; interface *

Note: The rules are applied in descending order.

Note: A role can also specify what resources in terms of Interfaces, VLANs and VRFs the user is entitled to access. Let’s exercise the interface restriction.

N7K-1-pod1-S1(config-role)# interface ? policy Configure the interface policy for this role

N7K-1-pod1-S1(config-role)# interface policy deny N7K-1-pod1-S1(config-role-interface)# permit interface ethernet 2/1 ------

Very granular access control up to the single CLI command. Ability to deny access to interfaces

Note the “ * ”… matches all

Space before and after “;”


Note: Let’s verify the role and create a user to who attach the role.

N7K-1-pod1-S1# show role name nxos role: test

description: new role

vlan policy: permit (default)

interface policy: deny permitted interface Ethernet2/1 vrf policy: permit (default)

-------------------------------------------------------------------


-------------------------------------------------------------------

4 permit command conf t ; interface * 3 permit command ping * 2 permit read-write feature cdp 1 permit read

Step4c. Attach the role. Create a new user and attach the role. After that, please log out and

login as the rbac user and test the RBAC configuration.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)#username rbac password rbac role nxos N7K-1-pod1-S1(config)#end N7K-1-pod1-S1# exit

Step4d. Using the puTTY, ssh as rbac into the management interface… “show running int

mgmt 0” will tell you the IP address of the your management interface

Password: rbac Cisco Data Center Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac <omitted output> N7K-1-pod1-S1# ? clear Reset functions

configure Enter configuration mode

debug Debugging functions

debug Debugging function

end Go to exec mode


ping Test network reachability

show Show running system information

Note: Most of the commands are missing. Let’s check the commands this user has been allowed to use.

N7K-1-pod1-S1# ping 192.168.100.250 vrf management

PING 192.168.100.250 (192.168.100.250): 56 data bytes







--- 192.168.100.250 ping statistics ---



N7K-1-pod1-S1# N7K-1-pod1-S1# debug ? cdp Configure CDP debugging

Note: Only the CDP debug is actually available.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# ? cdp CDP Configuration parameters end Exit configuration mode


interface Configure Interfaces

Note: Only the “cdp” commands are available.

N7K-1-pod1-S1(config)# cdp ?

advertise Highest CDP version supported on the switch

enable Enable/disable CDP on all interfaces

format Device ID format for CDP

holdtime CDP hold time advertised (in seconds)

timer CDP refresh time interval (in seconds) N7K-1-pod1-S1(config)#

Note: Let’s try to access an interface for which we don’t have the permission. -------

N7K-1-pod1-S1(config)# interface ethernet 2/2 % Interface permission denied N7K-1-pod1-S1(config)# interface ethernet 2/1 N7K-1-pod1-S1(config-if)# no shut N7K-1-pod1-S1(config-if)#

The step is completed you can now close the puTTY terminal you were just using.


Step 5 Configuration Rollback

NX-OS fully supports Configuration Rollback. This functionality allows you to revert to a

previous configuration state, effectively rolling back configuration changes. Let’s verify its

functionality within NX-OS.


- Create a checkpoint for the current configuration

- Modify the configuration for an interface

- Rollback the configuration

- Verify the interface configuration

N7K-1-pod1-S1# checkpoint ? <CR>

WORD Checkpoint name (Max Size 75)

file Create configuration rollback checkpoint to file

N7K-1-pod1-S1# checkpoint nxos

Note: Processing the Request... Please Wait

........Done

N7K-1-pod1-S1# N7K-1-pod1-S1# show checkpoint summary Checkpoint Summary

---------------------------------------------------------------------------

1) nxos:

Created by admin

Created at Wed, 01:04:48 31 Dec 2008

Size is 7,021 bytes

Let’s now modify the configuration of an interface.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# interface e2/15 N7K-1-pod1-S1(config-if)# ip address 1.1.1.1/24 N7K-1-pod1-S1(config-if)# no ip redirects N7K-1-pod1-S1(config-if)# ip proxy-arp N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# end

N7K-1-pod1-S1# sh running-config int e2/15 version 4.1(3)

interface Ethernet2/15

ip address 1.1.1.1/24

no ip redirects

ip proxy-arp

no shutdown

N7K-1-pod1-S1#

Odd Pods use interface e2/31

Finally the slash notation


Let’s check the difference between the current configuration and the checkpoint we created

before.

N7K-1-pod1-S1# show diff rollback-patch checkpoint nxos ?

checkpoint Use checkpoint as destination configuration

running-config Use running configuration as destination

startup-config Use startup configuration as destination N7K-1-pod1-S1# show diff rollback-patch checkpoint nxos running-config Processing the Request... Please Wait

!!

!


ip address 1.1.1.1/24

no ip redirects

ip proxy-arp

no shutdown

N7K-1-pod1-S1#

Let’s now rollback the configuration…

N7K-1-pod1-S1# rollback running-config checkpoint nxos atomic

Processing the Request... Please Wait

Generating the Rollbackpatch... Please Wait

Executing the patch... Please Wait

`conf t`

`interface Ethernet2/15`

`shutdown`

`no ip proxy-arp`

`ip redirects`

`no ip address 1.1.1.1/24`

N7K-1-pod1-S1# sh running-config int e2/15 version 4.1(3)


Step 6 Links up with Spanning Tree

It is time to bring up the interfaces and configure the Spanning Tree Protocol. Rapid

Spanning Tree Protocol (RSTP) is standardized in IEEE 802.1w. Cisco's implementation of

RSTP in both NX-OS and IOS provides a separate spanning tree instance for each active

VLAN, which permits greater flexibility of Layer 2 topologies in conjunction with IEEE 802.1Q

trunking. This implementation is also referred to as Rapid Per-VLAN Spanning Tree (Rapid-

PVST). Rapid-PVST is the default spanning tree mode for NX-OS, so it does not need to

be explicitly enabled.

During the rollback process the CLI commands are undone and shown to the user


Best practices dictate controlling the placement of the spanning tree root switch in the

network for each VLAN to ensure that it does not inadvertently end up by the election process

on a small switch in the access layer that creates a sub-optimal topology or may be more

prone to failure.

We will bring up few port-channels so we first need to enable the service for the LACP

protocol.

N7K-1-pod1-S1 (config)# feature lacp

Note: NX-OS is a fully modular operating system; most software modules don’t run unless the correspondent service is enabled. We refer to these features that need to be specifically enabled as “conditional services”. Once the service is enabled, the CLI becomes visible and the feature can be used and configured.

N7K-1-pod1-S1(config)# spanning-tree vlan 1-4 priority <..>

N7K-1-pod1-S1(config)# int po 10

N7K-1-pod1-S1(config-if)# switchport

N7K-1-pod1-S1(config-if)# switchport mode trunk

N7K-1-pod1-S1(config-if)# switchport trunk allowed vlan 1-4

N7K-1-pod1-S1(config-if)# spanning-tree port type network ----

N7K-1-pod1-S1(config-if)# spanning-tree guard loop

N7K-1-pod1-S1(config-if)# description link to the other Nexus7000

N7K-1-pod1-S1(config-if)# no shutdown

N7K-1-pod1-S1(config-if)# int e2/1-2

N7K-1-pod1-S1(config-if-range)# switchport

N7K-1-pod1-S1(config-if-range)# switchport mode trunk

N7K-1-pod1-S1(config-if-range)# switchport trunk allowed vlan 1-4

N7K-1-pod1-S1(config-if-range)# no shutdown

N7K-1-pod1-S1(config-if-range)# channel-group 10 mode active

Check the status of the port-channel… N7K-1-pod1-S1(config-if-range)# show port-channel summary

Flags: D - Down P - Up in port-channel (members)

I - Individual H - Hot-standby (LACP only)

s - Suspended r - Module-removed

S - Switched R - Routed

U - Up (port-channel)

---------------------------------------------------------------------------

Group Port- Type Protocol Member Ports

Channel

---------------------------------------------------------------------------

10 Po10(SU) Eth LACP Eth2/1(P) Eth2/2(P)

Bring up the interfaces facing the Access Layer…

N7K-1-pod1-S1(config-if-range)# int e2/9-10

4096 for N7K1 (Student 1) 8192 for N7K2 (Student 2)

LACP is a conditional service


N7K-1-pod1-S1(config-if-range)# switchport

N7K-1-pod1-S1(config-if-range)# switchport mode trunk

N7K-1-pod1-S1(config-if-range)# switchport trunk allowed vlan 1-4

N7K-1-pod1-S1(config-if-range)# no shutdown

Check the spanning-tree from both the Nexus 7000 and the Nexus 5000 (for the latter use the

puTTY link on the Desktop).

N7K-1-pod1-S1(config-if-range)# show spanning-tree vlan 3

VLAN0003

Spanning tree enabled protocol rstp

Root ID Priority 4099

Address 0022.5579.d2c2

This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 4099 (priority 4096 sys-id-ext 3)


Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- -----------------------------

Po10 Desg FWD 1 128.4105 Network P2p

Eth2/9 Desg FWD 2 128.265 P2p

Eth2/10 Desg FWD 2 128.266 P2p

N5K-1# show spanning-tree vlan 3 VLAN0003




Cost 2

Port 129 (Ethernet1/1)



Address 000d.eca4.0081



---------------- ---- --- --------- -------- -----------------------------

Eth1/1 Root FWD 2 128.129 P2p

Eth1/2 Altn BLK 2 128.130 P2p

Step 7 HSRP

To provide redundancy for the IP default gateway services, several protocols exist, which are

commonly referred to together as First Hop Redundancy Protocols (FHRPs). Cisco NX-OS

supports implementations of multiple FHRPs: Hot Standby Router Protocol (HSRP), Gateway

Load Balancing Protocol (GLBP), and Virtual Router Redundancy Protocol (VRRP).

You will configure HSRP in this step.

The link between the N5K and the N7K-2 is blocked as expected


Let’s create an SVI for VLAN 2 and VLAN 3 and configure HSRP:

N7K-1-pod1-S1(config)# feature interface-vlan N7K-1-pod1-S1(config)# feature hsrp

Note: Both the SVI service and the service for the HSRP protocol are “conditional”. Their code does not run unless the feature is explicitly enabled with the “feature” command.

N7K-1-pod1-S1(config)# int vlan 2 N7K-1-pod1-S1(config-if)# ip address 192.168.202.<Student #>/24 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# hsrp 1 N7K-1-pod1-S1(config-if-hsrp)# preempt delay minimum 180 N7K-1-pod1-S1(config-if-hsrp)# priority <...> N7K-1-pod1-S1(config-if-hsrp)# timers 1 3 N7K-1-pod1-S1(config-if-hsrp)# ip 192.168.202.3

N7K-1-pod1-S1(config-if-hsrp)# int vlan 3 N7K-1-pod1-S1(config-if)# ip address 192.168.203.<Student #>/24 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# hsrp 1 N7K-1-pod1-S1(config-if-hsrp)# preempt delay minimum 180 N7K-1-pod1-S1(config-if-hsrp)# priority <...> N7K-1-pod1-S1(config-if-hsrp)# timers 1 3 N7K-1-pod1-S1(config-if-hsrp)# ip 192.168.203.3

N7K-1-pod1-S1# show hsrp brief P indicates configured to preempt.

|

Interface Grp Prio P State Active addr Standby addr Group addr

Vlan2 1 40 P Active local 192.168.202.2 192.168.202.3

Vlan3 1 40 P Active local 192.168.203.2 192.168.203.3

40 for N7K1 (Student 1) 20 for N7K2 (Student

40 for N7K1 (Student 1) 20 for N7K2 (Student 2)


Step 8 Moving the Topology from STP-based to vPC-based

The “virtual Port Channel” (vPC) functionality provides the following benefits:

• Allows a single device to use a port channel across two upstream devices

• Eliminates Spanning Tree Protocol (STP) blocked ports

• Provides a loop-free topology

• Uses all available uplink bandwidth

• Provides fast convergence if either the link or a device fails

• Provides link-level resiliency

• Assures high availability

The topology will change as follow:

The terminology used for vPCs is as follows:

• vPC — The combined port channel between the vPC peer devices and the

downstream device.

• vPC peer device — One of a pair of devices that are connected with the special port

channel known as the vPC peer link.

• vPC peer link — The link used to synchronize states between the vPC peer devices.

Both ends must be on 10-Gigabit Ethernet interfaces.

• vPC domain — This domain is formed by the two vPC peer link devices. It is also a

configuration mode for configuring some of the vPC peer link parameters.

• vPC fault-tolerant link — The fault-tolerant link is a Layer3 link between the vPC

peer devices used to ensure that both devices are up. The fault-tolerant link sends

configurable, periodic keepalive messages between devices connected by the vPC

peer link on an out-of-band link.

• vPC member port — Interfaces that belong to the vPCs.

N7K-1 N7K-2

N5K

Current STP Topology

N5K

N7K-2 N7K-1

vPC Topology

vPC peer link vPC Member

vPC Member

vPC fault tolerant link


vPC will be available in the NX-OS 4.1(3) software in the Q1CY09.

During this step you will:

- Enable the vPC

- Create the vPC domain

- Configure the peer-link port channel, and place it in vpc peer-link mode

- Configure the access layer facing port channels, and place them in vPC mode

N7K-1-pod1-S1#conf t

N7K-1-pod1-S1(config)# feature vpc

Next we’ll enable the vPC domain. This domain ID is used to differentiate multiple vPC tiers,

allowing for an L2 unique Link Aggregation ID for LACP based configuration. We will also

configure the “role” so that the primary vPC device is the same device which is also the STP

root and the HSRP primary device. This is the recommended configuration

N7K-1-pod1-S1(config)# vpc domain 1 N7K-1-pod1-S1(config-vpc-domain)# role priority <...>

First thing to setup is the fault-tolerant link connection. For the fault-tolerant link we

recommend a separate port, preferably 1GigE, between the vPC peer devices (it does NOT

need to be a direct link). This port should belong to a separate VRF.

Another alternative is to use the Out-of-Band management network through the Supervisor’s

management interface and this is what we’ll do in this lab.

N7K-1-pod1-S1(config-vpc-domain)# peer-keepalive dest 192.168.100.<...>

source 192.168.100.<...> vrf management

Let’s check the status of the fault-tolerant link (peer-keepalive).

N7K-1-pod1-S1(config-vpc-domain)# show vpc peer-keepalive

vPC keep-alive status : peer is alive

--Destination : 192.168.100.21

--Send status : Success

--Receive status : Success

--Last update from peer : (0 ) seconds, (40 ) msec

N7K-9-pod9-S1(config-vpc-domain)#

Enter your partner’s mgmt0 IP address

1000 for Student 1 2000 for Student 2

Enter your mgmt0 IP address


Now that the base vPC domain is configured, we can configure the peer-link, and then we

can validate that the base vPC infrastructure is running (assuming your Partner has done

the same configuration steps on the other Nexus7000 in your Pod).

N7K-1-pod1-S1(config-int)# int port-channel 10 N7K-1-pod1-S1(config-int)# vpc peer-link

N7K-1-pod1-S1(config-int)# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1

Peer status : peer adjacency formed ok


Configuration consistency status: success

vPC role : primary

vPC Peer-link status

---------------------------------------------------------------------

id Port Status Active vlans

-- ---- ------ --------------------------------------------------

1 Po10 up 1-4

The STP status hasn’t changed on the Nexus 5000.

N5K# show spanning-tree vlan 3

VLAN0003



Address 001b.54c2.b1c2

Cost 2

Port 129 (Ethernet1/1)






---------------- ---- --- --------- -------- ----------------------------

Eth1/1 Root FWD 2 128.129 P2p

Eth1/2 Altn BLK 2 128.130 P2p

Now that the peer-link is running and the vPC is up, we can add in the access facing vPC

links.

N7K-1-pod1-S1(config)# int po 20 N7K-1-pod1-S1(config-int)# switchport N7K-1-pod1-S1(config-int)# switchport mode trunk N7K-1-pod1-S1(config-int)# switchport trunk allowed vlan 1-4 N7K-1-pod1-S1(config-int)# no sh


N7K-1-pod1-S1(config-int)# vpc 20

Let’s now add the port facing the Access Layer (Nexus 5000) to the port-channel.

N7K-1-pod1-S1(config-int)# int e2/9 N7K-1-pod1-S1(config-int)# channel-group 20 mode active

Let’s check the vPC status.

N7K-1-pod1-S1(config-if)# show vpc brief

Legend:


vPC domain id : 1




vPC role : primary


---------------------------------------------------------------------


-- ---- ------ --------------------------------------------------

1 Po10 up 1-4

vPC status

----------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

-- ---- ------ ----------- -------------------------- ------------

20 Po20 down* failed Consistency Check Not - Performed

The vPC status is “down” because we haven’t configured the port-channel on the Nexus5000

yet; in fact the port is in “individual” state from a LACP prospective.

N7K-1-pod1-S1(config-if)# sh port-channel summary






-------------------------------------------------------------------------


Channel

--------------------------------------------------------------------------

10 Po10(SU) Eth LACP Eth2/1(P) Eth2/2(D)

20 Po20(SD) Eth LACP Eth2/9(I)

If your teammate has reached this point as well, one of you can go on the N5K and configure

the port-channel.


N5K(config-if)# int e1/1-2

N5K(config-if-range)# channel-group 20 mode active Let’s check the STP and the port-channel status.

N5K(config-if-range)# show spanning-tree vlan 3

VLAN0003




Cost 1

Port 4115 (port-channel20)






---------------- ---- --- --------- -------- ----------------------------

Po20 Root FWD 1 128.4115 P2p

N5K(config-if-range)# show port-channel summary

Flags: D - down U - up in port-channel

I - Individual S - suspended

H - Hot-standby (LACP only)

R - Module-removed

--------------------------------------------------------------------------


Channel

--------------------------------------------------------------------------

20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U)

The Nexus 5000 has now a port-channel connected to two different upstream devices.

Let’s check the status of the vPC and the STP on the Nexus7000.

N7K-1-pod1-S1(config-if)# sh vpc brief

Legend:


vPC domain id : 1




vPC role : primary


---------------------------------------------------------------------


-- ---- ------ --------------------------------------------------

1 Po10 up 1-4

vPC status


----------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

-- ---- ------ ----------- -------------------------- ------------

20 Po20 up success success 1-4

N7K-1-pod1-S1(config-if)# show spanning-tree vlan 3

VLAN0003




This bridge is the root






---------------- ---- --- --------- -------- ----------------------------

Po10 Desg FWD 1 128.4105 (vPC peer-link) Network P2p Po20 Desg FWD 1 128.4115 (vPC) P2p Eth2/10 Desg FWD 2 128.266 P2p

The vPC topology is now up and running!

Step 9 vPC Failure Scenario

One of the advantages of the vPC approach to loop management is that failure recovery on a

link or of an entire switch relies on port-channel failover rather than on STP re-learning the

entire network. With port-channel failover, recovery is often sub-second. This alone is a key

reason why vPC provides an efficient scaling mechanism relative to STP managed Layer 2

topologies. In this step we will bring down the vPC peer-link. In the unlikely case that both

ports and line cards in the peer-link fail (being that two ports on two different line cards are

the recommended minimum for the peer-link) the vPC software will look to the fault-tolerant

link (the keep-alive link) to determine if the failure is a link level failure (perhaps a UDLD

failure of some nature), or if in fact the remote peer has failed entirely.

In the case that the remote peer is still alive (peer-keepalive messages are still being

received), to avoid loops the vPC secondary switch will disable its vPC member ports

and any Layer 3 interfaces attached to a vPC associated VLAN.


We will bring down the peer-link interfaces on the vPC primary device and observe what

happens on the vPC secondary and on the Nexus 5000 devices.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# int e2/1-2 N7K-1-pod1-S1(config-if-range)# shutdown

On the Nexus 5000 we can see how the port-channel port got suspended.





R - Module-removed

--------------------------------------------------------------------------


Channel

--------------------------------------------------------------------------

20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(D) 30 Po30(D) Eth NONE --

While on the vPC secondary you should see the following:

%VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on

secondary

N7K-2-pod1-S2(config-if)# show int vlan 2

Vlan2 is down, line protocol is down Hardware is EtherSVI, address is 001b.54c2.af42

Internet Address is 192.168.202.2/24

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

<omitted output>

N7K-2-pod1-S2(config-if)# show port-channel summary






-------------------------------------------------------------------------


Channel

-------------------------------------------------------------------------

10 Po10(SD) Eth LACP Eth2/1(D) Eth2/2(D)

20 Po20(SD) Eth LACP Eth2/9(D)


Now we can bring the peer-link interfaces on the vPC primary back up and check again the

Nexus5000 first.

N7K-1-pod1-S1(config)# int e2/1-2 N7K-1-pod1-S1(config-if-range)# no shutdown

After few seconds you should see, the link back up:





R - Module-removed

-------------------------------------------------------------------------


Channel

-------------------------------------------------------------------------

20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U)

Also on the vPC secondary the SVIs are back up:

N7K-2-pod1-S2(config-if)# show int vlan 2 Vlan2 is up, line protocol is up

<omitted output>

Step 10 OSPF Configuration

OSPF is fully implemented in NX-OS as part of the “Enterprise” License (however you can

use the feature leveraging the grace-period mode for 120 days). In this step we will configure

OSPFv2 and we will see how the configuration is interface centric vs. the network centric IOS

based OSPF configuration.

These are the steps for this exercise:

- Turn the OSPFv2 service on

- Configure the Loopback interfaces

- Instantiate an OSPF process

- Verify OSPF configuration by issuing few show command

N7K-1-pod1-S1(config)# interface loopback0 N7K-1-pod1-S1(config-if)# ip address 10.1.255.<Student #>/24 N7K-1-pod1-S1(config-if)# feature ospf N7K-1-pod1-S1(config)# router ospf 1 N7K-1-pod1-S1(config-router)# log-adjacency-changes


N7K-1-pod1-S1(config-router)# auto-cost reference-bandwidth 1000000

Note: As you may have noticed the “network x.x.x.x area y” configuration lines are not present. This is a big different from IOS. OSPF, as well as other IGP protocols, are interface centric, as we will see with the next few commands.

Let’s now configure the interfaces.

N7K-1-pod1-S1(config)# int e1/13 N7K-1-pod1-S1(config-if)# description link to the Cat6k N7K-1-pod1-S1(config-if)# ip address 192.168.<Student #>.1/30 N7K-1-pod1-S1(config-if)# ip ospf hello-interval 2 N7K-1-pod1-S1(config-if)# ip ospf dead-interval 6 N7K-1-pod1-S1(config-if)# ip ospf network point-to-point N7K-1-pod1-S1(config-if)# ip router ospf 1 area 0 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)#

Note: In the NX-OS the OSPF configuration is interface centric. The membership to an OSPF area is specified at the interface configuration level. This approach is more intuitive and manageable.

Now we can check the OSPF configuration we have been working on.

N7K-1-pod1-S1# sh running-config ?

<CR>

> Redirect it to a file

aaa Display aaa configuration

all Current operating configuration with defaults

am Display am information

arp Display arp information

bgp Display bgp information

<snip>

l3vm Display l3vm information

license Display licensing configuration

msdp Display msdp information

netflow Show NetFlow configuration

ospf Display ospf information ospfv3 Display ospfv3 information

pim Display pim information

pim6 Display pim6 information

<snip> N7K-1-pod1-S1# sh running-config ospf

version 4.1(3)

feature ospf

router ospf 1

auto-cost reference-bandwidth 1000000


Odd Pods interface e1/25


ip ospf dead-interval 6

ip ospf hello-interval 2

ip ospf network point-to-point

ip router ospf 1 area 0.0.0.0

Let’s check now the complete OSPF configuration with its default values.

N7K-1-pod1-S1# sh running-config ospf all

version 4.1(3)

feature ospf

snmp-server enable traps ospf rate-limit 10 7

snmp-server enable traps ospf 1 rate-limit 10 7

router ospf 1

graceful-restart

graceful-restart grace-period 60

timers lsa-arrival 1000

distance 110

maximum-paths 8

auto-cost reference-bandwidth 1000000

ip ospf event-history size small

ip ospf event-history cli size small

ip ospf event-history redistribution size small

ip ospf event-history spf size small

ip ospf event-history lsa size small

ip ospf event-history flooding size small

ip ospf event-history ha size small

ip ospf event-history event size small

ip ospf event-history adjacency size small


ip ospf dead-interval 6

ip ospf hello-interval 2

ip ospf network point-to-point

ip ospf priority 1

ip ospf retransmit-interval 5

ip ospf transmit-delay 1

ip router ospf 1 area 0.0.0.0

N7K-1-pod1-S1# sh ip ospf neighbors

OSPF Process ID 1 VRF default

Total number of neighbors: 1

Neighbor ID Pri State Up Time Address Interface

192.168.100.5 1 FULL/ - 00:08:58 192.168.201.2 Eth1/13

N7K-1-pod1-S1#


Step 11 State-full Process Restart

NX-OS is a modern operating system. NX-OS continuously checks the health of each

software module making sure that if a process crashes or hangs the right action is taken to

allow service continuity and availability. NX-OS has been designed around the concept of

zero service destruction.

All Layer2 protocols (STP, CDP, LACP etc) and OSPF support the State-full Process Restart

leveraging our PSS (Persistent Storage Service) architecture.

With this exercise we will see how the system recovers from an OSPF crash in a seamless

way. You will see how the connected Cat6K won’t even realize that the process crashed and

restarted.


- Display the OSPF process ID

- Kill the OSPF process

- Verify that the OSPF process has been restarted with a new process ID

- Check the Cat6K screen

Using the puTTY icon on the Desktop connect to the 6K so that you have both terminals open

one on the Nexus 7000 and one on the Catalyst 6500.

Note: Only one student can log into the Catalyst 6500 at a given time.

Just to show that the OSPF adjacency goes down as expected, shutdown the link on the N7K

N7K-1-pod1-S1(config)# int e1/13 N7K-1-pod1-S1(config-if)# shutdown

As you can see on the 6K terminal the link and the OSPF adjacency went down.

Now bring the interface back up on the Nexus 7000.

N7K-1-pod1-S1(config-if)# no shutdown

The interface is now up and the OSPF adjacency is back up. Now let’s kill OSPF.

N7K-1-pod1-S1# show process | inc ospf

1959 S 778f727b 1 - ospf - NR - 0 - ospfv3

- NR - 0 - ospf

- NR - 0 - ospfv3

- NR - 0 - ospf

Notice the PID on the left (you will need it in the killing process) and the number of restarts

(bold, blu and underlined fonts).


The “x” in the following CLI commands is the number of your Pod, i.e “1” for Pod1, “2” for Pod2.

N7K-1-pod1-S1# N7K-1-pod1-S1# copy bootflash:proc.res bootflash:proc<x>.res N7K-1-pod1-S1# N7K-1-pod1-S1# N7K-1-pod1-S1# load bootflash:procx.res

load_isanimg: entry

load_isanimg: uri_info:0x809ba90

load_isanimg: type:0x8

Loading plugin version 4.0(2)

###############################################################

Warning: debug-plugin is for engineering internal use only!

For security reason, plugin image has been deleted.

###############################################################

Successfully loaded service restart debug-plugin!!!

Commands Available: help kill <pid> exit

Enter Commands:

kill <ospf pid> killing …

2008 May 12 21:22:35 N7K-C1-1-pod1 %SYSMGR-2-SERVICE_CRASHED: Service

"__inst_001__ospf" (PID 19700) hasn't caught signal 9 (no core).

exit N7K-1-pod1-S1# sh process | inc ospf 16066 S 778f727b 2 - ospf - NR - 0 - ospfv3

- NR - 0 - ospf

- NR - 0 - ospfv3

- NR - 0 - ospf

- NR - 0 - ospfv3

- NR - 0 - ospf

- NR - 0 - ospfv3

Notice how the OSPF process has now a new process ID and how, looking at the Cat6K

terminal, the neighbor didn’t even realized that our OSPF process was killed and restarted.

Step 12 Wireshark

Wireshark used to be known as Ethereal®. Wireshark® is the world's foremost network

protocol analyzer and is the de facto (and often de jure) standard across many industries and

educational institutions.

NX-OS offers an integrated packet capture tool for packets directed to the control plane.

This packet analyzer is built on top of Wireshark and it is called Ethanalyzer.


The primary function of this protocol analyzer is to be able to capture and analyze control

packets, but it can also be leveraged to look at data traffic in its “acl-log” mode. When

analyzing data traffic, such traffic will reach the Supervisor after being rate limited in

hardware.

During this step we will capture regular control traffic, and then we will set up an ACL just to

show the procedure for capturing data-plane traffic, we won’t actually capture data traffic

during this lab.

Ethanalyzer can be used only from the default-VDC.

To start access the default-VDC by opening the “Device Access” folder located in the “My

Documents” folder and double click on the “N7K# default” ssh connection, where # is 1 for

Student1 and 2 for Student 2.

N7K-1# ethanalyzer local interface ? inband Inband/Outband interface

mgmt Management interface

N7K-1# ethanalyzer local interface inband ? <CR>

> Redirect it to a file

>> Redirect it to a file in append mode

brief Display only protocol summary

capture-filter Filter on ethanalyzer capture

decode-internal Include internal system header decoding

display-filter Display filter on frames captured

limit-captured-frames Maximum number of frames to be captured (default is

100)

limit-frame-size Capture only a subset of a frame

write Filename to save capture to

| Pipe command output to filter

The “brief” option will show one-liner info. N7K-1# ethanalyzer local interface inband brief capture-filter "udp" limit-captured-frames 10 Capturing on eth0

10 packets captured

2009-01-08 07:09:45.84 192.168.203.2 -> 224.0.0.2 HSRP Hello (state Standby)


2009-01-08 07:09:45.89 192.168.202.1 -> 224.0.0.2 HSRP Hello (state Active)







2009-01-08 07:09:47.90 192.168.203.1 -> 224.0.0.2 HSRP Hello (state Active) N7K-1#


To see the entire packet remove the “brief” keyword. N7K-1# ethanalyzer local interface inband capture-filter "udp" limit-captured-frames 1 | no-more Capturing on eth0

1 packets captured

Frame 1 (62 bytes on wire, 62 bytes captured)

Arrival Time: Nov 19, 2008 01:06:08.834050000

[Time delta from previous captured frame: 1227056768.834050000 seconds]

[Time delta from previous displayed frame: 1227056768.834050000 seconds]

[Time since reference or first frame: 1227056768.834050000 seconds]

Frame Number: 1

Frame Length: 62 bytes

Capture Length: 62 bytes

[Frame is marked: False]

[Protocols in frame: eth:ip:udp:hsrp]

Ethernet II, Src: 00:22:55:79:be:42 (00:22:55:79:be:42), Dst:

01:00:5e:00:00:02 (01:00:5e:0

0:00:02)

Destination: 01:00:5e:00:00:02 (01:00:5e:00:00:02)

Address: 01:00:5e:00:00:02 (01:00:5e:00:00:02)

.... ...1 .... .... .... .... = IG bit: Group address

(multicast/broadcast)

.... ..0. .... .... .... .... = LG bit: Globally unique address

(factory default)

Source: 00:22:55:79:be:42 (00:22:55:79:be:42)

Address: 00:22:55:79:be:42 (00:22:55:79:be:42)

.... ...0 .... .... .... .... = IG bit: Individual address (unicast)

.... ..0. .... .... .... .... = LG bit: Globally unique address

(factory default)

Type: IP (0x0800)

<IP Header Omitted>

Cisco Hot Standby Router Protocol

Version: 0

Op Code: Hello (0)

State: Standby (8)

Hellotime: Non-Default (1)

Holdtime: Non-Default (3)

Priority: 20

Group: 1

Reserved: 0

Authentication Data: Default (cisco)

Virtual IP Address: 192.168.202.3 (192.168.202.3) N7K-1# Let’s capture and store the file on the bootflash, so we can copy it over and look at it on our Windows machine. N7K-1# ethanalyzer local interface inband limit-captured-frames 30 write bootflash:capture Capturing on eth0 30 N7K-1# copy bootflash:capture sftp://Student#@192.168.100.250/HOME/Desktop vrf management


Connecting to 192.168.100.250... [email protected]'s password: < Your Remote Desktop Password > sftp> put /bootflash/capture /HOME/Desktop Uploading /bootflash/capture to /HOME/Desktop/capture /bootflash/capture 100% 822 0.8KB/s 00:00 sftp> exit Now the capture is on your Desktop, launch Wireshark using the icon and load the file.

The following portion of the Wireshark step is optional... if you are running out of time jump to Step 13 “Virtual Device Context”!!!

Ethanalyzer can capture data traffic as well, so that network administrators can have an

embedded and easy to use tool for on the fly capture. Ethanalyzer gives network

administrators more visibility into applications behavior with few simple steps:

1. Identify the application characteristics

2. Create ad hoc ACL to match (and permit) the application flow between two servers

3. Use the “log” keyword to punt copies of matching packets to supervisor CPU

4. The original traffic gets forwarded with no impact

5. The copies sent to CPU are subjected to hardware rate limiter (100 pps by default)

6. These copies can be captured by our Ethanalyzer (Wireshark a.k.a Ethereal)

7. Ethanalyzer can output to screen or dump to file on flash which can be copied to PC for

GUI analysis

Let’s suppose to have an application using TCP port 5600 between the server 1.1.1.24 and

the client 1.1.1.16.

Let’s now create the ad hoc ACL and let’s apply it to the interface. We won’t actually

capture traffic in this example and you do NOT need to run this part of the config:

N7K-1(config)# ip access-list etha N7K-1(config-acl)# statistics per-entry N7K-1(config-acl)# permit tcp host 1.1.1.24 host 1.1.1.16 eq 5600 log N7K-1(config-acl)# show ip access-lists etha IP access list etha

statistics per-entry

10 permit tcp 1.1.1.24/32 1.1.1.16/32 eq 5600 log

N7K-1(config)# int e1/1 N7K-1(config-if)# ip access-group etha in N7K-2-pod1(config-if)# end

We can now capture selectively these packets and save the capture to the usb1 (so we could

use our laptop with the nice wireshark graphical interface):

N7K-1# ethanalyzer loc interf inband capture-filter "tcp port 5600" write bootflash:cap_acl_log


Step 13 Virtual Device Contexts

NX-OS introduces support for the Virtual Device Contexts (VDCs), which allow the

Nexus7000 to be virtualized at the device level. Each configured VDC presents itself as a

unique device to connected users within the framework of that physical switch. The VDC runs

as a separate logical entity within the switch, maintaining its own unique set of running

software processes, having its own configuration, and being managed by a separate

administrator.

This lab has used the VDC concept to allow multiple PODs to work on a single switch.


- Delete the VDC you were working on.

- Create a new VDC and allocate resources to it.

- “switchto” the newly created VDC and perform the initial configuration script

For this last step keep using the putty terminal you were using for the previous step.

You need to be in the “default-VDC”

N7K-1# show vdc

vdc_id vdc_name state mac

------ -------- ----- ----------

1 N7K-1 active 00:22:55:79:c4:41

2 pod1-S1 active 00:22:55:79:c4:42

3 pod2-S1 active 00:22:55:79:c4:43

You will now delete the Pod (that is VDC) you were working on.

N7K-1# conf t

N7K-1(config)# no vdc pod< y >-S< x > where “y” is your Pod number and ”x”

is “1” for Student1, “2” for Student2

Deleting this vdc will remove its config. Continue deleting this vdc? [no] yes

Note: Deleting VDC, one moment please ...

N7K-1(config)#

2009 Jan 8 07:43:34 N7K-1 %VDC_MGR-2-VDC_OFFLINE: vdc 2 is now offline

Now create a new VDC and allocate the following interfaces

POD# Interfaces

POD{ODD} e1/1-24, e2/1-16

POD{EVEN} e1/25-48, e2/17-32


N7K-1(config)# vdc pod< y >-S< x > where “y” is your Pod number and ”x” is

“1” for Student1, “2” for Student2

Note: Creating VDC, one moment please ...

2009 Jan 8 07:44:17 N7K-1 %VDC_MGR-2-VDC_LIC_WARN: Service using grace period will be shutdown in 30 day(s)

2009 Jan 8 07:44:34 N7K-9 %VDC_MGR-2-VDC_ONLINE: vdc 2 has come online

N7K-1(config-vdc)# ?

allocate Assign interfaces to vdc

end Go to exec mode


ha-policy Change HA policy for this VDC

limit-resource Resource configuration




template Change the template for this vdc


N7K-1(config-vdc)# allocate interface ethernet <check the table above>

Moving ports will cause all config associated to them in source vdc to be removed. Are you sure you want to move the ports? [yes] yes

Should a control plane failure occur, the administrator has a set of options that can be

configured on a per-VDC basis defining what action will be taken regarding that VDC.

There are three actions that can be configured: restart, bringdown, and reset.

The restart option will delete the VDC and then re-create it with the running configuration.

This configured action will occur regardless of whether there are dual supervisors or a single

supervisor present in the chassis.

The bringdown option will simply delete the VDC.

The reset option will issue a reset for the active supervisor when there is only a single

supervisor in the chassis. If dual supervisors are present, the reset option will force a

supervisor switchover.

The default VDC always has a high-availability option of reset assigned to it. Subsequent

VDCs created will have a default value of bringdown assigned to them. This value can be

changed under configuration control.

N7K-1(config-vdc)# ha-policy single-sup restart dual-sup restart N7K-1(config-vdc)# limit-resource ? m4route-mem Set ipv4 route memory limits

m6route-mem Set ipv6 route memory limits

monitor-session Monitor local session


port-channel Set port-channel limits

u4route-mem Set ipv4 route memory limits

u6route-mem Set ipv6 route memory limits

vlan Set VLAN limits

vrf Set vrf resource limits

N7K-1(config-vdc)# limit-resource vrf minimum 16 maximum 20

N7K-1(config-vdc)# show vdc pod< y >-S< x > detail vdc id: 2

vdc name: pod1-S1

vdc state: active

vdc mac address: 00:1b:54:c2:29:42

vdc ha policy: RESTART

vdc dual-sup ha policy: RESTART

vdc create time: Thu Aug 7 10:15:46 2008

vdc restart count: 0

N7K-1(config-vdc)# show vdc pod< y >-S< x > membership

vdc_id: 2 vdc_name: student1 interfaces:

Ethernet1/1 Ethernet1/2 Ethernet1/3





Ethernet1/16

N7K-1(config-vdc)# exit

It’s now time to “switchto” the newly created VDC. You will go through the initial script

configuration, which is similar to the one you would go through on a first time-booted

Nexus7000.

N7K-1# switchto vdc pod< y >-S< x >

---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no): no

Enter the password for "admin": Test Confirm the password for "admin": Test

---- Basic System Configuration Dialog VDC: 2 ----

This setup utility will guide you through the basic configuration of

the system. Setup configures only enough connectivity for management

of the system.

Please register Cisco Nexus7000 Family devices promptly with your

supplier. Failure to register may affect response times for initial

service calls. DC3 devices must be registered to receive entitled

support services.

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime


to skip the remaining dialogs.

Would you like to enter the basic configuration dialog (yes/no): yes

Create another login account (yes/no) [n]: Configure read-only SNMP community string (yes/no) [n]: Configure read-write SNMP community string (yes/no) [n]: Enter the switch name : pod< y >-S< x >

Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: Mgmt0 IPv4 address : 192.168.100.<...> Mgmt0 IPv4 netmask : 255.255.255.0 Configure the default gateway? (yes/no) [y]: IPv4 address of the default gateway : 192.168.100.1 Configure advanced IP options? (yes/no) [n]: Enable the telnet service? (yes/no) [y]: Enable the ssh service? (yes/no) [n]: Configure the ntp server? (yes/no) [n]: Configure default interface layer (L3/L2) [L3]: Configure default switchport interface state (shut/noshut) [shut]: Configure default switchport trunk mode (on/off/auto) [on]:

The following configuration will be applied:

switchname pod1nxos

interface mgmt0

ip address 192.168.100.20 255.255.255.0

no shutdown

vrf context management

ip route 0.0.0.0/0 192.168.100.1

exit

telnet server enable

no ssh server enable

no system default switchport

system default switchport shutdown

Would you like to edit the configuration? (yes/no) [n]: Use this configuration and save it? (yes/no) [y]: y

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

20 for Odd Pods – Student1 22 for Odd Pods – Student2 21 for Even Pods – Student1 23 for Even Pods – Student2


Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and http://www.opensource.org/licenses/lgpl-2.1.php

N7K-1-pod1# N7K-1<x>nxos# sh running-config version 4.0(3)

username admin password 5 $1$XpvaHAKS$OhTkzciBdKkE4FOM0epik/ role vdc-admin

telnet server enable

ssh key rsa 1024 force

no ssh server enable

snmp-server user admin vdc-admin auth md5 0x77306315bd719b5d121cdeb6f0a9d697

priv 0x77306315bd719b5d121cdeb6f0a9d697 localizedkey

vrf context management

ip route 0.0.0.0/0 192.168.100.1

switchname pod1nxos

<omitting interface config>

interface mgmt0

ip address 192.168.100.20/26

N7K-1-pod1# ping 192.168.100.250 vrf management PING 192.168.100.250 (192.168.100.250): 56 data bytes






--- 192.168.100.250 ping statistics ---



Congratulations!!! The lab is now complete!

Please LOG OFF from the Windows Machines (Click “Start” on the

bottom left corner and “Log Off” right above), do NOT just close the

Windows Remote Desktop window.


Recommended Reading

Cisco Nexus 7000 Series Switches:

www.cisco.com/en/US/products/ps9402/index.html

Cisco NX-OS Feature Navigator:

www.cisco.com/go/nxosnav

Cisco NX-OS Home Page:

www.cisco.com/go/nxos

Complete Your Online Session Evaluation

Cisco values your input. Give us your feedback! We read and carefully consider your scores and comments, and incorporate them into the content program year after year

Go to the Internet stations located throughout the Convention Center to complete your session evaluations

Thank you!

LABDCT 2001 (Guide) Nexus.7000

Documents

Transcript of LABDCT 2001 (Guide) Nexus.7000