Internet QoS Workshop

1 Greetings and welcome to Cisco training on Internet QoS.

1Cisco Systems Confidential V2.00

Internet QoS Workshop

Charles Mujie

PME - ISP Business Unit

[email protected]


Workshop Agenda

What is Internet QoS?

Internet QoS Building Blocks

Configuration Guidelines

Case Study

Demo

3 There are main two QoS initiavtives happening in Cisco. One is on the11.1CC train and the other on 11.2 train. Depending on the feature orfeatures you are looking for you select the appropriate train.


Cisco QoS Initiative

QoSQoS

11.1CC CAR

WRED

WFQ

NetFlow Services

BGP Policy Propagation

11.2 Queuing

FIFO, Priority, Custom, WFQ

Traffic Shaping

RED

RSVP

NetFlow Switching

4 Initially 11.1CC will be targeted for SP only. The features in 11.1CC willeventually be integrated to 12.0 at which time Enterprise customer will haveaccess to it.

Anyone wanting to use 11.1CC MUST be trained for CEF and QoS features.



11.1CCTargeted to SP who wants to provide differentiated levelsof services to their customers

Speed

Performance

11.2Targeted to Enterprise customers

This workshop will focus on features inThis workshop will focus on features in11.1CC only11.1CC only



11.1 11.2 11.3

11.1CA11.1CB (ISP8)11.1CC (FIB)11.1CD (ISP8+L3)11.1CE (FIB+L3)

11.1CC

12.0

WFQ RED WRED RSVP NetFlow Switching

CEF CAR/DCAR DWFQ DWRED BGP Policy Propagation NetFlow Services

CYH298

11.1R IP ATM CoS (Ph I)

12.0T IP ATM CoS (Ph II)

11.1CC will merge with 11.3 in 12.0. Extensive integration work is underway.


What is Internet QoS?

IP - Best effort

Internet QoS is a set of features thatallows a user to build an IP networkcapable of providing

Timely delivery of packets

Bandwidth guarantees

Improve loss characteristics


Layer 3 vs Layer 2 QoS Layer 3 == IPIP

Different approaches in providing QoSFrame Relay - Committed Information Rate (CIR)

ATM - Peak Cell Rate (PCR)

Sustainable Cell Rate (SCR)

Maximum Burst Size (MBS)

In delivering Layer 3 QoS a set offeatures has been developed

CAR, WFQ & WRED


Layer 3 vs Layer 2 QoS

In some cases layer 2 and layer 3 QoShas to work together to deliver theappropriate level of QoS to theapplication or user

This is especially true in the case whereFrame Relay or ATM is used in thenetwork

9 Please note that the emphasis on this workshop is Internet QoS and NOTFrame Relay or ATM QoS.

We wont be discussing interworking between IP QoS and Frame Relay orATM QoS as these features are either on a separate IOS train or underdevelopment.

The features that we will cover in this workshop are those of 11.1CC.


A Point to Note

The focus of this workshop isThe focus of this workshop ison IP QoS and NOT on Frameon IP QoS and NOT on Frame

Relay or ATM QoSRelay or ATM QoS

10



Internet scale performance

Packet classification

Access bandwidth management

Congestion management

Queue management

Granular measurements

11



Internet scale performanceInternet scale performance




Queue management


12


Network Architecture

Edge FunctionsEdge FunctionsPacket classification

Admission controlBandwidth management

Queuing

Services and traffic metering

Security filtering

Customer access aggregation

Backbone FunctionsBackbone FunctionsHigh-speed High-speed switching and transport

Congestion managementQueue management

Traffic management

QoS interworking

Backbone

Scaleable Solutions Require Scaleable Solutions Require Cooperative Edge andCooperative Edge andBackbone FunctionsBackbone Functions

POP

POP

POP

POP

13


Distributed Switching & Services The key to delivering scaleable and high

performance Internet QoS is the distributedprocessing capabilities on the 12000 (GSR)and 7500 family of routers

With the 12000 and 7500 packet forwarding(switching) and other services are off-loadedfrom the central processor to the linecard(12000) and VIP (7500)

Utilizes the Packet Engine and SRAM on thelinecard or VIP

14


Distributed Switching & Services

With VIP2-40 and distributed processingwe are able to deliver up line rate on aDS3 interface

The next generation VIP (VIP2-50) withdistributed processing we can scale upto OC-3/STM-1 (155Mbps) rates

15


Distributed Switching & Services

Distributed Services

CAR Packet classification

Rate Limiting

WFQ

WRED

NetFlow Services

BGP PolicyPropagation

Distributed Switching

Cisco ExpressForwarding (CEF)

16

Cisco Express Forwarding (CEF) technology for IP is a scaleable,distributed, layer 3 switching solution designed to meet the futureperformance requirements of the Internet and Enterprise networks. CEF isalso a key component of Cisco's Tag Switching architecture.

CEF replaces Route Caching. CEF creates a Forwarding Information Base(FIB) for the destination switching decision which mirrors the entire contentsof the IP routing table. i.e. there is a one-to-one correspondence betweenFIB table entries and routing table prefixes; therefore no need to maintain aroute-cache.

CEF feature:

Load balancing: Per destination (the default) and per packet overequal/unequal cost links for as many paths as known in the routingtopology

Traffic statistics: Byte and packet counts at a granularity of per-prefix,per-neighbor etc.

Media independence: CEF currently supports Packet over Sonet,ATM/AAL5, Frame Relay, Ethernet, FDDI, HDLC and mPPP.

Tunnelling: Generic Route Encapsulation (GRE).

Subinterface support: allowing for the flexibility of per subinterfaceconfigurations e.g. MTU.

Cache-Based Forwarding

SiSi

Cisco Express Forwarding (CEF)

First packet to destination processed by route processor

Forwarding cache entry made to switching engineSubsequent packets to same destination switchedwithout route processorTopology changes flush cache entries; refresh of cacheis traffic-driven

Optimized for longer flows and moderate number ofdestinations

Forwarding information automatically distributed toswitching engines

Route processor is no longer in data pathUpdates to forwarding information are topology, nottraffic drivenOptimized for shorter flows and large number ofdestinations

2)Cache Entry

1)First Packet

3)SubsequentPackets

Forwarding Information

Distributed Forwarding

Cisco Systems Confidential

17


CEF

RSPRSPFIBFIB

TableTableRoutingRoutingTableTable

Cisco 7500

RSPRSPForwardingForwarding

CacheCache

CyBus

First Packet Process Switched

Subsequent Subsequent PacketsFast Switched

VIPVIP

RoutingRoutingTableTable

Cisco 7500

All All Packets Forwarded by VIPs

DistributedDistributedFIBFIB

CyBus

Fast/Optimum/Flow CEF

VIPVIPDistributedDistributedForwardingForwarding

CacheCache

VIPVIP VIPVIP VIPVIP VIPVIP

18

A point to note, CEF only runs distributed if your 7500 configuration hasVIP2-40 or better

Packets switched from port-to-port on the same VIP does not leave the VIP.


CEF

CEF works betweenPort-to-port on the same VIP

VIP to VIP

VIP to xIP

xIP to VIP

xIP to xIP

For xIP to VIP and xIP to xIP packetforwarding decision is made on the RSP

19

For an independent test result on CEF read The Tolly Group report #7295October 1997.


CEF

CEF runs on existing RSP but to takeadvantage of distributed switching andhigher performance you will need aVIP2-40 or better

Available on 7200, 7500 and 12000(GSR) platform

Other platforms will be added in thefuture

20

Distributed services runs on the VIP.

Each VIP has its own processor, called the Packet Engine, which runs theIOS code and SRAM for packet memory.


Versatile Interface Processor (VIP)

PacketEngine PortPort

AdapterAdapter

PortPortAdapterAdapter

SRAM

PPCCII

PPCCII

CCyyBBuuss SiSiSiSi SiSi

RSP RSP

VIP

21

Next we will talk about Packet Classification and Access BandwidthManagement. These two functions are delivered through a feature calledCommitted Access Rate (CAR).




Packet classificationPacket classification

Access bandwidth managementAccess bandwidth management


Queue management


22


Committed Access Rate (CAR)

Committed Access Rate (CAR)Previously known as Weighted Rate Limiting(WRL)

Two functionsPacket ClassificationPacket Classification - IP precedence setting

Access Bandwidth ManagementAccess Bandwidth Management through ratelimiting

23

In the next few slides we will discuss the above items in detail starting withTraffic Matching Specification.


CAR - Overview

TrafficTrafficMatchingMatching

SpecificationSpecification

TrafficTrafficMeasurementMeasurement

InstrumentationInstrumentation NextPolicy

ActionActionPolicyPolicy

24

Accounting information for all of the above are available. For MACaccounting in 11.1CC we provide accounting information for up to 512peers.

Please note that doing IP access-list is slow as it uses the same code thatthe current IP access-list is using. The same rules apply as if you are doinga regular access-list.


CAR - Traffic Matching Specification

Identify packets of interest forprecedence setting or rate limiting orboth

Matching specification1) All traffic

2) IP Precedence

3) MAC Address

4) IP Access List - Standard & Extended (slow)

25

The differences between token bucket and leaky bucket schemes will bediscussed later.


CAR - Traffic Measurement

Uses the token bucket schemetoken bucket scheme as ameasuring mechanism

Tokens are added to the bucket at thecommitted rate and the number oftokens in the bucket is limited by thenormal burst size

Depth of the bucket determines theburst size

26

Packets arriving are said to conform if sufficient tokens are available and thecorresponding number of tokens are removed from the bucket.

Packets arriving at the bucket are said to exceed if insufficient tokens areavailable.



Packets arriving with sufficient tokensin the bucket are said to conformconform

Packets arriving with insufficient tokensin the bucket are said to exceedexceed

27



Packets arriving exceeding the normalburst but fall within the excess burstlimit is handled via a RED-like manageddrop policy

This is to reduce TCP Slow-Startoscillation

(when the exceed-action is to drop packets)

28

There is a burst counter that counts the packets in excess of the committedrate. Any packet that is in excess of the committed rate will cause the burstcounter to increment. Likewise, when the traffic is below the committed ratethe burst counter will reset back to zero.

When a packet arrives the burst counter is evaluated:

< burst-normal: conform-action

< burst-max: possibility of exceed-action proportional to burst value

> burst-max: exceed-action

In any given period a committed rates worth of traffic will always conform.

To calculate the probability

P(exceed) = (burst_counter - normal_burst) / (max_burst -normal_burst)



Token bucket configurable parameters Committed rate (bits/sec)

Configurable in increments of 8Kbits

Normal burst size (bytes)To handle temporary burst over the committed rate limitwithout paying a penalty

Extended burst size (bytes)

Burst in excess of the normal burst size

29

The token bucket accumulates token at the Committed Rate up to theburst level. When that happens the token overflows.

As a packet arrives if there is a matching token the packet is said to conformotherwise exceed.

Token comes in bytes.

The token size must match the packet size for a conform.

Committed Rate = increments of 8Kbits/milisec.


Token Bucket

pp

Tokens

BBOverflowTokens

Packetsarriving Conform

Exceed

B B - Burst size

p p - Token arrival rate

30


Extended Burst

Packet Discard %

100

BucketDepth

ExtendedBurst

NormalBurst

31

In 11.1CC the rate-limit list is not bounded.

Each rate-limit statement is checked sequentially for a match. When amatch is found the token bucket, if there is one, is evaluated.

If the action is a continue action it will go to the next rate-limit on the list tofind a subsequent match. If a match is found and a token bucket exists it isevaluated again.

If an end of rate-limit list is encounter without finding a match or continueaction the default behaviour would be to transmit.


CAR - Action Policies Configurable actions

Transmit

Drop

Continue (go to the next rate-limit in the list)

Set precedence and transmit (rewrite the IPprecedence bits and transmit)

Set precedence and continue (rewrite the IPprecedence bits and go to the next rate-limit in the list)

Rate-limit statement can be cascadedIf a match is not found the default is to transmit

32

32

CAR - Policy Examples

Drop

Drop

Per Application CARPer Application CAR

MultimediaMultimedia

Mission-CriticalMission-Critical

Recolour

Recolour

Cisco Systems Confidential

33

Definition of Traffic shaping: Traffic shaping is forcing your traffic to conformto a certain specified behavior. Usually the specified behavior is a worstcase or a worst case plus average case (i.e., at worst, this application willgenerate 100 Mbits/s of data for a maximum burst of 2 seconds and itsaverage over any 10 second interval will be no more than 50 Mbit/s). Byknowing precisely how the traffic is going to behave, it is possible to allocateresources inside the network such that guarantees about availability ofbandwidth and maximum delays can be given.

For those who wants more information read Gigabit Networking by CraigPartridge, Ch11 - Traffic Shaping, page 253 - 263.


Token vs Leaky Bucket

Token bucket Passes bursts

No buffering

Does not smoothes or shapes traffic

Leaky bucket Smoothes or shapes traffic, this is achieved by bufferingthe traffic

Generic traffic shaping feature uses this scheme

Used in ATM networks for traffic shaping and policingKnown also as Generic Cell Rate Algorithm (GCRA) in ATM

34

The leaky bucket algorithm uses a buffer of finite size that incoming traffic isplaced into. Traffic is allowed to drain out of the bucket and sent on thenetwork at a rate, p. Excess data that cannot fit into the buffer is discarded.The leaky bucket algorithm has the effect of shaping bursty traffic into a flowof equally spaced packets, each being emitted 1/p units of time after theprevious packet. The size of the buffer limits the packet delay.

Any packets that arrive when the bucket is full is dropped.


Leaky Bucket

Packetsarriving

BBOverflowPackets

ppB B - Burst size

p p - Leak ratePackets are leakedat a rate specifiedby pp

35

The diagram above shows the effects of traffic shaping.


Traffic Shaping

Traffic Shaping

Tra

ffic

Time

Traffic Rate

Tra

ffic

Time

Traffic Rate

36


CAR - Packet Classification

A function of CAR

Also known as colouring or labeling ofpackets

Partition network traffic into multiplepriority levels or Class of Service (CoS)

37

8 bits in IP header for ToS - precedence, delay, reliability, throughput

3 bit for precedence (RFC 791)

Network Control (7)

Internetwork Control (6)

CRITIC/ECP (5)

Flash Override (4)

Flash (3)

Immediate (2)

Priority (1)

Routine (0)

Precedence 6 and 7 are reserved for routing protocol and cannot be used



Uses the 3 bits precedence field in theIP header

Up to 6 CoS can be defined

05

The other two are reserved (per RFC791)

Classification is done using severalmethods

rate-limit or IP access list (Standard & Extended)

38

Note that precedence bits can be override.



Packets can be classified based on1) IP Address (source/destination)

2) Application port

3) IP Protocol

4) Interface

5) Other IP header information

Classification can also be over-ridden orre-classified

39


CAR

Ingress RouterIngress RouterPacket classificationToken bucketlike Frame RelayMultiple thresholdsActions:

Change class (precedence)Drop packet (RED-like)

Egress RouterEgress RouterPacket classificationToken bucketMultiple thresholdsActions:

Drop packet

L3 CARL3 CAR

40


CAR

CAR implementation in 11.1CC isavailable in either RSP or distributed

To run Distributed CAR (DCAR) you willneed a VIP2-40 or better

41

- Bits/secs

- bytes

- bytes

The upper bound for bps is 155000000, normal-burst is 2000000 andextended-burst is 8000000


Configuring CAR

Configuring CAR[no] rate-limit {input|output}

[access-group [rate-limit] ]

conform-action {drop|transmit|continue|

set-prec-transmit |

set-prec-continue }

exceed-action {drop|transmit|continue|

set-prec-transmit |

set-prec-continue }

42


Configuring CAR

CAR access-list[no] access-list rate-limit

[no] access-list rate-limit

CAR show commandshow interface [interface] rate-limit

43

In the above configuration a customer has a T3 link to an ISP and the ISPwants to rate-limit the customer to only allow them 20Mbps of the 45Mbps.Probably because the customer is only willing to pay 20Mbps worth of traffic.

We have also configured to allow them to burst up to 24000 bytes andanything beyond that we drop.


CAR Configuration Example

R2#write term.!interface Hssi0/0/0 description 45Mbps to R1 rate-limit input 20000000 24000 24000 rate-limit input 20000000 24000 24000 conform-action transmit exceed-action drop conform-action transmit exceed-action drop ip address 200.200.14.250 255.255.255.252!

R2 R1hssi0/0/0

44

This is the output when do a show interface [interface] rate-limit.


CAR Show Command

R2#sh int hssi 0/0/0 rate-limitsh int hssi 0/0/0 rate-limitHssi0/0/0 45Mbps to R1 Input matches: all traffic params: 20000000 bps, 24000 limit, 24000 extended limit conformed 8 packets, 428 bytes; action: transmit exceeded 0 packets, 0 bytes; action: drop last packet: 8680ms ago, current burst: 0 bytes last cleared 00:03:59 ago, conformed 0 bps, exceeded 0 bps

45

The above example show a rate-limit by application.

We rate-limit Web traffic to 20Mbps and if that conforms set the precedenceto 5, otherwise we set if to 0 (best effort).

Ftp is rate-limit to 10Mbps again if it conforms we set the precedence to 5and if it exceeds we drop it.

The last line in the rate-limit statement is a catch all. For the catch all werate-limit to 8Mbps, if it conforms set the precedence to 5 otherwise drop.


CAR - More ExamplesR1#write term.!interface Hssi0/0/0 description 45Mbps to R2 rate-limit input access-group 101 20000000 24000 32000 rate-limit input access-group 101 20000000 24000 32000 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 rate-limit input access-group 102 10000000 24000 32000 rate-limit input access-group 102 10000000 24000 32000 conform-action set-prec-transmit 5 exceed-action drop conform-action set-prec-transmit 5 exceed-action drop rate-limit input 8000000 16000 24000 rate-limit input 8000000 16000 24000 conform-action set-prec-transmit 5 exceed-action drop conform-action set-prec-transmit 5 exceed-action drop ip address 200.200.14.250 255.255.255.252!access-list 101 permit tcp any any eq wwwwwwaccess-list 102 permit tcp any any eq ftpftp!

R2 R1hssi0/0/0

46

Output of show interface [interface] rate-limit.


CAR - More ExamplesR1#sh int hssi 0/0/0 rate-limitsh int hssi 0/0/0 rate-limitHssi0/0/0 45Mbps to R2 Input matches: access-group 101 params: 20000000 bps, 24000 limit, 32000 extended limit conformed 3 packets, 189 bytes; action: set-prec-transmit exceeded 0 packets, 0 bytes; action: set-prec-transmit last packet: 309100ms ago, current burst: 0 bytes last cleared 00:08:00 ago, conformed 0 bps, exceeded 0 bps matches: access-group 102 params: 10000000 bps, 24000 limit, 32000 extended limit conformed 0 packets, 0 bytes; action: set-prec-transmit exceeded 0 packets, 0 bytes; action: drop last packet: 19522612ms ago, current burst: 0 bytes last cleared 00:07:18 ago, conformed 0 bps, exceeded 0 bps matches: all traffic params: 8000000 bps, 16000 limit, 24000 extended limit conformed 5 packets, 315 bytes; action: set-prec-transmit exceeded 0 packets, 0 bytes; action: drop last packet: 9632ms ago, current burst: 0 bytes last cleared 00:05:43 ago, conformed 0 bps, exceeded 0 bps

47

The above configuration show an example using rate-limit to control traffic inan Internet Exchange Point (IXP).

Lets say we have a connection to another ISP via FDDI (back-to-back FDDI)and we want to rate-limit the other ISP to 80Mbps out of the 100Mbps FDDIbandwidth. If they conform we set the IP precedence to 1and if they exceedwe set the IP precedence to 0. Notice that in both cases we continue to finda next rate-limit match.

The next rate-limit statement limits web traffic to 80Mbps with a normal burstof 56kbytes and extended burst of 72kbytes. If it conforms set the IPprecedence to 5 and transmit otherwise we set the IP precedence to 0 andtransmit.

The next rate-limit statement is a catch all where we are only allowing50Mbps for all other traffic (other than web traffic). Again if it conforms weset the IP precedence to 5 and transmit otherwise we set the IP precedenceto 0 and transmit.

The last rate-limit statement is an output rate-limit. What we are doing hereis rate-limit what we send to the other ISP to 80Mbps. Nothing more.


CAR - More ExamplesR2#write term.!interface Fddi2/1/0 rate-limit input access-group rate-limit 100 800000000 80000 160000rate-limit input access-group rate-limit 100 800000000 80000 160000 conform-action set-prec-continue 1 exceed-action set-prec-continue 0 conform-action set-prec-continue 1 exceed-action set-prec-continue 0 rate-limit input access-group 101 80000000 80000 160000 rate-limit input access-group 101 80000000 80000 160000 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 rate-limit input 50000000 50000 100000 rate-limit input 50000000 50000 100000 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 conform-action set-prec-transmit 5 exceed-action set-prec-transmit 0 rate-limit output 80000000 80000 16000 rate-limit output 80000000 80000 16000 conform-action transmit exceed-action drop conform-action transmit exceed-action drop ip address 200.200.6.1 255.255.255.0!access-list rate-limit 100 00e0.34b0.7777 00e0.34b0.7777!access-list 101 permit tcp any any eq wwwwww!

R2 R1FDDIfddi2/1/0

48

Output of show interface [interface] rate-limit. This is on the input. Nextslide/page will be the output.


CAR - More ExamplesR1#sh int fddi2/1/0 rate-limitsh int fddi2/1/0 rate-limitFddi2/1/0 Input matches: access-group rate-limit 100 params: 800000000 bps, 64000 limit, 80000 extended limit conformed 0 packets, 0 bytes; action: set-prec-continue exceeded 0 packets, 0 bytes; action: set-prec-continue last packet: 4737508ms ago, current burst: 0 bytes last cleared 01:05:47 ago, conformed 0 bps, exceeded 0 bps matches: access-group 101 params: 80000000 bps, 56000 limit, 72000 extended limit conformed 0 packets, 0 bytes; action: set-prec-transmit exceeded 0 packets, 0 bytes; action: set-prec-transmit last packet: 4738036ms ago, current burst: 0 bytes last cleared 01:02:05 ago, conformed 0 bps, exceeded 0 bps matches: all traffic params: 50000000 bps, 48000 limit, 64000 extended limit conformed 0 packets, 0 bytes; action: set-prec-transmit exceeded 0 packets, 0 bytes; action: set-prec-transmit last packet: 4738036ms ago, current burst: 0 bytes last cleared 01:00:22 ago, conformed 0 bps, exceeded 0 bps

49


CAR - More Examples

Output matches: all traffic params: 80000000 bps, 64000 limit, 80000 extended limit conformed 0 packets, 0 bytes; action: transmit exceeded 0 packets, 0 bytes; action: drop last packet: 4809528ms ago, current burst: 0 bytes last cleared 00:59:42 ago, conformed 0 bps, exceeded 0 bps

50

This is done to ensure that customers who did not pay for premium servicefor example buts sets their packet to premium doesnt get premiumtreatment.

Remember there is nothing stopping a customer from setting all their trafficto IP precedence 5 for example before sending it you. If you so happen touse IP precedence 5 as premium service than this customer who is onlypaying you standard service will get premium treatment.

It is also recommended to have this at the end of every rate-limit list.


Implementation Note

Cisco recommends that you set the IPprecedence for all traffic entering yournetwork

This is done to ensure that onlycustomers who pay for preferentialtreatment gets preferential treatment

51

Here we are assuming that IP precedence 0 is best-effort.


Implementation Note

Set or reset IP precedence to 0

!interface Serial 0/0/0 rate-limit input 155000000 155000 155000 rate-limit input 155000000 155000 155000 conform-action set-prec-transmit 0 exceed-action set-prec-transmit 0 conform-action set-prec-transmit 0 exceed-action set-prec-transmit 0!

52


BGP Policy Propagation Conveys IP precedence to be used in

forwarding to specified destinationprefix via BGP community tag

Allows ingress routers to prioritizeincoming traffic

Also allows IP precedence setting basedon AS-path attribute or access list

Inter-ISP Service Level Agreements(SLAs)

53



For this feature to work you will need torun

BGP

CEF

54



PremiumCustomer

TrafficSource

iBGP Peers

ServiceProvider AS

210.210.1.0/24

Prefix 210.210.1.0/24; Community 210:5

R1 R2

Prefix Next-hopPrecedence

210.210.1.0/24h0/0/0 5

210.210.2.0/24h0/0/0 0

FIB Table

IP HeaderData

Src Addr: x.x.x.x

Dest Addr: 210.210.1.1

IP Precedence: 5

55


Configuring BGP PolicyPropagation

Configuring BGP Policy Propagation[no] bgp-policy ip-prec-map

56


BGP Policy Propagation - Sample ConfigR2#write term!router bgp 210 neighbor 210.210.14.1 remote-as 210 neighbor 210.210.14.1 route-map comm-relay-prec out neighbor 210.210.14.1 send-communitysend-community!ip bgp-community new-format!access-list 1 permit 210.210.1.0 0.0.0.255!route-map comm-relay-prec permit 10 match ip address 1 set community 210:5!route-map comm-relay-prec permit 20 set community 210:0!

57


BGP Policy Propagation - Sample ConfigR1#write term!router bgp 210 table-map precedence-maptable-map precedence-map neighbor 200.200.14.4 remote-as 210 neighbor 200.200.14.4 update-source Loopback0!ip bgp-community new-formatip bgp-community new-format!ip community-list 1 permit 210:5!route-map precedence-map permit 10 match community 1 set ip precedence 5set ip precedence 5!route-map precedence-map permit 20 set ip precedence 0set ip precedence 0!

58


BGP Policy Propagation - Sample Config

!int hssi0/0/0 ip address 210.210.2.1 255.255.255.252 bgp-policy ip-prec-mapbgp-policy ip-prec-map!

59


BGP Policy Propagation - Inter-AS

AS200 AS210R1 R2

Prefix Community

210.210.1.0/24

210.210.2.0/24

210.210.3.0/24

200:5

200:4

200:0

R1 configuration!

router bgp 200

table-map AS210-precedence-map

neighbour R2 remote-as 210

!

ip bgp-community new-format

!

ip community-list 1 permit 200:5





!

route-map AS210-precedence-map permit 10

match community 1

set ip precedence 5


match community 2

set ip precedence 4


match community 3

set ip precedence 3


match community 4

set ip precedence 2


match community 5

set ip precedence 1


set ip precedence 0

!

60


BGP Policy Propagation - AS-path

AS200 AS210R1 R2

!router bgp 210 table-map as-path-precedence-map neighbor R1 remote-as 200!ip as-path access-list 101 permit $200^!route-map as-path-precedence-map match ip as-path 101 set precedence 3!interface hssi/0/0/0 bgp-policy ip-prec-map!

61






Congestion managementCongestion management

Queue management


62


The Problem of Congestion Uncontrolled, congestion will seriously degrade

system performanceThe system buffers fill up

Packets are dropped, resulting in retransmissions

This causes more packet loss and increased latency

The problem builds on itself until the system collapses

Throughput

Congestion

Controlled CongestionControlled Congestion

Uncontrolled CongestionUncontrolled Congestion

63


Affects of Tail Drop

Time

Queue Utilization100%

Tail Drop

64

An algorithm that cooperates with TCP to provide congestion avoidance.

Puts a big buffer in front of a congested link and signals the application ateither end of the congested link to back off in the event of a congestion.

If they are using a well behaved TCP implementation they will back off.

Trade off packets get buffered, introduces latency.

The amount of buffer required is x2 the round trip delay.

RED does not give you more bandwidth. What it does is allows you to betterutilize your available bandwidth.

The obvious solution to fix the congestion problem is to increase thebandwidth of the link, but sometimes due to cost or availability this is notpossible. Therefore you use RED to manage the congestion.


Random Early Detect/Drop (RED)

A congestion avoidance algorithm

Designed to work with a transport protocollike TCP

Not bias against bursty traffic

Avoids global synchronisation of manyconnections

Global synchronisation is many connectionsgoing through TCP Slow-Start mode at the sametime

65


Global Synchronization

Time

Queue Utilization100%

Tail drop

3 traffic flows start at different times

66


RED

RED RED reducesreduces overall network packet loss, overall network packet loss,maximizing goodput and minimizingmaximizing goodput and minimizinglatencylatency

RED accomplishes this by fine-tuning theTCP Slow-Start congestion windowmechanism to avoid oscillation andminimize retransmission

Result is optimized throughput, with minimalpacket loss

67


TCP & RED

TCP is a sliding window protocol that usesself-clockingself-clocking to adjust its use of thenetwork to match available bandwidth

Packet loss is a requirement for this towork

Key decisions: what packets to drop, when to drop them

A drop is an explicit signal to TCP to slowdown transmission

68


TCP & RED

In a well behaved TCP implementation,the sender upon detecting a packet losswill shrink its window size (i.e., slowdown its rate of transmission), and gointo Slow-Start mode

69


RED

Packet DiscardProbability

AverageQueue SizeMaximum

ThresholdMinimumThreshold

Adjustable

1

70

In 11.1CC implementation of RED the packet dropping portion is notrandom. Today we drop every 100th packet that exceeds the predefinedminimum threshold.


REDQueue

QueuePointer

Without RED when the queue fills up all packetsthat arrive are dropped

This is also referred to as tail dropstail drops

With RED as oppose to doing a tail drop the routermonitors the average queue sizeaverage queue size and usingrandomization choose connections to notify that acongestion is impending

Packetsarriving

71

The above algorithm was taken from Random Early Detection Gateways forCongestion Avoidance by Sally Floyd and Van Jacobson.

RED has two algorithms. One for computing Average Queue Size and theother for calculating packet-marking probability Both will be explained in thefollowing two slides.

The max-threshold and min-threshold parameters are user configurableparameters.


RED Algorithmfor each packet arrival calculate the average queue size (avgavg) if min_thresholdmin_threshold > avgavg queue arriving packet else if min_thresholdmin_threshold

72

The weight parameter is a user configurable parameter.


RED - Average Queue Size

Used to determine the degree ofburstiness that will be allowed in thequeue

Calculating average queue sizeavg = (1 - 1/weight) * avg + 1/weight * current_queue_size

73


RED - Packet-drop Probability

Determines how frequent packets aredropped given the current level ofcongestion

The objective is to drop packets at a fairlyevenly-spaced intervals

This is to avoid biases and globalsynchronisation

Packets are dropped sufficiently frequentlyto control the average queue size

74

The packet-drop probability is a function of the average queue sizediscussed earlier.

The mark_probability, min_threshold and max_threshold parameters areuser configurable.


RED - Packet-drop Probability

Calculating packet-drop probabilityprobability = mark_probability (avg - min_threshold) /

(max_threshold - min_threshold)

The probability that a packet is droppedfrom a connection is proportional to theamount of packets sent by the connection

75


Weighted RED (WRED) WRED combines IP precedence IP precedence with RED

to implement multiple service classesmultiple service classes withdefined drop rates

Precedence applied at the edge or prior to entering network

Administered in the core

In a congestion situation, higher priority trafficis given precedence without exacerbating thecongestion problem

Lower priority traffic is throttled more aggressively

RED is applied to all levels of traffic tomanage congestion

Result: overall network traffic optimized,giving precedence to high-priority traffic

76


WRED Service Profile Example

AverageQueue SizeStandard

MinimumThreshold

PremiumMinimumThreshold

MaximumThreshold

StandardServiceProfile

PremiumServiceProfile


Two ServiceLevels are Shown;

Up to SixCan Be Defined

Adjustable

1

77


Where/When should I use WRED?

Congested long-haul links (e.g. trans-oceanic links)

Not recommended for campus networks

Where the bulk of your traffic is TCP asoppose to UDP

Remember only TCP will react to a packet dropRemember only TCP will react to a packet dropUDP will notUDP will not

78


DWRED

WRED implementation in 11.1CC runsdistributed only on the VIP

DWRED (Distributed WRED)

It utilizes the processor and SRAMmemory on the VIP

This feature requires VIP2-40 or better

79

In most cases to turn on DWRED all you need to do is type the random-detect enable interface command. The IOS will figure the rest out.


Configuring DWRED

Enabling DWRED[no] random-detect enable

Configuring weight factor for movingaverage queue size calculation

random-detect queue-weight

80


Configuring DWRED

Configuring DWRED max thresholdrandom-detect max-threshold

Configuring DWRED to not drop anypackets below to avoid droppingof TCP ACKs

random-detect min-mark-packet-size

81

Note that the min-threshold parameter is a percentage of max-threshold.The mark-probability parameter is by default 100.

To calculate the mark-probability for a particular precedence level;

p = 1/ mark-probability


Configuring DWRED

Configuring WRED parameters for aspecific CoS

random-detect precedence

Show commandshow interface [interface] random-detect

82


WRED CLI

R3#conf termR3(config)#int hssi 0/0/0R3(config-if)# random-detect ?random-detect ? enable Enable DWRED on this output interface max-threshold Maximum threshold min-mark-packet-size Minimum packet size subject to marking precedence Parameters for each precedence value queue-weight Packet weight for queue depth average

83


WRED Configuration Example

R3#conf termR3(config)#int hssi 0/0/0R3(config-if)# random-detect enablerandom-detect enableR3(config-if)#^Z

R3 R1hssi0/0/0

84


WRED Configuration ExampleR3#write terminal!interface Hssi0/0/0 description 45Mbps to R1 ip address 200.200.14.250 255.255.255.252 random-detect enable random-detect max-threshold 256 random-detect min-mark-packet-size 50 random-detect queue-weight 1024 random-detect precedence 0 12 100 random-detect precedence 1 25 100 random-detect precedence 2 37 100 random-detect precedence 3 50 100 random-detect precedence 4 62 100 random-detect precedence 5 75 100 random-detect precedence 6 87 100 random-detect precedence 7 100 100!

85


WRED Show CommandR3#sh int hssi0/0/0sh int hssi0/0/0Hssi0/0/0 is up, line protocol is up Hardware is cyBus HSSI Description: 45Mbps to R1 Internet address is 200.200.14.250/30 MTU 4470 bytes, BW 45045 Kbit, DLY 200 usec, rely 255/255, load 1/255 Encapsulation HDLC, loopback not set, keepalive set (10 sec) Last input 00:00:02, output 00:00:03, output hang never Last clearing of "show interface" counters never Queueing strategy: fifo Packet Drop strategy: VIP-based weighted REDVIP-based weighted RED Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 1976 packets input, 131263 bytes, 0 no buffer Received 1577 broadcasts, 0 runts, 0 giants 0 parity 4 input errors, 4 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 1939 packets output, 130910 bytes, 0 underruns 0 output errors, 0 applique, 3 interface resets 0 output buffers copied, 0 interrupts, 0 failures 1 carrier transitions

86


WRED Show CommandR3#sh int hssi 0/0/0 random-detectsh int hssi 0/0/0 random-detect Hssi0/0/0 queue size 0

packets output 3, drops 0 WRED: queue average 0, max threshold 256 weight 1/1024, minimum mark packet size 50 Precedence 0: 32 min threshold, 1/100 mark weight 3 packets output, drops: 0 random, 0 threshold Precedence 1: 64 min threshold, 1/100 mark weight

(no traffic) Precedence 2: 96 min threshold, 1/100 mark weight






(no traffic)

87


WRED Show Command


AverageQueue SizeMaximum

ThresholdMinimumThreshold

Random Drop

Threshold Drop

If you are seeing a lot of threshold drop you are no longer doing RED. Whatyou need to do is adjust your min-threshold to a point where you are nolonger seeing a lot of threshold drop.

88







Queue managementQueue management


89

Packet scheduling algorithm determines the order in which buffered packets aresent out to a transmission link.

An example of a packet scheduling algorithm is FIFO.

Before we go on to talk about WFQ let us briefly examine what Fair Queueing (FQ)is. FQ was first introduced by J. Nagle [RFC970] in 1985 and later enhanced by A.Demers, S. Keshav and S. Shenker [Analysis and Simulation of a Fair QueuingAlgorithm; ACM SIGCOMM 1989], Zhang [Virtual Clock; A New Traffic ControlAlgorithm for Packet Switching Networks; ACM SIGCOMM 1990], and McKenney[Stochastic Fairness Queuing; Internetworking Research and Experience].

The goal of FQ as introduced by Nagle was to protect the network from hosts thatare badly-behaved in the presence of both well-behaved and badly-behaved hosts.This is to insure that well behaved hosts gets better service than badly-behavedhosts.

With FQ each source host gets an equal fraction of the bandwidth. This is done byhaving multiple queues on the outgoing interface (as oppose to a single queuewhere all traffic gets queued on a single queue) one for each source host. Thebacklog queues are serviced in a round robin fashion.

Because each source hosts has a queue of its own well-behaved hosts will beprotected from badly-behaved hosts. Badly-behaved hosts can send as manypackets as they want, but this will not increase their share of the bandwidth. All thatdoes is it fills up its own queue and when that happens their packets are dropped.

The problem with Nagles approached was a queue is required for every sourcehosts on the network.

Demers, Keshav and Shenker did further studies on Nagles work and found thatthe same effect can be archived by separating the traffic into flows andguaranteeing that each flow gets an equal share of the bandwidth.

The W in WFQ was introduced by L. Zhang at about the same time as the workdone by Demers, Keshav and Shenker. The original algorithm introduced by Zhangwas called Virtual Clock (VC), later named Weighted Fair Queuing (WFQ). Thegoal of FQ and VC was somewhat different, but they both have a common goalwhich is to share resources fairly between a variable number of sources.

The objective of WFQ is to provide a packet based approximation of the theGeneralised Processor Sharing (GPS) model. That is providing queue service thatsupports bandwidth allocation and delay bounds while providing fairness andprotection for connections and retains packet switching efficiency.

The GPS model also yields a delay bounds both for queueing delay at a singlerouter based on allocated buffer length for the associated traffic class and for end-to-end queueing delay when the traffic source is constrained by a traffic contractsuch as token bucket or leaky bucket mechanism.


Weighted Fair Queuing (WFQ)

What is WFQ? Packet schedulingalgorithm on the transmitpath

Approximates theGeneralised ProcessorSharing (GPS) algorithm

RSPRSP

Cisco 7500

(V)IP(V)IP (V)IP(V)IP VIPVIP

90

forwardingengine

Output

Fair queuing; one queue per flow

forwardingengine Output

Normal queuing; one queue per output interface


Packet Scheduling

An algorithm thatdetermines the orderin which packets aresent out to thetransmission link

Examples of packetscheduling schemes

FIFO

Round Robin

Priority

RSPRSP

Cisco 7500

(V)IP(V)IP(V)IP(V)IP VIPVIPVIPVIP

91

The ideal algorithm is to serve each queue in proportion to its weight forexample for every 6 bits take 3 bits from the blue queue, 2 bits from the redqueue and 1 bit from the amber queue. Unfortunately though we deal in thepacket world. So the above is not practical.

What WFQ does is it approximates the GPS algorithm.


Generalised Processor Sharing(GPS)

RSPRSP

Cisco 7500

(V)IP(V)IP VIPVIP

Assign a weight foreach queue

Backlog queues aresevered inproportion to theirweight 11 22 33

(V)IP(V)IP

92


Why use WFQ?

Provides relative bandwidth guarantees Fair Queuing (FQ) provides fair shareallocation of bandwidth

Weighted Fair Queuing (WFQ) allows forunequal allocation of bandwidth

93

The absolute delay here talks about the delay on the transmit side.

The admission control algorithm and traffic descriptor has been discussedearlier on; CAR.

To provide absolute delay you will have to be able to bound the queue sizeand guarantee a service rate on the queue. This can only be done if youhave a traffic descriptor for the traffic you are dealing with.

guarantee delay == maximum delay.


Why use WFQ?

Provides absolute bandwidth/delayguarantees

Good for real-time applications (e.g.audio/video) and bandwidth provisioning

But requires cooperation of admission controlalgorithm and use of traffic descriptor todetermine the traffic characteristics of theapplication

Example:- average rate and burstiness of the traffic

94


DWFQ

WFQ implementation in 11.1CC runsdistributed only on the VIP

DWFQ (Distributed WFQ)

It utilizes the processor and SRAMmemory on the VIP

This feature requires VIP2-40 or better

95


DWFQ

In 11.1CC WFQ supportsFlow-based WFQ (default)

Class-based WFQ

96

Packets with the same IP source and destination address, TCP or UDPsource and destination port and Type-of-service (ToS) field belongs to thesame flow.

In 11.1CC each interface has a total of 512 queues; fix.


Flow-based WFQ

A flow ID is computed for each packetThe flow ID is a hash computed on source anddestination IP address, source and destinationTCP/UDP port and ToS field

Based on the flow ID the packet is thenclassified to the appropriate queue

In 11.1CC there are a total of 512 queues foreach interface

97


Flow-Based WFQ

Packetsarriving

.

Compute hash

Flow Queues

0 1 510 511

98


Class-based WFQ

Packets can be classified into one of thefollowing

1) IP Precedence

2) TCP/UDP Port

3) IP Protocol

4) Source Interface

99


Class-based WFQ

For IP precedence the classes followsdirectly from the precedence value

For other class-based methods theclasses are defined by mapping aparameter to a class

This is a user configurable parameter

Class range is from 031

100


Class-based WFQ

Packetsarriving

Class-based(IP Precedence).

0 1 6 7

101


Class-based WFQPacketsarriving

Class-based(IP Protocol).

TCP(6) UDP(17) GRE(47)IPinIP(4)

.

0 1 30 31

102


Weight Fair Queuing

Ones the packets are classified (toeither flow or class-based) a timestampis computed for each packet

The timestamp is computed based onflow/class weights

This timestamp is used for packetscheduling decision

103


Weighted Fair QueuingPacketsarriving

Flow or class-based queueswith timestamp entries

Output Queue

104

As of writing (Nov 97) only precedence-based class-based WFQ issupported. All the other options are currently not available and may changein future release.


Configuring WFQ

Enabling flow-based WFQ[no] fair-queue enable

Enabling class-based WFQ[no] fair-queue class-based

105


Configuring WFQ

Setting queue depthfair-queue max-queue-depth

Where

is the per interface buffer limit (innumber of packets)

is the per flow or per class limit (innumber of packets)

106


Configuring WFQ

Changing the weights for each precedence-based class

fair-queue prec-weight

Show commandshow interface [interface] fair-queue

107

The Congestive Discard Threshold parameter above is not used in the VIPimplementation of WFQ.


DWFQ CLI

R1#conf termR1(config)#int hssi 0/0/0R1(config-if)# fair-queue ?fair-queue ? Congestive Discard Threshold class-based Enable class-based DWFQ on this output interface max-queue-depth Set maximum global and local queue depth prec-weight Set weight for each precedence-based class

108


DWFQ Configuration Example

R1#conf termR1(config)#int hssi 0/0/0R1(config-if)# fair-queue enablefair-queue enable

Flow-based WFQ

Class-based WFQ

R1#conf termR1(config)#int hssi 0/0/0R1(config-if)# fair-queue class-basedfair-queue class-based

109


DWFQ Configuration ExampleR1#write term!interface Hssi0/0/0 description 45Mbps to R2 ip address 200.200.14.250 255.255.255.252 fair-queue enable fair-queue class-based fair-queue max-queue-depth 401 200 fair-queue prec-weight 0 8 fair-queue prec-weight 1 7 fair-queue prec-weight 2 6 fair-queue prec-weight 3 5 fair-queue prec-weight 4 4 fair-queue prec-weight 5 3 fair-queue prec-weight 6 2 fair-queue prec-weight 7 1!

110


DWFQ Show CommandR1#sh int hssi 0/0/0sh int hssi 0/0/0Hssi0/0/0 is up, line protocol is up Hardware is cyBus HSSI Description: 45Mbps to R2 Internet address is 200.200.14.250/30 MTU 4470 bytes, BW 45045 Kbit, DLY 200 usec, rely 255/255, load 1/255 Encapsulation HDLC, loopback not set, keepalive set (10 sec) Last input 00:00:09, output 00:00:00, output hang never Last clearing of "show interface" counters never Queueing strategy: VIP-based fair queuingVIP-based fair queuing Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 2011 packets input, 133587 bytes, 0 no buffer Received 1604 broadcasts, 0 runts, 0 giants 0 parity 4 input errors, 4 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 1971 packets output, 133082 bytes, 0 underruns 0 output errors, 0 applique, 3 interface resets 0 output buffers copied, 0 interrupts, 0 failures 1 carrier transitions

111


DWFQ Show Command

R1#sh int hssi 0/0/0 fair-queuesh int hssi 0/0/0 fair-queue Hssi0/0/0 queue size 0

packets output 35, drops 0 WFQ: global queue limit 401, local queue limit 200 Precedence 0: weight 8 Precedence 1: weight 7 Precedence 2: weight 6 Precedence 3: weight 5 Precedence 4: weight 4 Precedence 5: weight 3 Precedence 6: weight 2 Precedence 7: weight 1

112







Queue management

Granular measurementsGranular measurements

113


Granular Measurements

NetFlowBilling and accounting

Planning

Traffic monitoring

MIB supportIP Precedence

MAC Accounting

CAR

WRED

114

NetFlow switching operates by creating a flow cache that contains theinformation needed to switch and perform access list check for all activeflows. The NetFlow cache is built by processing the first packet of a flowthrough the standard switching path (fast or optimum). As a result, each flowis associated with an incoming and outgoing interface port number and witha specific security access permission and encryption policy. The cache alsoincludes entries for traffic statistics that are updated in tandem with theswitching of subsequent packets. After the NetFlow cache is created,packets identified as belonging to an existing flow can be switched based onthe cached information and security access list checks bypassed. Flowinformation is maintained within the NetFlow cache for all active flows.

NetFlow switching is based on identifying packet flows and performingswitching and access list processing within a router. It does not involve anyconnection-setup protocol either between routers or to any other networkingdevice or end station and does not require any change externallyeither tothe traffic or packets themselves or to any other networking device.

Note NetFlow does consume additional memory and CPU resourcescompared to other switching modes; therefore, it is important to understandthe resources required on your router before enabling NetFlow.


NetFlow

Only first packet is processed by multiple tasks

NetFlow is definedwith specific service requirements

Single switching task applies network services and collectstraffic statistics

RouteTable

AcctgData

AccessList

FirstPacket

SubsequentPackets

SwitchingTask

NetFlowCache

NetFlowNetFlowSwitchingSwitching

TaskTask

NetFlowNetFlowStatisticsStatistics

AcctgTask

SecurityTask

NetFlow DataNetFlow DataExportExport

115


NetFlowA Point to Note With 11.1CC NetFlow is no longer a

switching mode as in 11.1CA & 11.2

In 11.1CC NetFlow is a service whichprovides call record accountinginformation for a IP network independentof the switching mode used

The switching mode in 11.1CC is CEF

Distributed NetFlow and flow export

CEF and NetFlow integration

116

Flow export by default if you do not specify origin-AS or peer-AS does notexport the AS information.

Origin-AS is where the prefix originated and peer-AS is where you learnedthe prefix from.

If you see an AS #0 in you cache entry or data export AS #0 is:

1. Local traffic

2. Traffic destined for the router

3. Flows which are unroutable (flow where there was not an entry inthe route cache for the source or destination)

4. If source AS = 0 and source prefix mask = 0 then it indicates theabsent of route entries

Normally the default size of the NetFlow cache will meet your needs.However, you can increase or decrease the number of entries maintained inthe cache to meet the needs of your NetFlow traffic rates. The default is 64Kflow cache entries. Each cache entry is approximately 64 bytes of storage.Assuming a cache with the default number of entries, approximately 4MB ofDRAM would be required. Each time a new flow is taken from the free flowqueue, the number of free flows is checked. If there are only a few free flowsremaining, NetFlow attempts to age 30 flows using an accelerated time-out.If there is only one free flow remaining, NetFlow automatically ages 30 flowsregardless of their age. The intent is to ensure free flow entries are alwaysavailable.

Before attempting to changing the number of entries read the release notes.


Configuring NetFlow

Enabling NetFlow on an interface[no] ip route-cache flow

Exporting flowsVersion 1[no] ip flow-export

Exporting flowsVersion 5[no] ip flow-export

version 5 {origin-AS|peer-AS}

Show commandshow ip cache flow

117

Flow Flow ConsolidationConsolidation

Flow SwitchingFlow Switchingand Data Exportand Data Export Flow CollectionFlow Collection

Flow Profiling

Accounting/Billing

Network Planning

Network Monitoring

Flow ConsumersFlow Consumers

NetFlow Metering Infrastructure

117Cisco Systems Confidential

118


Source IP AddressSource IP AddressDestination IP AddressDestination IP Address

Next Hop AddressNext Hop AddressSource AS NumberSource AS NumberDestination AS NumberDestination AS Number

Input Interface PortInput Interface PortOutput Interface PortOutput Interface Port

Type of ServiceType of ServiceTCP FlagsTCP FlagsProtocolProtocol

Packet CountPacket CountByte CountByte Count

Start TimestampStart TimestampEnd TimestampEnd Timestamp

Source TCP/UDP PortSource TCP/UDP PortDestination TCP/UDP PortDestination TCP/UDP Port

Usage

QoS

Timeof Day

Application

RoutingandPeering

PortUtilization

NetFlow Data Record (V5)

119

On the router you specify an IP address and a UDP port number to exportthe flow data to. The UDP port number can be one UDP port number whereall routers export their flow data to or it can be a UDP port number for arouter and anything in between that.


NetFlow FlowCollector

Receive flow export data from router(s) onpredefined UDP port(s)

Supports both version 1 and 5 NetFlowrecords

Filtering (permit/deny) on the fly

Summarize/aggregate as needed

Periodically flush summarized data to disk

Filesystem management

120


FlowCollector Architecture

Filter

Storage

NetFlow Exports

FlowCollector

UserUserInterfaceInterface

ConfigConfigFilesFiles

Flow ConsumerApplications

Summarize

Workstation

121


Summarization/Aggregation Objective to reduce the data to be stored and

prepare it for the end applicationHostMatrix (conversion-pairs)

DetailHostMatrix (HostMatrix + application infomation +start/end timestamps)

CallRecord (NetFlow usage record)

Template for usage-based (CoS, time-of-day, etc.) billing

Gives host IP addresses + ports + protocols +ToS and total time spent in the router on switching +start/end stamps

DetailInterface, traffic per interface-pair as well asnexthop, useful in planning resources, trending etc.

SourceNode, DestNode, SourcePort, Protocol etc.

122


FlowCollector v1.0

FCS - Oct 97

Platform supportedSolaris 2.5

HP-UX - 10.2

123


FlowCollector - Sample Config

Filter allow-web-serverPermit Srcport 80

Filter deny-icmp-trafficDeny Prot 1Permit Dstaddr 0.0.0.0 255.255.255.255

See Appendix F for a complete list of FlowCollector Attritubes and theirmeaning.

124


FlowCollector - Sample ConfigThread DAM

Aggregation DetailASMatrix Period 30Port 9992State ActiveDataSetPath /usr1/netflow/data/r1DiskSpaceLimit 1000FileRetain 32

Thread CALLRECAggregation CallRecord Period 30Port 9991State ActiveDataSetPath /usr1/netflow/data/r2DiskSpaceLimit 1000FileRetain 0

125


NetFlow FlowAnalyzer

GUI front end to FlowCollector

Web basedJava applet running in a html file

Formats and display data

Data analysis

Charts

Spreadsheet data export capability

126


FlowAnalyzer

ServerUnix platform (SUN & HP-UX)

ClientUnix

PCs

MACs

127


FlowAnalyzer Architecture

UNIX Workstation

FlowCollectorFlowCollector

AggregationProcessing

FlowAnalyzer DisplayFlowAnalyzer Display



FlowAnalyzer Server

128


FlowAnalyzer Server Components

FlowAnalyzer DisplayFlowAnalyzer Display

Communications (Java)

Aggregation Processing

129


FlowAnalyzer Client Components

How it works

FlowAnalyzer ServerFlowAnalyzer Server

Java AWT 1.02

Communications

Netchart: Graphs

Microline: Trees, Tabs, & Spreadsheet

130


FlowAnalyzer v1.0 Features

Displays results of all aggregationschemes except raw

Set time ranges for viewing data

Table and graph displays

Sorting capability

Save data in Excel spreadsheet format

Online help

131


Platforms Supported

Client

Solaris 2.5.1Netscape 3.0 and 3.0Gold

Windows 9.5 & MACNetscape 3.0, 3.0 Goldand MSIE

Server

Solaris 2.5.1

HP-UX 10.2

Java 1.0.2

132


FlowAnalyzer v1.0

FCS - Oct 97

Bundled with FlowCollector v1.0

Demo copy for FlowCollector andFlowAnalyzer available

http://www.cisco.com/kobayashi/sw-center/netmgmt/nf-planner.shtml

133


Netsys Technologies and NetFlowProactive Planning/Design

Reactive Analysis and Diagnosis

Views and ReportsViews and Reports Link, LAN, router utilizations Application mix Communicating pairs

What-If AnalysesWhat-If Analyses Bandwidth/provisioning Topology Configuration tuning

FlowCollector

NetsysWorkstation

TokenRing

FDDIDualRing

134


MIB Support

CAR MIBS NetFlowMIBS

WFQ MIBSWRED MIBSIP Accounting &Statistics MIBS

CAR ConfigurationTable

CAR StatisticsTable

WRED GlobalConfiguration Table

WRED PrecedenceConfiguration Table

WRED QueueLength Table

WRED StatisticsTable

WRED BackingStore StatisticsTable

MAC AccountingTable

IP PrecedenceAccounting Table

None None

135


CAR MIBS

CAR Configuration Table CAR Statistics Table

Rate Limit Direction

Rate Limit TypeAccess List Index

Committed Rate

Burst Limit

Excess Burst Limit

Conform Action

Exceed Action

Packets Switched*

Bytes Switched

Packets Filtered

Bytes Filtered

Current Burst

* For rate limit

136


WRED MIBSWRED GlobalConfiguration Table

WRED BackingStore StatusTable

WRED StatisticsTable

WRED PrecedenceConfiguration Table

WRED QueueLength Table

No Discard Size

Average Queue LengthDecay Constant

Precedence

Queue MinimumDepth Threshold

Queue MaximumDepth Threshold

Average QueueLength

Byte Switched

PacketSwitched

Backing StoreQueue Depth

Queue DepthPackets Filteredfrom MinimumDepth Threshold

Drop ProbabilityPackets Filteredfrom MaximumDepth Threshold

Packets Filtereddue to BackingStore Exhaust

137


MAC/Precedence Accounting MIB

MAC Accounting MIB IP Precedence Accounting MIB

Packet Direction (input oroutput)

MAC Address

Packets Switched

Bytes Switched

Packets Direction

IP Precedence

Packets Switched

Bytes Switched

138

Putting It All Together

L3 CARL3 CAR

PacketClassifier

Apply IngressRate Thresholds

DeterminePacket Class

AdministerPacket Class

Apply EgressRate Thresholds

Traffic Metering

WRED/WFQ

Premium Medium Standard

138Cisco Systems Confidential

139


Configuration GuidelinesFeature RequirementsOperationPlatform Performance

CAR PacketClassification

CAR RateLimiting

7500 on RSP or distributed, 7200

7500 distributedonly

T3/E3 per VIP

Input orOutput side

N/Arequires VIP2-40 or betterto run distributed, requiresCEF, requires BGP forprecedence propagation

RED/WRED Outputside

Availability

WFQ

NetFlow

CEF

11.1CC

11.1CC 7500 on RSP or distributed, 7200 T3/E3 per VIP

requires VIP2-40 or betterto run distributed, requires CEF

11.1CC

11.1CC7500 distributedonly

T3/E3 per VIP

T3/E3 per VIPOutputside

Inputside

11.1(12)CA,[or greater]

11.1CC

11.1CC

7500 on RSP ordistributed, 7200

requires VIP2-40 or betterto run distributed, requires CEFrequires VIP2-40 or betterto run distributed, requires CEFrouter NetFlow license,optional FlowCollector &FlowAnalyzer

requires VIP2-40 or betterwith 32M DRAM to rundistributed

7500 on RSP or distributed, 7200

N/A

N/A N/A

BGP PolicyPropagation

11.1CC 7500 on RSP or distributed, 7200

T3/E3 per VIPInputside

requires VIP2-40 or betterto run distributed, requires CEF and BGP

140


The End

Q & A

141

Please note that the above URL can only be accessed from within Ciscosinternal network.


Resources

PM for NetFlow & Internet QoSDavid Powell ([email protected])

Internet QoS web pagehttp://corewww.cisco.com/core/html/qosindex.html

[email protected]

142


Case Study

143


Case Study

Application based rate-limiting

Premium bandwidth delivery

Subrate IP service

IX traffic control

Web hosting service

144


Application Based Rate-Limiting

Rate limit a particular type of traffic (e.g., Web) toa portion of the bandwidth

This is done so that the application does not takeup the entire pipe

Can be applied either on the outgoing or incomingpath or both

FTPTelnet

WWW 50%

145


Premium Bandwidth Delivery

PremiumCustomers

StandardCustomers

Broadband Pipe

ISP1

ISP2

Customer

Premium bandwidth allocationenforced by WRED or WFQ

Premium charging via NetFlow

Bi-directional premium traffic viaBGP policy propagation

Standard traffic bursts to fill capacity

146


IP Subrate Service

Fractional bandwidth pipes via rate limiting by port

Upgrade to higher speed without physical reconfig

Discard or recolour excess traffic

NetFlow metering for reporting and charging

Business customer or ISP application

147


Exchange Point Traffic Control

Downstream ISP &peering bandwidthcontrol

Rate limit by MACaddress

Discard excess traffic

MAC accounting

Peer A

Peer B

Peer C

148


ISP Web Hosting

Standard TrafficPremium Traffic

Medium Traffic

ISP Network

Multiple classes of hosting customers

Rate limit or allocate bandwidth toeach server

Classify traffic from/to each server

Measure and bill with NetFlow andCAR MIB

149


Internet QoS Demo

150

The router configuration for this demo can be found in appendix E.


Demo

Demo objectives

Topology

Configuration

151


Demo Objective

The objective of this demo is toThe objective of this demo is todemonstrate how Internet QoSdemonstrate how Internet QoS

works and how the different IOSworks and how the different IOSInternet QoS feature tie togetherInternet QoS feature tie together

152


CAR Demo

30,000pps

f0/0/0 f1/0/030,000pps

rate-limit 20,000pps

This quick demo illustrates how rate-limiting works.

For config see R1 config in Appendix E.

153


Topology

HSSI HSSI

HSSIR2 R3

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

FE

FEFE

FE

R1

FDDI

R4Traffic

GeneratorTraffic

Generator

TrafficGenerator

FE

FE

154


TopologyAS200 (200.200.240.0/20)

AS210 (210.210.240.0/20)

HSSI HSSI

HSSIR2 R3

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

FE

FEFE

FE

R1

FDDI

R4Traffic

GeneratorTraffic

Generator

TrafficGenerator

FE

FE

155


Demo 1

HSSI HSSI

HSSIR2 R3

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

FE

FEFE

FE

R1

FDDI

R4Traffic

GeneratorTraffic

Generator

TrafficGenerator

FE

FE

CAR CAR

DWFQ

Premium

Standard

Direction of Traffic

3

4

2

17

8

5

6

156


Demo 2

HSSI HSSI

HSSIR2 R3

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

TrafficGenerator

FE

FEFE

FE

R1

FDDI

R4Traffic

GeneratorTraffic

Generator

TrafficGenerator

FE

FE

CAR CAR

Premium

Standard

Direction of Traffic

DWFQ

DWRED

3

4

2

17

8

5

6

157

APPENDIX

APPENDIX........................................................................................................................................................157

APPENDIX A - CEF COMMAND SYNTAX ..................................................................................................158

INTRODUCTION ...............................................................................................................................................158PLATFORM REQUIREMENTS..............................................................................................................................158CEF CONFIGURATION/SHOW/DEBUG COMMANDS ..............................................................................................158

APPENDIX B - BGP POLICY PROPAGATION ............................................................................................162

APPENDIX C - LAB HARDWARE CONFIGURATIO N...............................................................................164

APPENDIX D - LAB IP ADDRESS LAYOUT ................................................................................................165

APPENDIX E - LAB ROUTER CONFI G .......................................................................................................167

ROUTER - R1...................................................................................................................................................167ROUTER - R2...................................................................................................................................................170ROUTER - R3...................................................................................................................................................173ROUTER - R4...................................................................................................................................................176

APPENDIX F - FLOWCOLLECTOR ATTRIBUTES LIS T ..........................................................................178

158

APPENDIX A - CEF Command Syntax

Introduction

Cisco Express Forwarding (CEF) is a new form of scaleable switching intended totackle the problems associated with demand caching. With CEF switching, theinformation which is conventionally stored in a route cache is now split up over severaldata structures. The CEF code is able to maintain these data structures in the RSP,and also in slave processors such as the VIP2. The data structures include:

A CEF table, containing all IP prefixes from the main routing table. An adjacency table, containing layer 2 rewrite strings. Shadow copies of hardware and software interface information, as needed for

maintaining the CEF, and also for switching packets.

With the CEF code, IP packets can be switched at interrupt level, just like fast,optimum, and flow switching. This packet switching can be performed strictly on theRSP, or it can also occur in a distributed mode (like DFS), where both the RSP and VIPprocessors can concurrently switch IP packets. When CEF is configured in adistributed mode, each VIP has a separate copy of the above mentioned datastructures.

Platform Requirements

Currently CEF is supported in 7500 and 7200. VIP2 is needed for distributed CEF. Forfull Internet routing table, VIP2's should have 32M memory.

CEF configuration/show/debug commands

The following is a brief description of the commands that are added with CEF switching.

I. Configuration Commands:

Global:

[no] ip cef switching

Enable CEF on the RSP

[no] ip cef distributed switching

Enable distributed CEF

159

[no] ip cef accounting [per-prefix] | [per-adjacency]

Enable per-prefix/per-adjacency accounting on both VIP and RSP

Interface:

no ip route-cache distributed

To disable fib switching on interface. Can be used only when express-cefis already configured on the router.

ip load-sharing [per-packet] | [per-destination]no ip load-sharing per-packet

To specify the type of load-sharing on an interface.

II. Show Commands:

show ip cef [unresolved] | [summary]

unresolved : Display all prefixes which are unresolved at the momentsummary : Display summary info on the CEF table: size of table (intbytes), number of nodes, leaves, number of routes, unresolved routes,etc.

Available both in RSP and on VIPs.

show ip cef [[] [] []] [internal]

detail : Provide detailed information on a destination prefix : Detailedinformation for a prefix includes the nexthop, nexthop interface, number ofdependencies, the nature of the cached adjacency, packet and bytestransferred to this prefix and the gateway via which this destination can bereached.internal : Displays data stored in the loadinfo structure used for load-sharing.

If no prefix is specified all the fib entries are displayed.

If the keyword 'longer-prefix' is specified after the mask of a prefix then allthe longer (more specific) prefixes of this prefix is displayed. Available onboth RSP and VIPs.

show ip cef adjacency [detail] |[internal]

160

Display info on prefixes resolving (directly or recursively) resolvingthrough regular adjacency specified by and

show ip cef adjacency glean | discard | drop | punt | null [detail] | [internal]

Display info on prefixes resolving through the special adjacencies - glean,discard, drop, punt, null

show cef interface [detail] | [stat]

Displays express-forwarding related interface information, whether thisinterface can express-forward the packet or not and why, the type of load-sharing configured, the transmit queue pointer etc.

The 'stat' keyword is available only the VIPs. Provides a in/out pkt/bytecount per interface on VIP.

Available on both RSP and VIPs

show cef [drop] | [not-cef-switched]

drop : Classifies packets dropped at each VIP. Packets are dropped atthe VIPs because of encapsulation failure, no route, no adjacency.not-cef-switched : Classifies packets sent to next slower switchingbecause cef was unsupported, packets were locally destined for the box,packet has IP OPTIONS, etc

Available on RSP only.

show cef linecard [] [detail]

Shows CEF information pertaining to VIPs. Displays the number ofprefix/adjacencies queued up by route-processor for updates, messagessent by RSP, total packets and bytes transferred by VIP.

Available on RSP only.

show adjacency [detail] | [internal]

Shows the adjacency specific information , protocol from which it waslearnt, timers, and other internal data structures.

161

III. Clear Commands:

clear cef linecard [] [adjacency] | [interface | prefix]

Available on RSP only. Reload either the adjacency, distributed interface,or CEF database information. If a slot number is specified, only performthe reload for that particular VIP slot, otherwise all VIP slots receive thereload operation.

clear ip cef [ []] | [*] statistics

Clear the packet/byte count for the specific prefix. If * is specified thenclear all prefix statistics.

clear adjacencyClean up the adjacency database.

IV. Debug Commands:

debug ip cef [table] | [events] | [interface-ipc] | [prefix-ipc] | [drops]

162

APPENDIX B - BGP Policy Propagation

Currently we have mechanism to set Precedence based on the inbound interface andsource IP address. With this new feature, we can set Precedence on the packet basedon the destination IP address. This uses the BGP attributes (AS-path or Community) toconvey Precedence value indirectly for different prefixes via BGP updates. Thisapproach is scaleable as the Precedence for destinations are learnt via routingprotocol.

For example each Precedence value is assigned BGP community value and prefixesare tagged with appropriate Community value. BGP will perform bestpath selection andinstall the best path in the IP routing table. The 'table-map' BGP router configurationcommand can be used to map the Community value to IP Precedence when installingthe prefix in the IP routing table. The Precedence value is populated in the FIB tablealong with the prefix. When packets are switched by FIB, Precedence for thedestinations are picked from FIB entry and set in the packets.

To support the above functionality, route-map is enhanced to support Precedencesetting.

For example the following will set Precedence 5 for prefixes with community 1000:5,and Precedence 4 for prefixes with community 1000:4.

!! to support new Community format!ip bgp-community new-format!!router bgp 1000table-map precedence-mapneighbor x.x.x.x ......!ip community-list 1 permit 1000:5ip community-list 2 permit 1000:4!!route-map precedence-map permit 10match community-list 1set ip precedence 5!

163

route-map precedence-map permit 20match community-list 2set ip precedence 4!

In the following example, as-path access-list is used in the route-map to setprecedence. This will set precedence of packets going thru AS 109, AS 120 to haveprecedence 5 and for packets destined to AS 130 to have precedence 4.

!!router bgp 100table-map precedence-mapneighbor x.x.x.x ...!!ip as-path access-list 101 permit _109_ip as-path access-list 101 permit _120_ip as-path access-list 102 permit _130$!!route-map precedence-map permit 10match as-path 101set ip precedence 5!route-map precedence-map permit 20match as-path 102set ip precedence 4!route-map precedence permit 30!

Verification

use the 'show ip bgp x.x.x.x' to verify if correct community is set on the prefixes

use the 'show ip bgp community-list ' to verify if the correct prefixes areselected.

use the 'show ip route x.x.x.x' to verify if the correct Precedence values are set onthe prefixes.

use the 'show ip cef x.x.x.x' to verify if FIB has the correct Precedence value for theprefix.

164

Appendix C - Lab Hardware Configuration

R1 7505Slot Card Slot 0 Slot 1

4 RSP4 (64MB)3 VIP2-40 PA-H PA-H2 VIP2-40 PA-F-MM PA-FE-TX1 VIP2-40 PA-FE-TX Empty0 VIP2-40 PA-FE-TX Empty


4 RSP4 (64MB)3 VIP2-40 PA-FE-TX Empty2 VIP2-40 PA-FE-TX Empty1 VIP2-40 PA-H Empty0 VIP2-40 PA-H Empty


4 RSP4 (64MB)3 VIP2-40 PA-FE-TX Empty2 VIP2-40 PA-FE-TX Empty1 VIP2-40 PA-H Empty0 VIP2-40 PA-H Empty

R4 7206Slot Card

0 NPE-200/7200-I/O (64MB)1 PA-F-MM2 PA-FE-TX3 PA-FE-TX4 PA-4E5 Empty6 Empty

165

Appendix D - Lab IP Address Layout

AS200 (200.200.0.0/20)R1 - 7505/RSP4

Interface Subnet IP Address Subnet Mask RemarksLoopback 0 200.200.14.1/32 200.200.14.1 255.255.255.255FastEthernet 0/0/0200.200.1.0/24 200.200.1.1 255.255.255.0 to Smartbit #1FastEthernet 1/1/0200.200.2.0/24 200.200.2.1 255.255.255.0 to Smartbit #2Fddi 2/0/0 200.200.5.0/24 200.200.5.1 255.255.255.0 to R4FastEthernet 2/1/0200.200.6.0/24 200.200.6.1 255.255.255.0 to 10/100 Ether SwitchHssi 3/0/0 200.200.14.252/30200.200.14.253255.255.255.252to R2Hssi 3/1/0 200.200.14.248/30200.200.14.249255.255.255.252to R3

R4 - 7206/200Interface Subnet IP Address Subnet Mask Remarks

Loopback 0 200.200.14.2/32 200.200.14.2 255.255.255.255Fddi 1/0 200.200.5.0/24 200.200.5.2 255.255.255.0 to R2FastEthernet 2/0 200.200.3.0/24 200.200.3.1 255.255.255.0 to Smartbit #3FastEthernet 3/0 200.200.4.0/24 200.200.4.1 255.255.255.0 to Smartbit #4

AS210 (210.210.0.0/20)R2 - 7505/RSP4

Interface Subnet IP Address Subnet Mask RemarksLoopback 0 210.210.14.1/32 210.210.14.1 255.255.255.255Hssi 0/0/0 200.200.14.252/30200.200.14.254255.255.255.252to R1Hssi 1/0/0 210.210.14.252/30210.

Internet QoS Workshop

Documents

Transcript of Internet QoS Workshop