BRKRST-3371 - Advances in BGP (2013 Orlando) - 90 Mins
description
Transcript of BRKRST-3371 - Advances in BGP (2013 Orlando) - 90 Mins
-
Advances in BGP BRKRST-3371
Gunter Van de Velde
Sr. Technical Leader
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What is BGP?
3
Without BGP the Internet would not exist in its current stable and simple form
It is the plumbing technology of the Internet
What a Google search bgp abbreviation finds?
Source: http://www.all-acronyms.com/BGP
Border Gateway Protocol Bacterial Growth Potential Battlegroup Becker, Green and Pearson
Bermuda grass pollen Berri Gas Plant beta-glycerophosphate biliary glycoprotein blood group bone gamma-carboxyglutamic acid protei bone gamma-carboxyglutamic acid-contai bone gla protein bone Gla-containing protein Borders Group, Inc. brain-type glycogen phosphorylase Bridge Gateway Protocol Broader Gateway Protocol Bureau de Gestion de Projet Brain Gain Program
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What is BGP? What it truly is?
4
The Bloody Good Protocol
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Agenda
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
5
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Agenda
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
6
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP started in 1989
Motivation and Development of BGP: When the Internet grew and moved to an autonomous system (AS) mesh architecture it was needed to have stable, non-chatty and low CPU consuming protocol to connect all of these ASs together.
In June 1989, the first version of this new routing protocol was formalized, with the publishing of RFC 1105, A Border Gateway Protocol (BGP).
7
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Service Provider Routing and Services progress
Multimedia, Mobile Internet and Cloud Services will generate massive bandwidth explosion
Prefix growth is almost a linear curve
Evolution of offered BGP services go from basic technologies to very advanced infrastructures
8
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Control-plane Evolution Most of services are progressing towards BGP
9
Service/transport 2008x and before 2013 and future
IDR (Peering) BGP BGP (IPv6)
SP L3VPN BGP BGP + FRR + Scalability
SP Multicast VPN PIM BGP Multicast VPN
DDOS mitigation CLI BGP flowspec
Network Monitoring SNMP BGP monitoring protocol
Security Filters BGP Sec (RPKI), DDoS Mitigation
Proximity BGP connected app API
SP-L3VPN-DC BGP Inter-AS, VPN4DC
Business & CE L2VPN LDP BGP PW Sign (VPLS)
DC Interconnect L2VPN BGP MAC Sign (EVPN)
MPLS transport LDP BGP+Label (Unified MPLS)
Data Center OSPF/ISIS BGP + Multipath
Massive Scale DMVPN NHRP / EIGRP BGP + Path Diversity
Campus/Ent L3VPN BGP (IOS) BGP (NX-OS)
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Why BGP is so successful ?
Robustness: Run over TCP
Low Overhead protocol: sends an update once and then remains silent
Scalability: Path Vector Protocol, allows full mesh
High Availability: NSR, PIC,
Well Known : Tons of engineers know BGP
Simplicity: BGP is simple (even if knobs make BGP BIG and sometimes less trivial to read)
Multi-protocol: IPv4, IPv6, L2VPN, L3VPN, Multicast
Incremental: easy to extend: NLRI,Path Attribute, Community
Flexible: Policy
10
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Agenda
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
11
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Scale & Performance Enhancements
Update Generation Enhancements Update generation is the most important, time-critical task
Is now a separate process, to provide more CPU Quantum
Parallel Route Refresh Significant delay (up to 15-30 minutes) seen in advertising incremental updates while RR is servicing route refresh requests or
converging newly established peers
Refresh and incremental updates run in parallel
Keepalive Enhancements Loosing or delayed keep-alive message result in session flaps
Hence keep-alive processing is now placed into a separate process using priority queuing mechanism
Adaptive Update Cache Size Instead of using a fixed cache size, the new code dynamically adapts to the address family used, the available router memory
and the number of peers in an update group
BGP Scaling
12
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Scale & Performance Enhancements
PE Scaling PE-CE Optimization In old code slow convergence was experienced with large numbers of CEs Improved by intelligently evaluating VPN prefixes based upon the prefixes in the CEs VRF
VRF-Based Advertise Bits Increased memory consumption when number of VRFs was scaled on a PE Smart reuse of advertise bit space for VRF
Route Reflector Scaling Selective RIB Download A Route-Reflector needs to receive the full RIB, however not all prefixes MUST be in the Forwarding Information Base (FIB) So, we now allow by using user policy to only download selected prefixes in the FIB
More about BGP Performance tuning in BRKRST-3321
13
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Resiliency/HA Enhancement
Issue: Slow peers in update groups block convergence of other
update group members by filling message queues/transmitting slowly
Persistent network issue affecting all BGP routers
Two components to solution
Detection
Protection
Detection
BGP update timestamps
Peers TCP connection characteristics
Slow Peer Management
14
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Resiliency/HA Enhancement
Protection
Move slower peers out of update group
Separate slow update group with matching policies created
Any slow members are moved to slow update group
Detection can be automatic or manual with CLI command
Automatic recovery
Slow peers are periodically checked for recovery
Recovered peers rejoin the main update group
Isolation of slow peers unblocks faster peers and lets them converge as fast as possible
Slow Peer Management
15
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Resiliency/HA Enhancement
Static protection [no] neighbor slow-peer split-update-group static
Dynamic detection [no] bgp slow-peer detection [threshold ]
Dynamic protection
[no] neighbor slow-peer detection [threshold ]
[no] bgp slow-peer split-update-group dynamic [permanent]
[no] neighbor slow-peer split-update-group dynamic [permanent]
Slow Peer Management
16
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
ASR1000 RP2, RP1, ASR1001 and 7200 BGP Route and Session Scalability Comparison - RR
Tested with BGP selective download feature for ipv4/ipv6 for dedicated RR application. This feature prevents ipv4/ipv6 BGP routes to be installed in RIB and FIB. It reduces memory usage per ipv4/ipv6 prefix and CPU utilization
ASR 1000 with RP1 allocates ~1.7GB to IOSd, ASR 1001 with 4GB allocates ~1.4GB to IOSd, whereas on NPE-G2 entire 2G is used by IOS
7200 NPE-
G2 (2GB)
ASR1000
RP1 (4GB)
ASR1001
(4GB)
ASR1001
(8GB)
ASR1001
(16GB)
ASR1000
RP2 (8GB)
ASR1000
RP2 (16GB)
ipv4 routes 4M 7M* 2M* 9M* 17M* 12M* 29M*
vpnv4 routes 7M 6M 2M 8M 16M 10M 24M
ipv6 routes 2M 5M* 2M* 8M* 15M* 9M* 24M*
vpnv6 routes 6M 5M 1.5M 7.5M 14.5M 9M 21M
BGP
sessions
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
ASR 1000 RP1 and RP2 Convergence Performance Comparison - RR
Tested with peer groups (1K RR clients per peer group) 7200 NPE-G2 can not converge in the above test cases. ASR1000 RP2 converges about twice faster than 7200 NPE-G2 based on RR customer profile testing CPU utilization below 5% after convergence Link to Isocore report http://www.cisco.com/en/US/prod/collateral/routers/ps9343/ITD13029-ASR1000-RP2Validationv1_1.pdf
Tested with 1M Total Unique
Routes
Total Routes Reflected by RR
to All Clients (Number of
routes x Number of Clients)
ASR1000 RP1 (4GB)
Convergence
(in seconds)
ASR1001 (16GB)
Convergence
(in seconds)
ASR1000 RP2 (16GB)
Convergence
(in seconds)
ipv4 (1K RR clients) 1Billion 220 133 75
vpnv4 (1K RR clients, 8K RT) 1Billion 680 489 221
ipv6 (1K RR clients) 1Billion 720 393 194
vpnv6 (1K RR clients, 8K RT) 1Billion 877 811 293
ipv4 (2K RR clients) 2 Billion 375 270 138
vpnv4 (2K RR clients, 8K RT) 2 Billion 1285 797 394
ipv6 (2K RR clients) 2 Billion 1126 897 284
vpnv6 (2K RR clients, 8K RT) 2 Billion 1766 1691 551
for your reference
18
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Agenda
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
19
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What Happened in XR Landscape?
20
4.0 4.1 4.1.1 4.2 4.2.1 4.2.3 4.2.4 4.3.0 4.3.1
Add Path Support Accumulated
Interior Gateway
Protocol (AIGP)
Metric Attribute
Unipath PIC for non-VPN address-
families
(6PE/IPv6/IPv4
Unicast)
RT-Constraint
BGP Accept Own
Multi-Instance/Multi-AS
BGP 3107 PIC Update for Global Prefixes
Prefix Origin Validation based on RPKI
PIC for RIB and FIB
Attribute Filtering and Error handling
DMZ Link Bandwidth for Unequal Cost Recursive
Load Balancing
Selective VRF Download 6PE/6vPE over L2TPv3 Next-Generation
Multicast VPN
BGP Based DDoS Mitigation
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What Happened in IOS Landscape?
21
15.2(1)S 15.2(2)S 15.2(4)S 15.3(1)S 15.3(2)S
Origin AS Validation
Gracefull Shutdown
iBGP NSR
mVPN BGP SAFI 129
NSR without Route-Refresh
Additional Path
Attribute Filtering and Error Handling
Diverse Path
Graceful Shutdown
IPv6 client for Single hop BFD
IPv6 PIC Core and Edge
RT Constraint
IP Prefix export from a VRF into global Table
mVPNv6 Extranet Support
Local-AS allow-policy
RT/VPN-ID Attribute Rewrite Wildcard
VRF Aware Conditional Announcement
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What Happened in XE Landscape?
22
3.8 3.9
Multicast VPN BGP Dampening
Multiple Cluster IDs
VPN Distinguisher Attribute
IPv6 NSR
Local-AS Allow-policy
RT or VPN-ID Rewrite Wildcard
VRF Aware Conditional Advertisement
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What Happened NXOS Landscape?
23
5.2 6.0 6.1 6.2
Default information originate support
Flexible distance manipulation with
Inject map
Unsupress map
as-format command for AS-plain & AS-dot
Enhancements for removal of private AS
enable route target import-export in default VRF
InterAS option B-lite
BGP Authentication for Prefix-based neighbors
BGP AddPath
BGP send community both
BGP Neighbor AF weight command
BGP med confed and AS multipath-relax
BGP next hop self for route reflector
Prefix Independent Convergence (Core)
local-as
AS Override (allowas-in)
Disable 4-byte AS advertisement
MP BGP MPLS VPNs, 6PE, MDT
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Agenda
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
24
The Bloody Good Protocol
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
PIC Edge Feature Overview
25
Internet Service Providers provide strict SLAs to their Financial and Business VPN customers where they need to offer a sub-second convergence in the case of Core/Edge Link or node failures in their network
Prefix Independent Convergence (PIC) has been supported in IOS-XR/IOS for a while for CORE link failures as well as edge node failures
BGP Best-External project provides support for advertisement of Best-External path to the iBGP/RR peers when a locally selected bestpath is from an internal peer
BGP PIC Unipath provides a capability to install a backup path into the forwarding table to provide prefix independent convergence in case of the PE-CE link failure
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow 10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PIC Edge: PE-CE Link Protection
PE3 configured as primary, PE4 as backup PE3 preferred over PE4 by local preference
CE2 has different RDs in VRFs on PE3 and PE4
PE4: advertise-best-external, to advertise route via PE4-CE2 link
PE3: additional-paths install, to install primary and backup path
BGP Resiliency/HA Enhancement
26
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow 10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PIC Edge: Link Protection
PE3 has primary and backup path
Primary via directly connected PE3-CE2 link
Backup via PE4 best external route
What happens when PE3-CE2 link fails?
BGP Resiliency/HA Enhancement
27
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow 10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PIC Edge: Link Protection
CEF (via BFD or link layer mechanism) detects PE3-CE2 link failure
CEF immediately swaps to repair path label Traffic shunted to PE4 and across PE4-CE2 link
BGP Resiliency/HA Enhancement
28
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow
10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
Withdraw route via PE3
PIC Edge: Link Protection
PE3 withdraws route via PE3-CE2 link
Update propagated to remote PE routers
BGP Resiliency/HA Enhancement
29
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow
10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
Withdraw route via PE3
PIC Edge: Link Protection
BGP on remote PEs selects new bestpath
New bestpath is via PE4
Traffic flows directly to PE4 instead of via PE3
BGP Resiliency/HA Enhancement
30
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow 10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PIC Edge: Edge Node Protection
PE3 configured as primary, PE4 as backup
PE3 preferred over PE4 by local preference
CE2 has different RDs in VRFs on PE3 and PE4
PE4: advertise-best-external, to advertise route via PE4-CE2 link
PE1: additional-paths install, to install primary and backup path
BGP Resiliency/HA Enhancement
31
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow
10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PIC Edge: Edge Node Protection
PE1 has primary and backup path
Primary via PE3
Backup via PE4 best external route
What happens when node PE3 fails?
BGP Resiliency/HA Enhancement
32
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow
10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PE3s /32 host route
removed from IGP
PIC Edge: Edge Node Protection
BGP Resiliency/HA Enhancement
BGP Resiliency/HA Enhancement
33
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
10.1.1.0/24 VPN1 Site #1
MPLS Cloud
Traffic Flow
10.2.2.0/24 VPN1 Site
#2
PE1
PE2
PE3
PE4
CE1 CE2
Primary
Backup
RR
PE3s /32 host route
removed from IGP
PIC Edge: Edge Node Protection
PE1 detects loss of PE3s /32 host route in IGP
CEF immediately swaps forwarding destination label from PE3 to PE4 using backup path
BGP on PE1 computes a new bestpath later, choosing PE4
BGP Resiliency/HA Enhancement
34
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Enabling BGP PIC Enabling IP Routing Fast Convergence
BGP PIC leverages IGP convergence Make sure IGP converges quickly
IOS-XR: IGP Timers pretty-much tuned by default
IOS: Sample OSPF config:
35
process-max-time 50
ip routing protocol purge interface
interface
carrier-delay msec 0
negotiation auto
ip ospf network point-to-point
bfd interval 100 min_rx 100 mul 3
router ospf 1
ispf
timers throttle spf 50 100 5000
timers throttle lsa all 0 20 1000
timers lsa arrival 20
timers pacing flood 15
passive-interface Loopback 0
bfd all-interfaces
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Enabling BGP PIC Edge: IOS-XR
Two BGP-PIC Edge Flavors: BGP PIC Edge Multipath and Unipath
Multipath: Re-routing router load-balances across multiple next-hops, backup next-hops are actively taking traffic, are active in the routing/forwarding plane, commonly found in active/active redundancy scenarios.
No configuration, apart from enabling BGP multipath (maximum-paths ... )
Unipath: Backup path(s) are NOT taking traffic, as found in active/standby scenarios
36
route-policy backup ! Currently, only a single backup path is supported
set path-selection backup 1 install [multipath-protect] [advertise]
end-policy
router bgp ...
address-family ipv4 unicast
additional-paths selection route-policy backup
!
address-family vpnv4 unicast
additional-paths selection route-policy backup
!
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Enabling BGP PIC Edge: IOS
As in IOS-XR, PIC-Edge w/ multipath requires no additional configuration
PIC-Edge unipath needs to be enabled explicitly ...
37
router bgp ...
address-family ipv4 [vrf ...]
or
address-family vpnv4
bgp additional-paths install
http://www.cisco.com/en/US/docs/ios/iproute_bgp/configuration/guide/irg_bgp_mp_pic.html
http://www.cisco.com/en/US/docs/ios/ios_xe/iproute_bgp/configuration/guide/irg_best_external_xe.html
... or implicitly when enabling best external
router bgp ...
address-family ipv4 [vrf ...]
or
address-family vpnv4
bgp advertise-best-external
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Question: How will my PEs learn about the alternate Paths?
By default my RR Only-Reflects the Best-Route
38
RR
PE2
PE3
Z NH:PE3, P:Z
NH:PE2, P:Z
PE1
Prefix Z
Via PE2
E0
E0
Prefix Z
Via E0
Prefix Z
Via E0
NH:PE2, P:Z
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Diverse BGP Path Distribution Shadow Session
Easy deployment no upgrade of any existing router is required, just new iBGP session per each extra path (CLI knob in RR1)
Diverse iBGP session does announce the 2nd best path
39
RR1
NH:PE2, P:Z
NH:PE2, P:Z
Prefix Z
Via PE2
Via PE3 NH:PE3, P:Z
NH:PE3, P:Z
PE2
PE3
Z
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Add-Path
Add-Path will signal diverse paths from 2 to X paths
Required all Add-Path receiver BGP router to support Add-Path capability.
40
RR1 NH:PE2, P:Z AP 1 NH:PE2, P:Z
Prefix Z
Via PE2
Via PE3 NH:PE3, P:Z AP 2
NH:PE3, P:Z
PE2
PE3
Z PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Add-path flavors
IETF defines 5 flavors of Add-x-Path. 2 are implemented by Cisco:
Add-n-path: with add-n-path the route reflector will do best path computation for all paths and send n best to BR/PE.
Usecase: Primary + n-1 Backup scenario. (n is maximal for IOS-XR 2 and 3 for IOS).
Add-all-path: with add-all-path, the route reflector will do the primary best path computation (only on first path) and then send all path to BR/PE.
Usecase: Large DC ECMP load balancing, hot potato routing scenario
Cisco innovation: Add-all-multipath and Add-all-multipath+backup in XR 4.3.1
41
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Add-Path Applications
Fast convergence / connectivity restoration As the ingress routers have visibility to more paths, they can switch to the backup paths faster once the primary path goes away. Requires backup paths to be sent.
Load balancing As the ingress routers have visibility to more paths, they can do ECMP on multiple paths. Requires either backup paths or all paths to be sent.
Churn reduction since alternate paths are available, withdraws can be suppressed (implicit update).
Route oscillation see RFC 3345 for scenarios. Requires group best paths (in some cases all paths) to be sent.
42
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Add-Path Configuration IOS-XR
Enable in global address-family mode Enables for all IBGP neighbors
Enable/Disable in neighbor mode
43
router bgp 100
address-family ipv4 unicast
additional-paths send
!
address-family vpnv4 unicast
additional-paths send
!
neighbor 1.1.1.1
remote-as 100
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
!
neighbor 2.2.2.2
remote-as 100
capability additional-paths send disable
address-family ipv4 unicast
!
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Add-Path Configuration IOS-XR
Enable in global address-family mode Enables for all IBGP neighbors
Enable/Disable in neighbor mode
44
router bgp 100
address-family ipv4 unicast
additional-paths receive
!
address-family vpnv4 unicast
additional-paths receive
!
neighbor 1.1.1.1
remote-as 100
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
!
neighbor 2.2.2.2
remote-as 100
capability additional-paths receive
disable
address-family ipv4 unicast
!
!
!
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Add-Path Configuration IOS-XR
Path selection is configured in a route-policy
Configuration in VPNv4 mode applies to all VRF IPv4-Unicast AF modes unless overridden at individual VRFs
45
route-policy ap1
if community matches-any (1:1) then
set path-selection backup 1 install
elseif destination in (150.0.0.0/16, 151.0.0.0/16) then
set path-selection backup 1 advertise install
endif
end-policy
!
route-policy ap2
set path-selection all advertise
end-policy
!
route-policy ap3
set path-selection backup 1 install
end-policy
!
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Add-Path Configuration IOS-XR
Add-Path Path Selection
46
router bgp 100
address-family ipv4 unicast
additional-paths selection route-policy ap1
!
address-family vpnv4 unicast
additional-paths selection route-policy ap2
!
vrf foo
rd 1:1
address-family ipv4 unicast
additional-paths selection route-policy ap3
!
!
vrf bar
rd 2:2
address-family ipv4 unicast
!
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
PIC Edge: Test Results
Test Setup Node Failure Link Failure
No PIC Edge, No BFD 12-14 sec 8-17 sec
BFD Only 10-12 sec 6-12 sec
PIC Edge Only 8 sec 4 sec
PIC Edge, BFD 0 sec 0 sec
BGP Resiliency/HA Enhancement
47
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Automated Route Target Filtering
Increased VPN service deployment increases load on VPN routers 10% YOY VPN table growth
Highly desirable to filter unwanted VPN routes
Multiple filtering approaches New RT filter address family
Extended community ORF
BGP Feature
48
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Automated Route Target Filtering
Derive RT filtering information from VPN RT import lists automatically
Exchange filtering info via RT filter AF or extended community ORF
Translate filter info received from neighbors into outbound filtering policies
Generate incremental updates for received RT update queries
Incremental deployment possible/desirable
49
BGP Feature
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
PE-1
PE-2
PE-3
PE-4
RR-1 RR-2
VRF- Blue
VRF- Red
VRF- Red
VRF- Green
RT-Constraint:
NLRI= {VRF-Blue, VRF-Red}
RT-Constraint:
NLRI= {VRF-Green, VRF-Purple}
RT-Constraint:
NLRI= {VRF-Purple, VRF-Blue}
RT-Constraint:
NLRI= {VRF-Red, VRF-Green}
RT-Constraint:
NLRI= {VRF-Blue, VRF-Red, VRF-Green}
RT-Constraint:
NLRI={VRF-Green, VRF-Purple, VRF-Blue}
VRF- Green
VRF- Purple
VRF- Purple
VRF- Blue
Automated Route Target Filtering
50
Improves PE and RR scaling and performance by sending only relevant VPN routes
router bgp as-number
address-family rtfilter unicast
neighbor {ip-address | peer-group-name} activate
neighbor {ip-address | peer-group-name} send-community extended
end
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Accept own
51
This feature allows movement from a PE-Based service provisioning model to a centralized router reflector (RR)-based service provisioning model. With this feature, you can define route TO service-VRF mapping within a centralized route reflector and then propagate this information down to all the PE clients of that RR. Without this feature, you would define the route TO service VRF mapping in all PE devices, thereby incurring a high configuration overhead, which could result in more errors.
This feature enables a route reflector to modify the Route Target (RT) list of a VPN route that is distributed by the route reflector, enabling the route reflector to control how a route originated within one VRF is imported into other VRFs.
router#configure
router(config)#router bgp 100
router(config-bgp)#neighbor 10.2.3.4
router(config-bgp-nbr)#address-family vpnv4 unicast
router(config-bgp-nbr-af)#accept-own
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
AIGP (Accumulated IGP Metric Attribute for BGP)
http://tools.ietf.org/html/draft-ietf-idr-aigp-09
Optional, non-transitive BGP path attribute
BGP attribute to provide BGP a way to make its routing decision based on the IGP metric, to choose the shortest path between two nodes across different AS.
The main driving force for this feature is to solve the IGP scale issue seen in some ISP core network.
Mainly to be deployed to carry nexthop prefixes/labels across different AS within the same administrative domain.
The remote ingress PE select its best path using the modified best path selection process using AIGP metric.
Overview AIGP
52
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Overview AIGP
Sending/Receiving AIGP attribute Per-session configuration Enabled for iBGP session by default Disabled for eBGP session by default, a knob to enable the AIGP
capability
AIGP attribute received on an AIGP-disabled sessions should be treated as an unrecognized non-transitive attribute.
Origination of AIGP metric By configuration
Redistribution IGP or static BGP network Inbound/outbound policy
for your reference
53
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Overview AIGP
Modification of AIGP attribute By Originator
A new BGP update should be issued
Configurable threshold to minimize IGP instability not in 4.0 By non-originator
When NH is not changed no change for the AIGP attribute value
When NH is changed to non-recursive IGP or static route increase the AIGP attribute value by the NH distance
When NH is changed to recursive BGP-learned or static route increase the AIGP attribute value by recursively resolving and increasing the AIGP attribute value of the NHs until either the NH is non-recursive or the NH is a BGP route without AIGP attribute
AIGP value change triggers new AIGP computation for the route AIGP carried across different AS with different IGP domain may not offer a
meaningful result.
for your reference
54
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Overview AIGP
Modified best path calculation Modifications in the tie breaking procedures Changes made after local_preference comparison When a route has AIGP attribute
Remove from considering routes without AIGP attribute
- this can be overruled by configuring a knob
Compare routes of the cumulative AIGP value When the NH has AIGP attribute
Compute the interior cost as the cumulative AIGP value for the NH
Compare routes using the modified IGP cost
Update generation Different update groups for neighbors of AIGP-capable, non-AIGP capable or
neighbors enabled to send AIGP value in cost-community.
BGP update is generated upon AIGP value change
for your reference
55
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Overview AIGP
Passing AIGP attribute to non-AIGP capable neighbors Translate AIGP into cost-community 2 POI of pre-best-path and igp-cost are supported A transitive keyword to make cost-comm transitive to eBGP neighbors
Redistribute BGP (with AIGP) into IGP Translate AIGP value into BGP MED
Other software components Route installation for BGP to tag AIGP metric during route installation NH notification when AIGP metric changed
Update generation throttling is not supported in XR4.0
It is highly recommended to deploy BGP best-external and Additional-path in conjunction with the AIGP attribute, to effectively achieve the desired routing policy.
56
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
AIGP: Originating AIGP
router bgp 1
address-family ipv4 unicast
redistribute ospf 1 route-policy set_aigp_1
route-policy set_aigp_1
if destination in (61.1.1.0/24 le 32) then
set aigp-metric 111
elseif destination in (2100::1:0/112,
2100::2:0/112) then
set aigp-metric igp-cost
Endif
end-policy
AIGP is enabled between iBGP neighbors by default
AIGP between eBGP neighbors need to be enabled
AIGP can be originated by using redistribute ospf, redistribute isis, redistribute static or the BGP network command.
AIGP can also be originated using neighbor address-family inbound or outbound policy to set AIGP to be the IGP cost or to a fixed value.
for your reference
57
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
AIGP capability verification #1:
RP/0/0/CPU0:router-RR#show bgp neighbor 110.33.33.3 BGP neighbor is 110.33.33.3 Remote AS 1, local AS 1, internal link Remote router ID 110.30.30.3 Cluster ID 110.50.50.5 BGP state = Established, up for 3w4d NSR State: NSR Ready Last read 00:00:24, Last read before reset 00:00:00 Hold time is 180, keepalive interval is 60 seconds Configured hold time: 180, keepalive: 60, min acceptable hold time: 3 Last write 00:00:55, attempted 19, written 19 Second last write 00:01:55, attempted 19, written 19 Last write before reset 00:00:00, attempted 0, written 0 Second last write before reset 00:00:00, attempted 0, written 0 Last write pulse rcvd Aug 6 11:48:49.296 last full Jul 12 12:05:24.042 pulse count 72908 Last write pulse rcvd before reset 00:00:00 Socket not armed for io, armed for read, armed for write Last write thread event before reset 00:00:00, second last 00:00:00 Last KA expiry before reset 00:00:00, second last 00:00:00 Last KA error before reset 00:00:00, KA not sent 00:00:00 Last KA start before reset 00:00:00, second last 00:00:00 Precedence: internet Non-stop routing is enabled Graceful restart is enabled Restart time is 120 seconds Stale path timeout time is 360 seconds
For Address Family: IPv4 Unicast BGP neighbor version 34101 Update group: 0.3 Route-Reflector Client AF-dependent capabilities: Graceful Restart capability advertised and received Neighbor preserved the forwarding state during latest restart Local restart time is 120, RIB purge time is 600 seconds Maximum stalepath time is 360 seconds Remote Restart time is 120 seconds Additional-paths Send: advertised Additional-paths Receive: advertised and received Route refresh request: received 0, sent 0 0 accepted prefixes, 0 are bestpaths Cumulative no. of prefixes denied: 0. Prefix advertised 31470, suppressed 0, withdrawn 3525 Maximum prefixes allowed 524288 Threshold for warning message 75%, restart interval 0 min AIGP is enabled An EoR was received during read-only mode Last ack version 34101, Last synced ack version 34101 Outstanding version objects: current 0, max 4 Additional-paths operation: Send
Neighbor capabilities: Route refresh: advertised and received Graceful Restart (GR Awareness): received 4-byte AS: advertised and received Address family IPv4 Unicast: advertised and received Address family IPv4 Labeled-unicast: advertised and received Address family VPNv4 Unicast: advertised and received Address family IPv6 Labeled-unicast: advertised and received Address family VPNv6 Unicast: advertised and received Received 36025 messages, 0 notifications, 0 in queue Sent 42771 messages, 0 notifications, 0 in queue Minimum time between advertisement runs is 0 secs
for your reference
58
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
RP/0/1/CPU0:olympic-12c-lr1#sh bgp 61.1.1.0/24 bestpath-compare BGP routing table entry for 61.1.1.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 31709 31709
Last Modified: Aug 6 06:05:44.392 for 00:26:12
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
Local
110.11.11.1 (metric 2) from 110.55.55.5 (110.10.10.1)
Origin incomplete, metric 3, localpref 100, aigp metric 111, valid, internal, best, group-best
Received Path ID 1, Local Path ID 1, version 31709
Originator: 110.10.10.1, Cluster list: 110.50.50.5
best of local AS, Overall best
Path #2: Received by speaker 0
Not advertised to any peer
Local
110.22.22.2 (metric 2) from 110.55.55.5 (110.20.20.2)
Origin incomplete, metric 3, localpref 100, aigp metric 211, valid, internal, backup, add-path
Received Path ID 3, Local Path ID 3, version 31709
Originator: 110.20.20.2, Cluster list: 110.50.50.5
Higher AIGP metric than best path (path #1)
AIGP metric verification #2: receive route with AIGP metric from RR best-path calculation considered AIGP metric
RP/0/1/CPU0:olympic-12c-lr1#sh route 61.1.1.0/24
Routing entry for 61.1.1.0/24
Known via "bgp 1", distance 200, metric 113 (AIGP metric)
Number of pic paths 1 , type internal
Installed Aug 6 06:05:44.152 for 00:33:50
Routing Descriptor Blocks
110.11.11.1, from 110.55.55.5
Route metric is 113
110.22.22.2, from 110.55.55.5, BGP backup path
Route metric is 113
No advertising protos.
for your reference
59
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What is Multi-Instance BGP?
60
A new IOS-XR BGP architecture to support multiple instances along the lines of OSPF instances
Each BGP instance is a separate process running on the same or a different RP/DRP node
The BGP instances do not share any prefix table between them
No need for a common adj-rib-in (bRIB) as is the case with distributed BGP
The BGP instances do not communicate with each other and do not set up peering with each other
Each individual instance can set up peering with another router independently
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What is Multi-AS BGP?
61
It will be possible to configure each instance of a multi-instances BGP with a different AS number
Global address families cant be configured under more than one AS except vpnv4 and vpnv6
VPN address-families may be configured under multiple AS instances that do not share any VRFs
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Why Multi-Instance/Multi-AS?
It provides a mechanism to consolidate the services provided by multiple routers using a common routing infrastructure into a single IOS-XR router
It provides a mechanism to achieve AF isolation by configuring the different AFs in different BGP instances
It provides a means to achieve higher session scale by distributing the overall peering sessions between multiple instances
It provides a mechanism to achieve higher prefix scale (especially on a RR) by having different instances carrying different BGP tables
IOS-XR CRS Multi-chassis systems can be used optimally by placing the different BGP instances on different RP/DRPs
It is the base of Ciscos SP DDoS Mechanism
62
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Configuration Example for your reference
63
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Show Command Example RP/0/0/CPU0:ios#sh bgp instances
Number of BGP instances: 4
ID Placed-Grp Name AS VRFs Address Families
--------------------------------------------------------------------------------
0 v4_routing ipv4 1 0 IPv4 Unicast
1 bgp2_1 ipv6 1 0 IPv6 Unicast
2 bgp3_1 vpn1 3 1 VPNv4 Unicast
3 bgp4_1 vpn2 3 1 VPNv4 Unicast
RP/0/0/CPU0:ios#sh bgp instance ?
WORD Specify the bgp instance name
all Choose all BGP instances
RP/0/0/CPU0:ios#sh bgp instance all ?
A.B.C.D IPv4 network
A.B.C.D/length IPv4 network and masklength
advertised Show advertised routes
af-group Show config information on address family groups
all Both ipv4 and ipv6 address families
attribute-key Display networks with their associated attribute key index
cidr-only Display only routes with non-natural netmasks
community Display routes matching the communities
convergence Test an address family for convergence
for your reference
64
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Show Command Example
RP/0/0/CPU0:ios#sh bgp instance all sessions
Wed Sep 28 20:45:56.917 PDT
BGP instance 0: 'ipv4'
======================
Neighbor VRF Spk AS InQ OutQ NBRState NSRState
10.0.101.1 default 0 1 0 0 Established -
BGP instance 1: 'ipv6'
======================
Neighbor VRF Spk AS InQ OutQ NBRState NSRState
10.0.101.2 default 1 1 0 0 Established -
BGP instance 2: 'vpn1'
======================
Neighbor VRF Spk AS InQ OutQ NBRState NSRState
20.0.101.1 default 2 200 0 0 Established -
BGP instance 3: 'vpn2'
======================
Neighbor VRF Spk AS InQ OutQ NBRState NSRState
20.0.101.2 default 3 200 0 0 Established -
for your reference
65
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Attribute Filtering and error-handling
Attribute filtering Unwanted optional transitive attribute such as ATTR_SET, CONFED segment in
AS4_PATH causing outage in some equipments.
Prevent unwanted/unknown BGP attributes from hitting legacy equipment Block specific attributes Block a range of non-mandatory attributes
Error-handling draft-ietf-idr-optional-transitive-04.txt
Punishment should not exceed the crime
Gracefully fix or ignore non-severe errors
Avoid session resets for most cases
Never discard update error, as that can lead to inconsistencies
66
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Architecture
67
Invalid
Attribute Contents
Wrong Attribute
Length Unknown Attributes Unwanted Attributes
Malformed BGP Updates Transitive Attributes
Attribute Filtering
Error-handling
NLRI processing
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Attribute filtering
First level of inbound filtering
Filtering is configured as a range of attribute codes and a corresponding action to take (Note: Never Discard Update as that can lead towards inconsistencies)
Actions Discard the attribute
Treat-as-withdraw
Applied when parsing each attribute in the received Update message When a attribute matches the filter, further processing of the attribute is stopped and
the corresponding action is taken
68
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Error-handling
Comes into play after attribute-filtering is applied
When we detect one or more malformed attributes or NLRIs or other fields in the Update message
Steps Classification of errors
Actions to be taken
Logging
69
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Error-handling details
Classification of errors Minor: invalid flags, zero length, duplicates, optional-transitive attributes
Medium: Non-optional-transitive attributes, inconsistent attribute length
Major: Invalid or 0 length nexthop
Critical: NLRI parsing, inconsistent message / total attributes length
Actions taken Local repair
Discard attribute
Treat-as-withdraw
Reset session
Discard Update message
70
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Support client functionality of RPKI RTR protocol Separate database to store record entries from the cache
Support to announce path validation state to IBGP neighbors using a well known path validation state extended community
Modified route policies to incorporate path validation states
BGP Origin Validation
71
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Prefix hijacking
Announce someone elses prefix
Announce a more specific of someone elses prefix
Either way, you are trying to steal someone elses traffic by getting it routed to you
Capture, sniff, redirect, manipulate traffic as you wish
72
Source: nanog 46 preso
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does the Solution look like?
73
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Configuration sample
router bgp 64726
bgp always-compare-med
bgp log-neighbor-changes
bgp deterministic-med
no bgp default ipv4-unicast
bgp rpki server tcp 217.193.137.117 port 30000 refresh 60
bgp rpki server tcp 2001:918:FFF9:0:250:56FF:FE15:159 port 8282 refresh 60
bgp rpki server tcp 2001:918:FFF9:0:250:56FF:FE15:159 port 30000 refresh 60
bgp rpki server tcp 217.193.137.117 port 8282 refresh 600
neighbor 2001:428:7000:A:0:1:0:1 remote-as 64209
neighbor 2001:428:7000:A:0:1:0:1 description "To Qwest MPLS"
74
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Valid vs Unknown vs Invalid routes?
JSV-ASR#sho bgp sum
BGP router identifier 66.77.8.142, local AS number 64726
BGP table version is 11688639, main routing table version 11688639
Path RPKI states: 38286 valid, 1574331 not found, 4558 invalid
404300 network entries using 59836400 bytes of memory
1617175 path entries using 103499200 bytes of memory
66778/66761 BGP path/bestpath attribute entries using 9081808 bytes of memory
62642 BGP AS-PATH entries using 2273670 bytes of memory
1347 BGP community entries using 70456 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 174761534 total bytes of memory
808583 received paths for inbound soft reconfiguration
BGP activity 744131/330548 prefixes, 7084275/5448612 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
63.231.216.9 4 64726 17784 17789 11688639 0 0 1d01h 3
65.119.97.101 4 64209 0 0 1 0 0 16:57:38 Idle (Admin)
66.77.8.129 4 209 216390 4021 11688634 0 0 2d12h 404293
66.77.8.130 4 209 212278 4020 11688634 0 0 2d12h 404290
66.77.8.150 4 64726 70180 227968 11688639 0 0 1d16h 3
JSV-ASR#
75
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
What do you see in the BGP table?
JSV-ASR#sho bgp
BGP table version is 11698585, local router ID is 66.77.8.142
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
V*> 0.0.0.0/1 0.0.0.0 0 32768 i
V* i 66.77.8.150 0 100 100 i
N* 0.0.0.0 66.77.8.130 0 1000 209 i
N*> 66.77.8.129 0 1000 209 i
N* 1.0.0.0/24 66.77.8.130 7800038 1000 209 15169 i
N*> 66.77.8.129 7800038 1000 209 15169 i
N* 1.0.4.0/22 66.77.8.130 8000039 1000 209 4323 7545 7545 7545 7545 56203
i
N*> 66.77.8.129 8000039 1000 209 4323 7545 7545 7545 7545 56203
i
N* 1.0.16.0/23 66.77.8.130 8000039 1000 209 2914 2519 i
N*> 66.77.8.129 8000039 1000 209 2914 2519 i
N* 1.0.18.0/23 66.77.8.130 8000039 1000 209 2914 2519 i
N*> 66.77.8.129 8000039 1000 209 2914 2519 i
N* 1.0.20.0/23 66.77.8.130 8000039 1000 209 2914 2519 i
76
for your reference
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Multicast VPN Solution Space
(complete solution is now available)
77
LSM Encapsulation
/Forwarding IP/GRE
P2MP TE (pt-mpt)
PIM (pt-mpt)
Core Tree
Signaling MLDP
(pt-mpt | mpt-mpt)
mVPN
IPv4
Native
IPv6
mVPN
IPv6
Service Native
IPv4
BGP PIM C-Multicast
Signaling
PORT
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP customer-multicast signaling and BGP
auto-discover is now added to the multicast
VPN solution.
Multicast VPN BGP Signaling
BGP as overlay allows Service Providers to
capitalize on a single protocol
Auto-Discovery of PEs and
Core tree/tunnel information
PE1
PE2
PE3
PE4
CE1 CE3
RR
Receiver Source
CE4 Receiver CE2
RP
BGP Auto-Discovery
BGP C-mroutes
PIM C-Join
(*,G) or (S,G)
PIM C-Join
(*,G) or (S,G)
Advertisement of Customer
Multicast routes
BGP
78
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP Graceful Shutdown
RFC 6198 April 2011
Old Behaviour If session drops then BGP will
withdraw all prefixes learned over that session
BGP has no mechanism to signal prefix will soon be unreachable (for maintenance for example)
Historically RRs have worsened the issue as they tend to hide the alternate path as they only forward the best path
79
BGP Graceful Shutdown allows to do maintenance on router without service disruption.
This new knob allows a router to notify neighbor to redirect traffic to other paths and after some time will drop BGP sessions.
The notification could be done using Local Preference attribute
or user community attribute
#Graceful Shutdown
Please wait
BGP/ Prefix 10.45 / localpref : 10
1 2
Traffic is
redirected
3
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Graceful Shutdown
GSHUT well-known community
The GSHUT community attribute is applied to a neighbor specified by the neighbor shutdown graceful command, thereby gracefully shutting down the link in an expected number of seconds
The GSHUT community is specified in a community list, which is referenced by a route map and then used to make policy routing decisions.
80
http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_bgp/configuration/15-s/irg-15-s-book.pdf
neighbor {ipv4-address | ipv6-address | peer-group-name} shutdown graceful seconds {community
value [local-preference value] | local-preference value}
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS Mitigation a stepstone approach
Phase III
Dynamic application aware redirection and traffic handling
Phase II
Malicious traffic mitigation
Cleaning of Malicious traffic
Dirty and clean traffic handling
Usage of Multi-instance BGP
Phase I
ACL
RTBH
PBR
uRPF
IOS-XR 4.3.1
IOS-XE partial
81
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS Overview
Distributed denial-of-service (DDoS) attacks target network infrastructures or computer services by sending overwhelming number of service requests to the server from many sources.
Server resources are used up in serving the fake requests resulting in denial or degradation of legitimate service requests to be served
Addressing DDoS attacks
Detection Detect incoming fake requests
Mitigation Diversion Send traffic to a specialized device that removes the fake packets from the traffic stream while retaining the legitimate packets
Return Send back the clean traffic to the server
82
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDOS impact on Customer Business
83
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDOS impact on customer Business
Enterprise customer cant defend themselve, when DDoS hit the FW its already too late.
SP could protect enterprise by cleaning DDoS traffic at ingress peering point.
New revenue for SP.
Mandated service to propose to Financial and visible customers.
for your reference
84
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS trends (Nanog source)
Any Internet Operator Can Be a Target for DDoS
Ideologically-motivated Hacktivism and On-line vandalism DDoS attacks are the most commonly identified attack motivations
Size and Scope of Attacks Continue to Grow at an Alarming Pace
High-bandwidth DDoS attacks are the new normal as over 40% of respondents report attacks greater than 1 Gbps and 13% report attacks greater than 10Gbps
Increased sophistication and complexity of layer-7 DDoS attacks, multi-vector DDoS attacks becoming more common
First-Ever Reports of IPv6 DDoS Attacks 'in the Wild' on Production Networks
for your reference
85
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS mitigation architecture 1. Detection (no DDoS)
DDOS
scrubber
Security
Server
DDOS
Analyser
Sample
Netflow
Scan Netflow data
to detect DDOS attacks
86
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS mitigation architecture 2. Detection (DDOS)
DDOS
scrubber
Security
Server
DDOS
Analyser
Sample
Netflow
Scan Netflow data
Find DDOS signature
87
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS mitigation architecture 3. Redirect traffic to DDOS scruber
DDoS
scrubber
Security
Server
DDoS
Analyser
Scan Netflow data
Find DDoS signature
BGP DDoS Mitigation
Action: redirect to DDoS
scrubber
88
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
DDoS Mitigation: Architecture Considerations
Normal traffic flow when there is no attack
Redirect traffic from any edge PE to any specific DDoS scrubber Including the PE that is connected to the host network
Granular (prefix level/network) diversion Customers buy DDoS mitigation service for some prefixes
Pre-provisioned DDoS service for those prefixes (using policy such as standard community flag)
Centralized controller that injects the diversion route
VPN based Labeled return path for the clean traffic To prevent routing loops
Solution support redirection of BGP less/more specific prefixes or local originated prefixes (static route, redistributed route)
Support for multi-homed customers During attack, send clean traffic from DDOS scrubber to multiple PEs
89
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
The concept
90
Traffic under normal conditions
Internet users
Traffic under normalized
conditions Traffic takes shortest path Upstream and downstream traffic follow
traditional routing
Server
Scrubber
ISP
Pre-provisioned DDoS
instrumentation Traffic Scrubber
Separate clean and malicious traffic
Security Analyser Analyses Netflow/IPFIX statistics from the
traffic flows
Security server Actions upon traffic analysis by
communication to infrastructure routers
Security analyser
Security server
PE3
PE2
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
BGP based DDoS
91
Traffic under DDoS condition
Internet users
Traffic under DDoS condition Traffic is redirected to a scrubber Scrubber separates the clean from
the malicious traffic
Clean traffic is returned to original destination server
Goal Do not drop all traffic Collect traffic intelligence Operational simplicity Easy to remove redirect when traffic
normalizes
Server Scrubber
ISP
Security analyser
Security server
PE3
PE2
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
92
Normal traffic condition
Internet users
All PEs peer with the RR All PEs exchange both Global
Internet and VPN prefixes
All PE interfaces are non-VPN Security analyser is performing
doing analyses
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security analyser
Security server
Destination Next-hop
1.1.1.1/32 2.2.2.2
PE3
PE2
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
93
Server is under DDoS
Internet users
Flow is detected as dirty by Security analyser
Result: Server is under attack Traffic needs to be redirected to the
scrubber to mitigate the attack
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security analyser
Security server
Destination Next-hop
1.1.1.1/32 2.2.2.2
PE3
PE2
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
94
Internet users
DDoS Route-Reflector was pre-visioned
Mitigation route to 1.1.1.1/32 is injected on the DDoS RR by the
Security server
Mitigation route to 1.1.1.1/32 is pointing to 3.3.3.3 on DDoS
mitigation RR
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security server
DDoS
Route-Reflector
6.6.6.6
Destination Next-hop
1.1.1.1/32 3.3.3.3
PE3
PE2
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
95
Internet users
Mitigation route to 1.1.1.1/32 is pointing to 3.3.3.3 is signalled to all
PEs All PEs receive the mitigation route
from the DDoS Mitigation RR
Each PE will now have 2 routes to reach 1.1.1.1/32
Which route will the PE use?
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security server
DDoS
Route-Reflector
6.6.6.6
Destination Next-hop
1.1.1.1/32 3.3.3.3
Destination Next-hop
1.1.1.1/32 2.2.2.2
1.1.1.1/32 3.3.3.3
Destination Next-hop
1.1.1.1/32 ????????????
BGP Table Routing Table
PE3
PE2
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
96
Internet users
Trick # 1 The DDoS mitigation route will
ALWAYS be preferred, even if
Both prefix lengths are the same
DDoS prefix is shorter Original prefix has better
administrative distance
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security server
DDoS
Route-Reflector
6.6.6.6
Destination Next-hop
1.1.1.1/32 2.2.2.2
1.1.1.1/32 3.3.3.3
Destination Next-hop
1.1.1.1/32 3.3.3.3
Routing Table BGP Table
PE3
PE2
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
97
Internet users
The mitigated traffic flows towards PE3 (3.3.3.3)
PE3 is sending the dirty flow towards the scrubber
The scrubber will Handle and remove the dirty
traffic within the original flow
Send the cleaned traffic towards the original destination
(1.1.1.1 at PE2 (2.2.2.2))
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
DDoS
Route-Reflector
6.6.6.6
Destination Next-hop
1.1.1.1/32 2.2.2.2
1.1.1.1/32 3.3.3.3
Destination Next-hop
1.1.1.1/32 3.3.3.3
Routing Table BGP Table
PE3
PE2
Clean
traffic
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
98
Internet users
Problem Scrubber sends traffic to PE3 PE3 does routing lookup for 1.1.1.1
and finds that it is directly attached
ROUTING LOOP!!! How do we fix this?
We use a new isolated routing table for the clean traffic
This routing table is Pre-provisioned Inside a VPN
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
DDoS
Route-Reflector
6.6.6.6
Destination Next-hop
1.1.1.1/32 2.2.2.2
1.1.1.1/32 3.3.3.3
Destination Next-hop
1.1.1.1/32 3.3.3.3
Routing Table BGP Table
PE3
PE2
Clean
traffic
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
99
Internet users
Server Scrubber
ISP
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
Destination Next-hop
1.1.1.1/32 2.2.2.2
1.1.1.1/32 3.3.3.3
Destination Next-hop VPN
1.1.1.1/32 3.3.3.3 Global
1.1.1.1/32 2.2.2.2 Clean
Routing Table BGP Table
PE3
PE2
The clean traffic will be injected upon PE3 on an interface member of VPN Clean
PE3 will now do a routing destination lookup for 1.1.1.1 in VPN Clean
The matching routing table entry is pointing towards PE2 at 2.2.2.2
The clean flow, which is now part of VPN Clean is sent towards PE2 reachable at
2.2.2.2
VPN Clean
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
100
Internet users
Server Scrubber
ISP
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
PE3
PE2 CE1
Destination Next-hop VPN
1.1.1.1/32 3.3.3.3 Global
1.1.1.1/32 CE1 Clean
Routing Table PE2 receives the clean flow within VPN clean
PE2 does a destination address routing lookup in VPN clean
A matching route is found in VPN clean
Flow is forwarded towards CE1 onwards to Server
HOLD on a minute! PE2 does not have any interface part of VPN clean
All interfaces on PE2 are global interfaces
so how did that clean route for 1.1.1.1 get into VPN
clean?
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
How does it work?
101
Internet users
Server Scrubber
ISP
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
Destination Next-
hop
VPN
1.1.1.1/32 CE1 Global
1.1.1.1/32 3.3.3.3 Global
1.1.1.1 CE1 clean
BGP Table
PE3
PE2 CE1
Trick # 2 Copy the locally BGP inserted route
directly into VPN clean BGP table
Neighbour details are inherited from the global table (i.e.)
Outgoing interface Next-hop
Interface pointing towards CE1 is NOT VPN aware
This VPN clean distributed as normal VPN
New CLI command to do that import from default-vrf route-policy ddos
advertise-as-vpn
Destination Next-hop VPN
1.1.1.1/32 3.3.3.3 Global
1.1.1.1/32 CE1 Clean
Routing Table
PE1
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Going back to traditional traffic flow
102
Internet users
Remove the routing entry on the Mitigation DDoS RR
No more route is remaining on the DDoS Mitigation RR
Traffic flows normally again
Server Scrubber
ISP
Internet and VPN
Route-Reflector
1.1.1.1/32
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
Security server
DDoS
Route-Reflector
5.5.5.5
Destination Next-hop
1.1.1.1/32 3.3.3.3
PE1
Server is under DDoS
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Configuration (1)
router bgp 99 instance ddos
bgp router-id 3.3.3.3
bgp read-only
bgp install diversion
address-family ipv4 unicast
!
router bgp 99
bgp router-id 2.2.2.2
address-family ipv4 unicast
!
Creation of DDoS BGP
instance
Allows config of 2th IPv4 or IPv6 instance
Suppresses BGP Update Generation
Triggers BGP ddos instance to install
diversion path to RIB, so that the paths
are pushed down to FIB
for your reference
103
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Configuration (2)
vrf clean
address-family ipv4 unicast
import from default-vrf route-policy ddos advertise-as-vpn
export route-target
111:1
!
!
address-family ipv6 unicast
import from default-vrf route-policy ddos advertise-as-vpn
export route-target
111:1
!
!
!
Importing the global routes in the clean VRF
104
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
show commands
RP/0/0/CPU0:hydra-prp-A#show route Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, su - IS-IS summary null, * - candidate default U - per-user static route, o - ODR, L - local, G - DAGR A - access/subscriber, a - Application route, (!) - FRR Backup path Gateway of last resort is not set O 1.0.11.0/24 [110/2] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 O 1.1.1.1/32 [110/2] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 L 2.2.2.2/32 is directly connected, 00:37:24, Loopback0 O 3.3.3.3/32 [110/2] via 87.0.1.2, 00:36:19, GigabitEthernet0/2/1/9 O 4.4.4.4/32 [110/3] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 [110/3] via 87.0.1.2, 00:36:19, GigabitEthernet0/2/1/9 B 5.5.5.5/32 [200/0] via 1.1.1.1, 00:34:22 B > [200/0] via 123.0.0.2, 00:34:22 [...]
for your reference
105
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
show commands (1)
RP/0/0/CPU0:hydra-prp-A#show route Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, su - IS-IS summary null, * - candidate default U - per-user static route, o - ODR, L - local, G - DAGR A - access/subscriber, a - Application route, (!) - FRR Backup path Gateway of last resort is not set O 1.0.11.0/24 [110/2] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 O 1.1.1.1/32 [110/2] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 L 2.2.2.2/32 is directly connected, 00:37:24, Loopback0 O 3.3.3.3/32 [110/2] via 87.0.1.2, 00:36:19, GigabitEthernet0/2/1/9 O 4.4.4.4/32 [110/3] via 13.0.3.1, 00:36:19, GigabitEthernet0/2/1/5 [110/3] via 87.0.1.2, 00:36:19, GigabitEthernet0/2/1/9 B 5.5.5.5/32 [200/0] via 1.1.1.1, 00:34:22 B > [200/0] via 123.0.0.2, 00:34:22 [...]
for your reference
106
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
show commands (2) RP/0/0/CPU0:hydra-prp-A#show route 5.5.5.5/32
Routing entry for 5.5.5.5/32
Known via "bgp 2394-ro", distance 200, metric 0, type internal
Installed Feb 19 22:56:45.896 for 00:34:33
Routing Descriptor Blocks
1.1.1.1, from 1.1.1.1
Route metric is 0
123.0.0.2, from 101.0.0.4, Diversion Path (bgp)
Route metric is 0
No advertising protos.
RP/0/0/CPU0:hydra-prp-A#show cef 5.5.5.5/32 det
5.5.5.5/32, version 60652, internal 0x14000001 (ptr 0xaf6e3840) [1], 0x0 (0x0), 0x0 (0x0)
Updated Feb 19 22:56:46.723
local adjacency 87.0.1.2
Prefix Len 32, traffic index 0, precedence n/a, priority 4
gateway array (0xae07a310) reference count 2, flags 0x8020, source rib (5), 0 backups
[1 type 3 flags 0xd0141 (0xae10f8c0) ext 0x420 (0xaec261e0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
via 123.0.0.2, 2 dependencies, recursive [flags 0x6000]
path-idx 0 [0xaf6e3c00 0x0]
next hop 123.0.0.2 via 123.0.0.0/24
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/2/1/9 87.0.1.2
for your reference
107
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
show commands (3)
RP/0/0/CPU0:hydra-prp-A# show route 123.0.0.2
Routing entry for 123.0.0.0/24
Known via "ospf 100", distance 110, metric 2, type intra area
Installed Feb 19 22:54:48.363 for 00:39:01
Routing Descriptor Blocks
87.0.1.2, from 3.3.3.3, via GigabitEthernet0/2/1/9
Route metric is 2
No advertising protos.
RP/0/0/CPU0:hydra-prp-A#
RP/0/0/CPU0:hydra-prp-A#show route 1.1.1.1
Routing entry for 1.1.1.1/32
Known via "ospf 100", distance 110, metric 2, type intra area
Installed Feb 19 22:54:49.259 for 00:49:20
Routing Descriptor Blocks
13.0.3.1, from 1.1.1.1, via GigabitEthernet0/2/1/5
Route metric is 2
No advertising protos.
for your reference
108
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Summary
109
Bloody Good Protocol
Motivation to Enhance BGP
Scale and Performance Enhancements
What happened in BGP Landscape?
Some new Cool features that may interest you
-
110
-
2013 Cisco and/or its affiliates. All rights reserved. BRKRST-3371 Cisco Public
Maximize your Cisco Live experience with your
free Cisco Live 365 account. Download session
PDFs, view sessions on-demand and participate in
live activities throughout the year. Click the Enter
Cisco Live 365 button in your Cisco Live portal to
log in.
Complete Your Online Session Evaluation
Give us your feedback and you could win fabulous prizes. Winners announced daily.
Receive 20 Cisco Daily Challenge points for each session evaluation you complete.
Complete your session evaluation online now through either the mobile app or internet kiosk stations.
111